Making JavaScript Charts work with Stored Procedures and Entity Framework


, , , , , , ,

After adding one of the AmCharts examples in the CSHTML source, I had the graphics rendering code and a static array providing the chart data, and everything was displayed as it should. This application needed to be adapted so it displayed the metrics relating to the messages being processed by systems using Service Broker.
Again I used a stored procedure. This returns a table of dates against the number of messages processed on those dates, and takes as inputs startDate and endDate, both of which could be null. We’ll call this stored procedure ‘prMessagesProcessedByDay‘.

What I needed to achieve here was: a) Use Entity Framework to model the stored procedure and its response body, b) Add a controller to pass data between the Entity Framework model and the view layer, and c) Add some JavaScript to call the controller and render the data set as a chart.

Entity Framework Model
Right-click the Models folder in the Solution Explorer, and add new item. Here we want to add ADO.NET Entity Data Model, which will be a Database First (EF Designer from database).
When generating the model Entity Framework should have assigned the returned data as a ‘Complex Type’, which didn’t happen for some reason. In the Model Browser, I right-clicked on the ‘Complex Types‘ object for the model, and ‘Add New Complex Type…‘.

After selecting the stored procedure to import, the Context.cs code should look something like this:


Again both input variables are nullable, as the entire table should be returned if no date range is specified, and I should have the option of adding a feature for doing this later. And the returned variables were also nullable in case there was an empty table:


The next problem is that Entity Framework runs on the application server, whereas the JavaScript executes in the client’s browser, so the application would need to fetch the data through a controller that calls the stored procedure whenever the page loads.

Web API Controller
The way I’m doing this is through a Web API controller. Apparently it handles JSON and serialisation, which is required for the JavScript to populate an array. Autogenerating an empty Web API controller gives us the following:


When doing this, you might encounter error messages about variable types. The first thing to check is whether the Stored Procedure is assigning a primary key to the returned table – especially if the Web API template includes select, edit and delete actions. Here I needed to modify the stored procedure by prefixing one of the instructions with ‘SELECT NEWID()as Id‘.

Second problem that might be encountered is an HTTP 404 error when attempting to call the Web API when the application’s running. Removing all the NuGet packages and re-installing them fixed the problem.

Thirdly, the controller needed to perform some typecasting, as it didn’t work well with ‘complex types’. It needed the object GetprMessagesProcessedByDay_Result() to be declared as a list.

Eventually I ended up with something like this:


View Layer and Testing the Controller with JavaScript
Now there’s hopefully an Entity Framework model that’s accesible to the Web API controller, the next requirement is some JavaScript to send requests to it. The code for that would look something like:


This JavaScript section was repurposed from another tutorial, just to ascertain there was was a JSON response. After a few modifications it passed the following when the application was run:


Loading the Data into AmCharts
The chartData array included with the AmCharts example is in the same format as the JSON response, so switching the two should be straightforward.


To adapt the AmCharts code, I imported dataloader.min.js and inserted the following JSON request code in place of the dataProvider section.


And set the categoryField and valueField variables to the JSON response field names. Here’s the prototype:






Waterstones unfortunately didn’t have much in the way of Aristotle’s writings, so I settled on Thomas More’s Utopia. It’s a paperback of just 150 pages, including Professor Baker-Smith’s notes. However, it’s not an easy read: Though the language was simple, I found myself actually disagreeing, in places, with humanist ideas that I know were based on rationality.
Given More was educated and a very devoted Catholic, I expected his work would borrow rather heavily from Aristotle, Thomas Aquinas and Plato, and indeed all three were mentioned in places, and as I anticipated, More challenged the professed idealism of Christian society by contrasting that with the the reality of the European culture in which the Church actually was dominant.

A statement in the closing pages sums up the theme of the book:
[…] when I survey and assess all the different political systems flourishing today, nothing else presents itself – God help me – but a conspiracy of the rich, who look after their own interests under the name and title of the commonwealth.

And so More examined the nature of true commonwealth, and envisioned an economic, social, political and religious system with that as its primary goal. And as I read the book, it seemed that More was presenting another question: How much are we actually willing to sacrifice in order to achieve such a society, given that he appeared to argue that commonwealth and private property (and possibly by implication individuality) are mutually exclusive?

This is expanded on further in Appendix I, in which More made references to what seemed a common adage among Socrates, Cicero, Erasmus and Euripides: ‘Between friends all is common’. I read this as an assertion that private property is at the root of all conflict, that commonwealth is therefore necessary for lasting peace. But this seems unattainable. As multiple failed attempts to implement Marxism were to demonstrate, there would always be a minority who guard their wealth, power and status jealously. Appendix II might have been added to make the point that true commonwealth is only found in primitive societies.

The First Book
It’s quite possible the foundations of this work were decided upon during a real world conversation between Thomas More and his friend Peter Giles, while they were in the Netherlands.
Roughly in the style of Plato’s Atlantis, Book One opens with an account of how More met a traveller called Raphael Hythloday, who told him of some undiscovered nation of Utopia. Raphael, in turn, recounts a fictional debate between himself and Bishop John Morton. Raphael speaks as someone who cares little for status or wealth, and as someone with a pretty low opinion (mainly contempt) of the political class, landowners and aristocracy.
If every political system is a conspiracy of the rich, where better to start than with the European justice system? This fictional debate started with a criticism of the idea of returning a crime of theft with the death penalty, the general point being that crime (or certain crimes) is driven by poverty, poverty in turn is caused by the inequality of power between the labourers and land owners, and therefore the existing laws were ineffective and unjust. Not only that, but in returning theft with more severe punishments, More argued (through the character of Raphael, of course), this form of justice caused more damage than the original crime.
Raphael also pointed out that such a law set a precedent that undermined its moral position: ‘[…] where human law permits the death penalty, then what’s to prevent men from settling among themselves just how far rape, adultery and prejury ought to be tolerated?

And that argument is still relevant today. Too often society fails to uphold the sanctity of life and basic freedoms as principles, and masses of protestors and campaign groups lobby today for legislation without examining why such laws should be considered progressive or detrimental in the long term.

Throughout the conversation, Utopia is sporadically used as an ideal that the existing contemporary political and economic solutions are contrasted with. But it turns out this idyllic place isn’t so idyllic after all. Raphael goes onto describe Utopia in greater detail, and thereby Thomas More explains that the eradication of inequality and irrationality comes with a price…

The Second Book
In the second book we are presented with Utopia, a society in which all things are held in common – it is a true commonwealth. What isn’t there to like about this system of government?
Firstly, humans are irrational, and we’re all driven by the need for personal gratification. One of our tendencies is to acquire and store wealth, which is, of course, the beginning of inequality. Utopia was designed from the start to eliminate this.

Reading through this section, one gets the impression that the state of Utopia is so delicately balanced that it requires conformity and micro-management to keep itself in existence. Although there are relatively few laws, which aren’t interpreted in the legalistic way we’re familiar with in the real world, almost every aspect of the citizens’ lives are regulated. Even the clothing is uniform among most the citizens, quotas determine whether a citizen would live in whichever city, and their activities are governed by the state schedule.
Despite all this planning and regulation, meticulously envisioned by More, it isn’t long before the beginnings of another class system, and inequality of privilege, become apparent – though to a barely noticeable degree compared to the inequality that exists in the real world. Deference is given to the leaders, some of them getting better food, and priests distinguish themselves by wearing more colourful clothes – and yes, the state religion is used as one method of controlling the citizenry. Utopia also retains an underclass of slaves, who are held in such contempt that they’re made to wear gold as a method of devaluing it. Notably, the slaves are made to do the jobs too gruesome for the citizenry of Utopia. To a lesser extent there is the distinction between the manual workers and those selected for promotion to the ranks of intellectuals from which the state officials are drawn.

I definitely recommend having a look at this book, in which you’d find much more in the way of argument against the current establishment, a few explanations of why politicians make decisions that are seemingly bad, and also the absurdity of extreme rationalism. All of it’s still very relevant today.

Just Your Average Site Defacement


, , ,

Given the following sub-headline in The Independent’s reporting of the NHS site defacements yesterday morning, and the description of it as an ‘unprecedented attack’ the public would probably be wondering how bad the threat is to their medical information:

‘One analyst, says the hacks ‘appear to be deliberately targeted at a British public institution and in particular at an institution dealing with something which affects every member of the public, their health’

Calling themselves the ‘Tunisian Fallaga Team’, these people did a few other sites back in 2015, some of them for leisure centres in Ireland and suchlike that are often considered easy targets. The sites would have displayed something like this:



There’s actually nothing unprecedented about this, and how exactly did The Independent make the conection between the Tunisian Fallaga Team and ISIS, when it was essentially an anti-war message? So, the article opens with classic scaremongering.
Since The Independent doesn’t reveal which sites were defaced (were they even on the .nhs domain?), they could well have been sites only incidentally related to the NHS, such is the level of privatisation in England – that nobody drew attention to the incidents for about three weeks, until yesterday, is telling.

But what about the impact? Well, there isn’t much, apart from some embarrassment. Almost all the NHS public-facing sites are completely segregated from the systems that store and process patient records, and there are multiple layers of protection for the latter. In short, the data is only accessible from within the network, and using clinical applications that are authenticated. The threats I’m far more worried about are a) insiders, and b) private firms who are given access to the data.

OpenPGP for Windows Phone


, , , , , ,

A couple of interesting PGP applications are available for Windows Phone, which would be useful if you trust the Windows 10 operating system enough to not to upload your cryptographic keys to Microsoft’s servers. I trust it enough for my secondary, less sensitive, email accounts.
OpenPGP for Windows is used for managing the user’s PGP keys and encrypting/decrypting messages. The companion application, OpenPGP Contacts, is used for fetching and storing the public keys for contacts.

Generating a PGP Key
If you already have a public/private key pair, it can be imported in OpenPGP for Windows. If not, it can be created in this application.


And key parameters:


However, I strongly recommend generating the keys on a physical Linux machine.

Previously I have submitted two public keys to, and have added that server in the OpenPGP Contacts settings, which enabled me to store both keys here.

Encrypt Message
The application has a basic text editor in which to type messages. Whatever is contained here is converted to ciphertext that’s copied to whichever messaging application.


Pressing the ‘+’ button in the ‘To‘ field will display the public keys stored by OpenPGP Contacts, and the text entered here will be encrypted using that key. The encrypted message can be shared with other messaging applications on the device, to send it as an email or text message.


Decrypt Message
To decrypt, simply copy and paste from the messaging application to OpenPGP application, and select the private key for the decryption.



PGP Message Encryption for Web Mail (Part II)


, , , ,

Halfway through doing my last blog post on PGP, this Web mail pop-up caught my interest:


Mailvelope is a browser extension which ‘brings OpenPGP encryption to webmail services such as Gmail, Yahoo and others’. Does it provide any advantage over the other method of encrypting text files? Well, kind of. Mailvelope stores PGP keys locally and converts a plaintext message into ciphertext without the user having to leave the browser – that alone is a major improvement on usability. I had a few problems with the setup, as the recovery key for GMX’s encryption didn’t export properly to PDF (basically I’ve lost it).

Mailvelope’s FAQ page states that keys are only stored locally, and explains that security depends on the endpoints being resistant to attack. It is transparent about the extension scanning all Web pages to determine whether it’s a Webmail service being displayed.
What’s less obvious is the correct procedure must be followed when composing emails in order for Mailvelope to provide confidentiality. Web mail services (especially Google) typically save messages as you type, which means the plaintext is stored on their servers, and that makes pretty much the entire conversation accessible to third parties. This means you absolutely must compose emails in Mailvelope’s pop-up editor instead of the Web mail interface. We’ll come to this.

Creating or Importing Keys
Since I already have generated keys for my Web mail accounts, I’ll import them here. The way to import a key from a file in the Import Keys page is to open whatever private key in a text editor, and copy it into the ‘Import key as text‘ box. The public key will be derived from this.



The key can be managed and the public key uploaded to Mailvelope’s server. Because the verification email needs to be decrypted using another program, and the plaintext link is malformed, I submitted my public keys to the Ubuntu key server instead.


Back to Web Mail
According to the setup dialogues, GMX doesn’t have access to the crypto key. When setting this up, do not lose the recovery key, as the encryption cannot be reset without contacting GMX’s technical support team.

Mailvelope also works with Outlook, displaying a small pad icon in the top-right of the content box. Click that icon before doing anything else.


The following window should then be displayed:


Compose the email in the following window and click the ‘Encrypt‘ button.