Ordering Results in DataTables.js by JSON-Defined Columns


, , , , ,

The main DataTables.js API call is straightforward:

function SetupDataTable(data) {
var c = JSON.parse(data);
var url = ResolveUrl("~/Search/JsonResults");

table = $("#SearchResultsTable").DataTable({
"pagingType": "full_numbers",
"pageLength": 10,
"lengthMenu": [[10, 25, 50, -1], [10, 25, 50, "All"]],
"processing": true,
"serverSide": true,
"ajax": url,
"columns": c,

Aside from calling the API to render the DataTable, this JavaScript function makes a request to the /Search/JsonResults controller action when a page load is triggered. The controller action, in turn, will fetch the subsequent x number of records from the source. Set the ‘serverside‘ attribute to ‘true‘ for this.

The (abbreviated) controller action for this originally looked something like:

Ordering results in DataTables should be easy, yes? Actually, no, it wasn’t, since the application here used a single DataTable instance to present tabled data for doctors, organisations, third-party providers, other personnel, sites, tests, etc. In this case, the data fields/columns are defined by the JSON response from the controller, and they’ll change according to the dataset set being selected by the user.

This was my initial code, simply to get a working LINQ query and to create a placeholder for a dynamic variable:

var rows = results.DataSet.AsEnumerable().OrderBy(r =>
r.Field("Organisation Postcode")).Skip(model.start - 1).Take(model.length);

The above is a LINQ statement to sort the data (ascending) by Organisation Postcode – I picked that variable arbitraily from the JSON response. We use DataSet to create an in-memory representation of the data being fetched, so the DataAdapter would limit the read volume to the number of records being requested by DataTables.js. LINQ is used to query this in-memory data instead of the database itself.

What we need to do now is replace the ‘Organisation Postcode‘ with something that could dynamically represent any column name in the JSON response. The LINQ queries I ended up with were:

var sortedResults = order.column == 0 ? results.DataSet.AsEnumerable() :

order.dir == "desc" ? results.DataSet.AsEnumerable().OrderByDescending(o=> o.ItemArray[order.column] is DBNull ? DateTime.MinValue : o.ItemArray[order.column]) :

results.DataSet.AsEnumerable().OrderBy(o => o.ItemArray[order.column] is DBNull ? DateTime.MinValue : o.ItemArray[order.column]);

And the Skip() and Take() operations have been shifted to the row creation loop:

Here we have roughly the same thing as before:

Passing a somewhat different LINQ statement:

o => o.ItemArray[order.column] is DBNull ? DateTime.MinValue : o.ItemArray[order.column]

In the first part, ‘o => o.ItemArray[order.column]‘, ‘o‘ represents the DataRow in the DataSet. It means pick value ‘order.column‘ from the ItemArray, and order the output by that value.

The local variable is derived from:

var order = new Order() { column = int.Parse(Request["order[0][column]"]), dir = Request["order[0][dir]"] };

This line, in turn, creates a new instance of the following class:

public class Order
public int column { get; set; }
public string dir { get; set; }

Now, it might be the case that null values are returned for a column, which screws the ability to pick that column as a sorting criteria, so we use the following to populate the null fields with whatever is derived from DateTime.MinValue:
is DBNull ? DateTime.MinValue

The above is a short-hand way of creating an if/then statement. If the value is null, set the value to DateTime.Min.

Note: Parsing JSON Response to Get Columns

There is another method in the SearchController:

Set the variable ‘result‘ simply as the ‘[‘ character to begin with, to mark the beginning of an array.
In the next instruction, if the result is not ‘[‘, in other words, if there’s already something in the array, we append a comma to mark the end of the last element before executing the next instruction. If we’re not at the start of an empty array, ‘result‘ is appended with:
{ "data": "{0}", column.logical_name }

The ‘column.logical_name‘ refers to a string coming from the database table that identifies it as a column name.
After the array has been populated, ‘]‘ is appended to ‘result‘, to mark the end of the array, and it is returned to the calling function as a JSON response.


The Book of Samurai: The Collected Scrolls of the Natori-Ryu


, , , ,

Antony Cummins seems pretty knowledgeable about the Samurai and warfare in feudal Japan, and, intrigued by his attempt to recreate the Natori-ryu as a modern-day Samurai school and his claims to have translated its syllabus from the 17th century, I bought myself The Book of Samurai.

Generally I like his material very much, but the one serious criticism I really should mention, since there’s a lot of this on YouTube, is his demonstration of rather careless handling of Samurai swords. Yes, I know that one doesn’t need training to cut things with a katana, but those things are bloody dangerous. An untrained person screwing around with one is liable to seriously injure themselves and others around them. Several times Cummins came dangerously close to severing his axillary artery during his ‘speed draws’.
Before doing anything with a 3ft live blade, one should spend a few years (at least) practising with a bokken under the guidance of a Sensei who knows the history and is fussy about traditions, etiquette, techniques, discipline, etc. These are observed for very good reasons. Learn to develop and ‘extend’ Ki so there’s control and precision in the bokken’s movements, develop the constant mindfulness of everything around you.

Anyway… the premise of The Book of Samurai is the Natori-Ryu was founded sometime in the 1500s, to circulate the teachings of the then Samurai with a syllabus that focussed on things related, directly and tangentially, to warfare.
It was after 1654, we’re told, that a Natori Masazumi became the master of the Natori-Ryu. He was to collate the Samurai teachings into a syllabus and create an encyclopaedia of all things Samurai, and this expanded into many other areas than warfare, tactics and strategy. It is within the scrolls that we find information about the Ninja/Shinobi arts, though I think you’d find that in Cummins’ other book, The Book of Ninja.

The Book of Samurai appears to be a pretty decent and credible work. My Aikido instructor, who’s far more knowledgeable about Samurai-related literature, thinks it’s legit. The caveats are that Cummins himself isn’t conversant or literate in Japanese, and the absence of illustrations/photographs of the source material makes it difficult to verify independently.
Personally I believe, at the very least, the book reveals to us subjects that were addressed in the source material, and what was generally written. This is why, even though it’s excellent as a syllabus and educational, I approach it with skepticism.

As a stand-alone book, the content is excellent. Aside from what we’d expect in a Samurai manual, much of the syllabus is remarkably practical today. For example, there’s an etiquette and strategies for preventing conflict, its escalation and other problems that might ensue. There’s advice on travelling at night, and what to do when confronted by a group of potential attackers. You’ll also find sections on emergency preparedness, with advice that’s very similar to what’s given in a corporate fire safety training session. Anyone who’s into the whole Ninjutsu tradecraft thing would certainly like it.

A Solution for Synchronising and Sharing Encrypted Text


, , ,

The SafeNotes application caught my interest the other day, while casually browsing the Play Store. Initially I thought it was simply the usual password protected notes thing with a nice UI. The application is pretty straightforward to use. Just press on the key icon to set a password for a note, and the padlock icon to encrypt or decrypt it.

Then I noticed the ‘Add existing note‘ option. It turns out the application can be used to sync notes with a site called ‘ProtectedText.com‘, that stores the ciphertext.


Visiting the URL, we get a rather basic-looking site, which I haven’t come across before. It’s a service for storing encrypted notes, which can be used with or without Safe Notes. The encryption is handled by a main.js file which encrypts and decrypts the content locally, so the plaintext should never be stored on the server.
I’ve briefly examined the JavaScript, and it appears pretty legit. A WireShark capture during a browser session with one of my sites here revealed nothing in plaintext beyond a DNS lookup for protectedtext.com.

Each text file is referred to as a ‘site’, and there is a reason for this. Entering a unique name in the text field will create a new ‘site’ at the URI, and each tab could be viewed as an individual page. If several people know the full address and the password, they could use the site as a ‘drop box’ or a private Web site.

To sync the Safe Notes application with a site, just press the ‘Add existing note‘ option, enter the note’s URL, and then enter the password for it.

After the application is synched, it’s simply a matter of uploading or downloading changes. I’m not sure if there’s a version control system for multiple users.

OpenPGP Encryption using Gpg4win and Kleopatra


, , , , , , ,

  • Downloading and Installing Gpg4win
  • Creating and Importing Your OpenPGP Keys
  • Encrypting a Message
  • Decrypting a Message

I’ve covered PGP encryption in more depth elsewhere on this blog, for anyone who might be interested in the finer details. Essentially, here we’re going to use an application called Kleopatra to create an OpenPGP ‘key pair’, which consists of a public key and a private key. To keep things simple, since this is a beginners’ guide, the general rule is:

  • The public key is used only for encrypting a message
  • The private key is used only for decrypting a message

Kleopatra is particularly useful because it works with most Web mail services that don’t natively support PGP encryption. Once everything’s set up, it’s also relatively easy to use as a matter of routine.

Downloading and Installing Gpg4win

Kleopatra is the graphical interface for the collection of components and programs that make up the Gpg4win system. To install this, we have the option of donating to the Gpg4win project and getting the latest version, or going directly to the download page linked to older releases (look for a gpg4win.exe file).
After installation, there’ll be a Kleopatra icon somewhere on the desktop or the applications menu. This is primarily a key management application, but as we’ll see, it can be used to encrypt and decrypt messages.

Creating and Importing Your OpenPGP Keys

The first thing you’ll need is a key pair – your public and private key. You’ll also want to import others’ public keys at some point.

Select ‘File‘ and ‘New Key Pair…‘. In the next window, we want to create a personal OpenPGP key pair.

In the advanced settings you get the option of using RSA or DSA or EDCSA. EDCSA is less widely supported, and if I remember correctly, DSA is more reliant on a sufficiently good random number generator for its security. But the main comparisons are in speed of encryption/decryption, which isn’t something I’m overly concerned with. So here I’ve gone with RSA with a key size of at least 2,048 bits.

You might notice the Certification, Signing and Encryption options are set by default. The ‘Authentication’ option, I think, is used when creating keys to secure SSH and Remote Desktop connections, instead of messages.

Lastly, ensure you set a reasonably strong password when prompted, and I strongly recommend making a backup of the keys on another storage device that’s unlikely to be lost or mislaid.

At the top menu, there is also an ‘Import…‘ button. Sometimes you’ll find public keys attached to emails, whether as a signature or as a file attachment. If it’s a signature, copy and paste it into a text editor, save the file with the .asc extension and import it into Kleopatra.

Exporting and Sharing a Public Key

For others to encrypt messages addressed to you, it’s necessary to share or publish your public key. To do this in Kleopatra, right-click on the entry, and select ‘Export…‘.

This will export the public key as a .asc file, which can be opened in a text editor, and everything in the file can be used as your email signature, posted on a Web site, etc. Ensure the header says ‘PUBLIC KEY’, not ‘PRIVATE KEY’.

Encrypting a Message

Assuming you have the public key for the person you wish to communicate with, it’s possible to encrypt messages to that person within the Kleopatra application, and copy that ciphertext into a Webmail interface when composing an email.

Click the ‘Notepad‘ button in the top menu to access Kleopatra’s text editor. Enter the message in the text field, and click the Recipients tab when done. Here we have drop-down menus from which the intended recipients can be selected.

Now click ‘Sign / Encrypt Notepad‘, and the contents of the text field under the Notepad tab will be encrypted.

Now, it’s simply a matter of sending this ciphertext as an email. The recipient should be able to use his/her private key to decrypt the message.

Decrypting Messages

A similar process to the above is used for decrypting messages. Imagine, if you will, that someone has emailed you ciphertext similar to what was generated above. This email can be copied and pasted into Kleopatra’s text editor, but this time we click the ‘Decrypt / Verify Notepad‘ option. Kleopatra will request the password for your private key, and the ciphertext will be replaced by the plaintext message.

Improving COMSEC With ProtonMail


, , , ,


  • Introduction
  • Some Tips on Registering an Account
  • Encryption for Non-ProtonMail Users
  • ProtonVPN

ProtonMail is an email service that provides a level of security surpassing that of most the alternatives. Pretty much the entirety of its security model is based on well-implemented encryption – to be more specific, AES 128, RSA with 4,086-bit key and OpenPGP. All messages are encrypted on ProtonMail’s servers, and are only ever decrypted in the user’s browser or client application – certain features extend this to communication between ProtonMail users and recipients using another service. This means the service provider cannot turn over the messages to a third-party, and the information is protected even if the mail servers are compromised. The servers, by the way, physically exist in the company’s data centre in Switzerland – they’re not virtual machines provisioned by a third party.

Additional security features include a ProtonVPN service, availability through TOR, and the option to register accounts without supplying personal information such as cellphone number. This is very useful if we’re using publicly accessible (but, of course, not well advertised) list servers to communicate, or we’re concerned about the security of our organisation’s extranet/portal. Many of us have already taken that security measure with other services.

So, what’s the catch? In comparison to Outlook and Gmail, ProtonMail’s free service accounts are limited in storage capacity, features and the number of messages that can be sent, though it’s still very usable. The company’s business model is reliant on subscriptions to the paid services, which are comprehensive for an annual subscription of 48 Euros.
Also, while it provides decent end-to-end encryption, ProtonMail by itself is not suitable for communicating highly sensitive information – additional layers of security would be required.

Some Tips on Registering an Account

A prerequisite for communicating securely, of course, is to use a computer that’s reasonably secure. This means having the latest updates installed for your operating system and anti-virus software. Also, you need a browser that’s regularly updated. And avoid using a public access computer. In short, take the same precautions you would as when doing online banking.

Only three things are required for registering a ProtonMail account: a chosen email address, a password, and your usual email address for validation/recovery. The latter can be deleted from your account settings after, if you choose. After registering my account, I deleted the email containing the verification code, so my ProtonMail address wouldn’t be revealed and its account reset in the unlikely event my usual account is compromised.

Choose a strong password, as ProtonMail’s encryption is only as good as this. Here I’ve chosen a long string of random letters, numbers and special characters. Instead of writing this password on paper, I’ve used an offline password manager, which stores login details in an encrypted container file – there are several that are freely available, including Password Safe, KeePass and F-Secure Key. It’s a good idea to make a backup of the encrypted password manager file.

Regarding your email address, you might wish to keep that private among a few associates for PERSEC reasons. Don’t re-use it across third-party Web sites, if you don’t want others to associate it with you or your social media profile. Since registration and access of your account is over an HTTPS connection, nobody on the outside should be able to determine your full email address. ProtonMail’s VPN service enables access to the extranet/portal without others knowing, if you’re away from the home network.

In the settings there are several security-related configurations that relate to access and recoverability of the account: It is possible to remove the email address used for resetting your password – removing it protects your ProtonMail account if the recovery account is compromised, but it renders the ProtonMail account permanently inaccessible if the password for that is forgotten.
Slightly related, there is the option to configure two-factor authentication (commonly referred to as ‘2FA’). Enabled, this protects the account if the password is compromised, as an authenticator application would be required to complete the login. If the device running the authenticator application is lost, that might also be an issue.

Encryption for Non-ProtonMail Users

Message security is very strong between ProtonMail addresses, but what if either the sender or recipient is using another email service provider? There are two ways of encrypting emails in this case.

The first method involves a password that’s shared beforehand with associates. When sending an email to a recipient who’s not using ProtonMail, compose the message in the normal way, but click the ‘Encryption‘ button at the foot of the window. All the recipient would need to do is open the email, follow the link to a Web application and enter the password to decrypt the message within the browser.

ProtonMail allows the recipient to send a protected reply, which is again encrypted in the browser, and independently of other email service providers.

Sending from a non-ProtonMail address is a little more awkward, as it requires installing an application for managing OpenPGP encryption. Essentially it’s the same procedure we’re all familiar with for using PGP encryption, but ProtonMail automatically decrypts the messages.
In the Settings, in the ‘Keys‘ page under the ‘Email encryption keys‘ heading, there are links for downloading your public key as a .asc file. This can be opened in a text editor, and the contents pasted as your email signature. Recipients could use this to encrypt emails to your address, using an OpenPGP application, such as Mailvelope (a Firefox/Chrome browser extension) or gpg4win. Only the intended recipient with the correct private key could decrypt and read the email.


Some of us already use VPNs and Tor to access the extranet/portal, so ProtonMail’s VPN service, which is just as easy to use as the Tor Browser, is another welcome addition. This is more to add another layer of protection to your login details and the fact you’re accessing the portal, especially if you’re using free WiFi services ‘in the field’ or otherwise are away from the home network – DNS Leak Protection is always enabled here, by the way.

Using the ProtonVPN service this is simply a matter of downloading the client, entering the ProtonMail account details and clicking the ‘Connect‘ button.

There are a couple of settings to look at here. You can create profiles, to save the trouble of reconfiguring the connection for a given use case.

Here we have the option to use Tor or P2P or a conventional VPN service, at least for the duration of the trial period. ProtonVPN client can be configured to automatically start and connect when the local machine is switched on.