I’ve always wanted to do a post (perhaps someday even a book) on cryptography throughout the 20th century, but what really inspired this was my observation that today’s symmetric encryption works in fundamentally same way as the Enigma machines. None of the information that follows here is classified, and I’ve taken care to mention only equipment that’s been obtained, dissected and publicly displayed by collectors across several countries.
Enigma and its Predecessors
Cryptology has a history stretching back at last 2,000 years, but up until 1900, high-level communications were generally encoded with the use of cipher disks, the Caesar Cipher, Discret, etc. There were a few very clever exceptions, but generally it took time to prepare the ciphertext, communicate it, and for the recipients to decrypt the message. The probability of a given message being intercepted by the adversary was also much lower.
From then on, the volume and rate of information being communicated on the battlefield rapidly increased, and so did the reliance on combat information systems. This progression, still ongoing, is the very reason there’s so much talk of ‘cyber warfare’ now. But the point here is that faster and more sophisticated methods of protecting the confidentiality of that information were needed, and this is where modern cryptography began to take form.
The earliest rotor-based cryptographic systems I could find (the ‘Enigma A’) were manufactured from 1923, and were based on a set of rotors with internal wiring that substituted plaintext characters. The wiring would form a circuit between the keyboard and an array of bulbs, which would light up when a character was pressed. It was essentially a substitution cipher, but with much less predictability than ciphers that merely shifted all characters x number of positions.
The image below, from Richard Brisson’s Cryptographic Artifacts collection, shows the typical rotors for one of the lesser-known device in their storage case, and we can see each has electrical contacts for the internal wiring.
It was on this idea that the more famous Enigma variants were based. With five rotors of 26 characters each (earlier versions had three), there were 11,881,376(-1) possible key settings ( or 26^n -1) – far too many to bruteforce manually. To make things much harder for cryptanalysts, one of the rotors would change position each time a plaintext character was repeated, thereby changing the ciphertext the plaintext characters were being mapped to.
What this meant was that adversaries would need a machine that replicated this operation exactly in order to break the encryption, and they did. The British were using a very similar device from 1937 called the Typex. In fact, it was so similar that at least one was used at Bletchley Park for the latter stages of decrypting the Enigma intercepts. They remained in service until the early 1960s.
The rotor-based design was also used in the United States Army’s field kit, with the M-209. This was a cheap, simple and portable device that printed a letter to paper tape when a character was selected and a lever pressed. Unfortunately the cipher was quite weak, having a fixed substitution sequence for each key setting, and so it was only used for short-duration messages between ground units.
The Cold War
Although the Enigma cipher itself was eventually compromised through reverse engineering and the Colossus project, machines based on the same principle were used by the West and the Soviets right up until the early 1970s. The later machines in use during the Cold War had more rotors (the Soviets’ Fialka had ten), increasingly complex wiring schemes and additional features that made their ciphers harder to break.
According to Richard Aldrich’s book, GCHQ: The Uncensored Story of Britain’s Most Secret Intelligence Agency, the encryption provided by a string of Hagelin/Crypto AG products was solid enough that the NSA, until 1975, had to engage in some complicated effort, involving other agencies, to get the manufacturer to subvert them. Aldritch doesn’t say precisely how, but it made me wonder exactly how much codebreaking was actually done after World War II.
From the Electrical to the Electronic
Another thing that happened during the Cold War was that electrical-mechanical cipher machines were gradually replaced with electronic-mechanical successors – they were enhanced by discrete components mounted on circuit boards for real-time communications over the telecommunications infrastructure, but at the core still based on the Enigma design.
At this time, computers used paper tape for input and output, with data encoded as 1s and 0s. This led to the XOR operation becoming an underlying feature of the next generation cryptographic systems. Basically, if messages can be encoded as 1s and 0s (not necessarily ASCII), they can be XORed with a keystream. Doing this would encrypt the message until someone else determined the keystream and repeated the XOR operation. If the keystream has a limited length, there’ll be some repeating pattern in the ciphertext and the encryption is breakable. However, if the plaintext stream and keystream are of equal length, and the keystream is used only once, the encryption becomes totally unbreakable. Cryptomachines that did this are often referred to as ‘mixers’.
Then the electronic systems appeared. Sometime around the mid-1970s (I’m unsure of the exact date), the recently mothballed CLANSMAN was introduced into the British Army as its integrated comms system. Most of it consisted of your typical det, vehicle-mounted and manpack radios, but it was also designed to support primitive low-bandwidth data communications. Consequently, the CLANSMAN range had to include electronics for encrypting that data. But how did we get from electronic-mechanical to purely electronic encryption?
The answer lies with the Telsy T-500, which marked the separation between cipher and cryptographic key hardware components in 1976. From the primitive PCB shown below, we can see the copper tracks functioned as a ‘substitution box’ – directly analogous to both the Enigma rotors and the substitution boxes of the modern DES cipher. Today’s cryptographic chips work precisely this way, with multiple blocks of nanometre tracks re-arranging and XORing bits between input and output pins.
Telsy TS-500 Cryptographic Key, Crypto Museum
However, the transition from the mechanical to the electronic, which occured within the space of a decade, created new problems, and handling crypto material became a complicated process. Kerckhoff’s Principle wasn’t really applied until much later, which in practical terms meant the security depended on adversaries knowing as little as possible about the hardware being used, and not solely on how cryptographic keys themselves were protected. It’s for this reason that those who operated in the INFOSEC/COMSEC field prior to it becoming an industry remain carefully guarded about the hardware. Even some of the older ciphers used by the NSA remain classified for this reason, taking into account adversaries might have been archiving intercepts.
Bear this in mind whenever commercial vendors boast about their ‘military grade’ encryption – true ‘military-grade’ encryption was fragile, not being open to review, inspection, discussion or criticism.
Commercial Technologies and Standardisation
The next major developments came shortly after, when advances in electronics manufacturing gave us consumer integrated circuits and the microprocessor, and wide area networking began to connect government departments, academic institutions and large private entities. The commercial world was communicating information digitally, and it needed to protect that with sufficiently strong encryption. The NSA set about working with IBM to develop something based on the Lucifer algorithm that could be implemented on mass-produced hardware. The outcome was the Data Encryption Standard (DES).
Soon the information systems technology itself became more standardised, which made for better interoperability across government, military and private sector comms. From around 2000, a batch of hardware-based cryptographic systems appeared that could be deployed with virtually any common telecommunications method, such as fax, GSM, dial-up Internet, etc. and they used industry standard interfaces for key storage devices, such as Smart Card readers and PCMCIA.
The Crypto Communities
The Internet brought together a global cryptological community that was prepared to openly share their ideas, and they were certain to pick holes in any cryptographic system that became publicly-known. Meanwhile a wider community of hackers often found holes in how they were implemented. Modern encryption was no longer limited to the realm of the Official Secrets Act and suchlike – it became one of the most openly, intelligently and heavily debated subjects, and as a result, more superior, peer reviewed cryptographic systems became freely-available to the masses. Sometimes the community outsmarted attempts by the US government to regulate this, for example by publishing vulnerabilities in the mandated Clipper Chip key escrow system, and by wearing the PGP algorithm on T-shirts to defeat export restrictions. It became clear the three/four-letter agencies weren’t necessarily the sharpest knives in the drawer.
And so it was probably for this reason that the effort to establish a successor to DES was a collaborative one involving pretty much anyone capable of submitting a decent algorithm. After two years of review, it announced the Rijndael cipher as the Advanced Encryption Standard, which is arguably stronger, more versatile and resilient than ciphers that came before it (although NIST appears to have chosen the fastest algorithm rather than the strongest). This approach was actually successful enough that specific implementations of AES encryption are currently deemed suitable for protecting Top Secret information.
Spot the difference: Enigma rotor wiring (left) and the Rijndael cipher (right)
Throughout the history of encryption throughout the past 100 years, we can see a logical progression from mechanical to electro-mechanical, to electrical, to electronic and finally software implementations. Perhaps the future of cryptography lies in the next layer of abstraction – pure data.
Useful Links
GESSLER, N. 2005. Collections in Cryptology – Soviet Machines: Russian Fialka. [WWW]. https://web.duke.edu/isis/gessler/collections/crypto-fialka.htm. (12th September 2013).
REUVERS, P. SIMON, M. 2013. Crypto Museum: Enigma A. [WWW]. http://www.cryptomuseum.com/crypto/enigma/a/index.htm. (16th September 2013).
REUVERS, P. SIMON, M. 2013. Crypto Museum: Siemens M-190. [WWW]. http://www.cryptomuseum.com/crypto/siemens/m190/. (12th September 2013).
REUVERS, P. SIMON, M. 2013. Crypto Museum: Telsy TS-500. [WWW]. http://www.cryptomuseum.com/crypto/telsy/ts500/index.htm. (12th September 2013).
RANDOMBIT.NET. 2013. Crypto discussion list. [WWW]. http://lists.randombit.net/mailman/listinfo/cryptography. (12th September 2013).
SIMPSON, R. 2012. Crypto Machines: Russian M-125 Fialka. [WWW]. http://www.jproc.ca/crypto/russian_m125_fialka.html. (12th September 2013).