003 | Al-Kindī and Frequency Analysis

“When the Letters Gave Themselves Away”

The message may be disguised. The alphabet may have changed its clothes. But underneath, language still has habits, and habits are the kind of thing attackers collect in killing jars.

Our first artefact, the scytale, made meaning depend on a physical staff. The second, the Caesar cipher, made meaning depend on a small numerical shift. Both are about concealment. Al-Kindī and frequency analysis introduce something different: systematic attack.

This is one of the birth moments of cryptanalysis.

People occasionally comment that I have an impressive and memorable name. But Al-Kindī replies: “Hold my inkpot.” Although often referred to simply as Al-Kindī, or even Al-Kindi for diacritically-challenged typists, his full name is rather impressive.

Abū Yūsuf Yaʻqūb ibn ʼIsḥāq ibn aṣ-Ṣabbāḥ ibn ʻImrān ibn Ismāʻīl ibn al-Ashʻath ibn Qays al-Kindī

أبو يوسف يعقوب بن إسحاق بن الصباح بن عمران بن إسماعيل بن الأشعث بن قيس الكندي

Yaʻqūb, father of Yūsuf, son of ʼIsḥāq, descended through aṣ-Ṣabbāḥ, ʻImrān, Ismāʻīl, al-Ashʻath, and Qays, of the tribe of Kindah.

What’s in a name? In al-Kindī’s case, quite a lot.

Al-Kindī lived in the intellectual world of ninth-century Abbasid Baghdad. He was a philosopher, mathematician, scientist and scholar, associated with the translation movement that brought Greek philosophical, mathematical and scientific texts into Arabic. A polymath of the first water: philosophy, optics, medicine, astronomy, music, mathematics, metaphysics, physical science, ethics, and, of direct interest here, cryptanalysis.^[1]

The work that matters here is usually rendered in English as A Manuscript on Deciphering Cryptographic Messages. Its Arabic title is often given as Risāla fī Istikhrāj al-Muʿammā, a treatise on extracting or uncovering obscured messages. It is one of the earliest known systematic accounts of how to break encrypted writing. Not guess. Not pray. Not declare that the gods have whispered the plaintext through a conveniently positioned curtain.

Count.

The method is wonderfully blunt. If you know the language of the message, take a long enough sample of ordinary writing in that language and count how often each letter appears. Rank them: most common, second most common, third most common, and so on. Then count the symbols in the encrypted message. The most common ciphertext symbol is a candidate for the most common plaintext letter, the next most common symbol for the next most common letter, and so forth. Try the likely mappings. Look for words. Adjust. Repeat until the message begins to talk.^[2]

This works because a simple substitution cipher changes the names of letters but not their behaviour. If one plaintext letter is common, its disguised ciphertext replacement will also be common. The cipher has hidden the face but left the walk.

That is frequency analysis.

In English, we are used to saying that E is common. In Arabic, the relevant letters and combinations are different, because languages have their own statistical fingerprints. That is the point. Frequency analysis is not about one magic alphabet. It is about the idea that language is not evenly random. It is lumpy, patterned, repetitive and full of tell-tale habits, like most human systems when left unsupervised.

Al-Kindī did not merely wave vaguely at this. Later study of the Arabic cryptological tradition notes that he calculated Arabic letter frequencies using a sample of 3,667 letters. That detail matters. It turns “some letters seem common” into measurement. It makes cryptanalysis empirical. The ciphertext is no longer a mystical veil; it is data.^[3]

This is why al-Kindī belongs in a history of cybersecurity rather than just a history of puzzles. Cybersecurity is not only about designing controls. It is about testing claims. Does the protection still work when the attacker knows the method? Does the system leak structure? Does the evidence support the confidence? Can the defender’s assumption be measured, modelled and broken?

Frequency analysis says: let us find out.

It is also a tidy answer to the false comfort of “there are too many possibilities”. A general monoalphabetic substitution over a large alphabet has a huge number of possible mappings. Trying every one by brute force is absurd. Frequency analysis avoids the whole swamp. It does not try every key. It exploits the structure left behind by the plaintext.

This is the difference between blind search and informed attack.

Imagine an encrypted message where one symbol appears far more often than the others. Another appears frequently in certain positions. A pair of symbols turns up again and again. Perhaps one symbol often begins a word. Perhaps another often ends one. If the attacker knows the language, the writing conventions, the likely subject, the expected greeting, or the names involved, the wall starts developing cracks.

A cipher may hide letters. It may not hide language.

That lesson keeps returning. Modern cryptography works very hard to make ciphertext look statistically useless. Good encryption should not preserve obvious structure from the plaintext. If encrypted medical records, diplomatic messages or disk sectors retain visible patterns, then they may be vulnerable to partial or full exposure. We have more mathematics now, and faster machines, and standards with stern document numbers, but the underlying demand is recognisable: the output must not betray the input.

Frequency analysis also teaches the limits of an attack. It is not magic.

But it is powerful.

It needs enough ciphertext. Short messages may not show the expected distribution. It helps if the attacker knows the language. It is aimed at simple substitution; more complex systems can deliberately flatten frequencies or split common letters across several symbols. Nulls can be inserted. Homophones can be used. Codes can replace whole words or names. Polyalphabetic ciphers, such as the later Vigenère system, were partly a response to this sort of attack, though repetition would, eventually, betray them too. Security history is largely one side inventing a rug and the other side learning where the dust went.

Nor is frequency analysis only about single letters. Later Arabic cryptologists looked at combinations too: pairs, triples, common openings, likely endings, and the habits of particular kinds of text. That is already recognisably modern. Attackers do not attack the ideal cipher in a textbook; they attack real messages, written by real people, in real contexts, with predictable phrases, repeated formats, standard greetings, known names and deadlines that make everyone careless.

The broader Arabic cryptological tradition matters here. Al-Kindī was not a lone genius in a vacuum, because nobody is, however much statues try to imply otherwise. Arabic scholarship had already developed intense attention to language: grammar, lexicography, poetry, Qur’anic study, administration and translation all encouraged close examination of words, letters and patterns. Al-Farāhīdī, the eighth-century linguist and lexicographer, is an important predecessor. His work on Arabic words, phonology and combinatorics helped form the intellectual soil from which later cryptology grew.^[4]

This is one of the places where popular Western histories need a firm correction. Western intellectual history often remembers the Greek sources and the later Latin scholastics, while treating the Arabic thinkers who preserved, transformed, and extended that material as a rather inconvenient middle chapter. Al-Kindī is one of the casualties of that bad habit. The story is often told as if cryptography passes from classical Rome to Renaissance Europe with a vague misty gap in between. The Arabic and Islamic cryptological tradition was not an intermission. It was central, and it was transformative.

One modern summary of Arabic cryptology explicitly describes it as a school: scholars learned from one another, built on earlier work, travelled across regions, and contributed over centuries. The list does not stop at al-Kindī. It includes names such as Ibn Wahab al-Kātib, Ibn ʿAdlān, Ibn al-Durayhim and al-Qalqashandī, among others. The tradition developed classification of ciphers, letter-frequency work, bigrams and trigrams, and techniques for making frequency attacks harder.

That matters for attribution. It is good to name al-Kindī. It is also dangerous to pick just one person, standing on the shoulders of giants so to speak, and with later individuals standing upon his. To name just him would be like talking about Watson & Crick when describing the discovery of the structure of DNA, quietly erasing the enormous contribution of Rosalind Franklin to the discovery.^[5]

There are other missing people too. Who copied the manuscripts? Who preserved them? Who prepared the paper and ink? Who served as secretaries in the chancery systems where secrecy mattered? Who carried messages? Who did the tedious counts? Who noticed errors in copying? Who taught the language skills that made the analysis possible? Who maintained the libraries and archives through which the text survived? History often gives us the scholar’s name and then quietly ignores the labour that made the scholarship possible.

The translation movement around al-Kindī also complicates any tidy cultural ownership story. The “Kindi circle” involved translators, patrons and scholars, including many Syrian Christians or people of Syrian extraction. Knowledge moved through languages, faiths, courts, libraries, administrative needs and personal networks. The archive is not a single heroic mind. It is a complex supply chain with cleanly legible handwriting.^[1]

It is also worth noting that manuscript cultures, elite courts and later histories frequently preserve the names of key men while obscuring the work of everyone else. Silence in the record is not proof of absence. It is often proof of who was considered worth recording.

Al-Kindī’s importance, then, is not just that he broke ciphers. It is that he changed what breaking meant.

Before this, a hidden message might look like a contest of tricks. After this, it becomes an object of analysis. Count the letters. Compare the distributions. Test a hypothesis. Revise the mapping. Use language against itself. The defender’s mechanism is no longer judged by how clever it looks, but by what it leaks under attack.

That idea survives everywhere in cybersecurity.

Password cracking uses frequency, probability and human habit. Malware detection looks for patterns in behaviour. Traffic analysis studies what communication reveals even when content is hidden. Side-channel attacks exploit measurable leakage. Compression, randomness tests, cryptographic distinguishers and modern statistical attacks all live somewhere downstream of the same unpleasant truth: systems leak structure, and attackers count.

The Caesar cipher taught us that a small secret is not enough.

Al-Kindī taught us that a large-looking secret may still have fingerprints.

The message changes costume. The language gives itself away.

And the adversary, irritatingly, has learned arithmetic.

References

[1] Peter Adamson, “Al-Kindi”, Stanford Encyclopedia of Philosophy, for al-Kindī’s broader intellectual context, the ninth-century translation movement, and the Kindi circle. (Stanford Encyclopedia of Philosophy)

[2] Lyle D. Broemeling, “An Account of Early Statistical Inference in Arab Cryptology”, The American Statistician 65:4, 2011, for al-Kindī’s use of relative frequency analysis and the framing of this as early statistical inference in Arab cryptology. (JSTOR)

[3] “Arabic Origins of Cryptology”, drawing on Muḥammad Mrayati and colleagues’ work on al-Kindī’s treatise, for the 3,667-letter frequency count, Arabic cryptology context, al-Farāhīdī, cipher classification and the wider Arab school of cryptology. (Muslim Heritage)

[4] 1001 Inventions, “Code Breaking a Thousand Years Ago”, for an accessible account of the rediscovered manuscript and a paraphrasable version of al-Kindī’s method: count letters in a known plaintext sample, count ciphertext symbols, then compare the rankings. (1001 Inventions)

[5] Franklin had done much of the hard experimental and interpretive work. Watson and Crick supplied the successful model-building synthesis, helped by access to Franklin’s data that she had not freely handed them, including their use of her Photo 51 without her permission, and without full acknowledgement at the time.