Editing Frequency analysis (section)

== Frequency analysis for simple substitution ciphers ==
In a simple [[substitution cipher]], each letter of the [[plaintext]] is replaced with another, and any particular letter in the plaintext will always be transformed into the same letter in the ciphertext. For instance, if all occurrences of the letter {{samp|e}} turn into the letter {{samp|X}}, a ciphertext message containing numerous instances of the letter {{samp|X}} would suggest to a cryptanalyst that {{samp|X}} represents {{samp|e}}.

The basic use of frequency analysis is to first count the frequency of ciphertext letters and then associate guessed plaintext letters with them. More {{samp|X}}s in the ciphertext than anything else suggests that {{samp|X}} corresponds to {{samp|e}} in the plaintext, but this is not certain; {{samp|t}} and {{samp|a}} are also very common in English, so {{samp|X}} might be either of them. It is unlikely to be a plaintext {{samp|z}} or {{samp|q}}, which are less common. Thus the cryptanalyst may need to try several combinations of mappings between ciphertext and plaintext letters.

More complex use of statistics can be conceived, such as considering counts of pairs of letters (''bigrams''), triplets (''trigrams''), and so on. This is done to provide more information to the cryptanalyst, for instance, {{samp|Q}} and {{samp|U}} nearly always occur together in that order in English, even though {{samp|Q}} itself is rare.

=== An example ===

<!-- decryption alphabet
ABCDEFGHIJKLMNOPQRSTUVWXYZ
kxpqaycdefbhijvlwsouzrntgm
-->
Suppose [[Alice and Bob|Eve]] has intercepted the [[cryptogram]] below, and it is known to be encrypted using a simple substitution cipher:     
 {{not a typo|LIVITCSWPIYVEWHEVSRIQMXLEYVEOIEWHRXEXIPFEMVEWHKVSTYLXZIXLIKIIXPIJVSZEYPERRGERIM
 WQLMGLMXQERIWGPSRIHMXQEREKIETXMJTPRGEVEKEITREWHEXXLEXXMZITWAWSQWXSWEXTVEPMRXRSJ
 GSTVRIEYVIEXCVMUIMWERGMIWXMJMGCSMWXSJOMIQXLIVIQIVIXQSVSTWHKPEGARCSXRWIEVSWIIBXV
 IZMXFSJXLIKEGAEWHEPSWYSWIWIEVXLISXLIVXLIRGEPIRQIVIIBGIIHMWYPFLEVHEWHYPSRRFQMXLE
 PPXLIECCIEVEWGISJKTVWMRLIHYSPHXLIQIMYLXSJXLIMWRIGXQEROIVFVIZEVAEKPIEWHXEAMWYEPP
 XLMWYRMWXSGSWRMHIVEXMSWMGSTPHLEVHPFKPEZINTCMXIVJSVLMRSCMWMSWVIRCIGXMWYMX}}
For this example, uppercase letters are used to denote ciphertext, lowercase letters are used to denote plaintext (or guesses at such), and {{samp|X}}~{{samp|t}} is used to express a guess that ciphertext letter {{samp|X}} represents the plaintext letter {{samp|t}}.

Eve could use frequency analysis to help solve the message along the following lines: counts of the letters in the cryptogram show that {{samp|I}} is the most common single letter,<ref>{{Cite web |url=http://www.asecuritysite.com/security/Challenges/scramb |title=A worked example of the method from bill's "A security site.com" |access-date=2012-12-31 |archive-url=https://web.archive.org/web/20131020173206/http://www.asecuritysite.com/security/Challenges/scramb |archive-date=2013-10-20 |url-status=dead }}</ref> {{samp|XL}} most common [[bigram]], and {{samp|XLI}} is the most common [[trigram]]. {{samp|e}} is the most common letter in the English language, {{samp|th}} is the most common bigram, and {{samp|the}} is the most common trigram. This strongly suggests that {{samp|X}}~{{samp|t}}, {{samp|L}}~{{samp|h}} and  {{samp|I}}~{{samp|e}}. The second most common letter in the cryptogram is {{samp|E}}; since the first and second most frequent letters in the English language, {{samp|e}} and {{samp|t}} are accounted for, Eve guesses that {{samp|E}}~{{samp|a}}, the third most frequent letter. Tentatively making these assumptions, the following partial decrypted message is obtained.

 {{not a typo|heVeTCSWPeYVaWHaVSReQMthaYVaOeaWHRtatePFaMVaWHKVSTYhtZetheKeetPeJVSZaYPaRRGaReM
 WQhMGhMtQaReWGPSReHMtQaRaKeaTtMJTPRGaVaKaeTRaWHatthattMZeTWAWSQWtSWatTVaPMRtRSJ
 GSTVReaYVeatCVMUeMWaRGMeWtMJMGCSMWtSJOMeQtheVeQeVetQSVSTWHKPaGARCStRWeaVSWeeBtV
 eZMtFSJtheKaGAaWHaPSWYSWeWeaVtheStheVtheRGaPeRQeVeeBGeeHMWYPFhaVHaWHYPSRRFQMtha
 PPtheaCCeaVaWGeSJKTVWMRheHYSPHtheQeMYhtSJtheMWReGtQaROeVFVeZaVAaKPeaWHtaAMWYaPP
 thMWYRMWtSGSWRMHeVatMSWMGSTPHhaVHPFKPaZeNTCMteVJSVhMRSCMWMSWVeRCeGtMWYMt}}

Using these initial guesses, Eve can spot patterns that confirm her choices, such as "{{samp|that}}". Moreover, other patterns suggest further guesses. "{{samp|Rtate}}" might be "{{samp|state}}", which would mean {{samp|R}}~{{samp|s}}. Similarly "{{samp|atthattMZe}}" could be guessed as "{{samp|atthattime}}", yielding {{samp|M}}~{{samp|i}} and {{samp|Z}}~{{samp|m}}. Furthermore, "{{samp|heVe}}" might be "{{samp|here}}", giving {{samp|V}}~{{samp|r}}. Filling in these guesses, Eve gets:

 {{not a typo|hereTCSWPeYraWHarSseQithaYraOeaWHstatePFairaWHKrSTYhtmetheKeetPeJrSmaYPassGasei
 WQhiGhitQaseWGPSseHitQasaKeaTtiJTPsGaraKaeTsaWHatthattimeTWAWSQWtSWatTraPistsSJ
 GSTrseaYreatCriUeiWasGieWtiJiGCSiWtSJOieQthereQeretQSrSTWHKPaGAsCStsWearSWeeBtr
 emitFSJtheKaGAaWHaPSWYSWeWeartheStherthesGaPesQereeBGeeHiWYPFharHaWHYPSssFQitha
 PPtheaCCearaWGeSJKTrWisheHYSPHtheQeiYhtSJtheiWseGtQasOerFremarAaKPeaWHtaAiWYaPP
 thiWYsiWtSGSWsiHeratiSWiGSTPHharHPFKPameNTCiterJSrhisSCiWiSWresCeGtiWYit}}

In turn, these guesses suggest still others (for example, "{{samp|remarA}}" could be "{{samp|remark}}", implying {{samp|A}}~{{samp|k}}) and so on, and it is relatively straightforward to deduce the rest of the letters, eventually yielding the plaintext.

 {{not a typo|hereuponlegrandarosewithagraveandstatelyairandbroughtmethebeetlefromaglasscasei
 nwhichitwasencloseditwasabeautifulscarabaeusandatthattimeunknowntonaturalistsof
 courseagreatprizeinascientificpointofviewthereweretworoundblackspotsnearoneextr
 emityofthebackandalongoneneartheotherthescaleswereexceedinglyhardandglossywitha
 lltheappearanceofburnishedgoldtheweightoftheinsectwasveryremarkableandtakingall
 thingsintoconsiderationicouldhardlyblamejupiterforhisopinionrespectingit}}

At this point, it would be a good idea for Eve to insert spaces and punctuation:

 Hereupon Legrand arose, with a grave and stately air, and brought me the beetle
 from a glass case in which it was enclosed. It was a beautiful scarabaeus, and, at
 that time, unknown to naturalists&mdash;of course a great prize in a scientific point
 of view. There were two round black spots near one extremity of the back, and a
 long one near the other. The scales were exceedingly hard and glossy, with all the
 appearance of burnished gold. The weight of the insect was very remarkable, and,
 taking all things into consideration, I could hardly blame Jupiter for his opinion
 respecting it.

In this example from "[[The Gold-Bug]]", Eve's guesses were all correct. This would not always be the case, however; the variation in statistics for individual plaintexts can mean that initial guesses are incorrect. It may be necessary to [[backtracking|backtrack]] incorrect guesses or to analyze the available statistics in much more depth than the somewhat simplified justifications given in the above example.

It is possible that the plaintext does not exhibit the expected distribution of letter frequencies. Shorter messages are likely to show more variation. It is also possible to construct artificially skewed texts. For example, entire novels have been written that omit the letter {{samp|e}} altogether &mdash; a form of literature known as a [[lipogram]].