Editing Shannon–Hartley theorem (section)

===Noisy channel coding theorem and capacity===
{{main|Noisy-channel coding theorem}}
[[Claude Elwood Shannon|Claude Shannon]]'s development of [[information theory]] during World War II provided the next big step in understanding how much information could be reliably communicated through noisy channels. Building on Hartley's foundation, Shannon's [[noisy channel coding theorem]] (1948) describes the maximum possible efficiency of [[error-correcting code|error-correcting methods]] versus levels of noise interference and data corruption.<ref>{{cite book | first = C. E.|last=Shannon | author-link = Claude E. Shannon | title = The Mathematical Theory of Communication | location = Urbana, IL|publisher=University of Illinois Press | orig-year = 1949| year = 1998|url=https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf}}</ref><ref>{{cite journal |first = C. E. |last=Shannon |author-link = Claude E. Shannon |title = Communication in the presence of noise |journal = [[Proceedings of the Institute of Radio Engineers]] |volume = 37 |issue = 1 |pages = 10–21 |date = January 1949 |doi=10.1109/JRPROC.1949.232969 |s2cid=52873253 |url=http://www.stanford.edu/class/ee104/shannonpaper.pdf |archive-url=https://web.archive.org/web/20100208112344/http://www.stanford.edu/class/ee104/shannonpaper.pdf |archive-date=8 February 2010 |url-status=dead}}</ref> The proof of the theorem shows that a randomly constructed error-correcting code is essentially as good as the best possible code; the theorem is proved through the statistics of such random codes.

Shannon's theorem shows how to compute a [[channel capacity]] from a statistical description of a channel, and establishes that given a noisy channel with capacity <math>C</math> and information transmitted at a line rate <math>R</math>, then if

:<math> R < C </math>

there exists a coding technique which allows the probability of error at the receiver to be made arbitrarily small. This means that theoretically, it is possible to transmit information nearly without error up to nearly a limit of <math>C</math> bits per second.

The converse is also important. If

:<math> R > C </math>

the probability of error at the receiver increases without bound as the rate is increased, so no useful information can be transmitted beyond the channel capacity. The theorem does not address the rare situation in which rate and capacity are equal.

The Shannon–Hartley theorem establishes what that channel capacity is for a finite-bandwidth [[continuous-time]] channel subject to Gaussian noise. It connects Hartley's result with Shannon's channel capacity theorem in a form that is equivalent to specifying the ''M'' in Hartley's line rate formula in terms of a signal-to-noise ratio, but achieving reliability through error-correction coding rather than through reliably distinguishable pulse levels.

If there were such a thing as a noise-free analog channel, one could transmit unlimited amounts of error-free data over it per unit of time (Note that an infinite-bandwidth analog channel could not transmit unlimited amounts of error-free data absent infinite signal power). Real channels, however, are subject to limitations imposed by both finite bandwidth and nonzero noise.

Bandwidth and noise affect the rate at which information can be transmitted over an analog channel. Bandwidth limitations alone do not impose a cap on the maximum information rate because it is still possible for the signal to take on an indefinitely large number of different voltage levels on each symbol pulse, with each slightly different level being assigned a different meaning or bit sequence. Taking into account both noise and bandwidth limitations, however, there is a limit to the amount of information that can be transferred by a signal of a bounded power, even when sophisticated multi-level encoding techniques are used.

In the channel considered by the Shannon–Hartley theorem, noise and signal are combined by addition. That is, the receiver measures a signal that is equal to the sum of the signal encoding the desired information and a continuous random variable that represents the noise. This addition creates uncertainty as to the original signal's value. If the receiver has some information about the random process that generates the noise, one can in principle recover the information in the original signal by considering all possible states of the noise process. In the case of the Shannon–Hartley theorem, the noise is assumed to be generated by a Gaussian process with a known variance. Since the variance of a Gaussian process is equivalent to its power, it is conventional to call this variance the noise power.

Such a channel is called the Additive White Gaussian Noise channel, because Gaussian noise is added to the signal; "white" means equal amounts of noise at all frequencies within the channel bandwidth. Such noise can arise both from random sources of energy and also from coding and measurement error at the sender and receiver respectively. Since sums of independent Gaussian random variables are themselves Gaussian random variables, this conveniently simplifies analysis, if one assumes that such error sources are also Gaussian and independent.