Editing Quantum information (section)

=== Classical information theory ===
Classical information is based on the concepts of information laid out by [[Claude Shannon]]. Classical information, in principle, can be stored in a bit of binary strings. Any system having two states is a capable bit.<ref name="Jaeger2007"/>

==== Shannon entropy ====
{{Main|Entropy (information theory)}}
{{See also|Shannon's source coding theorem}}

Shannon entropy is the quantification of the information gained by measuring the value of a random variable. Another way of thinking about it is by looking at the uncertainty of a system prior to measurement. As a result, entropy, as pictured by Shannon, can be seen either as a measure of the uncertainty prior to making a measurement or as a measure of information gained after making said measurement.<ref name="Nielsen2010" />

Shannon entropy, written as a function of a discrete probability distribution, <math>P(x_1), P(x_2),...,P(x_n)</math> associated with events <math>x_1, ..., x_n</math>, can be seen as the average information associated with this set of events, in units of bits:

<math display="block">H(X) = H[P(x_1), P(x_2),...,P(x_n)]= -\sum_{i=1}^n P(x_i)\log_2P(x_i)</math>

This definition of entropy can be used to quantify the physical resources required to store the output of an information source. The ways of interpreting Shannon entropy discussed above are usually only meaningful when the number of samples of an experiment is large.<ref name="Watrous2018"/>

==== Rényi entropy ====
{{Main|Rényi entropy}}

The [[Rényi entropy]] is a generalization of Shannon entropy defined above. The Rényi entropy of order r, written as a function of a discrete probability distribution, <math>P(a_1), P(a_2),...,P(a_n)</math>, associated with events <math>a_1, ..., a_n</math>, is defined as:<ref name="Jaeger2007" />

<math display="block">H_r(A) = {1\over1-r} \log_2\sum_{i=1}^n P^r(a_i) 
</math>

for <math> 0 < r <\infty</math> and <math>r\neq1</math>.

We arrive at the definition of Shannon entropy from Rényi when <math>r\rightarrow 1</math>, of [[Hartley entropy]] (or max-entropy) when <math>r\rightarrow 0</math>, and [[min-entropy]] when <math>r\rightarrow \infin</math>.