Editing Entropy (information theory) (section)

==== Discussion ====
The rule of additivity has the following consequences: for [[positive integers]] {{math|''b''<sub>''i''</sub>}} where {{math|''b''<sub>1</sub> + ... + ''b''<sub>''k''</sub> {{=}} ''n''}},
<math display="block">\Eta_n\left(\frac{1}{n}, \ldots, \frac{1}{n}\right) = \Eta_k\left(\frac{b_1}{n}, \ldots, \frac{b_k}{n}\right) + \sum_{i=1}^k \frac{b_i}{n} \, \Eta_{b_i}\left(\frac{1}{b_i}, \ldots, \frac{1}{b_i}\right).</math>

Choosing {{math|''k'' {{=}} ''n''}}, {{math|''b''<sub>1</sub> {{=}} ... {{=}} ''b''<sub>''n''</sub> {{=}} 1}} this implies that the entropy of a certain outcome is zero: {{math|Η<sub>1</sub>(1) {{=}} 0}}. This implies that the efficiency of a source set with {{math|''n''}} symbols can be defined simply as being equal to its {{math|''n''}}-ary entropy. See also [[Redundancy (information theory)]].

The characterization here imposes an additive property with respect to a [[partition of a set]]. Meanwhile, the [[conditional probability]] is defined in terms of a multiplicative property, <math>P(A\mid B)\cdot P(B)=P(A\cap B)</math>. Observe that a logarithm mediates between these two operations. The [[conditional entropy]] and related quantities inherit simple relation, in turn. The measure theoretic definition in the previous section defined the entropy as a sum over expected surprisals <math>\mu(A)\cdot \ln\mu(A)</math> for an extremal partition. Here the logarithm is ad hoc and the entropy is not a measure in itself. At least in the information theory of a binary string, <math>\log_2</math> lends itself to practical interpretations.

Motivated by such relations, a plethora of related and competing quantities have been defined. For example, [[David Ellerman]]'s analysis of a "logic of partitions" defines a competing measure in structures [[Duality (mathematics)|dual]] to that of subsets of a universal set.<ref>{{cite journal |last1=Ellerman |first1=David |title=Logical Information Theory: New Logical Foundations for Information Theory |journal=Logic Journal of the IGPL |date=October 2017 |volume=25 |issue=5 |pages=806–835 |doi=10.1093/jigpal/jzx022 |url=http://philsci-archive.pitt.edu/13213/1/Logic-to-information-theory3.pdf |access-date=2 November 2022 |archive-date=25 December 2022 |archive-url=https://web.archive.org/web/20221225080028/https://philsci-archive.pitt.edu/13213/1/Logic-to-information-theory3.pdf |url-status=live }}</ref> Information is quantified as "dits" (distinctions), a measure on partitions.  "Dits" can be converted into [[Shannon (unit)|Shannon's bits]], to get the formulas for conditional entropy, and so on.