The entropy cost of an overlapping pair of conjunction features

The extremal four-outcome distribution is (5/8, 1/8, 1/8, 1/8). Costs use natural logarithms, consistently with WeightBound; divide by Real.log 2 to obtain bits. The saving compares this joint cost with two quarter-biased marginal costs, rather than asserting a mutual-information lower bound.

noncomputable def Algebraic.Cutwidth.Aggregate.Geometry.Entropy.pairEntropyCost : ℝ

Natural-log entropy of the extremal overlapping two-conjunction table.

Equations

• Algebraic.Cutwidth.Aggregate.Geometry.Entropy.pairEntropyCost = 3 * Real.log 2 - 5 / 8 * Real.log 5

noncomputable def Algebraic.Cutwidth.Aggregate.Geometry.Entropy.pairEntropySaving : ℝ

The saving over separately charging two quarter-biased Boolean messages.

Equations

• Algebraic.Cutwidth.Aggregate.Geometry.Entropy.pairEntropySaving = 2 * Real.binEntropy (1 / 4) - Algebraic.Cutwidth.Aggregate.Geometry.Entropy.pairEntropyCost