ptwm_core
ptwm_core::entropy
Entropy: measurement and coding.
ptwm_core::entropy
Entropy: measurement and coding.
ptwm_core::entropy
crates/ptwm-core/src/entropy/mod.rs:1mod entropyEntropy: measurement and coding.
This module gathers everything PTWM does at the *information-theoretic*
layer of compression:
- [`shannon`] / [`histogram`] — measure per-plane entropy in bits/byte.
The PPG layer uses these to decide whether a chain is worth running;
the explorer uses them to score candidate plane factorisations.
- [`huffman`] and [`rans`] — two pure-Rust byte-oriented entropy coders
with the same public interface (`compress` / `decompress` /
`compress_outcome`) and a peer position in the dispatcher's per-plane
trial encode. Both share the 4-stream interleaved framing in
[`stream_frame`] and return the [`outcome::CompressOutcome`] enum so
the plane-codec wrapper can distinguish "encoded N bytes" from
"not beneficial" / "incompressible" / "dst too small" / "len overflow".
Neither codec is the default. The dispatcher trial-encodes every plane
against both (plus Identity and Zstd) and keeps the smallest payload,
so the per-plane winner is determined by the data shape rather than a
configuration choice.
ptwm_core::entropy::histogram
crates/ptwm-core/src/entropy/mod.rs:34pub fn histogram(data: &[u8]) -> [u32; 256]Byte-frequency histogram for `data`. Saturates at `u32::MAX` per bucket — fine for any realistic tensor (`u32::MAX` bytes ≈ 4 GiB).
ptwm_core::entropy::shannon
crates/ptwm-core/src/entropy/mod.rs:43pub fn shannon(data: &[u8]) -> f64Base-2 Shannon entropy of `data` in bits/byte. Empty input yields 0.0.