entropy
ptwm_core::entropy::tans
Tabled ANS (tANS / Finite State Entropy) entropy codec.
ptwm_core::entropy::tans
Tabled ANS (tANS / Finite State Entropy) entropy codec.
ptwm_core::entropy::tans
crates/ptwm-core/src/entropy/tans/mod.rs:1mod tansTabled ANS (tANS / Finite State Entropy) entropy codec.
A third entropy coder peer alongside [`crate::entropy::huffman`] and
[`crate::entropy::rans`]. Where Huffman wastes the fractional bits that
integer code lengths can't represent, tANS — like rANS — codes at
(near-)entropy cost; the *tabled* variant does it with a single
table lookup per symbol on both encode and decode, the construction
Zstd uses internally (FSE). See the 2024 ANS survey
(<https://arxiv.org/abs/2408.07322>) for the modern treatment.
## Design
Single-stream, byte-alphabet, with an 11-bit normalized frequency
table (`TABLE_LOG = 11`, `L = 2048`) to match rANS's precision. The
coder is the textbook tANS with an explicit fixed start state, which
keeps the encode/decode bit accounting exactly symmetric (`n` encode
transitions ↔ `n` decode transitions, one flush ↔ one init) and
sidesteps the asymmetric first-symbol initialization micro-optimization
that the Zstd/FSE reference uses. The decoder recovers the start state
and verifies the bitstream is fully consumed, so a corrupt payload
fails rather than producing garbage.
4-stream interleaving (the shared `stream_frame` framing the Huffman /
rANS coders use for parallel decode) is a deliberate follow-up — this
single-stream form is a complete, correct dispatcher peer.
## Blob layout (what [`compress`] writes / [`decompress`] reads)
```text
[ table_log : u8 ]
[ nsym : u16 LE ] number of symbols with freq > 0
nsym × [ sym : u8 ][ freq : u16 LE ] normalized frequencies (sum = 2^table_log)
[ total_bits : u64 LE ] length of the FSE bitstream in bits
[ bitstream : ⌈total_bits / 8⌉ bytes ]
```
ptwm_core::entropy::tans::compress
crates/ptwm-core/src/entropy/tans/mod.rs:273pub fn compress(dst: &mut [u8], src: &[u8]) -> Result<usize, PtwmCoreError>tANS-encode `src`. Returns the encoded length written to `dst`, or 0 to signal "store raw" (incompressible, degenerate alphabet, or the encoded form would not be smaller than the input).
ptwm_core::entropy::tans::decompress
crates/ptwm-core/src/entropy/tans/mod.rs:372pub fn decompress(dst: &mut [u8], src: &[u8]) -> Result<usize, PtwmCoreError>Decode a [`compress`] blob into `dst`. `dst.len()` is the symbol count.
stream_frame
Shared framing helpers for the 4-stream interleaved entropy codecs (huffman::compress / rans::compress). Both codecs split their input into 4 roughly-equal segments, encode each segment as an independent stream, and record per-stream compressed lengths in a fixed-size jump table so the decoder can locate each stream without scanning.
error