ptwm_core
ptwm_core::transcode
PTWM container ⇄ canonical PTWM-JSON manifest transcoder.
ptwm_core::transcode
PTWM container ⇄ canonical PTWM-JSON manifest transcoder.
ptwm_core::transcode
crates/ptwm-core/src/transcode/mod.rs:1mod transcodePTWM container ⇄ canonical PTWM-JSON manifest transcoder.
The binary `.ptwm` container is structurally *(a)* a registry of
codecs/ops/shared-state/chains, *(b)* per-tensor records, and *(c)* per-plane
compressed payloads. This module lifts that into a declarative manifest
(see [`manifest`]) plus loose binary members, and back again.
* [`explode`] parses a container into `(registry_json, per-tensor manifests,
loose members)`.
* [`implode`] reverses it, re-driving the deterministic container writer so
that `implode(explode(blob)) == blob` byte-for-byte for every container the
compression engine produces.
The encoder may build a container transiently in memory; the stored artifact
(WebDataset shard / LMDB exploded value) never contains one.
ptwm_core::transcode::explode
crates/ptwm-core/src/transcode/mod.rs:236pub fn explode(blob: &[u8]) -> Result<Exploded, PtwmCoreError>Parse a `.ptwm` container into a registry manifest, per-tensor manifests, and loose binary members.
ptwm_core::transcode::Exploded
crates/ptwm-core/src/transcode/mod.rs:43pub struct ExplodedResult of exploding a container.
ptwm_core::transcode::ExplodedTensor
crates/ptwm-core/src/transcode/mod.rs:53pub struct ExplodedTensorptwm_core::transcode::implode
crates/ptwm-core/src/transcode/mod.rs:497pub fn implode(registry_json: &str, tensor_jsons: &[String], members: HashMap<String, &[u8]>) -> Result<Vec<u8>, PtwmCoreError>Reconstruct a `.ptwm` container from a registry manifest, per-tensor manifests (in file order), and loose binary members.