ptwm
ptwm.random_access
Random-access loader for .ptwm containers.
ptwm.random_access
Random-access loader for .ptwm containers.
ptwm.random_access
python/ptwm/random_access/__init__.py:1module ptwm.random_accessRandom-access loader for ``.ptwm`` containers.
:class:`TensorIndex` opens a compressed bundle and decompresses individual
tensors on demand. The index is tiny (a manifest of name → offset/length
records), so opening a bundle and fetching one tensor touches only that
tensor's bytes.
idx = TensorIndex.open("model.safetensors.ptwm")
weight = idx.get_tensor("layer_5.weight")
for name, tensor in idx.stream_tensors(names_of_interest):
...
ptwm.random_access.TensorEntry
python/ptwm/random_access/__init__.py:29class TensorEntry:One entry from a ``.ptwm`` tensor index.
ptwm.random_access.TensorEntry.__init__
python/ptwm/random_access/__init__.py:1def __init__(self, name: str) -> Noneptwm.random_access.TensorEntry.name
python/ptwm/random_access/__init__.py:33name: strptwm.random_access.TensorIndex
python/ptwm/random_access/__init__.py:36class TensorIndex:Name-indexed random-access loader over a ``.ptwm`` container.
ptwm.random_access.TensorIndex.__init__
python/ptwm/random_access/__init__.py:39def __init__(self, path: str | Path) -> Noneptwm.random_access.TensorIndex.__iter__
python/ptwm/random_access/__init__.py:125def __iter__(self) -> Iterator[str]ptwm.random_access.TensorIndex.entries
python/ptwm/random_access/__init__.py:66def entries(self) -> list[TensorEntry]Return every :class:`TensorEntry` in manifest order.
ptwm.random_access.TensorIndex.entry
python/ptwm/random_access/__init__.py:70def entry(self, name: str) -> TensorEntryReturn the :class:`TensorEntry` for ``name`` or raise ``KeyError``.
ptwm.random_access.TensorIndex.get_bytes
python/ptwm/random_access/__init__.py:79def get_bytes(self, name: str) -> bytesReturn the raw decompressed bytes for ``name``.
ptwm.random_access.TensorIndex.get_tensor
python/ptwm/random_access/__init__.py:88def get_tensor(self, name: str) -> torch.TensorDecompress ``name`` and return a :class:`torch.Tensor`.
Shape/dtype come from the CBOR shape metadata written at compression
time; tensors without metadata raise :class:`TypeError`.
ptwm.random_access.TensorIndex.names
python/ptwm/random_access/__init__.py:62def names(self) -> list[str]Return tensor names in manifest (write) order.
ptwm.random_access.TensorIndex.open
python/ptwm/random_access/__init__.py:50def open(cls, path: str | Path) -> SelfOpen ``path`` for random-access tensor lookup.
ptwm.random_access.TensorIndex.path
python/ptwm/random_access/__init__.py:58path: PathFilesystem path this index was opened from.
ptwm.random_access.TensorIndex.stream_tensors
python/ptwm/random_access/__init__.py:108def stream_tensors(self, names: Iterable[str] | None = None) -> Iterator[tuple[str, torch.Tensor]]Yield ``(name, tensor)`` pairs.