Parity with Python#

The TypeScript package is a port of the Python tensorcode package (version 0.4.0a4). It has the same operations, tools, tracing, training and artifact formats. The Python package is the reference: where the two disagree on something the remaining differences do not list, Python's behavior is the correct one and the difference is a bug.

This page says how parity is checked, what matches, what you can move between the two languages and the few places where they still differ.

How parity is checked#

• Fixtures from the Python stack. Generators in scripts/fixtures run the Python package (with PyTorch 2.14, transformers 5.17, diffusers 0.40, torchvision 0.29 and Pillow 12) and write reference outputs to test/fixtures. The TypeScript tests compare against them: layer outputs and gradients, optimizer steps, random streams and freshly initialized weights, safetensors and PyTorch checkpoint bytes, tokenizer output, decoded images, native model outputs, generation, tool receipts and error messages, training losses, experience files, checkpoints and session files.
• Ported tests. The suite in test/ ports the Python tests area by area. It runs with npm test and needs neither Python nor network access.
• Gradient checks. Every differentiable operation in the numerical core is checked against finite differences.
• Real checkpoints. Tests that need a cached Hugging Face checkpoint (for example google/flan-t5-small, google/electra-small-discriminator, openai/clip-vit-base-patch32, cross-encoder/nli-deberta-v3-small or HuggingFaceTB/SmolVLM-256M-Instruct) run when it is in the local cache and skip otherwise.
• Examples. Every ported example was run beside its Python original with the same inputs and seeds; see examples.

By area#

What moves between the languages#

These files are interchangeable. A file written by one package loads in the other:

• Model artifacts (tensorcode_config.json, model.safetensors and the model card), saved with savePretrained / save_pretrained, locally or on the Hugging Face Hub. A Python artifact loaded in TypeScript and saved again has a byte-identical manifest. Weights are byte-identical too, except for models with two or more tied parameter aliases (T5): Python writes that metadata in a nondeterministic order, so even two Python saves differ there. The tensor bytes are identical.
• Configuration-only operation artifacts (tensorcode.operation).
• Experience files written by trace.save(...) in either language.
• Standalone checkpoint files (tensorcode.checkpoint) and directory checkpoints (training.json plus a tensor file), including their PyTorch and CPython random states.
• Tool session files: chat, ranking, cognitive session and state, trajectories and JSON memory. Python session files load in TypeScript and save back byte-identically.
• The Hugging Face cache. Both packages use the huggingface_hub cache layout, so a model downloaded by one is found by the other.

Numbers and dictionaries#

JavaScript numbers do not distinguish 1 from 1.0, and JavaScript objects list integer-like keys first. TypeScript keeps Python's view anyway:

• Values read from Python files (artifacts, experience, checkpoints, sessions, JSON memory) remember whether each whole number was an int or a float and the order of dictionary keys, so they save and fingerprint exactly as Python wrote them. A Python caller who passed an int for a float field (hidden_dropout_prob=0) gets the same fingerprint in both languages.
• In your own code, float(0) and int(0) from tensorcode say which one you mean (float(0) is written 0.0). Unmarked whole numbers follow the field's Python type: configuration fields that Python stores as floats are written as floats, everything else as ints.
• Use orderedObject([['2', ...], ['1', ...]]) from tensorcode (or a Map where an option expects a mapping, such as Retrieve items) for a dictionary whose integer-like keys must keep their order. Python dictionaries with int, bool, None or tuple keys decode to a Map that keeps those key types.

Random numbers#

The random generator is PyTorch's CPU generator, reproduced exactly: manualSeed(n) is torch.manual_seed(n), getRngState()/setRngState() exchange the same 5056 bytes as torch.get_rng_state()/torch.set_rng_state(), and rand, randn, normal_, uniform_, randint, randperm, bernoulli, exponential_ and multinomial produce the same bits. init mirrors torch.nn.init, and native models initialize in transformers' post_init order, so seeded fresh construction of every tool, operation and native model gives Python's weights bit for bit; loading a foundation draws only for weights the checkpoint lacks, as from_pretrained does. Dropout masks match bit for bit. Generation sampling (do_sample, nucleus and beam sampling) makes the same draws as torch.multinomial, so sampled tokens match Python's whenever the float32 token probabilities agree, which they do on the fixtures. PythonRandom reproduces CPython's random.Random.

Naming#

The API follows TypeScript conventions. Saved and reported data keeps Python's spelling, which is what makes the files above interchangeable.

Keyword arguments become one trailing options object. Anything that reads or writes files or uses the network returns a Promise; pure computation is synchronous, and so are the blocking provider calls (complete, op.call, ask), as in Python.

Remaining differences#

Everything not listed here behaves as in Python.

Compute#

TypeScript ships its own dependency-free tensor and autograd core (tensorcode/nn) instead of PyTorch, and runs on the CPU only: device options accept 'cpu', and there is no CUDA or other GPU backend, so a checkpoint that carries CUDA generator states is rejected, as a CPU-only Python process rejects it. Float32 products, convolutions, attention, softmax, layer norm and activations run on WebAssembly SIMD kernels on worker threads (setNumThreads, TENSORCODE_THREADS); browsers run them on one thread. Products accumulate in float32 like PyTorch's CPU kernels, so computed results agree with Python within float tolerance rather than bit for bit, and they are deterministic and independent of batch size and thread count. float16 results are computed in float32 and rounded once, where PyTorch's float16 CPU convolution accumulates in float16, so they agree to a few units in the last place.

Speed and memory, measured against PyTorch 2 on the same 20-core AArch64 CPU: an electra-base forward pass (8 x 128 tokens) takes as long as in PyTorch, and a training step about 1.5 times as long. A SmolVLM-256M interpretation of one 640 x 480 image takes about 7 s for one token (PyTorch: 5 s) and 9 s for 32 tokens (PyTorch: 12 s), with a similar peak memory (3.6 GB, PyTorch 3.2 GB). On Linux, part of the remaining time goes to page faults on fresh result buffers; starting Node with GLIBC_TUNABLES=glibc.malloc.hugetlb=1 (glibc 2.35 or newer) backs large buffers with transparent huge pages, which made the SmolVLM vision encoder about 10% faster.

PyTorch initializes CUDA during a training step on a machine with a GPU, even when every tensor is on the CPU, so a directory checkpoint that Python saves there carries CUDA generator states and TypeScript rejects it. To move such a run to TypeScript, save the checkpoint from a Python process that cannot see the GPU (CUDA_VISIBLE_DEVICES=), or use a standalone checkpoint file, which stores no generator states.

Reference platform#

Bit-exact random streams and image resampling follow PyTorch, glibc 2.39 and Pillow on AArch64, where the fixtures are generated. PyTorch itself is not reproducible across platforms: on x86-64 with AVX2 its float32 normal_ uses a different vectorized kernel and glibc picks different math routines, so Python's own samples differ there. Two branches the samplers never reach in practice are not bit-exact: trunc_normal_ with less than 30% of the mass inside its bounds (PyTorch computes that log with a vectorized kernel), and sin/cos of arguments beyond 105414350 in magnitude.

Native architectures#

Python loads any transformers AutoModel for a text foundation. TypeScript implements ALBERT, BERT, RoBERTa, Electra, DistilBERT, DeBERTa-v2, T5, ViT, CLIP, Llama and Idefics3 natively and raises ValueError for other model_types. A foundation's tokenizer comes, as in AutoTokenizer, from its tokenizer.json, from slow vocab.txt (BERT, ELECTRA, DistilBERT) or vocab.json/merges.txt (RoBERTa, CLIP, GPT-2) files, or, when the checkpoint has no tokenizer files at all, from the class defaults. Checkpoints whose only tokenizer file is a SentencePiece model (spiece.model, tokenizer.model) fail in both packages: Python converts them only when the optional sentencepiece or tiktoken package is installed.

Scene language mode#

fromLanguageFoundation writes the processor assets Python writes for TokenizersBackend (including PreTrainedTokenizerFast and unknown class names), LlamaTokenizer, GPT2Tokenizer, T5Tokenizer, AlbertTokenizer, DebertaV2Tokenizer, BertTokenizer, RobertaTokenizer and CLIPTokenizer, with or without Fast (the Idefics3 checkpoints on the Hub use GPT2Tokenizer or PreTrainedTokenizerFast). Other transformers tokenizer classes with their own construction (for example Qwen2Tokenizer, GemmaTokenizer, CodeLlamaTokenizer, XLMRobertaTokenizer) raise NotImplementedError there. Image processors other than Idefics3ImageProcessor (also named ...Fast or ...Pil) raise NotImplementedError as well.

Chat templates#

Processor chat templates render with a built-in Jinja implementation that follows transformers' sandboxed jinja2 environment. Recursive for loops, call blocks, the filters attr, filesizeformat, groupby, pprint, random, slice, striptags, urlencode, urlize, wordwrap and xmlattr, and the tests escaped, filter and test raise TemplateError. The chat templates of the Idefics3 checkpoints on the Hub use none of them.

Images#

openImage reads PNG, JPEG, GIF, WebP and BMP; other formats Pillow opens (TIFF, ICO, PPM and so on) raise ValueError('cannot identify image file'). Rare Pillow modes (RGBX, YCbCr, LAB, HSV, I;16B) are not supported, and truncated progressive JPEGs are decoded without libjpeg-turbo's block smoothing. Where torchvision itself returns uninitialized memory (palettes below 8 bits in UNCHANGED mode, palette PNGs without transparency in the alpha modes, GIF frames beyond the canvas), the pixels cannot match. Some decoder error messages are worded differently from the C libraries; the error types match.

Providers#

complete, op.call and ask block on a worker thread. Runtimes that cannot block a thread (browser main threads, edge runtimes) raise SynchronousCallUnavailable (a NotImplementedError) there, and a provider built with an injected fetch function is asynchronous only, because a function cannot move to another thread; acall and aask always work. LocalModel runs Transformers.js (ONNX Runtime) rather than PyTorch, so its answers can differ from Python's where two tokens score almost equally, and its blocking complete loads a second copy of the model in the worker thread.

Hub conversion pull requests#

When a public repository has an open conversion pull request from an author other than SFconvertbot, transformers asks the conversion Space for a new one and may then load the first matching pull request whoever opened it. TypeScript only loads pull requests opened by SFconvertbot on public repositories, so it never loads weights a third party proposed.

JavaScript values#

• JavaScript cannot tell a closure from a module function, so a saved callback (for example combine) always needs an explicit configuration().
• Unmarked whole numbers in your own code follow the field's Python type (see numbers and dictionaries); write float(1) where Python code writes 1.0, including optimizer options (new AdamW(params, { weightDecay: float(0) })).
• Decoded dictionaries with tuple keys are Maps whose tuple keys compare by identity.
• A plan action's keyword parameters are read from the properties its source destructures from args. A function whose source does not show them (bound, native or transpiled functions) accepts every keyword unless it is declared with withSignature.
• Generation settings keep the Python kind of top-level whole numbers read from a file, so "repetition_penalty": 2 or "top_k": 5.0 fail as they do in transformers. Inside nested lists only floats are recorded, so an int bias in sequence_bias ([[[3], 5]]) is accepted where transformers rejects it.

Not ported#

The Python examples that require the CUDA training host (compare_cognitive_verifiers.py, train_hypotheses.py, train_realization.py, train_verifier.py, prepare_response_quality.py, train_response_quality.py) and the optional Stable Diffusion image path of pretrained_latent_lifecycle.py. Every other Python example has a TypeScript port in examples that reproduces its results.

The design notes in DESIGN.md map every Python module to its TypeScript file.