Distilled programs and weight patches#

tensorcode.distill turns a workload you already run through a large model into a small program that TensorCode compiles, scores against that model's answers and serves on the CPU, escalating what it does not claim. tensorcode.weights records how tensors change as hash-pinned, data-only patches, renders weights as text a language model can read, and applies the edit statements it writes back.

Both are mechanisms, not claims about any particular authoring model. The loop measures agreement with the teacher you supply on cases the author never saw; it does not establish that a frontier model can author a program for your task. The measured example below uses recorded author replies.

The loop#

from tensorcode.distill import distill
from tensorcode.integrations import OpenAICompatibleModel

author = OpenAICompatibleModel(base_url="http://localhost:8000/v1", model="your-served-model")
result = distill(
    "Route a support ticket from twelve 0/1 keyword indicators: ...",
    inputs={"features": (12,)},
    workload=[{"features": row} for row in feature_rows],
    targets=[{"route": float(route)} for route in teacher_routes],  # or oracle=callable
    author=author,
    attempts=4,
    holdout=0.5,
    seed=0,
    min_coverage=0.6,
)
print(result.report["design"])      # what the author was scored on
print(result.report["acceptance"])  # cases the author never saw
result.save("./ticket-router")

distill splits the workload by seed into design cases, which the author sees with the teacher's outputs and gets feedback on, and acceptance cases, which it never sees. Each attempt sends the task, the input and output contracts, design examples, the language reference and the previous attempts' results, and receives one program. The compiler parses the reply and never executes it. The best attempt on the design set, by fewest wrong-while-covered answers and runtime errors and then by coverage, is scored once on the acceptance set after the loop ends. The report states this policy in feedback_policy.

Exactly one of oracle, a callable called once per workload case, or precomputed targets is required. truth optionally supplies ground truth for the acceptance cases, and the report then separates program-versus-teacher from program-versus-truth and teacher-versus-truth, since a faithful program copies the teacher's mistakes. DistillationError carries every attempt when none compiles.

What the author writes#

The author receives the bounded subset in tensorcode.distill.SUBSET_SPEC: one function over the declared inputs, assignments, literal-range loops, arithmetic, comparisons that produce 0/1 arrays, where, the unary functions and tensor methods of the algorithm compiler, and a dict return. Indexing does not compile; elements are read with x.slice(i, i + 1).reshape().

The program returns the teacher's output names and shapes plus a scalar covered output: 1.0 when it claims the input, 0.0 when the input should go to the fallback. A wrong answer while covered is the failure the loop minimizes; abstaining is allowed and costs a fallback call. Coverage is authored logic, not a calibrated probability.

The author may declare parameters in a json block and take them as extra arguments. They start at the written values and are the only part a later gradient fit changes:

from tensorcode.distill import fit

tuned = fit(result, cases, targets, steps=200, lr=0.05)  # needs tensorcode[algorithm]

Comparisons carry no gradient, so a fit adjusts thresholds and weights inside the author's structure; it does not discover a new program.

Report fields#

Agreement is empirical over the supplied cases at atol/rtol; it is not a proof over the input domain. Runtime errors never count as agreement.

Serve, route, repair#

from tensorcode.distill import Distillation, repair

program = Distillation.load("./ticket-router")
router = program.router(lambda inputs: {"route": float(frontier_route(inputs))})
answer = router({"features": row})       # answer["escalated"] names the path taken
print(router.rate())                     # share of calls that went to the fallback

cases, targets = router.cases()          # escalations with the fallback's answers
revised = repair(program, author=author, cases=cases, targets=targets, attempts=3)

Router calls the program and, when it abstains or raises, calls the fallback you supply and records the case. repair adds those cases to the design set, shows the author the deployed program as its starting point, and scores the revision on the same hidden acceptance set. Cases identical to a hidden case are excluded and counted in report["repair"]["excluded_hidden"].

Saved programs are two JSON files: the validated algorithm artifact and a record with the source text, parameters, report, every attempt's prompt and reply, and the design and acceptance cases. Loading validates the artifact and never evaluates the source.

Weight patches#

from tensorcode import weights

shown = weights.render(weights.algorithm_parameters(algorithm), digits=3)
base = weights.replace_parameters(algorithm, shown.rounded)   # the network as shown
# shown.text goes to a model; it replies with edit statements such as
edits = "w[6, 122] = 80\nw[:, 122:128] = 0\nb[3] += -5\n"
patch = weights.Patch.from_edits(
    shown.rounded, edits, scales=shown.scales, rationale="...", evidence=[{"source": "forward:3"}]
)
edited = weights.patch_algorithm(base, patch)   # hashes checked against the rounded base

render writes each tensor as integers times one scale, with digits significant digits of its largest entry, and returns the rounded tensors that text shows. Edits apply to that rounded network, so make it the base. The edit grammar is name[index] = value, += or -=, with integer or start:stop index parts and a number or list on the right; values are in the tensor's rendered units when scales is given. Statements are parsed with a fixed grammar, never evaluated.

A Patch records, per tensor, the base and edited SHA-256 and one of four forms:

diff(base, edited) chooses sparse or dense per tensor; build(base, changes) makes the other forms. patch.apply(mapping) works on any name-to-tensor mapping, including a PyTorch state dictionary, and refuses a base whose hash differs. The artifact is JSON with a rationale, evidence records and author description; summary() and fraction_changed(tensors) give report numbers. Only named parameters of an algorithm are addressable; constants are not.

Measured example#

examples/distill/route_tickets.py distills an authored routing policy over 240 generated tickets with recorded author replies: the first reply indexes with [] and fails to compile, the second is accepted. The program reads twelve keyword indicators and abstains on tickets without a keyword, which the teacher routes from text the program cannot see.

These numbers describe an authored teacher, a fixed lexicon and recorded replies. They show the loop's mechanics and reporting, not that a live model would write this program or that keyword features suffice for real tickets.

Boundaries#

• Inputs are numeric tensors with declared shapes. Reading text or images into features is the caller's step; the example uses a keyword lexicon.
• Programs are stateless; stateful algorithms use compare(..., rollout=True) in the algorithm guide directly.
• The author is whatever you supply, a Model with complete(ModelRequest) or a callable from prompt text to reply text. Its competence is not measured here.
• A high agreement with the teacher inherits the teacher's errors; supply truth when you have it.
• Rendering is lossy by design; patches pin the rounded network, not the original.
• No TypeScript port of these two modules exists yet; the algorithm artifacts they produce load there as before.