Evaluation records#

These JSON files preserve the original measured outputs, settings, hashes and failures. The older records were moved here without changing their bytes; owned-tool records contain current evaluation summaries and provenance. Commands embedded in a record describe the historical run and may name an older output location. Model weights, source datasets and captured training artifacts stay outside the repository.

Record	Evidence	Source
latent-foundations-a2.json	Canonical alpha artifacts recreated under public operation identities, with old/new weight, output and training-continuation checks	GB10 recreation of the recorded runs
latent-foundations.json	Original FLAN/ViT/diffusion native parity and four-pair adapter lifecycle before the a2 API boundary	Historical source revision recorded in the report
cognition-hotpot.json	Frozen complete pipeline, 32 final questions, manual response review, evidence controls and episodic retrieval	evaluate_cognition.py
cognition-diagnostic.json	Earlier development failures used to change components; not an untouched final test	evaluate_cognition.py
hypotheses-qa2d.json	Human declaration supervision, article-disjoint test, factual errors and post-hoc NLI limitations	train_hypotheses.py
realization-qa2d.json	Preserving an already selected statement; train/development measurement, not QA inference	train_realization.py
scene-language.json	Owned pretrained VLM, real spatial-caption judgments, blank/different-image controls and unverified descriptions	evaluate_scene_language.py
verifier-snli.json	GB10 NLI fine-tune, held-out temperature fit, test probability metrics and exact reload; foundation already trained on SNLI	train_verifier.py
action-outcomes.json	Executed-choice feedback, simulated service recovery and complete learning restart; authored status classes	learn_action_outcomes.py
scene-vsr.json	Negative real-photo spatial-caption result, image/workspace ablations and restart	train_scene.py
pretrained-releases.json	Published tool IDs, pinned revisions and Hub reload verification	Hugging Face model cards and complete artifacts
chatbot-hotpot.json	Owned FLAN-based QA fine-tune and workspace ablations; oracle supporting evidence	train_chatbot.py
investigator-hotpot.json	Owned Electra-based support ranking with multi-positive targets and ablations	train_cognitive_tools.py
typed-decisions-flan-t5-base.json	Zero-shot generated-JSON vs likelihood decoding: Banking77 (385 rows) and three response-quality axes on reused development candidates	evaluate_typed_decisions.py
planner-hotpot.json	Document-read relevance learning, not measured action utility	train_cognitive_tools.py
banking77-restart.json	Capture, baseline evaluation, restarted training and checkpoint evaluation in separate processes; 77-label held-out classification	banking77_restart.py
banking77-in-process.json	Earlier in-process supervised run; retained historical measurement	Historical `examples/banking77.py` at commit `7827ac0`
mutag.json	Historical, retired neural graph adapter: fixed graph-disjoint molecule split, losses, accuracy, parameter changes and split IDs	Historical mutag.py at d8188ed
multimodal-smolvlm-256m.json	Real image/text model answers, incorrect descriptions/counting and rejected structured outputs	local_multimodal.py
multimodal-qwen3-vl-2b.json	Real model answers, partial visual successes and failed structured outputs including an explicit formatting follow-up	local_multimodal.py

Validation and scope explains the measurements and their practical limits. The results establish behavior on the recorded inputs and settings; they do not claim calibrated confidence, universal reasoning, inferred chemistry or reliable local structured decisions.

The graph neural implementation and applications were removed in favor of reserved symbolic operation contracts. MUTAG records are historical evidence only. The retired dependency-impact application likewise used static import parsing and authored graph callbacks; it has no learned-cognition result record. Neither application represents an implemented symbolic graph path today.

Response-quality comparisons preserve failed JSON, paired-input and evidence-intervention development runs. These are assistant-reviewed diagnostics, not qualified checkpoints or final evaluations.

Foundation and prompt comparisons record assistant-reviewed historical-case outputs, including failed source-copying proposals. These measure inherited generation, not complete cognitive-tool quality.

Complete XL development pipeline records seven correct answers, one incorrect attribution and 24 abstentions, plus source withdrawal/conflict controls. Frozen generative quality judgments also fail promotion despite stronger positive retention.

Source-wise XL comparison improves coverage on the same known questions, but includes wrong, incomplete and ambiguous answers. Claim verbalization loses question restrictions or changes meaning; NLI accepts 25 nonfaithful rewrites. Workspace-only quality adaptation approves every development candidate despite exact training continuation and reload. None of these checkpoints is qualified for release.

Connected OUTPUT_ENCODING learning demonstrates collecting and replaying an encoder → decoder training graph with real FLAN weights, updating readout/bridge parameters and reloading the operations exactly. It measures the training lifecycle on 16 supplied training pairs, not held-out semantic alignment or answer quality.

Stage diagnosis separates generated answers, verification eligibility, ranking and realization on those same cases. Candidate-ranking adaptation addresses the ranker's document-versus-answer training mismatch, but its calibration choices worsen and it cannot repair verifier exclusions. No checkpoint is promoted.

Fixed quality gates apply the predeclared numerical criteria to complete prediction receipts, preserving unknown labels and counting excluded cases against retention. A numerical pass would still require evidence-use checks and complete-tool validation before qualification.

Joint foundation/workspace adaptation also fails: active screening accepts all 91 development candidates, including 28 known failures, while the adapted foundation with workspace bypass rejects all 91. Changed weights and exact optimizer continuation do not establish useful quality judgment. These weights are not promoted.

Collapse diagnosis finds workspace residuals over 1,000 times larger than native token representations on four selected calibration rows. Float32 retains the collapse and native decoder parity is exact. This identifies an unbounded update, without establishing that a magnitude bound alone will learn useful answer verification.

Bounded workspace adaptation prevents that collapse but accepts one additional known development failure and five additional calibration failures. Exact reload confirms the result; no useful workspace improvement or checkpoint qualification is established.