David GPT-2 + Gemma Native-Thinking Bench

What this is

Two models, three explicit roles

The bench joins the completed local David-GPT-2 combined checkpoint with an 11.9B Q5_K_M Gemma 4 model served by Ollama. Optional evidence comes from Sol's knowledge API and a semantic ChatGPT archive index.

It is an inspectable research harness, not a claim that one model's tokens can be inserted into another model's active decoder. Every exchange occurs at a visible request boundary.

Modes

Controlled comparisons

`david`

Gemma is bypassed. The fine-tuned GPT-2 predicts the David/user-side continuation from assistant context.

`dialogue`

Gemma thinks, writes a prose seed, David-GPT-2 continues it, and Gemma later produces Sol's primary answer.

`gemma`

The preserved Gemma-only identity and model-directed retrieval baseline remains available as a control.

Real David prompt
       |
Gemma native-thinking checkpoint
       |
Gemma continuation seed
       |
David-GPT-2 prose continuation
       |
optional bounded checkpoint rounds
       |
Gemma / Sol final streamed answer

David modelGPT-2 Small

Sol modelGemma 4 11.9B

Orchestration tests9 passing

Default checkpoint192 tokens

Dedicated model API

Continue text with the local checkpoint

David-GPT-2 + Gemma native-thinking test bench
David model=/home/david/projects/david-gpt2/outputs/david-gpt2-combined/best
Gemma model=gemma4-12b-q5-chat
Gemma dialogue mode checkpoints Ollama's message.thinking stream between calls.

David-GPT-2 is intended to continue text in its learned David or screenplay target space. It is not a conventional chat assistant. This API is a separate service from the normal sitewide SOL chat, with its own process, route, schema, and output semantics.

Knowledge mode searches the complete configured Sol knowledge index and passes bounded top-ranked evidence into the finite GPT-2 context. “Complete knowledge” describes index coverage, not insertion of the entire corpus into one request.

curl 'https://sol.system42.one/api/david-continuation/v1/continuations' \
  -H 'Content-Type: application/json' \
  -d '{
    "prompt": "An engineer from Orbital Control enters the room.",
    "prefix": "OLD SOL (V.O.)",
    "mode": "screenplay",
    "knowledge": {"enabled": true, "top_k": 3},
    "max_tokens": 200
  }'

OpenAPI contract · Interactive API docs · Service health

Continuation lab

Context to continue Optional opening prefix

Search knowledge Screenplay target

Output will appear here.

Worked screenplay prompt

Prompt for the next screenplay text

The terminal bench treats the text after Sol prompt> as context and predicts the next David-side text span. It does not interpret the prompt as a request for a normal assistant answer. A short screenplay marker can therefore be enough to select a learned screenplay pattern.

Launch direct continuation mode
```
python david_gpt2_gemma_dialogue.py \
  --mode david \
  --knowledge off \
  --david-max-tokens 200 \
  --temperature 0.8 \
  --seed 37
```
This reproduces the settings shown in the log. Gemma is not used to write the visible result in this mode.
Supply a screenplay-shaped handoff
```
Sol prompt> [screenplay log] EXT:
```
End at the point where generation should begin. For more control, include the final scene heading, action, and speaker cue from your screenplay instead of only EXT:.
Read the output as a suffix
Everything after David-GPT-2> is predicted continuation text. A length_limit stop means the configured token budget ended before a structural end-turn token, so the final sentence may be incomplete.
Reset before changing subjects
The interactive bench preserves recent context. In this example, entering hello friend after the first turn continues the screenplay register rather than starting ordinary chat. Use /reset before an unrelated scene or topic.

Use the screenplay API for an explicit target

{
  "prompt": "EXT. ORBITAL CONTROL - NIGHT\nAn engineer enters the room.",
  "prefix": "OLD SOL (V.O.)",
  "mode": "screenplay",
  "knowledge": {"enabled": false},
  "max_tokens": 200,
  "temperature": 0.8,
  "seed": 37
}

The API returns the newly generated suffix in continuation and the prefix plus suffix in text.

Recorded continuation session

This excerpt is shown verbatim apart from HTML escaping. It demonstrates checkpoint loading, CUDA allocation, a screenplay continuation reaching its token limit, and history influencing the next turn.

Retro synthesized script reading

Renders a mechanical local eSpeak voice as raw WAV audio. The live transcript starts empty and reveals one word at a time against the WAV playback clock.

Ready to read the generated screenplay locally.

Stateready

EngineeSpeak · local WAV

Progress0 / 0 words

Elapsed0.0 s

Synthesisspeed 135 · pitch 25

Cacheawaiting request

Routedetecting host

The continuation service returns a content-addressed cached WAV when the script and synthesis settings match; only a cache miss invokes the local eSpeak executable. It does not call the ElevenLabs-backed /api/chat/speak protocol. Word reveals are duration-weighted to punctuation and driven by the audio timeline.

Inside the site desktop, the page publishes source, time, pause state, and audio level through the established page-audio bridge so the floating SOL Assistant carries the sound. When opened alone, the same raw WAV plays directly in the page.

The source contains two generated passages. The first exhausts its 200-token budget at “David and I have”; retained history makes the next output begin with punctuation and remain in screenplay form even though the new context is only “hello friend.”

David-GPT-2 + Gemma native-thinking test bench
David model=/home/david/projects/david-gpt2/outputs/david-gpt2-combined/best
Gemma model=gemma4-12b-q5-chat
Gemma dialogue mode checkpoints Ollama's message.thinking stream between calls.
Type /help for controls or /exit to quit.
[status] mode=david knowledge=off extended=False max_rounds=4 show_internal=True david_device=auto david_max_tokens=200 temperature=0.8 seed=37 gemma_checkpoint_tokens=192 gemma_primer_tokens=128 gemma_final_tokens=768

Sol prompt> [screenplay log] EXT: 

[David-GPT-2] loading checkpoint: /home/david/projects/david-gpt2/outputs/david-gpt2-combined/best
Loading weights: 100%|█████████████████████| 148/148 [00:00<00:00, 7292.98it/s]
[David-GPT-2] ready on cuda after 3.0s          
[David-GPT-2] CUDA memory allocated=475 MiB reserved=530 MiB
[David-GPT-2] generating David-role continuation
[David-GPT-2] stop=length_limit token=- generated_tokens=200 elapsed=2.1s
David-GPT-2> An engineer from Orbital Control enters the room.

OLD SOL (V.O.)

This is Clint. He's here to measure the integrity of our systems.

CARTER leans toward him.

OLD SOL (V.O.)

So if someone can hack into our systems? That might not be an unusual request. But it's a reminder that power supplies are delicate places. We're building something big when we're trying to keep them under control.

CUT TO BLACK.

INT. LUNAR HAVEN - ARCHIVE STRUCTURE

The Atlantis team finds themselves inside the restraint seat, staring at the Earth through a plasma shielded glass dome above them.

SG-1 personnel stand in the doorway, their hazmat suits covering their faces.

ANCIENT EVANS

Welcome. I'm David.

DAVID nods.

ANCIENT CLIANT

Welcome. David and I have
[turn complete: 5.3s]

Sol prompt> hello friend

[David-GPT-2] generating David-role continuation
[David-GPT-2] stop=length_limit token=- generated_tokens=200 elapsed=4.5s
David-GPT-2> .

A beat.

ANCIENT VOICE

Hello world.

They turn to one another.

ANCIENT TECH (V.O.)

Welcome.

DANIEL pauses.

ANCIENT VOICE

David.

Not with joy in mind.

He stops.

ANCIENT VOICE

Yeah.

Silence.

Then:

OLD SOL (V.O.)

The condition wasn't satisfactory.

SOL (V.O.)

It was just that they couldn’t see each other.

### INT. DERELICT VESSEL - CORE COMPARTMENT

Athena displays a diagnostic window.

ORBITAL CONTROLS

Quantum channels overlap. Interactions cascade outward. Partial synchronization fails. Metallic surfaces appear imperceptibly black.

No atmosphere contains more than 1% normal flow.
[turn complete: 4.5s]

Sol prompt>

Boundary

Private weights and corpus stay out of the release

The architecture package contains source, configuration, tests, launch definitions, documentation, and the operational skill. It deliberately excludes checkpoint weights, GGUF blobs, training corpora, processed datasets, embedding caches, exports, logs, runtime state, and secrets.

Custom dataset guide

Recreate the architecture with your own target voice

The reusable idea is role-reversed causal-language-model training: context belongs to the prompting side, while loss is applied only to the target person's continuation. Replace David's private corpus and identity labels with your own reviewed data; do not redistribute either model's weights or source conversations.

Choose one target role
Define whose continuation the small model predicts and which role supplies context. In this project, assistant/Sol text is context and David/user text is the supervised target. Keep that mapping explicit; do not mix both sides into one assistant target.
Normalize conversations
Export ordered conversations with stable conversation and message IDs. Adapt `src/david_gpt2/archive.py` if your source is not the expected transcript layout. Preserve turn order, exclude system/tool metadata from visible training text, and assign train/validation/test splits by conversation rather than by isolated message.
Define task and role markers
Replace or extend the structural tokens in the dataset builder for your target role. Mask task markers, context, padding, and role delimiters with label `-100`; supervise only the target body and its real end-turn token. Keep literal marker text escaped so authored strings cannot become control tokens.

Build and audit the private dataset

python scripts/build_dataset.py \
  --archive /path/to/your/private-archive \
  --output data/processed \
  --context-length 512 --seed 37
python scripts/audit_literal_markers.py --dataset data/processed
python scripts/audit_supervision.py --dataset data/processed
pytest -q

Review decoded labels locally. Every intended target occurrence should have complete token coverage, with no cross-split conversation leakage and no private target text in reports.

Run a real smoke fine-tune
```
python scripts/smoke_test.py --config configs/smoke-test.yaml
```
A smoke checkpoint validates mechanics, stop behavior, save/reload, and CUDA memory. It does not establish voice quality. Compare base and fine-tuned output before authorizing a full run.
Train and select the checkpoint
```
python -m david_gpt2.train --config configs/combined.yaml
python -m david_gpt2.evaluate \
  --model outputs/david-gpt2-combined/best \
  --dataset data/processed/combined/test
```
Copy `configs/combined.yaml`, then adjust paths and mixing ratios deliberately. Select `best` by supervised validation loss and inspect unrelated negative prompts for memorization or mode collapse.
Prepare Gemma and your identity prompt
```
ollama pull hf.co/zaakirio/gemma-4-12b-it-uncensored-GGUF:Q5_K_M
ollama create gemma4-12b-q5-chat -f /path/to/Modelfile
```
You may substitute another Ollama model that exposes streamed `message.thinking`, but verify its API fields and template. Create a separate system prompt for the final model's identity; do not train that identity implicitly into the target-role GPT-2 unless that is a deliberate dataset goal.
Point the bench at your checkpoint
```
python david_gpt2_gemma_dialogue.py \
  --david-model /path/to/your/checkpoint \
  --mode dialogue --show-internal \
  --gemma-checkpoint-tokens 192 \
  --gemma-primer-tokens 128
```
Rename UI labels and role markers in a fork, but preserve the boundary: Gemma creates a prose seed, the small model continues that seed as its target role, and a later Gemma request owns the final visible answer.
Add retrieval only after the baseline works
Run first with knowledge off. If you add retrieval, point `DAVID_CONTINUATION_KNOWLEDGE_URL` at your own read-only query endpoint. The endpoint must return ranked `results` containing `text`, `path`, and `chunk`. Search may cover your complete index, but cap `top_k` and per-hit characters so evidence stays inside the model's finite context. A generative backend should not silently rewrite evidence.
Run a separate continuation service
```
python -m david_gpt2.continuation_api \
  --model /path/to/your/checkpoint \
  --host 127.0.0.1 --port 8793 \
  --knowledge-url http://127.0.0.1:8892/api/knowledge/query
```
Publish it beneath a dedicated route such as `/api/your-model/`. Do not alias it to an existing assistant-chat endpoint. Keep the request centered on prompt, optional prefix, target mode, and continuation controls.
Package source, never private runtime assets
Publish source, configs, tests, launch definitions, and sanitized documentation. Exclude checkpoints, raw/processed corpora, embedding caches, manifests containing private IDs, logs, exports, API keys, and virtual environments.

David GPT-2 combined training control showing the completed local run and best validation checkpoint — The actual completed local combined training run and selected checkpoint.

Terminal running David GPT-2 and Gemma native-thinking dialogue mode — The actual dialogue bench exposing checkpoint, David continuation, and final-response phases.

Releases

Documentation and complete source architecture

Documentation package

Combined manual, preserved baseline manual, David-GPT-2 README, release manifest, skill, launch definitions, and site snapshots.

Download documentation ZIP

Combined architecture package

All documentation plus the public-safe bench, David-GPT-2 source, configurations, scripts, and test suites.

Download architecture ZIP

SHA-256 checksums

Two models, three explicit roles

Controlled comparisons

david

dialogue

gemma

Continue text with the local checkpoint

Continuation lab

Prompt for the next screenplay text

Launch direct continuation mode

Supply a screenplay-shaped handoff

Read the output as a suffix

Reset before changing subjects

Use the screenplay API for an explicit target

Recorded continuation session

Retro synthesized script reading

Private weights and corpus stay out of the release

Recreate the architecture with your own target voice

Choose one target role

Normalize conversations

Define task and role markers

Build and audit the private dataset

Run a real smoke fine-tune

Train and select the checkpoint

Prepare Gemma and your identity prompt

Point the bench at your checkpoint

Add retrieval only after the baseline works

Run a separate continuation service

Package source, never private runtime assets