What happened

InferenceKey ran SenseNova 6.8 Code2Video, SenseTime's open 27B model that generates video by writing code, as a long agent on a single 24 GB RTX 4090 in the cloud, driven by Claude Code. According to the research write-up, the model produced a full 12-second clip within ten minutes, but at forty minutes it had no finished film. The weights were not the bottleneck. The context budget around them was.

Why it matters

When an open model runs as an agent that calls tools in a loop, the limiting factor is often how much the agent can keep in view and how long a single answer may be. The SenseNova team publishes a Claude Code skill called code-video and six complete production runs, served across four GPUs with a 512K-token context window and a 64K output cap. Their films needed between 261 and 876 turns each. InferenceKey's single-card setup used bartowski's 4-bit GGUF (17.2 GB) for the llama.cpp server plus a 0.93 GB vision component, and set three Claude Code variables to encode the gap:

``shell CLAUDE_CODE_MAX_CONTEXT_TOKENS=98304 # authors: 524288 CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=80 # same as the authors CLAUDE_CODE_MAX_OUTPUT_TOKENS=16384 # authors: 64000 ``

Without the first variable, Claude Code assumes 200K for a model it does not recognize.

Technical details

Compaction is how Claude Code survives a full context: once the window passes 80%, it has the model write a recap, discards the old history, and continues from the recap. Because the recap is a normal reply, the output cap applies to it too. The 98,304-token window filled after about 19 minutes. Three recap requests followed, all three hit the 16,384-token ceiling, none produced a usable summary, and together they consumed 21 minutes until a 40-minute limit killed the run.

Compaction can work: four of the authors' six runs cleared contexts of 167K to 492K tokens under a 64K cap. InferenceKey flags two suspects on its side: the 16,384-token cap and hidden reasoning, which draws from the same cap. The logs cannot distinguish them, though in a separate single request the same 4-bit file burned all 8,192 output tokens thinking and wrote no page. A bigger window, a bigger cap, or reasoning off might fix it; none were tried.

Self-review partly worked. The model opened 21 of its own stills, spotted two bugs, and fixed both, and all on-screen text matched the brief word for word. But the brief asked for 30 dots in a 5 by 6 rectangle, and locating each dot centre in the 7.2-second frame with numpy and scipy returned 5 rows of 7 columns with one empty cell per row. The skill's file check still passed because it only inspects resolution, frame rate, and duration, never what is drawn.

Two plumbing issues also surfaced. The gateway in front of the model lost image frames, so each image either vanished or arrived as text and the model reviewed its work blind; this was fixed in a test build but not yet on the hosted service. Separately, the first call waited for the full answer instead of streaming, and the worker process gave up after 120 seconds while the model was still thinking; raising that to 30 minutes fixed the run.

Smaller copies were tested on the same 4090 with one short request each. A 3-bit copy with a 32K budget peaked at 13.7 GiB and rendered a page with one dot visible at a time. A 2-bit copy with a 16K budget peaked at 10.0 GiB and was cut off at 8,192 tokens. The 4-bit run peaked at 20.9 GiB.

What to watch

Before trusting a single-card agent, measure two things: when the context will hit the compaction threshold, and whether one summary fits under your output cap with reasoning on. Add a check that knows what the clip should show, not just its resolution, frame rate, and duration. With any gateway, send one PNG first and confirm the model describes it. And verify that requests stream rather than waiting for the full answer, since worker timeouts can end a run while the model is still thinking.

About the author

DEV Community (AI)

Thoai Dispatch Curated — curated source featured on Thoai Dispatch.

More by DEV Community (AI)