Repository navigation
dsv41: COLI_VK_KV_QUANT -- the device KV mirrors in the checkpoint's cache formats (approximate, off by default) - #2701
Closed
jameswitney wants to merge 19 commits into
Closed
jameswitney wants to merge 19 commits into
jameswitney wants to merge 19 commits into
Conversation
The dense chain fit now runs before the mapped-weights window is allocated, so the trunk sees the device's real free memory instead of the ~0.5 GiB left after a 9/10-heap window ask. The window is then asked for what remains beside the trunk less COLI_VK_FREE_RESERVE_GB (1 GiB, floor+warn at <1 GiB ask; COLI_VK_WINDOW_ASK_MB still overrides), the tier sizes into it with the host pool off by default beside the trunk (COLI_VK_TIER_HOST=1 / _GB re-enable), and dense host copies stay given back. Other engines, the CPU-only path and the integrated/ Lavapipe defaults are unchanged; explicit COLI_VK_STAGED=1 still forces staging. Opt-in is deepseek_v41-only (incl. its GGUF path). A bounded (<=3 s) wait covers RADV reporting the probe's freed blocks back through VK_EXT_memory_budget before the tier reads it. Measured on RX 6900 XT 16 GiB: 40 of 40 trunk layers on the device (7.62 GiB), window 5 GiB, tier 3.28 GiB / 83 resident experts, no GTT host-pool mirror, no new swap.
Baseline origin/dev vs the trunk-first commit, one build apart, same serve parameters and the same warm start: decode 0.485 -> 0.786 tok/s median (+62%) on a single-NVMe ReBAR card, the tier's 11.5 GiB host pool on -> off with GTT at ~0.27 GiB, and 40 of 40 trunk layers on the device beside it. Harness committed beside the record.
…l context Track C fix for the --ram 28 / --ctx 131072 / 262144 hard-fail. The planner priced the v41 forward's workspace at batch == context, so the runtime reservation ballooned with the context (25 GiB at 131072) and the RAM expert cache came out with zero bytes; cap_for_arch then raised "RAM budget cannot hold one expert slot per layer". deepseek_v41.c prefill reads a prompt in bounded chunks (default 1280 rows), so a forward never holds the full-context buffers at once. Price the workspace at one chunk (COLI_VK_CHAIN_ROWS / COLI_PROMPT_SEGMENT, matching the qwen38 convention) so a feasible plan is produced again, degrading to fewer slots instead of hard-failing when the budget is genuinely tight.
…e chunk, and a 2-row stream threshold Everything here is allocation policy, packing and observability: no row of arithmetic changes. Measured on the 16 GiB RX 6900 XT with DeepSeek V4.1 Flash and the trunk-first placement. 1. The VRAM ledger (V41_STATS, or COLI_VK_LEDGER=1/0 on its own). v41c_ledger prints one line at a turn's end: the weight pool's blocks and the bytes of rows in them, the chain's dev_bytes and its three pools' blocks, the tier's residents and streaming slots (count and bytes) with the pool's blocks, the extra layers' pool, the KV mirrors, this process's device bytes against the pools, and the card's live mem_info_vram_used. vkc_pool_bytes and vkt_vram are the two accessors it needs. It answers the audit's open item: the 2.14 GiB a short-prompt run grew by was not the tier (whose pools do not move after the warm start) but 1262.5 MiB of token embedding + 150 MiB of the 40 routers + 673 MiB of the DSpark stages' matrices, all uploaded by the first forward and counted by neither dev_bytes nor the tier's budget; 52 MiB is the KV mirrors. The weight pool holds 9608.2 MiB of rows at the turn's end against the 7523.0 MiB the setup places. 2. The streaming slots: 64 -> 48 (COLI_VK_TIER_STREAM_SLOTS; the measured runs used 44 at most), and vkc_chunk_rows now reserves vkt_slot_reserve() before it reads the free memory, so the chunk and the slots no longer fight over the same headroom: the default chunk goes 1280 -> 1024 rows. The tier splits its budget between residents and slots, so the slot set alone frees nothing (64+91 and 48+107 fill the same 13 pool blocks of 256 MiB); what it and the reserve do is take the engine's own hold at the turn boundary from 15.53 to 15.26 GiB (chain 2133.7 -> 2060.0 MiB) with the prefill wall at +2% (289.95 -> 295.60 s). The card's sysfs peak does not move (15.907 -> 15.906 GiB) because the desktop yields: with the tier forced to 2.0 GiB the engine held 1.46 GiB less and sysfs fell 0.19 GiB, the compositor taking 1.27 GiB of it. 3. COLI_VK_TIER_STREAM_MIN=2: the rows from which a cold expert is streamed, measured instead of assumed (the engine's own line said the GEMM's 16 rows were never measured). One 21.1 MiB DeepSeek V4.1 expert, by the new bench (COLI_VK_CHAIN_BENCH= xbrows in tests/test_vk_chain.c): a device step costs 0.12 ms at 1 row, 0.21 at 2, 0.40 at 4, 0.76 at 16 (the tiled GEMM from there) against a 2.2 ms upload and the CPU's 2.05-2.52 ms a row, so two rows already pay for the upload; the GEMM route below 8 rows is slower than the GEMV, so COLI_VK_TIER_GEMM_ROWS stays 16. On the chunk-256 workload (COLI_VK_CHAIN_ROWS=256, the 2600-word prompt) the CPU's share of expert-rows falls 34.5% -> 1.6% and the prefill 226.7 -> 175.9 ms a row, at 2 TB of expert reads instead of 74.7 GB: it pays in disk bandwidth, not in time.
…'s extra layers upload A tier pool's limit and its blocks disagreed by a page. tier_pool_limit sized a block from the budget rounded up to a page (15360 -> 16384 B) but left the limit at the budget itself, which the pool cannot grow a block past; pool_suballoc then fell back to the limit floored to a page (12288 B) and counted a refusal. At deepseek_v41's tiny-budget case (COLI_VK_TIER_GB=0.00005: a three-expert tier, and one extra-layer expert's bytes -- 15360 B -- in the extra layers' pool) the pool could never hold its expert: the first MTP stage's upload was refused, the run's line read "failed 1 refused by the pool", and T.xmax fell to 0 with the extra layers left on the CPU. The expert needs exactly 15360 B (three tensors of 4096 rows + 1024 scales): the budget was right, the page floor of the block capacity lost the 3072 B that made it fit. The streaming-slot reservation commit is not this: with c615cca's six files reverted to 05e2c6d the same run fails identically (same "no room for streaming slots" line, same counters resident 2 / uploads 6 / evictions 4 / failed 1), and vkt_slot_reserve() is 0 in it -- the extra-layer take leaves the tier at fit_raw 2, so streaming is off and the chunk already reserves "0.00 GiB of it the tier's slots and staging". The exposure is older (the 9/10-of-the-heap mapped window, and the extra layers' share of a tiny budget: 9cfa502's trunk-first placement fixed the window for deepseek_v41 only) and it surfaced when the deepseek family was run on the discrete ICD, where the MTP stages are the tier's extra layers by default and not on the CPU as they are on a device that shares the RAM. 1. tier_pool_limit (backend_vulkan.c) rounds its limit up to a page, as its blocks already are, so a budget of N expert bytes holds N expert bytes -- at most a page (4095 B) over the tier's own budget, another page-granular figure. The tiny-budget case now keeps the extra layer on the device: extra layers (2) 4 of 48 routed experts, resident 1 (budget 1), uploads 7, evictions 4, failed 0, and the run's output is the CPU's. 2. vkt_slot_reserve (vk_tier.c) caps the reservation at what the budget holds beside the residents, and reserves nothing when the budget cannot hold the slots at all: it sizes a chunk, it is never a claim on the room an upload or an eviction needs (defensive: it is not what refused anything above, where it was 0). 3. cu_tier.c renames vkt_slot_reserve and vkt_vram for the CUDA build of vk_tier.c, which 'make -C c test' compiles: with the two accessors c615cca added, that build did not compile (redefinition against vk_tier.h's stubs). Measured on the 16 GiB RX 6900 XT: the default-budget serve (ctx 16384, short prompt, one turn) is unchanged -- 48 slots reserved, tier 1434.4 MiB of 1536.0 in 6 blocks as 68 residents, extra layers 168.8 MiB, process 11.28 GiB of 11.34 GiB in pools, chunk 768 rows with this desktop's 3.5 GiB already on the card (1024 on an idle one); the same run without this change gives the same figures.
…e that shares the RAM deepseek_v41's DSpark loop asserted that the MTP stages never go on the tier by default. That is true where the device's memory is the host's (Lavapipe): the tier's extra layers default to the CPU there. On a discrete card COLI_VK_TIER_MTP's default puts them on the tier, so the assertion fired on every DSpark case of the family when it ran on the radeon ICD, before any of them could be judged. The check now keys on the engine's own ", shared RAM," line, which says which of the two it is; the cases that ask for the stages on the tier (COLI_VK_TIER_MTP=1) are untouched and still require them there.
…tic cap buys twice the rows
The indexer's candidate mask is only ever read as a boolean -- the score kernel tests
it, the host's copy of it feeds a "kept or not" byte, nothing multiplies by it -- and it
was an fp32 word a column: rows x wcap of them beside rows x wcap fp32 scores. It is a
bit a column now (vk_chain.h: vkc_dsv4_cand writes it, vkc_dsv4_score reads it, mask_row
and mask_off in words of 32 columns), so the pair costs 4E + E/8 bytes a row instead of
8E. No row of arithmetic changes: the same blocks are picked and kept whole, and the
selection the top-k runs on is the same set of columns for every row (the indexer case in
tests/test_vk_chain.c compares the mask column for column, through the single-workgroup,
the wide-query and the grid kernels; the families give the CPU's tokens).
1. The chain (c/deepseek_v41_chain.h): isc stays fp32 at rows x wcap; mask and its host
copy are rows x ceil(wcap/32) words, and the host unpacks the bits where it builds
m->candidates for the CPU's layers after a partial chain. The chunk's quadratic cap
(v41c_forward_seg) was rows <= 32M/E, which is the 256 MiB the fp32 pair could take;
it is now the same 256 MiB over 4E + E/8 a row, i.e. ~62M/E (1.94x the rows), and it
prints one line when it binds -- nothing printed it before, which is why the audit had
to derive it.
2. Measured, the tiny V4.1 fixture at a 20000-position prompt: the chunk the tier's
stream line reports goes 1677 -> 3253 rows, the row cost the sizing prints falls 181
-> 105 KiB, and the new line reads "a prompt of 20000 positions caps a chunk at 3253
rows (80.6 KiB a row of scores and packed candidate mask, 256 MiB of 256)". On the
released checkpoint at E=3107 the row cost goes 1293 -> 1281 KiB a row (the mask was
12.4 KiB of that, one bit a column is 0.4).
3. Two items of the wave-2b list are not there, measured rather than argued (the
correction is in docs/deepseek-v41.md so it does not come back):
- the 1262.5 MiB the turn-end ledger attributes to the "token embedding" is the lm
head. tie_word_embeddings is false on this checkpoint, both tensors are 129280 x
5120 bf16, and the embedding has never been on the device: its only reader is
wb_row_f32, which copies the rows a step needs into the host's rows (one a token
when decoding, one a position for a chunk). A run with every device upload printed
shows head.weight as the 1262.5 MiB item and the embedding never uploaded at all.
The head is a GEMV of 662M multiply-adds a token, so it stays where it is, as do
the 40 routers' 150 MiB.
- the per-layer vkc_finish (deepseek_v41_chain.h, after the attention frame) does not
wait on anything the host does not read. A 1280-row chunk spends 28095 ms waiting
for the frame it reads and 0.5 ms for the rest: two frames are in flight and a
single queue runs submissions in order, every frame starting with a barrier on
everything submitted before it, so the previous layer's shared expert is always
through. Waiting only the read frame was implemented, measured at 0.5 ms a chunk and
dropped. The audit's R4 estimate (2-8 ms a chunk) was right; the wait that remains
is the attention itself, and the device is idle during moe_run_at instead.
Gates on the 16 GiB RX 6900 XT (RADV), against the same three families at 955a0f6:
tests/vulkan_engines.sh deepseek (29 OK), deepseek-chain (35 OK), prefill-deepseek (2
OK), make -C c vk-chain-check VK=1 (PASS), make -C c test (test-c green; test-python
3226 tests). Three failures and two more are identical at both commits and are
environmental: "deepseek_v4 session: no matmul ran on the device" and "chain deepseek_v4
4 experts: the chain never ran" (the deepseek_v4 engine's dense placement declines with
the desktop holding VRAM; its tokens still equal the CPU's), "pf deepseek_v41 staged:
logits" (max |logit diff| 3.22e-02 at both), and the two
test_harness_builds_without_warnings cases (a GCC ABI note in mistral_vision.h).
The commit before this one quoted the row cost the packed candidate mask saves as "1293 -> 1281 KiB a row" at E=3107. 1293 is the row cost at E=11 (the short-prompt arm), not at E=3107: the same 8 bytes a column make it 1317 KiB there, which is the number the audit prints as well, so the pair is 1317 -> 1305. Measured on the released checkpoint at E=1300 this run: 1303 -> 1298 KiB a row (the mask's 5.1 KiB a row of 41 packed words). The saving itself was never in doubt -- 8 bytes a column become 4 + 1/8 -- only the base it was subtracted from.
…hunk, not the context The planner priced the forward workspace at the whole context, so at --ram 28 a --ctx 131072 or 262144 serve came out with a zero-byte RAM expert cache and cap_for_arch() hard-failed with "RAM budget cannot hold one expert slot per layer". deepseek_v41's prefill reads a prompt in bounded chunks (default 1280 rows; COLI_VK_CHAIN_ROWS / COLI_PROMPT_SEGMENT set it), so the workspace is priced at one chunk and stays bounded as the context grows. The regression test (tests/test_resource_plan.py) builds a two-layer 128-expert checkpoint and checks that both long contexts keep a positive expert cache per tier. Python-only; no engine source changes.
…orkspace gives 05e7748 pinned the contexts chat_context() recommends for DeepSeek V4.1 when the prompt's workspace was priced at the whole context: 16384 on 16 GB, 65536 on 64 GB, 131072 only from 512 GB, and the refusal to start at 262144 (53.7 GB of workspace, no room for one expert slot). 008a01b prices the workspace at one prompt chunk, so the same table reads 65536 on 8 GB, 131072 on 16 GB and the family's 262144 ceiling from 64 GB. The workspace is still in the sum and still in the reason it prints (0.3 GB at every step); the comment says which share applies at each step and what the table was before. Caught by `make -C c test` on the merge of fix/ram-plan-128k: 7 failures, these five and the two known environmental test_harness_builds_without_warnings cases. With this change the target is green (3019 tests, 0 failures, 358 skipped).
…ts it tested What the wave after Track A and Track B measured, tied to the commits that carry it: the turn-end ledger (c615cca) closing the audit's 2.14 GiB as the lm head, 40 routers and the DSpark stages rather than the tier; the 48 streaming slots and the chunk reserve (0.27 GiB off the engine's own hold, 0.001 off the card); the 2-row stream threshold (chunk 256: the CPU's share of expert rows 34.5% -> 1.5%, the prefill 226.7 -> 178.0 ms a row, at 1.15 TB of expert reads a turn); the tier pool's page (90959d6); the packed candidate mask (12b2908: 1.94x the chunk for the same 256 MiB, 1677 -> 3253 rows at a 20000-position prompt) with a6486c7's correction of the base it was quoted against; and the merge of the RAM-plan fix (9508327) with the first GPU run it ever had: --ctx 131072 and 262144 plan 21 and 20 slots a layer, load, and finish a turn in 17.9 and 18.1 s with no OOM. The corrections are called out rather than folded in: the 1262.5 MiB is the lm head and the embedding never touches the device; the per-layer drain costs 0.5 ms a chunk and waits on nothing the host does not read; and the card's sysfs peak does not move when the engine's hold does, because the desktop takes the difference (1.46 GiB held less, 0.19 GiB read less, 1.27 GiB to the compositor). The comparison (Track A) and the audit (Track B) are copied in byte-identical, because their predictions are what the wave tested; the raw file carries every log line the record quotes, the planner's before/after table and the two serves above. The 50 Hz VRAM sample series stay as scratch: the record names them, nothing commits them.
The full gate on the merged tree at the previous commit: deepseek 29 OK, deepseek-chain 36 OK, prefill-deepseek 2 OK (one environmental failure each), vk-chain-check PASS, make -C c test with test-c ALL PASS and test-python 3227 tests / 294 skipped / the two environmental test_harness_builds_without_warnings failures. The section also records that this target's own totals move between runs (3227, 3019, 3227 on the same box), so a count alone is not a regression signal.
…nd gitignore with it 955a0f6's guard grepped ', shared RAM,' which no code path prints (vk_tier.c prints 'shared RAM (COLI_VK_TIER_GB)', 'shared RAM: a quarter of what the expert cache leaves' or '(shared RAM)'), so the DSpark-on-shared-RAM default assertion was silently skipped on every device. It now matches any 'shared RAM' wording and never 'device memory'. vulkan.md gains the tier's new defaults (STREAM_SLOTS 48, STREAM_MIN 2), the record fixes its quoted line, the pair-cost labels, the 12.1 KiB mask share, and lists the two instrumented runs its M1/M2 numbers come from; .gitignore covers the audit scratch.
mem_info_gtt_used beside mem_info_vram_used: system RAM behind the aperture is where the host-visible pools land when device-local is exhausted, and that spill is host-RAM pressure the process RSS does not show (a 256K-token prefill on a card that had lost 3.3 GiB of its device-local to a leak spilled pull and the mirrors there while RSS looked steady). One more field on the turn-end line, 0.00 when the driver does not export it.
… formats The chain's window rings, compressed rows and index keys mirrored as DeepInfer's paged cache stores them (c/dsv41_kvq.h, the host reference): an MXFP8 window row (e4m3, a ue8m0 scale a group of 32, 528 B), an FP4 compressed row (e2m1, an e4m3 scale a group of 16 as stored, 288 B) and FP4 index keys in block-wide sections (a ue8m0 scale a group of 32, a block of 32 tokens, 68 B a row on average). chain_dsv4.comp packs a row after its last fp32 reader (mode 13) and dequantizes it where read (the attention's flags 2, the index scores' quant field) and on the pull's write-back (mode 14), the scales' reciprocals as DI's bit-exact LUT math (a GPU's 1/x is a couple of ulps low, which the codecs' bit agreement cannot take). Off by default; the flag branches only at the mirrors' widths and stores, a forward's first check sums the mirrors' bytes against the flag's widths, and the KV split still decides as before: when it arms the mirrors are fp32 whole again, re-mirrored from the host.
…ode gated tests/test_vk_kv_quant holds the device's pack to dsv41_kvq.h's reference bit for bit (the edge rows of the scale math included: a zero amax, denormals, -0, an amax a power of two over the format's max, the clamp edges and the RNE ties), its unpack, a re-upload's little-endian lanes and the pack's idempotence (a fixpoint in values: a group at half its scale's span re-packs a step down to the same decoded row), and the index scores over quantized keys to the decoded rows' bits with the original rows' drift and top-k overlap reported. The kv-quant-deepseek family runs the tiny fixture at the released KV geometry (make_dsv41_tiny.py --kv-quant-dims: head_dim 512, index_head_dim 128, the formats' own widths) under the flag against a documented logits tolerance -- the flag-off run is the exactness proof, as V41_PREFILL_TAIL's tests put it -- with the token drift printed and the ledger's kv-mirror line pinned to the 528/288/68 B math.
…d save ENVIRONMENT.md's flag row; deepseek-v41.md a subsection by the trunk-first placement: the three formats, the ~6,400 -> ~890 B a position of mirror, the ~690 MiB at 128K tokens and 5.38 GiB of the peak at the 1M ceiling (the prefill-VRAM analysis's §10 figures), the approximate-mode caveat (a pull returns dequantized rows, the pack's idempotence bounds the hop) and the KV split's precedence.
…he index staging an index row wide Two bugs the tiny fixture's first quantized run found, both in the chain's integration rather than the codec (which the unit test had already pinned bit for bit): - vkc_write's offset is in FLOATS, like every chain offset -- the pack re-uploads passed bytes, so the index keys' scale section landed four times past its block and the frame died as a copy out of range. The three upload helpers now divide by 4. - the fp32 staging for a chunk's new index rows was sized with index_n_heads (4) where a row is index_head_dim floats (128): the idx_wk matmul wrote past 224 bytes of scratch. qik is an index row wide, and the chunk-row accounting with it. The kv-quant-deepseek gate with them: the first decode row is the one input the two runs share, so it is the row the tolerance gates (the argmax kept, the drift within the row's own largest |logit| -- the tiny fixture's random weights leave gaps narrower than the format's noise; past that row a flipped pick feeds the model its own choice), and the ledger's kv-mirror line comes from a one-turn serve run of each flag setting (the ledger ends a turn; the CLI never has one): 6.4 -> 1.6 MiB on the fixture, the 528/288/68 B math rounded to whole index blocks. The test's last line is the plain PASS the gates grep for.
…s, the gate as it gates The docs' 128K/1M figures now sit on the corrected 6,400/890 B a position basis (0.78 -> 0.11 GiB at 128K, 6.25 -> 0.87 GiB at 1M), with the audit's older savings-as-totals numbers named as such; the family's comment states the gate it runs (the input-shared first row keeps its argmax, drift within that row's own largest |logit|) instead of a tolerance it does not use; the idempotence claim says values, not bytes; the kvq fixtures and run CSVs are ignored; the wide score case's comment names mode 10's grid arm and that mode 9's quantized reader is the same expression.
Contributor
Author
|
Superseded by #2712, which is this branch in full plus the 9 commits that put the host's Nothing is lost: Closing in favour of #2712. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DeepInfer (the official deepseek implementation of v4.1) quantises the values stored in KV cache. Colibri uses fp32.
FP32 is undesirable for two reasons:
The following are changed:
Stacks on #2637 (trunk-first) and carries the prefill-VRAM wave it builds on (ledger, 48 streaming slots + budget ordering, 2-row stream threshold, page-granular pool limits, 1-bit candidate mask, chunk-priced RAM planner).
What
The chain's window rings, compressed rows and index keys can live in the same formats DeepInfer's paged cache uses, cutting mirror VRAM ~4-7x (geometry-dependent). A row is quantized once, after its last fp32 reader (
chain_dsv4.compmode 13), and dequantized where read -- the attention (flags bit 2) and the indexer's scores (modes 3/9/10,quantfield) decode inline; the pull's write-back dequantizes (mode 14), so the host's state stays fp32 holding dequantized rows and rollback/prefix-reuse/lost-device need nothing rebuilt. Scale reciprocals use the vendor's bit-exact reciprocal-LUT math on both sides (a GPU's1/xis ~2 ulp low, which bit-agreement cannot take).The flag branches only at the mirrors' widths and stores; the first forward sums mirror bytes against the flag's widths and aborts on a mismatch; the KV split decides exactly as before and, when it arms, the mirrors revert to fp32 whole and re-mirror from the host (logged, asserted). Flag-off is bit-identical: every existing oracle stays green.
Formats (DeepInfer
cache_common.py)[data|scales]~6,400 -> ~890 B a position: 0.78 -> ~0.11 GiB of mirrors at 128K tokens (~690 MiB saved), 6.25 -> ~0.87 GiB at the 1M ceiling (~5.4 GiB) -- the analysis in
docs/experiments/dsv41-prefill-vram-2026-11-06.mdand the docs note in this PR.Measured
Tests
c/tests/test_vk_kv_quant.c: codec bit-for-bit host vs device (edge rows: zero amax, negative zero, denormals, amax=2^k, the e4m3/e2m1 clamp edges, RNE ties, the 32-token block boundary, scales packed as int32), unpack exact, the re-upload little-endian lanes pinned end to end, pack o dequant a fixpoint in values, the score readers vs decoded keys with top-k overlap 92-100%.vulkan_engines.shfamilykv-quant-deepseek: flag-off is the exactness proof (the bit-exact families keep it byte for byte); under the flag the input-shared first decode row keeps its argmax (drift within that row's own largest |logit|), token drift printed not gated, and the ledger's kv-mirror line asserted against the 528/288/68 B math.deepseek,deepseek-chain,prefill-deepseek,kv-split-deepseek,make test,vk-chain-check).Caveats
Approximate mode, off by default (
COLI_VK_KV_QUANT): FP4 index keys can move a top-k pick, and the pull's write-back returns dequantized rows (bounded by the pack's idempotence). The planner is not re-priced under the flag; the dev2 mirrors quantize too but the ledger line remains device-0-only (pre-existing gap, annotated).Analysis lineage: the session's VRAM audit + the DeepInfer buffer contrast identified the mirrors as the only context-scaling lever with same-quantity precedent; every number above traces to
docs/experiments/dsv41-prefill-vram-2026-11-06*.