Repository navigation
Batch 2026-11-13-b: twenty-eight fixes (DeepSeek V4 CUDA, qwen38 chain and prefix store, server, Windows builds and tests) - #2808
Merged
Merged
Conversation
…lease capacity v4_moe_batch_union keeps dual_loader_lanes() reads in flight, and each of them holds an expert-cache lease until the expert is computed; the expert being computed holds one more. The hot store has min(experts, 6) slots per layer at its smallest, and without the prefill pool a layer can lease only its own slots: lookup_hot returns -1 once every slot in the partition is leased. With 9 lanes (the default; up to 16 via V4_LOADER_LANES) and 9 or fewer slots a layer, the union's later reads found no free slot and the prefill failed with "reading expert N from the expert store failed". The union now caps its lanes at coli_v4_expert_store_lease_capacity() - 1. The new function returns the layer's slots, or every slot while the prefill pool lends them to this layer (what hot_oldest_pool_victim's last-resort fallback can reach), and 0 for an alternative backend, which leaves the lanes as they were. Runs with more slots than lanes are unchanged. test_deepseek_v4: test_expert_store_lease_capacity pins the store contract the cap relies on: 6 concurrent leases on a 6-slot layer, the 7th refused, 12 while pooled (8 leased), 6 again after the pool. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ffer dsv4_cuda_upload_head_exact built the packed head through dsv4_cuda_upload_compressor: a full row-major copy, then the packed copy, then a free of the first. Both copies were live at once, so the upload peaked at twice the head (2020 MiB for V4-Flash's 129280x4096 bf16 head), and coli_v4_gpu_head_upload asked for rows*cols*4 bytes above the resident reserve before trying. Upload the head in chunks of at most 1024 rows into a device staging buffer and pack each chunk straight into the final allocation with the existing pack_bf16_rows32 kernel, at offset r*I (r is a multiple of 32). The peak is the head plus the staging buffer (1018 MiB for V4-Flash), and the placement check now asks for that. Same packed layout, same scoring kernels, same preconditions: the logits do not change. test_dsv4_head_cuda gains a 2080x256 shape in its --small set, so the small run crosses two chunk boundaries and a 32-row tail. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#2356 added qwen36 to env_for_engine's RAM_GB tuple because the gateway turns RAM_GB into Qwen's positional expert-cache cap. The gateway's cap planner reads RAM_GB the same way for qwen38 (openai_server: arch in deepseek_v41, mimo, qwen36, qwen38), but coli never forwarded the flag for that arch, so `coli serve/chat --ram N` on Qwen3.8 was ignored and the planner used an inherited RAM_GB or measured RAM instead. Add qwen38 to the tuple and run the existing regression test for both Qwen archs as subTests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/test_cuda_chain.cu and tests/test_cuda_dsv4c.cu call setenv/unsetenv, which MSVC does not declare. On Windows, nvcc uses cl.exe as its host compiler, so `make cuda-chain-check` (and `make gpu-compile`) stopped at `identifier "setenv" is undefined`. Both files now carry the same _WIN32 shim over _putenv_s that tests/test_backend_cuda.cu and tests/test_fp8_warp_cuda.cu already use. An empty _putenv_s value removes the variable, so unsetenv keeps its meaning. Test-only. Builds that do not define _WIN32 are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The V4.1 image expander joined a message's text parts and image placeholder spans with no separator. The checkpoint's encoding.py (process_image_messages) treats each text part and each image as its own block and joins the blocks with "\n\n", so a multi-part or image request reached the engine with different prompt bytes than the vendor encoder produces. Join with "\n\n" as the reference does. Tests: the reference's encoding case 5 (two images between three text parts, thinking mode, effort max) is now copied into tests/fixtures and run through expand_dsv41_images and the renderer; it is byte-identical after folding each placeholder span to the single placeholder the reference writes. The fixture reader opens the files as UTF-8, which the CJK text of case 5 needs on Windows. The text-part test in test_openai_server expects the blank line for deepseek_v41. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Makefile builds single-threaded on a Mac without libomp, but five tests included <omp.h> unguarded and test_glm53_live_prefix called omp_set_num_threads, so make check stopped there. Guard them with #ifdef _OPENMP like test_omp_tune.c, and check every tests/test_*.c for it (tests/test_omp_guard_source.py): CI only builds with OpenMP.
…ly global The Windows SDK UCRT defines INFINITY as ((float)(1e+300)), so device code that writes -INFINITY makes nvcc warn #221 at every instantiation (45 in coli_cuda.dll). Use -__int_as_float(0x7f800000) at the four sites, the spelling cuda_dsv4c.cuh already has; the value is the same -inf. g_cuda_raw_experts is read and written only under COLI_ANS; define it under the same guard so a COLI_CUDA build without COLI_ANS does not warn that it is unused. Co-Authored-By: Claude <noreply@anthropic.com>
#2652 added V4_LINK_STAMP = deepseek_v4.ldflags (Makefile.deepseek-v4), written into c/ by every `make deepseek-v4`. Its two siblings, deepseek_v4.cflags and deepseek_v4.cudaflags, are listed in the root .gitignore and in tools/clean.py; the new one is in neither. So a standalone V4 build leaves `?? c/deepseek_v4.ldflags` in `git status`, and `make clean` (tools/clean.py) leaves the file behind. Add the entry to both lists, and a test_clean_ownership case that reads every V4_*STAMP name from Makefile.deepseek-v4 and requires each one in clean.py's FILES and in .gitignore, so the next stamp cannot repeat it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
q38cc_mirror allocates every attention layer's K/V, index-key and pooled-key mirrors at the host's capacity. When one cc_buf failed it set cap = 0 and returned, leaving the layers before it (and the part of the failing layer that did allocate) on the card. q38cc_forward then logged "device memory for N rows refused; the per-matrix path" and set ch->failed, which no code resets and every chain entry point checks: the mirrors, the largest of the chain's buffers, stayed on the card unused until the process exited, VRAM the per-matrix path and the expert tier could no longer use (and, under WDDM, process commit). q38cc_mirror_free releases every attention layer's mirrors (the MTP head's slot included) and resets cap and the watermarks. q38cc_mirror calls it on a refusal, so the mirror is all or nothing, and q38cc_forward calls it before turning the chain off. The host's K/V rows are canonical, so nothing is lost; a later q38cc_mirror (prewarm, MTP head) uploads them again. test_qwen38_cuda_chain: a chain written by hand on the fake card with fake_chain_dev_cap set so the second attention layer's mirror is refused, then so every mirror fits and a row's scratch does not; both forwards return 0 with the card's bytes back where they were, and a later forward takes nothing. The three checks fail on dev. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s take upload() in DeepGEMM builds packed DeepGEMM scales for every fmt 8/9 tensor with O and I multiples of 128. The packing kernels work in packs of four 128-column groups (I/512), and the fmt 9 path splits O into 1024-row groups: - I < 512 (I = 128, 256, 384): groups/4 is 0, the packing grid is empty, the launch fails with "invalid argument" and the whole upload returns 0, so the tensor silently stays on the CPU. - fmt 9 with O < 1024: groups = O/1024 = 0 and O/groups divides by zero on the host, which kills the process. fmt 9 is attn.wo_a with the batched attention path on. No V4-Flash shape is affected, but small models (the V4 tiny fixture, all 128x128) fail the upload or crash. Every DeepGEMM consumer needs I % 512 == 0 (and the grouped wo_a path N == 1024), and refuses a tensor whose dg_scale is NULL, so pack only those shapes. Other tensors keep dg_scale NULL and run on the generic kernels. The packing of the real shapes is unchanged. test_dsv4_dense_batch_cuda now uploads small fp8 and fp8-bf16 tensors (128x128 up to 512x512) and checks a matvec on each against an exact host reference; on a DeepGEMM build it failed the 128x128 upload before this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r's budget
The GGUF path offers the chain's memory (q38cc_reserve_bytes) to the placer
as a component of its own ("chain") before the tier sizes its expert budget.
The native FP8 path did not: qt_init_fp8 took free minus 1 GB minus the
trunk as its budget, and the chain, set up after q38_tier_start, allocated
its K/V mirrors, DeltaNet state, scratch and own matrices into the room the
promotions were filling. Promotions run until a cudaMalloc fails, so the
chain's later growth (layer-major buffers, a verify's copies, larger
read-backs) is refused and q38cc_forward turns the chain off for the
process; with the driver's sysmem fallback the uploads spill to host memory
instead.
Move the GGUF branch's offer into q38_tier_offer_chain and call it from the
FP8 branch too. qt_reserve_stage stays GGUF-only. This is the qwen38
counterpart of what qwen36's main does with qt_reserve (#2741).
test_qwen38_tier_engine: a pass with COLI_CUDA_CHAIN=1 checks that the
chain's offer is placed on the card and that the expert budget is the
chain-off budget less its bytes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… card The capacity growth case the fix covers had no check: place the mirror at 32 positions, then grow it to 64 with room for the first attention layer's new mirrors only. q38cc_mirror returns 0 with cap 0, no mirror pointer and the card's bytes back where they were. On dev the last layer's old mirrors and the second layer's half stay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
q38cc_reserve_bytes's comment said the "chain" offer exists so a GGUF's arena leaves it; q38_tier_offer_chain now makes the same offer before the FP8 tier sizes its promotions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
q38_trunk_place_all quantizes every matrix it offers to the CUDA tier to int8 and kept its BF16 rows; they were freed only later, in q38_trunk_cpu_int8, after q38_tier_start returned. With COLI_CUDA=1, FP8 experts and a BF16 trunk, the load therefore held the whole BF16 trunk plus the int8 copy of every placed matrix at once, a transient the gateway's planner (which prices the trunk at its int8 steady state) does not budget. Every CPU read of a placed matrix already goes through q8 (q38_weight_matmul takes q8 before data), so free the BF16 rows right after placement, with the same resident_weight_bytes bookkeeping as q38_trunk_cpu_int8, which then skips those matrices. Gated on q38_trunk_cpu_int8_wanted(): Q38_TRUNK_CPU_INT8=0 still keeps BF16. Block-format (GGUF-quantized) trunk items skip this code; a GGUF's BF16/F32 trunk matrices take the new path and end in the same state. The placement log reports the GiB released. test_qwen38_tier_engine: after the trunk-on-the-fake-tier pass, run model_init + q38_tier_start directly and check that every placed matrix has q8, no BF16 rows and the int8 resident_weight_bytes, and that Q38_TRUNK_CPU_INT8=0 keeps the rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_qwen38_tier_engine, test_qwen38_dn_gpu and test_qwen38_cuda_chain #undef setenv inside their _WIN32 block. compat.h already maps setenv to compat_setenv there (and c6d6b10 dropped the per-test helpers for that reason), so after the #undef the tests call an undeclared setenv and do not compile with MinGW gcc. Keep the unsetenv override, drop the #undef. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… runtime search
dsv4_load_one glued COLI_DSV4_DLL onto the engine's folder, so an absolute
path (C:/x/coli_cuda_dsv4.dll or C:\x\...) never loaded as given; the by-name
fallback (application dir + System32) then found nothing or found the DLL
without its CUDA runtime on PATH: error 126, "continuing-CPU", a silent ~3x
slower run.
- A name with '\', '/' or ':' is an explicit file: GetFullPathNameA (which also
turns '/' into '\') and LOAD_WITH_ALTERED_SEARCH_PATH, so its imports resolve
beside it, then System32 and PATH. A path over MAX_PATH says "path too long
or invalid" instead of a stale error. Bare names keep the engine-folder load.
- SetDllDirectoryA("") before loading: the altered search order looks in the
CWD before PATH, so a cudart64_*.dll / cublasLt64_*.dll there was loaded
into the engine (silent CPU fallback on a bogus one).
- The failure names the CUDA runtime generically (cudart64_*.dll, plus
cublasLt64_*.dll for a DSV4_TC=1 build) and the path that was tried; the
success line prints the loaded DLL's path.
tests/test_dsv4_dll_path.py (make dsv4-dll-path-check DSV4_DLL=...): a real
DLL outside the engine's folder with the runtime only on PATH loads by '/'
and '\' path; runtime off PATH and an over-long path are named; the default
still loads beside the engine with a fake runtime in the CWD; a bare name
ignores a DLL in the CWD. Skips without DSV4_TEST_DLL.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A relative value (sub\coli_cuda_dsv4.dll) resolved against the engine's folder on dev; the previous commit sent any name with '\', '/' or ':' to GetFullPathNameA, which resolves it against the CWD instead. Only a drive path (C:\x, C:/x) or a UNC / \?\ path is now an explicit file; anything else is glued onto the engine's folder as before. test_relative_path_stays_under_engine: sub\<dll> beside a copied engine loads from there with the CWD elsewhere. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The DeepSeek V4 loader and resource_plan tell Pascal/Turing users to build with CUDA_ARCH=portable-pre-ampere NO_TC=1, but on Windows the DLL is built by c/Makefile's cuda-dsv4-dll, which passed the preset to nvcc as -arch (fatal: Unsupported gpu architecture) and read only DSV4_TC. The preset and NO_TC lived only in Makefile.deepseek-v4's Linux/macOS block. c/Makefile now expands portable-pre-ampere to the same gencode list as Makefile.deepseek-v4 and maps NO_TC=1 to DSV4_TC=0; the Windows DLL table in docs/deepseek-v4.md gains the row, and a make dry-run test pins it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ndows test_qwen38_int2_convert kept CovStats (and its np.memmap of cov.bin) alive when its TemporaryDirectory closed. test_qwen38_int4_convert runs convert(workers=1), which builds the worker's Source in the test process; Source opens the shards with os.open and never closes them, and every convert() call leaves a new one behind. Windows cannot delete an open file, so the temp-dir cleanup raised WinError 32 in 5 tests. Drop the memmap, and close the in-process Source's descriptors after each convert. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…it is recommended The preset's gencode list starts at compute_61, which CUDA 13 dropped (nvcc 13 lists compute_75 first), so nvcc stops at once with "Unsupported gpu architecture 'compute_61'". The loader's no-usable-DLL message, the backend's no-kernel-image reason, resource_plan's CUDA_TIER_FLOOR hint, Makefile.deepseek-v4 and docs/deepseek-v4.md all sent Pascal and Turing users to the preset without that limit. They now say "CUDA <= 12.9"; the Makefile comment and the doc add that a Turing card builds on CUDA 13 with CUDA_ARCH=sm_75 NO_TC=1. A test reads the preset's oldest compute_XX from Makefile.deepseek-v4, takes the last release that builds it from setup_flow.CUDA_RELEASES (the measured nvcc table) and checks every hint names that limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With a tokenizer loaded, a reference run printed the decoded text on the
"C engine :" line itself ("C engine : 1 2 3 Text : ..."), followed
by a blank line before "Matching tokens". The text now starts its own
line and the blank line is gone; without a tokenizer the output is
byte-identical.
spec_drafts_harness.engine_tokens now accepts only ids on that line
instead of cutting at "Text", so the qwen38/qwen36 MTP and spec-draft
gates (whose fixtures carry a tokenizer) catch it if text comes back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s it On a low-RAM box the RAM plan streams the BF16 head, and coli_v4_gpu_head_upload returned early without a host copy: the head ran on the CPU and read 1.01 GiB from disk every token, whatever the free VRAM. - coli_v4_gpu_head_upload reads a streamed head for the upload and frees the host copy afterwards, uploaded or refused. It reads only for a BF16 vocab x hidden head, a backend with the exact head, a device with room and head + 512 MiB of available host RAM; otherwise it logs why. - head_scores_impl tries the device head before streaming. - dsv4_cuda_head_exact_available: optional export (dsv4.def, loader), so a DLL built before this keeps the old behaviour. - test_v4_gpu_placement: borrowed head refused, uploaded, unreadable, and not read when the upload cannot happen. V4-Flash, RAM_GB=11 (head streamed): head time 289 -> 5.9 ms/token warm, 1280 -> 21 ms/token on the first turn, same text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…out the CR that Windows' stderr adds The engine's stderr lines end in \r\n under MinGW, and prefix_serve_harness decodes them without newline translation, so REUSED's "from (.+)" captured "the live state\r" and six tests failed on Windows. Stop the capture at \r or \n. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ave, hold the RAM floor across a save, keep a chain's root checkpoint - coli_ps_save refuses a save whose chain and state (coli_ps_cost: every position's rows and token, and the state) exceed the budget before it evicts anything. Before, it added the blocks one by one, evicting other entries for each, then failed at the checkpoint and left the blocks behind; every later save on that chain did the same. - A save that fails (the RAM floor, an allocation) takes back the blocks it added (coli_ps_unwind): without their checkpoint they are only RAM. - The free-RAM floor holds across a whole save: coli_ps_room lowers avail_mb by every grant and raises it by every eviction. It was checked per entry against the same value, so one save could commit the whole budget below COLI_PS_RAM_FLOOR_MB. - A chain's shallowest checkpoint (where its system prompt ends) is marked used whenever the chain is, by a save and by a lookup (coli_ps_touch_root), and between checkpoints used at the same time the deeper is evicted first. A turn's end no longer evicts the system prompt's checkpoint as the oldest entry. tests/test_prefix_store: one check for each; 5 of them fail on the old header. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y the blocks below the row the slot's next token made q38_ps_take kept in place every whole block the slot's record shares with the prompt. The MTP head's row at p pairs the streams at p with the token at p + 1, so its row at common - 1 was made with the record's token at common, not the prompt's. When the two part exactly at a block boundary, that row stayed. With the head on, keep is now at most common - 1: the block that ends there is loaded again from the store. Drafts are verified, so only their acceptance was affected, never the output. Test: test_mtp_head_row_where_the_slot_parts_from_the_prompt (a system prompt that differs from the stored one at token 32): 16 rows in place with the head on (32 on dev), 32 with it off; the answers a fresh read's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_EXCEEDED q38_serve_start attaches the request's image (q38_vision_attach) before the context check. The CONTEXT_EXCEEDED refusal returned without q38_vision_detach, and both callers (serve_one_turn and the KV_SLOTS mux) return on a failed start before their own detach. The picture's map stayed for the next request: its prompt read the picture's rows in place of its own tokens at those absolute positions (q38_embed_row), its prefix reuse was tainted, and the prefix store stayed off for it. The refusal now detaches the picture. test_qwen38_vision_serve.py sends the same text-only prompt before and after an image request refused for context and expects the same answer; turn() now keeps each DATA frame's bytes so two requests with the same id compare text, not only lengths. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…quest its IMAGE frame names An IMAGE frame was kept for whatever SUBMIT came next. A SUBMIT refused before the picture was attached (CONTEXT_EXCEEDED, EMPTY_PROMPT, BAD_REQUEST, SLOT_BUSY, DUPLICATE_ID) left it pending, and the next request, text only, went into mlv_attach and was answered ERROR BAD_IMAGE. The pending picture now keeps its IMAGE frame's id, as glm53 does; at SUBMIT a picture with another id is dropped with a stderr line before the attach. tests/test_ml4_pending_picture.py drives it on the tiny ML4 model and tower, wired into the Mistral-Large-4 CI step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s to E4M3 per 128 on the device coli_fp8_matvec_ref and coli_fp8_matmul_batch_ref round every fp8 input to E4M3 per 128 (coli_fp8_activation_qdq_ref) before they multiply, and the fused CUDA kernels run fp8_sim before run_mv<8>. A mirrored weight is sent to the GPU with the raw input, and dsv4_cuda_matvec / _matvec_grouped, dsv4_cuda_wo_decode (wo_a input and the bf16 intermediate), _proj_batch and the generic _matmul_batch multiplied it as it came. So CUDA decode used other operands than the CPU build and the fused path. act_quant_input runs fp8_sim per 128 on the device copy before those kernels: per group in _matvec_grouped (as the CPU group views), on the wo_a input and after bf16_round in _wo_decode, once in _proj_batch (all fp8), and in _matmul_batch on a device copy, so the caller's activation stays raw (dsv4_cuda_moe_activation does the same). The DeepGEMM branch of _matmul_batch is left alone: its kernels round. fp4 (run_mv<4>) is not touched. DSV4_CUDA_ACT_QUANT=0 keeps the raw input. tests/test_dsv4_act_quant_cuda.c checks each call bit for bit against coli_fp8_matvec_ref on operands whose sums are exact in any order, and that each call differs under DSV4_CUDA_ACT_QUANT=0; it joins dsv4-cuda-test. tests/test_dsv4_gpu_grouped.c rounded nothing in its CPU reference; it now rounds x per 128 per group unless DSV4_CUDA_ACT_QUANT=0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t-floor-unwind-root
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Twenty-eight PRs, fixes and the tests and builds around them, each merged with its own commits. None conflicts with another; one commit of ours on top (below).
What is in it
DeepSeek V4
COLI_DSV4_DLLpath is loaded as given, and the current directory is out of the CUDA runtime's search order.cuda-dsv4-dllon Windows acceptsCUDA_ARCH=portable-pre-ampereandNO_TC=1.portable-pre-ampereneeds CUDA <= 12.9, said wherever the preset is recommended.DSV4_CUDA_ACT_QUANT=0keeps the raw input).Qwen3.8 and Qwen3.6
Text :on its own line after the reference'sC engine :ids.CONTEXT_EXCEEDED.COLI_PREFIX_LOG, each checkpoint save says whether it was kept.Prefix store, shared by eight engines
Server and launcher
--ramreaches qwen38 asRAM_GBtoo.encoding.pydoes.keep_enginetakes the context andmax_tokensof the restarted engine.Builds and tests
c/deepseek_v4.ldflagsis ignored and cleaned.datapoint.pyrefuses non-finite memory budgets.test_qwen38_prefix_servereads the reuse source without Windows' CR.All by @KairaCode except #2737 (@divya0795), #2739 (@tylnexttime) and #2756 (@Stamina9).
Ours: #2755's new row in
docs/deepseek-v4.mdrecommends the pre-Ampere preset and #2760's test wants every such place to name CUDA <= 12.9. The row now does.Tests
make checkon the whole branch.check-envand every C test pass. The Python suite ran 3398 tests: the only failure was the docs row above, and with it fixedtest_deepseek_v4_cuda_arch_flagsandtest_makefile_platformpass.colibri.c,cuda_chain.cu,deepseek_v4.c,qwen38_core.h,openai_server.py,resource_plan.py,tests/test_openai_server.py; no conflict):make checkagain, every C test and 3405 Python tests pass.qwen38-tiny-fp8-check,qwen38-tier-engine-check,qwen38-dn-gpu-checkandqwen38-cuda-chain-checkpass, 8 of 8 tokens in every run (qwen38: keep the CUDA chain's device memory out of the FP8 expert tier's budget, as the GGUF arena already does #2746, qwen38 CUDA chain: a device-memory refusal gives the K/V mirror back instead of holding it for the process #2747, qwen38: free a placed trunk matrix's BF16 rows as soon as its int8 copy exists, not after the whole trunk #2748).make dsv4-cuda-testpasses, with deepseek-v4 CUDA: round the input of the generic 128-block fp8 matvecs to E4M3 per 128 on the device, as the CPU reference and the fused kernels do #2767's newtest_dsv4_act_quant_cuda(the 128-block fp8 calls bit for bit againstcoli_fp8_matvec_ref) and deepseek-v4: pack DeepGEMM scales only for shapes the DeepGEMM kernels take (no zero grid, no divide by zero) #2745's dense batch test.make cuda-chain-checkpasses (331ok), with tests: build the CUDA chain tests with nvcc's MSVC host (setenv shim) #2731's shim and cuda: build without nvcc #221 on Windows and an unused ANS-only global #2739's kernels.colibri,qwen38anddeepseek_v4build withCUDA=1.test_dsv4_head_cudaamong them, deepseek-v4: upload the exact CUDA head through a 1024-row staging buffer #2729 and deepseek-v4: upload the exact CUDA head even when the RAM plan streams it (read it for the upload, then free it) #2763): there is one card. No model run of DeepSeek V4 on CUDA, so deepseek-v4 CUDA: round the input of the generic 128-block fp8 matvecs to E4M3 per 128 on the device, as the CPU reference and the fused kernels do #2767's effect on speed is not measured. The Windows builds and tests (tests: build the CUDA chain tests with nvcc's MSVC host (setenv shim) #2731, cuda: build without nvcc #221 on Windows and an unused ANS-only global #2739, tests(qwen38): keep compat.h's setenv on Windows so the qwen38 tests build with MinGW #2751, deepseek-v4: load an absolute COLI_DSV4_DLL path as given; keep the CWD out of the CUDA runtime search #2752, tests: let the int2/int4 converter tests delete their temp dirs on Windows #2754, build: cuda-dsv4-dll on Windows accepts CUDA_ARCH=portable-pre-ampere and NO_TC=1 #2755, tests(qwen38): read the reuse source in test_qwen38_prefix_serve without the CR that Windows' stderr adds #2769) are left to CI's Windows jobs.