Skip to content

Batch 2026-11-08 D: grid IQ dot on the CPU, the DeepSeek V4 split, GLM-5.3's contract, the env registry and 8 more PRs - #2702

Merged
JustVugg merged 76 commits into
devfrom
batch/2026-11-08-d
Nov 8, 2026
Merged

JustVugg merged 76 commits into
devfrom
batch/2026-11-08-d

Conversation

@JustVugg

@JustVugg JustVugg commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

One CI run for twelve open PRs and six commits of our own, built on #2699, now in dev. Our first commit comes first, then the PRs (each merge commit names its PR), then the fixes the merge needed.

Our commits

  • CPU: the grid IQ types' dot product with the signs on the activation. IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS and IQ3_S put their sign bits on the decoded weights, and the integer dot product took them off again. The grid rows now go in as they are and the signs go onto the activation. Every product has the same two operands, so the bits are the same (tests/test_gq_signs against the kernels it replaces: 2304 rows and row counts, 0 failures). A sub-block's indices come from one load, and IQ1_M's decoder reads its scales and index bits a word at a time.

    • One Zen 4 core, x86-64-v3, a 27B gate matrix: IQ2_XXS 3.58 -> 4.37 GB/s, IQ3_XXS 3.45 -> 5.22, IQ2_XS 3.08 -> 3.53, IQ1_M 1.16 -> 1.90. Sixteen threads, IQ2_XXS: 20.7 -> 25.5.
    • Qwen3.8-27B UD-IQ2_XXS on the CPU, the same greedy answers, prompt / decode: a Ryzen 7 8700GE 2.7 / 2.4 -> 2.8 / 2.8 tok/s, a Core i5-13500 1.9 / 1.9 -> 2.0 / 2.2.
  • Env registry (Add a checked registry for engine environment variables #800): the variables dev and this batch read since Add a checked registry for engine environment variables #800 was written. check-env named 46 variables the code reads that coli_env.h did not list. Each row takes the engines of the binaries that read it. check-env: 772 variables, the registry matches the sources.

  • Env registry: no line for what coli serve or Windows set on every engine. Served, every qwen36 and qwen38 start printed three lines for variables nobody set for it: NGEN and SERVE_BATCH, which the gateway gives every engine, and COLI_SERVE_PORT, coli's own. On Windows, every engine started from cmd also named PROMPT (cmd's own prompt) and TEMP (the temporary folder, read as the deprecated temperature). Those rows carry CE_LAUNCHER, which drops the wrong-engine line for them and keeps it for what a user sets. A deprecated numeric variable is named only when its value is a number. test_env_registry covers the cases, built for Linux and run as a Windows executable.

  • deepseek_v4: the CUDA tier's unit includes the device-list parser. The split's integration gave v4_cu_open (the GGUF's CUDA tier) a line for DSV4_CUDA_DEVICES, read with coli_v4_gpu_devices_parse, whose header only the GPU unit included. make deepseek-v4 CUDA=1 stopped there. Every unit of deepseek_v4.c now compiles without a diagnostic with COLI_CUDA, COLI_VULKAN and both.

  • CI: the MSYS2 installs without the libgomp package. fix(windows): point at libgomp when -lgomp is missing under MSYS2 gcc 16 #2617 added mingw-w64-ucrt-x86_64-libgomp to the workflows' MSYS2 install lists. The package exists from gcc 16 on; the workflows' MSYS2 (update: false) has an older package list, where pacman stops with target not found and installs nothing, so every Windows job of this batch's first run failed. Its gcc still brings libgomp, so the lists go back to what they were. The quickstart says what the same message means on an MSYS2 that predates gcc 16 (pacman -Syu first, or leave the package out).

The PRs

Conflicts

Checked on the batch

  • An RTX PRO 4000 (CUDA and Vulkan builds of this tree, every engine built without a new warning):
    • tests/cuda_tier_engines.sh 37 OK, 0 FAIL; cuda-chain-check 317 ok; dsv4-cuda-test (dense, attention and MoE batches, the generic kernels bit for bit, bank bytes): passes;
    • vk-tier-check and test_vk_chain: 0 FAIL.
  • The DeepSeek V4 split's own CUDA tests (make -f Makefile.deepseek-v4 dsv4-cuda-tests) take two cards (DSV4_TEST_DEVICES), and no V4 checkpoint or GGUF is on our machines now: the split's speed and its byte-identical answers are the author's runs (2x RTX 3090 and an RTX 5070, in its PRs), and here the kernels' tests and CI's V4 oracles.
  • GLM-5.3 (unsloth's GGUF, imported): detect_chat_flavor gives the full model's contract; GLM-5.2's import keeps its own.
  • Served against batch 2026-11-08-c: Qwen3.8-27B on CUDA, Vulkan and the CPU, the CUDA hybrid wait bounded, tool-call literals, Vulkan hybrid and indexer work #2699 (dev), prompts of 1000 and 4000 tokens and decode, tok/s:
    • Qwen3.6-35B int4 on CUDA: prompt 2717 / 3347 -> 2929 / 3393, decode 106.5 -> 107.9; on Vulkan 2187 / 2402 -> 2221 / 2408, decode 98.8 -> 98.7;
    • Qwen3.8-Flash-Next UD-IQ2_XS on CUDA, three runs each way, alternating: prompt 1K 289 / 314 / 302 -> 282 / 289 / 291, 4K 1398 / 1369 / 1359 -> 1370 / 1373 / 1375, decode 65.9 / 67.8 / 64.8 -> 63.9 / 64.4 / 64.5 (a first pair gave 61.1 -> 65.4); on Vulkan prompt 126 / 127 -> 126 / 127, decode 34.1 -> 33.8.
  • make check on Linux, the C suite and 3348 Python tests: everything passes but the two test_harness_builds_without_warnings, which fail on dev as well with this compiler (a GCC note on mistral_vision.h).

ZacharyZcR and others added 30 commits August 31, 2026 15:24
Two failure modes, both silent today:

    COLI_PREFIL_CHUNK=512 ./colibri ...    # typo
    K3_BITS=8 ./colibri ...                # wrong engine

Neither prints anything. The run proceeds on the default, produces entirely
plausible output, and the number that gets written into a benchmark table is
for a configuration nobody set. Anyone who has tuned this engine has lost
an afternoon to one of these.

The cause is that nothing in the tree knew which variables exist: 214 of
them, read from 187 scattered getenv() sites, with no list anywhere. So:

  c/coli_env.h -- one table: name, value shape, which engines read it, and
  whether it is deprecated. coli_env_check() runs once at startup in all
  four engines and reports what it cannot use, with a "did you mean X?"
  from an edit-distance match:

    [env] unknown variable COLI_PREFIL_CHUNK -- did you mean COLI_PREFILL_CHUNK?
    [env] K3_BITS is not read by colibri (it belongs to kimi_k3) -- it will have no effect

  It warns, it does not exit: an unrecognised variable has never stopped a
  run and making it fatal would break scripts that export a knob for
  whichever engine they may launch. COLI_ENV_STRICT=1 makes it fatal for
  those who want the guarantee. COLI_ENV_DUMP=1 prints every variable the
  engine reads with its current value, which answers "is my export actually
  arriving" without a debugger.

  Variables we do not own are never touched -- only COLI_*, K3_* and INK_*
  are checked, so an unrelated EDITOR in the environment is not our
  business.

This is the shape vLLM arrived at (vllm/envs.py): one declaration point, one
prefix convention, validation at the boundary. Worth recording that vLLM
carries 284 of these to our 214 -- the count was never the problem in either
project, the missing registry was.

WHAT THIS DELIBERATELY DOES NOT DO

It does not rewrite the 187 getenv() call sites. That is mechanical but
large, and one mistyped default in the middle of it silently changes engine
behaviour; it belongs in its own commits, engine by engine, with this table
as the precondition for doing it safely.

So the table could drift from the code the moment it lands -- which is what
`make -C c check-env` (and a CI job) prevents. It fails on a getenv() with
no row, a row nothing reads, a duplicate, or a table that is not sorted
(coli_env_find binary-searches it, so an unsorted table silently stops
finding valid names). All three failure modes were verified by breaking the
table on purpose and confirming the check catches each one. It caught its
own first bug during development: COLI_ENV_DUMP was added to the code and
not to the table.

Only TEMP is marked deprecated -- it is the one the docs already call a
deprecated alias, and the one that actually bites, since $TEMP is the
temp-DIRECTORY path on Windows and under the ROCm runtime (#509). SNAP,
SNAP_MIRROR and PROMPT are fallbacks but remain the mainstream spelling
(README, CONTRIBUTING's oracle command and CI all use SNAP=), so flagging
them would have printed a warning on every existing run and every CI job for
no benefit. Verified: today's usage is completely silent.

All four engines build warning-free and parse under every backend
configuration (default, CUDA, Vulkan, Metal, ANS); make test-c passes; the
Python suite is 288 tests OK (18 skipped).
Rebasing on dev turned check-env red, which is the point of it:

  - read by the code but missing from coli_env.h: COLI_MTP_GUARD_PCT
  - read by the code but missing from coli_env.h: COLI_MTP_GUARD_WINDOW
  - read by the code but missing from coli_env.h: COLI_V4_EXPERT_PREFETCH
  - read by the code but missing from coli_env.h: CUDA_EXPERT_LOAD_BALANCE
  - read by the code but missing from coli_env.h: INK_METAL_SHARED
  - read by the code but missing from coli_env.h: V4_PREFIX_LOG

Types come from the call sites, not from the names: the three read
through atoi() are CE_INT, INK_METAL_SHARED and COLI_V4_EXPERT_PREFETCH
are tested as switches so CE_BOOL, and V4_PREFIX_LOG is presence-checked
like the INK_PREFIX_LOG row already in the table, so CE_STR.

Two of them belong to deepseek_v4.c, which #165 added and which has no
engine bit here, so add CE_DSV4. It is deliberately NOT folded into
CE_ALL: that flag means the four engines sharing route_trace.h, rans.h
and omp_tune.h, and deepseek_v4.c includes none of the three -- putting
it in CE_ALL would claim every shared-header knob is read by V4 too.

Also refresh the header comment, which had gone stale in the same way the
table would have: 212 -> 220. The "187 scattered getenv() call sites"
figure is removed rather than corrected -- nothing verifies it, so it can
only rot again. `make check-env` prints the live count and is the one
number in that comment that cannot go stale.

check-env: 220 variables, registry matches the sources.
Rebasing again turned check-env red again, which is the argument for the
PR restating itself: COLI_V4_{AUTOPIN,DIRECT,MARKOV_BLOCK,MARKOV_KEEP,
MARKOV_SPEC,PREWARM,SAVE_USAGE}, K3_CUDA, V4_{DRAFT,MTP,MTP_DRAFT,MTP_GB,
MTP_MIN,MTP_PARTIAL_KEEP,NGRAM,NGRAM_PARTIAL_KEEP} -- most of them from
#839's MTP/DSpark work, landed since the last rebase.

Types from the call sites: the ones tested as atoi()!=0 or atoi()==0
switches are CE_BOOL, V4_MTP_GB parses with atof() and defaults to 0.45
so CE_FLOAT, the rest read a number with atoi() so CE_INT. K3_CUDA is
CE_KIMI; every V4_/COLI_V4_ row is CE_DSV4.

Header count 220 -> 236.
- CE_MIMO, CE_LAYA and CE_GLINER, MIMO_ in the owned prefixes, and the
  three engines call coli_env_check/coli_env_dump like the others.
- The scanner knows the engines' wrappers that take the literal name
  (mimo's env_int, qwen38's q38_env_int_in_range/q38_env_positive_int,
  vk_tier's env_gb, vk_kvsplit's vkc_kv_env, coli_env_on,
  coli_omp_env_on), which hid MIMO_CAP, COLI_VK_KV_SPLIT and 13 others.
- 297 new rows (702 in all). Each row's engines are the binaries whose
  sources read it: the units `make -n` builds for every backend (CUDA,
  Vulkan, Metal, HIP, XDNA, the V4 GPU tier) and their includes on
  Linux, Windows and macOS. 55 existing rows gained engines this way,
  mostly through the shared CUDA and Vulkan tiers (COLI_CUDA, COLI_VULKAN,
  KV_SLOTS, TEMP, ...), which would otherwise warn "has no effect" on
  engines that do read them. No existing engine bit was removed.
- A row with no engine (COLI_TUI_SIXEL, COLI_API_KEY, LANG: read by
  coli tui, never by an engine) makes no engine warn.
- K3_VK_FILL_FRAC is gone: nothing reads it any more.
- The owner list in the "not read by" message is a table now, instead of
  one %s per engine.
The dense chain fit now runs before the mapped-weights window is
allocated, so the trunk sees the device's real free memory instead of
the ~0.5 GiB left after a 9/10-heap window ask. The window is then
asked for what remains beside the trunk less COLI_VK_FREE_RESERVE_GB
(1 GiB, floor+warn at <1 GiB ask; COLI_VK_WINDOW_ASK_MB still overrides),
the tier sizes into it with the host pool off by default beside the
trunk (COLI_VK_TIER_HOST=1 / _GB re-enable), and dense host copies
stay given back. Other engines, the CPU-only path and the integrated/
Lavapipe defaults are unchanged; explicit COLI_VK_STAGED=1 still forces
staging. Opt-in is deepseek_v41-only (incl. its GGUF path).

A bounded (<=3 s) wait covers RADV reporting the probe's freed blocks
back through VK_EXT_memory_budget before the tier reads it.

Measured on RX 6900 XT 16 GiB: 40 of 40 trunk layers on the device
(7.62 GiB), window 5 GiB, tier 3.28 GiB / 83 resident experts, no GTT
host-pool mirror, no new swap.
Baseline origin/dev vs the trunk-first commit, one build apart, same
serve parameters and the same warm start: decode 0.485 -> 0.786 tok/s
median (+62%) on a single-NVMe ReBAR card, the tier's 11.5 GiB host
pool on -> off with GTT at ~0.27 GiB, and 40 of 40 trunk layers on
the device beside it. Harness committed beside the record.
bobsummerwill and others added 28 commits November 7, 2026 01:00
…'s extra layers upload

A tier pool's limit and its blocks disagreed by a page. tier_pool_limit sized a block from
the budget rounded up to a page (15360 -> 16384 B) but left the limit at the budget itself,
which the pool cannot grow a block past; pool_suballoc then fell back to the limit floored
to a page (12288 B) and counted a refusal. At deepseek_v41's tiny-budget case
(COLI_VK_TIER_GB=0.00005: a three-expert tier, and one extra-layer expert's bytes -- 15360
B -- in the extra layers' pool) the pool could never hold its expert: the first MTP stage's
upload was refused, the run's line read "failed 1 refused by the pool", and T.xmax fell to 0
with the extra layers left on the CPU. The expert needs exactly 15360 B (three tensors of
4096 rows + 1024 scales): the budget was right, the page floor of the block capacity lost
the 3072 B that made it fit.

PR #2637 makes it reachable on a discrete device, where the DSpark stages are the tier's
extra layers by default (COLI_VK_TIER_MTP=1) and not on the CPU as they are on a device that
shares the RAM, so "deepseek_v41 tier a budget of three experts" fails its EVICT gate there
(tests/vulkan_engines.sh: "an upload failed (the budget shrank)"). The bug itself is in the
pool and not in the placement: the pool sized a block it could never grow, whatever put the
expert there.

tier_pool_limit (backend_vulkan.c) rounds its limit up to a page, as its blocks already are,
so a budget of N expert bytes holds N expert bytes -- at most a page (4095 B) over the
tier's own budget, another page-granular figure. block_bytes then follows the limit instead
of re-rounding the raw budget, so the two agree by construction. Nothing else changes: the
spare-block release at the end of the function is as it was, and the fallback in
pool_suballoc is untouched (it now only ever sees a limit the block size was derived from).

This is the core of 90959d6 (perf/prefill-vram), byte for byte: that commit's other two
files -- a defensive cap in vkt_slot_reserve() and cu_tier.c renames for a CUDA build of
vk_tier.c -- do not belong on this tree (vkt_slot_reserve() does not exist here, and the
renames only fix a build that this tree does not have). The fix was first validated there.

Measured on the 16 GiB RX 6900 XT (RADV NAVI21, the radeon ICD; Lavapipe is broken on this
host), the same gate cases at 05e2c6d and with this change, one case at a time so the
family's first failure does not hide it (tests/vulkan_engines.sh's own plumbing):

  before: FAIL deepseek_v41 tier a budget of three experts: an upload failed (the budget
          shrank) -- resident 2 (budget 2), uploads 6, evictions 4, failed 1, "failed: 1
          refused by the pool", extra layers (2) 0 of 48 on the device, resident 0 (budget 0)
  after:  OK -- resident 2 (budget 2), uploads 7, evictions 4, failed 0, extra layers (2) 4
          of 48 on the device, resident 1 (budget 1), device 8 of 588 routed experts, output
          = CPU

The other cases reached on this device fail identically before and after, with the same
counters: the DSpark default-placement case ("the stages went on the tier by default on
shared RAM", the shared-RAM guard trunk-first predates) and deepseek_v4's and olmoe's
three-expert budgets on the radeon ICD ("the budget forced no eviction", "no routed expert
ran on the device"), all discrete-device placement differences unrelated to this fix. The
deepseek_v41 dense and tier caps, its long-prompt arms and the COLI_VK_TIER_MTP=1 DSpark
arms pass before and after.

The harness's own pool test (backend_vulkan.c: run_tier_pool, a 10.5-expert budget that is
not a page multiple) passes both ways -- 10 experts in 1 block, 1 refusal -- and the block
it keeps goes from 913408 B (the budget floored to a page) to 917504 B (the budget rounded
up to one): the limit and the block are now the same figure. 'make -C c test-c' is green.
…s layerwise vk prompt beside the pool's page-granular budget

93 commits from dev and no conflict in the tree: dev touched neither backend_vulkan.c,
backend_vulkan.h nor vk_tier.c, so the trunk-first placement, the extra layers' pools and
tier_pool_limit's page rounding are this branch's, unchanged and byte for byte. What dev
brought lands beside them -- the CUDA chain (cuda_chain.cu, cuda_glm53c.cuh, cuda_dsv4c.cuh,
GLM-5.2/5.3's DSA selection, DeepSeek V4.1's layer ops, COLI_VK_DEV2 for a second card),
vk_chain.c's layerwise V4.1 prompt passes with shaders/chain_dsv4_pa.comp and its .spv,
v4_topk.h's exact radix top-k, Qwen3.8's long-prompt streaming, OLMoE's shards across
COLI_MODEL_DIRS, and their tests and docs.

c/.gitignore took both sides' additions (dev's cuda_dsv4c_test/cuda_glm53c_test binaries,
this branch's three shader .spv) and auto-merged, as did docs/ENVIRONMENT.md,
docs/deepseek-v41.md and deepseek_v41.c. The branch's experiment record under
docs/experiments/dsv41-trunk-first-vram-* is untouched.

Green on the CPU gate before the commit: make -C c colibri inkling, make -C c test-c (the
whole C suite) and make -C c cuda-host-syntax ARCH=x86-64.
…ypes and the Xe2 subgroup pin, V4.1's radix indexer top-k, beside the trunk-first window ask

dev advanced to b2c813b minutes before the previous merge's fetch, so this
re-merges it. Incoming: #2685 "vulkan: memory types the buffers take; Xe2
pins subgroups of 16", #2671 "deepseek_v41: the indexer's top-k through
v4_topk.h", #2660 "fix(vulkan): probe the secondary device mapped memory
window", and the batch merges.

deepseek_v41.c and docs/ENVIRONMENT.md merged clean; deepseek_v41.c carries
#2671's indexer (index_topk.h) and this branch's trunk-first calls untouched.

c/backend_vulkan.c conflicted in exactly the place both sides rewrote:
vk_window_decide, which this branch splits into a trunk-first ask and dev
makes device-parametric for the secondary device's window probe. The two
behaviours compose, and the composition is forced rather than cosmetic --
dev's probe now runs for device 2, so nothing of it may fall back to G:

  - vk_window_decide keeps dev's parameters and takes the trunk-first flag
    last:
      -static void vk_window_decide(VkUp *u, int trunk_first)                       (branch)
      -static void vk_window_decide(VkUp *u, VkPhysicalDevice phys, uint32_t memtype,
      -                             int has_budget, const char *tag)                (dev)
      +static void vk_window_decide(VkUp *u, VkPhysicalDevice phys, uint32_t memtype,
      +                             int has_budget, const char *tag, int trunk_first)
  - the ask is this branch's: trunk_first sizes `want` from the heap less
    coli_vk_device_used() and COLI_VK_FREE_RESERVE_GB and describes it in
    `ask`; otherwise 9/10 of the heap, dev's `want = ... * 9 / 10` kept as
    the else arm. Both arms set want and ask, so the two lines below read
    either.
  - the probe's alloc/free loop is dev's (memtype, u->dev), and this
    branch's budget wait for a driver that reports freed blocks late stays
    around it, on dev's parameters:
      -if (trunk_first && G.has_budget && got > 0)
      -    ... vk_heap_free_now(&bud0, &mp20, hi) ...
      +if (trunk_first && has_budget && got > 0)
      +    ... vk_heap_free_now(phys, &bud0, &mp20, hi) ...
  - vk_heap_free_now gains the device instead of reading G.phys: dev's
    window probe reaches it for device 2 as well, where G.phys is another
    device's heap.
  - the mapped-weights line keeps dev's "[VK] %smemory:" tag and this
    branch's "(%s ...)" ask, so a device-2 line is tagged and a trunk-first
    line says what it asked:
      +fprintf(stderr, "[VK] %smemory: mapped weights, the host-visible
       device-local memory gave the %d GiB asked (%s, less if its budget
       leaves less), found in %.1f s%s\n", tag, got, ask, ms / 1000.0, by);
  - coli_vk_init's window block is this branch's deferral on dev's call:
      if (g_win_trunk_first) g_win_trunk_deferred = 1;   /* the trunk first, the ask once it is placed */
      else vk_window_decide(&g_up[0], G.phys, G.memtype, G.has_budget, "", 0);
    dev's inlined dl_max/cf/hv_dl probe stays deleted: this branch moved it,
    unchanged, into vk_window_up_decided, which the block calls next.
  - coli_vk_window_trunk_ask passes device 0 and trunk_first = 1;
    coli_vk_init_dev2's probe passes trunk_first = 0, dev2 having no trunk.

This branch's tier_pool_limit page-granular budget, its __APPLE__ unistd
include and its why[256] are regions dev did not touch and came through as
they were.
IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS and IQ3_S applied their sign bits to
the decoded weights, and the integer dot product took them off again to
multiply by the activation. The grid rows now go in as they are (every
grid byte is positive and below 64) and the signs onto the activation:
the same two operands for every product, so the same bits. A
sub-block's indices come from one load, and IQ1_M's decoder reads its
scales and index bits a word at a time.

tests/test_gq_signs checks the new kernels against the ones they
replace, built as they were, for one to four activation rows and through
gq_matmul: 2304 rows and row counts, 0 failures.

One Zen 4 core, x86-64-v3, a 27B gate matrix: IQ2_XXS 3.58 -> 4.37
GB/s, IQ3_XXS 3.45 -> 5.22, IQ2_XS 3.08 -> 3.53, IQ1_M 1.16 -> 1.90;
sixteen threads IQ2_XXS 20.7 -> 25.5. Qwen3.8-27B UD-IQ2_XXS on the
CPU, the same greedy answers: Ryzen 7 8700GE prompt / decode 2.7 / 2.4
-> 2.8 / 2.8 tok/s, Core i5-13500 1.9 / 1.9 -> 2.0 / 2.2.
fix(install): ship chat UI and IQ3 runtime data

# Conflicts:
#	CHANGELOG.md
…s2-2612

fix(windows): point at libgomp when -lgomp is missing under MSYS2 gcc 16
…e-20261102

fix(api): select the full GLM-5.3 prompt contract from checkpoint metadata

# Conflicts:
#	c/openai_server.py
…nnect-20261102

fix(proxy): cancel confirmed resets and close upstream responses

# Conflicts:
#	ui/responses_shim.py
…-fixes

DSV4 split 1/6: device refusal fix and GGUF C-only helpers
…gpu-layers

DSV4 split 2/6: multi-GPU layer placement and the KV-owner guard

# Conflicts:
#	c/Makefile.deepseek-v4
DSV4 split 3/6: resident experts, budgeted auto and device routing
…kernels

DSV4 split 4/6: CUDA kernels (indexer, quant, compressor, wo_decode, head)

# Conflicts:
#	c/Makefile.deepseek-v4
…pool

DSV4 split 6/6: peer mirror pool, peer compute and CUDA fault handling

With #2668 (bobsummerwill/split-20261107/dsv4-mirror-vram-first, 5/6:
mirror-cache budget and VRAM-first decode), the lane this one builds on.
Where the two lanes meet (deepseek_v4.c, deepseek_v4_internal.h,
backend_cuda_dsv4.cu/.h, backend_loader_dsv4.c, dsv4.def,
docs/deepseek-v4.md and two tests) the files are #2670's, the
integration of both lanes on e24cd51; dev has not changed them since.
dsv41: trunk-first VRAM placement — place trunk first before Expert Cache
Add a checked registry for engine environment variables

# Conflicts:
#	c/Makefile.deepseek-v4
…itten

check-env on the merge named 46 variables read by the code and missing
from coli_env.h: the CUDA chain's and the Vulkan chain's newer switches,
the hybrid wait's bound (COLI_CUDA_HOST_WAIT_S, read by qwen36's and
qwen38's chains), the trunk-first reserve (deepseek_v41), G53_DIRECT,
and the DeepSeek V4 split's DSV4_CUDA_* and V4_* switches. Each row takes
the engines of the binaries that read it: the mask of the registered
variables read in the same file, or the one engine that includes the
file (glm53_cuchain.h, colibri_cuchain.h, deepseek_v41_cuchain.h and
deepseek_v41_chain.h). check-env: 772 variables, the registry matches
the sources.
The split's integration gave v4_cu_open (the GGUF's CUDA tier, in the
generate unit under COLI_CUDA) a line for DSV4_CUDA_DEVICES, read with
coli_v4_gpu_devices_parse, whose header only the GPU unit included: a
CUDA=1 build of deepseek-v4 (make deepseek-v4 CUDA=1, the cu_tier
objects linked) stopped with COLI_V4_GPU_MAX_DEVICES undeclared. The
header is static inline with include guards; the unit includes it too.
Every unit of deepseek_v4.c compiles without a diagnostic with COLI_CUDA,
COLI_VULKAN and both.
Served on an RTX PRO 4000, every qwen36 start printed three lines for
variables nobody set for it: NGEN and SERVE_BATCH, which the gateway
gives every engine it starts and qwen36 does not read, and
COLI_SERVE_PORT, which coli sets so that it can find a port's engine by
its environment and no engine reads. On Windows each engine started from
cmd also named PROMPT (cmd's own prompt) and TEMP (the temporary folder,
read as the deprecated temperature).

Rows the launcher sets for every engine (NGEN, SERVE, SERVE_BATCH, SNAP,
KV_SLOTS, CHAT, MAX_NEW, RAM_GB, COLI_DENSE_BITS, COLI_CUDA, COLI_VULKAN,
COLI_GPUS, CUDA_EXPERT_GB, K3_CUDA, COLI_TEMP, CLUSTER_WORKERS) and
PROMPT carry CE_LAUNCHER, which keeps the wrong-engine line for what a
user sets and drops it for those; COLI_SERVE_PORT is coli's own, not an
unknown variable; a deprecated numeric variable is named only when its
value is a number. test_env_registry checks the three cases against a
user's wrong-engine variable and a typo, built for Linux and run as a
Windows executable.
…t have

#2617 added mingw-w64-ucrt-x86_64-libgomp to the MSYS2 install lists of
check.yml, ci.yml and release.yml. That package exists from gcc 16 on;
the workflows' MSYS2 (setup-msys2 with update: false) has an older
package list, where pacman stops with "target not found:
mingw-w64-ucrt-x86_64-libgomp" and installs nothing, so every Windows job
failed. Its gcc still brings libgomp, so the lists go back to what they
were; the Makefile's warning and coli setup's check name the package on a
machine that needs it. The quickstart says what the same message means on
an MSYS2 whose package list predates gcc 16.
@JustVugg
JustVugg merged commit a2cf6c1 into dev Nov 8, 2026
232 of 234 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants