Repository navigation
windows: DeepSeek V4 built with clang and LLVM's libomp, whose barriers spin (#2286) - #2800
Conversation
…rs spin (JustVugg#2286) MinGW's libgomp is configured "mingw32 posix": its barriers wait on a semaphore with no spin loop, so OMP_WAIT_POLICY=active and GOMP_SPINCOUNT, which the launcher sets for deepseek_v4, do nothing on Windows, and each of a token's many small parallel regions pays a thread wake-up. On an i7-7700HQ (4 threads) an OpenMP barrier costs 22 us with libgomp whatever those variables say, 0.3 us with LLVM's libomp. - c/Makefile: clang's MinGW triple (x86_64-w64-windows-gnu) is Windows too, so `make deepseek-v4 CC=clang` reaches the V4 build; the static OpenMP archive check looks for libomp.a when the compiler is clang. - c/Makefile.deepseek-v4: with CC=clang on Windows, libomp.a is linked in (-static as before, no new DLL), no -pthread (clang's driver links winpthreads; the flag only warns), LTO off unless asked for (GNU ld has no LLVM plugin in MSYS2's clang package). The gcc line is unchanged. - c/deepseek_v4.c: drop v4_moe_bank_hash_layer, written and never read (clang's -Wunused-but-set-global), so the clang build has 0 warnings. - c/coli, c/openai_server.py: KMP_BLOCKTIME=200 beside GOMP_SPINCOUNT for deepseek_v4. With OMP_WAIT_POLICY=active alone libomp's team spins forever: a served clang engine burned 3 cores while idle (JustVugg#341). - release.yml, ci.yml: the Windows deepseek_v4.exe is built with CC=clang; the OpenMP package is installed under either of its MSYS2 names; the release checks the packaged engine is a libomp build importing no MinGW DLL. - docs and CHANGELOG. Tiny V4 fixture on the i7-7700HQ, 7 interleaved rounds, same tokens in every run: decode 500 -> 1547 tok/s, a 72-token prompt 67 -> 20 ms. test_deepseek_v4_tiny/_prefix/_brio pass with both builds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
JustVugg#2286) The launcher's KMP_BLOCKTIME=200 default for deepseek_v4 is now set on Windows only, and only when the resulting OMP_WAIT_POLICY is active. libomp gives passive its blocktime of 0 only while KMP_BLOCKTIME is unset, so the default turned a user's passive into 200 ms of spinning after every region. macOS's V4 also links libomp (Homebrew), where the change was not measured (JustVugg#707), and Linux builds V4 with libgomp. coli's GLM path, which sets the same active policy on Windows, gets the same line: a clang colibri.exe now builds with the Windows flags. ci.yml's Windows job builds the default gcc deepseek-v4 again, and the release's XDNA=1 VK=1 CC=clang one after it, with the Vulkan headers and shaderc installed, so the clang compile of the Vulkan units and the LTO-less link are built on every PR. The Makefile's missing libomp.a warning points to docs/deepseek-v4.md. Docs and CHANGELOG: same tokens on the tiny fixture's oracle, the issue's 1.5x came from an LTO build, the Windows CUDA tier now spins (not measured). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, the analysis in #2286 and here is convincing, and the clang build is the right direction. We took it into batch #2805, and CI's "All engines (windows)" job failed at the link of the clang build: The runner installed So #2800 is out of the batch for now, and the other seven PRs went ahead. Could you make the static link work with what MSYS2 ships today (22.1.7), or check for that version up front, and run the Windows job on your branch? Once it passes we will take it in the next batch. |
…2286) libomp.a defines omp_get_num_devices() only as a COFF weak symbol, and GNU ld leaves it unresolved with MSYS2's llvm-openmp 22.1.7 or binutils 2.46, which is what GitHub's windows runner installs: `make deepseek-v4 CC=clang` failed at the link in JustVugg#2805's CI, on libomp's own kmp_alloc and kmp_ftn_extra. Only llvm-openmp 23.x with binutils 2.47 links it. Makefile.deepseek-v4 now links one call to it the same way; only when that fails on this symbol does backend_loader_dsv4.c define it, returning 0, what libomp returns when no offload runtime is linked. Where it links, nothing is added, so there is no second definition. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for running it. The symbol isn't really missing: libomp.a defines The fix (000475a) is a link probe in I ran the Windows job on a fork branch with the same setup-msys2 settings and packages, in two configurations:
In both, Not checked there: upstream's full "All engines" loop; only the V4 builds and the tests ran. Side note: if ci.yml or release.yml ever switch to |
Base: dev @ b92eccd · Depends on: none
Summary
#2286 measured the shipped Windows
deepseek_v4.exe(MSYS2 gcc, libgomp) decoding DeepSeek V4 Flash 1.5x slowerthan a clang + libomp build of the same source, with the same output. The reporter's diagnosis holds: MinGW's
libgomp never spins at a barrier, so the
OMP_WAIT_POLICY=activeandGOMP_SPINCOUNT=200000that the launchersets for this engine do nothing on Windows. This PR makes the clang build a supported, tested build
(
make deepseek-v4 CC=clang), builds the release's Windowsdeepseek_v4.exethat way (libomp linked instatically, no new DLL), and on Windows adds
KMP_BLOCKTIME=200to the launcher's defaults when the wait policyis active, so the libomp team sleeps when the engine is idle instead of spinning forever.
Root cause
libgomp on MinGW is configured with
config_path="mingw32 posix". Its barriers areconfig/posix/bar.c: they waiton a semaphore with no spin loop. Only the Linux port (
config/linux/wait.h) spinsGOMP_SPINCOUNTtimes first.A DeepSeek V4 decode step runs hundreds of small parallel regions, and with libgomp each one ends in a semaphore
wake-up of every worker.
Microbenchmark (a scratch C program, not part of the PR):
#pragma omp barrierin a loop inside one parallelregion, and a parallel
forover a 1024 x 512 f32 matrix-vector product (the shape of a decode region), 20000iterations each. i7-7700HQ (4 cores, 8 threads), Windows 11, MSYS2 UCRT64 gcc 16.2.0 and clang 23.1.5 with LLVM
OpenMP 23.1.5, both
-O2 -march=x86-64-v3 -fopenmp -static,OMP_DYNAMIC=FALSE, 3 interleaved rounds (eachvalue below is the range of the 3):
OMP_WAIT_POLICY=active GOMP_SPINCOUNT=200000GOMP_SPINCOUNT=infiniteOMP_WAIT_POLICY=active GOMP_SPINCOUNT=200000(the launcher today)KMP_BLOCKTIME=200(this PR)OMP_WAIT_POLICY=passiveThe libgomp rows do not move with any setting, and libgomp never burns CPU while idle: it never spins. libomp's
barrier is 70x cheaper. The last column is why
KMP_BLOCKTIMEis part of this PR: withOMP_WAIT_POLICY=activeand no
KMP_BLOCKTIME, libomp's blocktime is infinite and the three idle workers spin at 100% (#341). With 200 msthey spin for 200 ms after the last region (0.6 s of the 1 s) and then sleep.
A second round with the same program (clang / libomp, 4 threads,
OMP_DYNAMIC=FALSE, 3 interleaved rounds,ranges of the 3) checked the
passivecase, which this PR leaves withoutKMP_BLOCKTIME:OMP_WAIT_POLICY=passiveOMP_WAIT_POLICY=passive KMP_BLOCKTIME=200OMP_WAIT_POLICY=active GOMP_SPINCOUNT=200000 KMP_BLOCKTIME=200(this PR, Windows)OMP_WAIT_POLICY=active GOMP_SPINCOUNT=200000(the launcher today)With
KMP_BLOCKTIME=200,passivebehaves likeactive: the team spins for 200 ms after each region and thensleeps. That is why the launcher adds it only when the policy is active.
A pure-gcc fix was considered and not taken. libgomp's barrier cannot be made to spin from outside, and LLVM's
libomp.afor MinGW has no GNU ABI entry points (noGOMP_*symbols), so gcc-compiled code cannot be linkedagainst libomp either. What is left is replacing the engine's ~46
#pragma ompregions with a hand-written threadpool and barrier, which is a much larger change than building with the compiler whose runtime already spins.
What changes
c/Makefilex86_64-w64-windows-gnu(MSYS2's clang and llvm-mingw), gccx86_64-w64-mingw32. Only the second was recognised, so withCC=clangthe Makefile treated Windows asan unknown platform:
make deepseek-v4 CC=clanganswered "supported only on x86-64/aarch64 Linux andWindows/MSYS2".
MINGWnow matches both.libomp.aand namesmingw-w64-ucrt-x86_64-llvm-openmpand
docs/deepseek-v4.mdwhen the compiler is clang, instead of warning thatlibgomp.ais missing. gcc'smessage is unchanged.
c/Makefile.deepseek-v4, Windows block, only whenCCis clang:LDFLAGS = -lm -fopenmp -static: libomp is linked in from MSYS2'slibomp.a, so the engine still importsonly system DLLs. No
-pthread: clang's MinGW driver adds-lpthreaditself, and the flag only drew"argument unused during compilation".
LTO ?= 0: with GNU ld, clang's LTO objects need LLVM's linker plugin (LLVMgold.dll), which MSYS2's clangpackage does not install; the default build stopped at the link with "error loading plugin".
LTO=1stillworks for anyone who has the plugin or lld.
c/deepseek_v4.c:v4_moe_bank_hash_layerwas written in two places and never read, and clang 23 warns aboutit (
-Wunused-but-set-global). Removed with its#define/#undef. With this,make deepseek-v4 CC=clanghas 0 warnings(the release's
VK=1adds 3 inbackend_vulkan.c, which this PR does not touch; see Correctness).c/coli(env_for_engine) andc/openai_server.py(tune_child_env):KMP_BLOCKTIME=200besideGOMP_SPINCOUNT=200000fordeepseek_v4,setdefaultlike its neighbours, inside the sameCOLI_NO_OMP_TUNEguard, on Windows only and only when the resulting
OMP_WAIT_POLICYisactive(any case). libgomp ignoresit; it is the value
omp_tune.handcolibri.calready set for libomp builds on Linux and FreeBSD.passive: libomp givespassiveits blocktime of 0 only whileKMP_BLOCKTIMEis unset, so alauncher default would turn a user's
OMP_WAIT_POLICY=passiveinto 200 ms of spin-yield after every region(measured above: 0.6 s of CPU in the idle second instead of 0).
blocktime has not been measured ([Bug]: OpenMP spin-wait tuning is inert on macOS — and applying it costs 2.2x decode on a 32 GB host #707 measured both knobs slower on Apple Silicon). Linux builds V4 with gcc
and libgomp; a Linux clang build keeps what it had.
c/coli'senv_for(the GLM path), which sets the same active policy on Windows, gets the same line: withthis PR
make colibri CC=clangbuilds with the Windows flags, and such a build would otherwise spin all itsworkers forever while idle.
.github/workflows/release.yml(Windows row):mingw-w64-ucrt-x86_64-clang, and LLVM's OpenMP package in a step of its own: MSYS2 renamed itfrom
openmptollvm-openmp, and the runner's package list (update: false) may predate the rename. A namepacman does not know makes the whole install fail (6b09123), so the step asks
pacman -Siwhich name thelist has.
make deepseek-v4 XDNA=1 VK=1 CC=clang(new matrix valuev4_args); the other engines stay gcc.deepseek_v4.exeis not a libomp build (noKMP_BLOCKTIMEstring init) or imports any
lib*.dllthe archive does not carry..github/workflows/ci.yml(engines-all-platforms, Windows row): the loop still builds the default gccdeepseek-v4(the buildmake deepseek-v4and source installs produce), and after it the job builds therelease's
make deepseek-v4 XDNA=1 CC=clang VK=1(new matrix valuev4_args; the flags stamp rebuilds theunits). It installs what that needs: clang, the OpenMP package (same step as release.yml),
vulkan-headersandshaderc. So every PR compilesbackend_vulkan.c,vk_tier.candvk_chain.cwithclang and links V4 the release's way (clang,
VK=1, no LTO); before, no job did.c/tests/test_makefile_platform.py:test_clang_mingw_triple_builds_for_windows(the windows-gnu triplegives
colibri.exeand the-static -lpsapilink) andtest_deepseek_v4_windows_link_per_compiler(dry runof
Makefile.deepseek-v4withOS=Windows_NT: clang's link line ends-lm -fopenmp -static, gcc's stillends
-static -pthread -flto). Both run undermake check(test-python).c/tests/test_cli_output.py:test_v4_libomp_team_sleeps_when_idle_on_windows,test_v4_blocktime_only_for_an_active_team_on_windows(an explicitPassive, macOS and Linux get noKMP_BLOCKTIME);KMP_BLOCKTIMEadded to the kill-switch test.c/tests/test_openai_server.py:test_direct_v4_server_libomp_team_sleeps_when_idle(Windows default, anexplicit value wins,
PASSIVEand macOS get none);KMP_BLOCKTIMEadded to the kill-switch test.c/tests/test_env_defaults.py(the GLM path): the Windows default includesKMP_BLOCKTIME=200,test_passive_keeps_libomp_blocktime_zero, and the kill-switch and non-Windows tests also checkKMP_BLOCKTIME.docs/deepseek-v4.md(Build: the clang build, why, the two packages, the numbers),docs/tuning.md(MinGW'slibgomp ignores the wait policy and the spin count),
CHANGELOG.md(Unreleased / Performance).New env flags: none. Defaults: the gcc build is unchanged; on Windows, when the wait policy is active, the
launcher adds
KMP_BLOCKTIME=200(deepseek_v4incoliandopenai_server.py, andcoli's GLM path),which libgomp ignores. Linux and macOS: unchanged.
Correctness
All on the same i7-7700HQ laptop, Windows 11, MSYS2 UCRT64 (gcc 16.2.0, clang 23.1.5, LLVM OpenMP 23.1.5),
Python 3.14, with the generated tiny V4 fixture (
tools/make_deepseek_v4_tiny.py, generator version 2, theref.jsoncommitted on dev).tests/test_deepseek_v4_tiny.py,tests/test_deepseek_v4_prefix.pyandtests/test_deepseek_v4_brio.py(
--binary <exe> --fixture <copy>) pass with both the gcc build and the clang build of this branch: 12, 7 and 5PASS lines, "all checks completed". That covers teacher forcing and greedy against the oracle, sessions, the
served engine, KV prefix reuse and the numeric channel.
round of it), for every build and every OpenMP setting.
tests/test_deepseek_v4(the C unit test) passes with the gcc build. With clang it does not link, and that isnot new: on Windows this test only links with LTO, which drops the GPU unit's references to units the test
does not link. gcc with
LTO=0fails the same way (25 undefined references tocoli_v4_layer_gpuandfriends).
tests.test_makefile_platform7 tests OK; on dev the 2 new ones fail (the clang triple is notWindows; clang's link line has
-pthread -flto).tests.test_cli_output -k v4 -k kill_switch -k explicit_setting6 OK,tests.test_openai_server -k direct_v44 OK; on dev the new test in each fails(no
KMP_BLOCKTIME).tests.test_env_defaultsOK.make deepseek-v4 CC=clang0 warnings (29 s);make deepseek-v4(gcc) prints the same 11-Wcast-function-typewarnings inbackend_loader_dsv4.cand thelto-wrappernote as dev (gcc 16.2). Theclang
deepseek_v4.exeimportsKERNEL32.dlland theapi-ms-win-crt-*sets only, like the gcc one.git archiveof the branch:tests.test_env_defaults32 tests OK,tests.test_makefile_platform7 OK,tests.test_family_registry63OK,
tests.test_cli_output tests.test_openai_server523 OK (1 skipped). The 9 tests selected by-k blocktime -k passive -k libomp -k kill_switch -k non_windows -k measured_defaults(the launcher's Windows default, anexplicit
Passive, macOS and Linux, the kill switches) all pass.fake-clanggets "fake-clang hasno libomp.a, so the -static -fopenmp link will fail with "cannot find -lomp". ... pacman -S --needed
mingw-w64-ucrt-x86_64-llvm-openmp (docs/deepseek-v4.md)",
fake-gccstill getslibgomp.a,mingw-w64-ucrt-x86_64-libgompanddocs/windows.md. None with the real clang or gcc.make deepseek-v4 XDNA=1 VK=1 CC=clang(LTO off; Vulkan-Headers 1.4.368 throughCPATH;GLSLC=true, so the committed.spvfiles are used), from a clean tree: builds and links in 39 s.3 warnings, all in
backend_vulkan.c, which this PR does not touch:-Wc23-extensions(a label followed bya declaration, line 5724) and
-Wunused-but-set-globalforg_vk_gemm_callsandg_vk_coop_calls. Thebinary imports
KERNEL32.dlland theapi-ms-win-crt-*sets only, and passes both checksrelease.ymladds(a
KMP_BLOCKTIMEstring, nolib*.dllimport).test_deepseek_v4_tiny,_prefixand_briopass with iton the CPU path: 12, 7 and 5 PASS lines, "all checks completed".
ci.yml's order in one tree:make deepseek-v4 XDNA=1(gcc, 87 s, 12warning:lines: the 11 and the noteabove), then
make deepseek-v4 XDNA=1 CC=clang VK=1: "build flags changed, rebuilding the units", 32 objectsrecompiled by clang, linked in 34 s, the same 3 warnings, and that binary passes
test_deepseek_v4_tiny(12PASS lines).
make -C c check(gcc, the default build), same laptop:portableandtest-cpass.test-pythonran 3473tests with 11 errors, and the same 11 fail on dev on this machine: 6 read a UTF-8 file with Python's Windows
default encoding (
UnicodeDecodeError: 'charmap' codec:test_v4_shared_expert_bindingsreadingdeepseek_v4.c,test_glm53_cap_launch_source,test_glm53_metal_clamped_source,test_openai_tools_v41_e2ex2,
test_qwenimage_gguf_import), and 5 intest_qwen38_int2_convert/test_qwen38_int4_convertcannot delete atemporary file Windows still holds open (
WinError 32). The new and changed tests are not among them.Performance
The tiny fixture, i7-7700HQ, engine team of 4 (the engine's own sizing: 4 physical cores of 8), every run from a
fresh copy of the fixture (so
.coli_usageis the committed one), 7 interleaved rounds per configuration. Decodetok/s = (generated - 1) /
after_first; the engine printsafter_firstin ms, so the clang values arequantized (99/0.064 = 1547, 99/0.063 = 1571). Median [min-max]:
LTO=0KMP_BLOCKTIMEOMP_WAIT_POLICY=passive"The launcher's settings" are
OMP_WAIT_POLICY=active GOMP_SPINCOUNT=200000 OMP_DYNAMIC=FALSE, plusKMP_BLOCKTIME=200for "new". The tiny model's matrices are small, so OpenMP's per-region cost is most of a tokenhere and the 3.1x overstates what a real checkpoint gains. The issue measured 1.5x on DeepSeek V4 Flash (Ryzen AI
Max+ 392, 12 threads, llvm-mingw clang 23 with libomp.dll) with an LTO build (
-flto, llvm-mingw links withlld). This PR's clang build has no LTO; that LTO makes no difference is shown only on the tiny fixture, where
OpenMP's overhead dominates.
KMP_BLOCKTIME=200costs nothing measurable in decode (rows 4 and 5 overlap).Idle served engine:
openai_server.Enginestarted on the fixture (so withtune_child_env's settings), one32-token request, then 5 s with no request, CPU time of the engine process over those 5 s, 2 rounds:
KMP_BLOCKTIME)KMP_BLOCKTIME=200)So the launcher change is required for the clang build: without it a
coli servewith this engine keeps all butone of its compute threads at 100% between requests.
No experiment manifest is attached: there is no real DeepSeek V4 checkpoint on the test machine. The full-model
figure is the issue's.
Validation (upstream template)
make -C c check(Windows, MSYS2 gcc):test-cpasses;test-pythonhas 11 errors, all of which failthe same way on dev on this machine (see Correctness). Not run on Linux or macOS.
make -C c cuda-test/dsv4-cuda-test: not run. No CUDA code changes; the clang engine loadscoli_cuda_dsv4*.dllthrough the samebackend_loader_dsv4.c(compiled by clang, 0 warnings), but it was notrun with a GPU (see Not tested).
and microbenchmark only, above).
Compatibility and risks
KMP_BLOCKTIMEdefault is Windows only, so macOS's V4 (Homebrew libomp, activepolicy from the launcher) keeps its infinite blocktime; whether 200 ms would help it there is not measured.
KMP_BLOCKTIME=200with theactive policy, which libgomp ignores. A user's own
KMP_BLOCKTIMEstill wins,OMP_WAIT_POLICY=passivegets no
KMP_BLOCKTIME(so libomp's blocktime 0), andCOLI_NO_OMP_TUNEstill removes all of them.deepseek_v4.exechanges compiler and OpenMP runtime. Same tokens on the tiny fixture'soracle, which is all that was checked: gcc contracts floating-point expressions into FMAs more freely than
clang by default (
-ffp-contract=fastagainston) and the clang build has no LTO, so the rounding differsand a real checkpoint can pick another token where two are nearly tied. The archive check fails the release if
the binary is not a libomp build or imports a MinGW DLL.
was the same speed with and without LTO (table above), and clang without LTO is 3x faster than gcc with it, but
only on the tiny fixture; the issue's 1.5x was an LTO clang build.
c/Makefilenow treats any*-windows-gnucompiler as MinGW. Before, such a compiler fell through everyplatform block; now
make <engine> CC=clangbuilds with the Windows flags. Onlydeepseek-v4was built andrun with clang for this PR (
coli's GLM path gets the sameKMP_BLOCKTIMEdefault, so a clangcolibri.exewould not spin forever when idle, but none was built).
slept. This is a behaviour change that was not measured: there is no GPU run here.
docs/tuning.mdrecordssevere regressions from active teams contending with CUDA dispatch (GLM), and
colibri.candomp_tune.hskip active waiting when a GPU is on. The precedent for V4 is Linux, where the launcher's active policy already
makes the team spin (bounded by
GOMP_SPINCOUNT) with the CUDA tier on.disk). This PR does not change the policy: the launcher already asked for active waiting on Windows, and only
now does the runtime honour it.
OMP_WAIT_POLICY=passiveremains the per-host override, also through thelauncher, and
COLI_NO_OMP_TUNE=1drops all of the launcher's OpenMP settings.Not tested:
where the launcher set
KMP_BLOCKTIME=200for V4 on every host and policy; on Windows with the active policy,the case measured there, this version sets the same environment, and the passive rows were measured without
KMP_BLOCKTIME, which is what it now leaves passive. The second round ran the unit tests, the builds and theompbar rounds above.
VK=1, no LTO) was built, linked and run on thetiny fixture's CPU path (Correctness), but no Vulkan device was used, and the shaders were the committed
.spvfiles (GLSLC=true): the runner'sglslcbuild of them was not run. The gate is this PR's own Releaserun (
release.ymlruns on pull requests that change it) and, from now on,ci.yml's Windows job: this PRshould not be merged before both are green.
ci.yml,release.yml) were not run, so neither was the OpenMP package step on therunner. Its fallback should rarely matter: MSYS2 renamed
openmptollvm-openmpwith LLVM 20.1.0 (March2025), and
llvm-openmpprovides and replacesopenmp, while the libgomp split that broke the runners'older package list (6b09123) came with gcc 16.2.0-4 in September 2026. Either package links statically: the
last
openmpon the MSYS2 mirror (18.1.2) and the firstllvm-openmp(20.1.0) both ship alibomp.aof 32kmp_*.cpp.objobjects that defines__kmpc_fork_call, beside thelibomp.dll.aimport library, as 23.1.5does.
coli_cuda_dsv4*.dllloaded at run time). The loader is plainLoadLibrary/GetProcAddressC and compiles warning-free with clang, but no GPU run was made.x86-64-v3, so the AVX-512 kernels never run on Windows) isnot addressed here.
Refs #2286
🤖 Generated with Claude Code