feat(windows): MSVC DLLs export, packages link with cl.exe, and module_extensions is followed everywhere - #453
Merged
Merged
Conversation
…thing follows it
`.ixx` is not built in and should not be: the extension set is configuration,
not a list mcpp grows one entry at a time. What that owes in return is that ONE
declaration is enough — and three places were not holding up their end.
**The scanner accepted what it could not classify.** An `.ixx` matched by
`sources` but declared nowhere produced a compile edge whose object nothing
links. Measured:
build obj/mathkit.ixx.o | gcm.cache/mathkit.gcm : cxx_object …
bmi_out = gcm.cache/mathkit.gcm ← the BMI was produced
build bin/app : cxx_link obj/main.o ← the object is not here
ld: undefined reference to `mk::answer@mathkit()'
Two answers to "is this a module interface" and only one of them read: the
scanner sees `export module` and records `provides`, which is why the edge got a
`bmi_out`; the classifier says `Other`, and the link set is built from the
classifier. Refused now, where the classification happens, naming the file, the
extension and the key.
**The lib-root convention hard-coded `.cppm`**, so packing an `.ixx` library
started its closure at a file that does not exist:
$ mcpp pack mathkit
Interface (headers only) ← the module interface, gone
Withheld (nothing)
Packed …-x86_64-linux-gnu ← the C-SURFACE tag
Silently wrong in both directions, and the second one is worse than the first:
an empty published set is precisely how the packer recognises a C surface, so
losing the interface also downgraded the compatibility tag and the gate stopped
checking compiler and stdlib. The convention now offers one candidate per
declared extension and the packer takes the one that exists.
**The generated manifest listed `.ixx` sources without saying what an `.ixx`
is**, so every consumer was refused by the classifier. The package now declares
it — computed from the published FILES, so it cannot disagree with `sources`,
and absent entirely for a `.cppm` package, whose manifest is unchanged.
⚠️ The probing resolver lives in `mcpp.manifest.toml`, not in
`mcpp.manifest.types` where its sibling is, and the reason is measured: probing
needs the extension table, and adding that import to `types` — a module nearly
everything depends on — made GCC 16.1 ICE while compiling `src/main.cpp`, an
unrelated file, with a cleared gcm.cache. Same module-poisoning shape this
project has hit before. The edge is not added; the function moves to where the
edge already is.
MSVC exports nothing from a DLL without `__declspec(dllexport)` or a `.def`, so mcpp refused `kind = "shared"` on that ABI. The reason was right and the conclusion was not: CMake has shipped `WINDOWS_EXPORT_ALL_SYMBOLS` since 3.4 and its `bindexplib` reads COFF directly — which matters here, because `dumpbin` lives in a Visual Studio developer environment and mcpp's default Windows toolchain is clang, so a plain `mcpp build` is not inside one. `mcpp.build.coff_exports` is that reader, as a pure function over bytes. Its filter follows bindexplib's semantics — external storage class, defined section, DATA for anything that is not code, and the skip list (destructor thunks, dotted managed names, ARM64EC bridges) each entry of which has its own reason. Past 65535 exports it refuses rather than truncates: PE addresses exports by 16-bit ordinal, and a truncated table links cleanly and then fails at whichever consumer needed the symbol that fell off the end. Pure over bytes so it can be tested everywhere: 16 unit tests build synthetic objects byte by byte, which is the only way to vary a storage class on demand, plus a committed real mingw-cross object — a reader fed only its own test's output agrees with itself and with nothing else, and macOS and Windows CI cannot produce COFF at all. In the graph it is `mcpp coff-def`, an edge whose inputs are the same objects the link consumes, so the exported surface cannot drift from what was compiled. A subcommand rather than a shell fragment for the reason `bmi-equal` is one: a generated POSIX-shell command is skipped entirely on Windows, the only platform this edge exists for. No edge is emitted for MinGW — its linker auto-exports, and a second source of truth for a DLL's surface is how the two come to disagree. Two limits survive that no tool can remove, and they are CMake's documented ones for the same mechanism: a consumer still needs `__declspec(dllimport)` to read exported DATA, and a class whose vtable is referenced must be marked whole. 258 exercises a function across the boundary and says why.
…xe can consume a package Two things the auto-`.def` needed to be usable rather than merely present. **An author who annotates keeps their surface.** `__declspec(dllexport)` makes the compiler write `/EXPORT:` directives into the object's `.drectve` section. Generating a list on top of that exports the same names twice (LNK4197) and, worse, exports everything else as well — replacing a chosen public surface with all of it. So mcpp reads `.drectve` and stands down. Detected rather than configured: a manifest key for "I annotated my exports" would be a second place to say what the objects already say, and the two could disagree. Tested against a real annotated object, because the point is recognising what a COMPILER emits. **A package can now be linked by native cl.exe.** The generated manifest already carried each leg's link line as `ldflags`, which is GNU spelling — cl rejects `-L`. It now also carries the dialect-neutral pair `[target.<pred>.runtime]` `link_library_dirs` / `libraries`, which mcpp renders as `/LIBPATH:` + `<n>.lib` or `-L` + `-l<n>` from the target. Not new vocabulary: the same two keys `[runtime]` has had at the top level, made per-target. BOTH spellings are emitted, and that is the load-bearing part. An older mcpp reads only the ldflags and silently ignores the runtime block (measured), so dropping the ldflags would leave every older client with no link line at all. A newer mcpp seeing the neutral form therefore IGNORES that leg's ldflags rather than adding to them — adding would put `-L` back on a cl command line, which is the whole thing being avoided. Scoped to distribution packages: a hand-written manifest that states both may well mean both.
Records the three deviations and their evidence: .ixx is NOT built in (the extension set is configuration); pack had to follow module_extensions too, which the design did not cover; and the export-macro layer became detection of `.drectve` rather than a new key, because a manifest key for 'I annotated my exports' is a second place to say what the objects already say. Also the two defects only measurement found — a .ixx library packed to a silently C-surface-tagged package with no interface at all, and a new module edge into mcpp.manifest.types making GCC 16.1 ICE on an unrelated file — and an honest list of what is still unverified.
…thing e2e 257 caught a regression the previous commit introduced, and the reason is worth keeping: a leg's link line is not always just a library reference. A PE/MinGW shared leg links with `-L… -Wl,-Bdynamic -lmathkit`, and `-Wl,-Bdynamic` only works IMMEDIATELY BEFORE the `-l` it enables — mcpp gives PE executables `-static`, which otherwise leaves ld in static-only mode where it refuses an import library and says `have you installed the static version of the mathkit library?`, naming neither the DLL nor `-static`. The first attempt cleared the leg's ldflags wholesale and lost the flag. The second kept it but rendered the library reference through the neutral channel, which puts it in a different slot on the command line — so the flag and its argument were separated and the same failure came back. Both are the same mistake: treating a hand-tuned link line as if it were a two-field record. So the neutral form is emitted only for legs where it is EQUIVALENT, and the PE/MinGW shared leg keeps the spelling that works. It costs nothing — a PE/GNU leg is not an MSVC-ABI leg, and cl.exe, the reason the neutral form exists, never reads it. On the consuming side only the library references are replaced, never the whole list.
… using Two things a read-through caught. `/bigobj` objects are a different container — machine 0 and `0xFFFF` where the section count would be — and the reader fell through to "unsupported COFF machine 0x0000". True and useless: it blames the reader for a flag the project passed. Named now, with the two ways out (drop /bigobj, or annotate the surface and need no generated .def at all). `kScnMemWrite` was left over from a first version whose DATA rule keyed on writability. It does not: a `const` in `.rdata` is a variable and consumers read it as one, so the rule keys on "not executable" instead. The constant was documenting a decision that is no longer taken.
Windows CI reported
error: build failed
FAIL: shared pack failed off ELF
and nothing else, because `mcpp pack` printed `BuildError::message` and dropped
`BuildError::diagnosticOutput` — the field that carries what the compiler
actually said. `mcpp build` has printed it all along; the two pack pipelines
were the only callers that did not.
That is a defect on its own terms, not just an inconvenience for this
investigation: a packaging failure is exactly when a maintainer has least context,
and three words is not a report. Both pipelines now print it.
Windows CI, once the packer stopped swallowing the compiler's output:
error: unsupported option '-fPIC' for target 'x86_64-pc-windows-msvc'
every MSVC-ABI shared build died in clang-scan-deps before compiling anything.
The comment beside the condition already said the right thing — "PE code is
position independent by design" — and the condition tested the DIALECT:
`!isMsvcDialect`. Windows' default toolchain is clang, which speaks the GNU
dialect while targeting the MSVC ABI, so the flag went out anyway.
Same shape as the shared-library guard this PR replaced: asking which COMPILER
when the question is which TARGET. It keys on the target format now.
Unreachable until this PR, because `kind = "shared"` was refused on that ABI —
the refusal was hiding an untested path, which is what refusals do. 257 pins the
absence on every Linux CI pass through mingw-cross rather than only on Windows:
the flag is equally meaningless for a PE target whichever compiler emits it, and
GCC merely ignores it where clang refuses.
Windows CI again, one layer further in:
lld-link: warning: ignoring unknown argument '--out-implib'
lld-link: error: could not open 'bin/mathkit-shared.lib': no such file
Clang targeting the MSVC ABI speaks the GNU DIALECT while driving lld-link, so a
dialect-keyed spelling handed the MSVC linker a MinGW flag. The `.def` had the
same problem from the other side: `/DEF:` was added only to the MSVC-dialect
rule, so a clang-driven MSVC-ABI link generated the file and never passed it.
Three flags in this PR made the identical mistake — `-fPIC`, the import library,
and `/DEF:` — and it is always the same one: asking which COMPILER when the
question is which TARGET. So the spelling leaves the dialect table entirely and
becomes `pe_link_flag`, which reads the target triple and wraps in `-Wl,` unless
the linker is invoked directly. The dialect table now says, where the entry used
to be, why it cannot answer this.
Verified in both directions locally: a MinGW target still gets
`-Wl,--out-implib,` and no def edge; ELF and Mach-O are untouched.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the three questions left open by 2026.8.18.1, from
.agents/docs/2026-08-18-windows-shared-library-and-module-extensions.md. Theyshared a shape: mcpp treated "the toolchain does not do this for us" as "this
cannot be done", and each had an industry answer older than a decade.
kind = "shared"works on the MSVC ABIMSVC exports nothing from a DLL without
__declspec(dllexport)or a.def, sothe import library came out empty and consumers failed with unresolved externals
for symbols visibly in the objects. The reason was right; the conclusion
was not. CMake has shipped
WINDOWS_EXPORT_ALL_SYMBOLSsince 3.4, and itsbindexplibreads COFF directly — decisive here, becausedumpbinlives in aVisual Studio developer environment and mcpp's default Windows toolchain is
clang, so a plain
mcpp buildis not inside one.mcpp.build.coff_exportsis that reader, a pure function over bytes so itcan be tested on hosts that cannot produce COFF at all. 16 unit tests build
objects byte by byte — the only way to vary a storage class on demand — plus two
committed real mingw objects, because a reader fed only its own test's output
agrees with itself and nothing else. Past 65535 exports it refuses rather than
truncates: a truncated export table links cleanly and fails at whichever
consumer needed the symbol that fell off the end.
Annotation wins. An object already carrying
/EXPORT:directives — what__declspec(dllexport)emits — makes mcpp stand down. Adding a list on top wouldexport the same names twice (
LNK4197) and export everything else besides,replacing a chosen public surface with all of it. Detected, not configured: a key
for "I annotated my exports" is a second place to say what the objects say.
Two limits survive that no tool removes, and they are CMake's documented ones for
the same mechanism — exported data still needs
dllimporton the consumer,and a class whose vtable is referenced must be marked whole. Both are in
docs/12 rather than left to be discovered.
A package can be consumed by native
cl.exeThe generated manifest now also carries the dialect-neutral
[target.<pred>.runtime]pair, which mcpp renders as/LIBPATH:+<n>.libor-L+-l<n>. Not new vocabulary — the same two keys[runtime]has had at toplevel, made per-target. Both spellings ship: an older mcpp reads only the
ldflagsand silently ignores the new block, so dropping them would leave everyolder client with no link line.
leg links with
-L… -Wl,-Bdynamic -lmathkit, and-Wl,-Bdynamiconly worksimmediately before the
-lit enables. Two attempts to route it through theneutral channel both failed with
have you installed the static version of the mathkit library?— first by clearing the ldflags, then by rendering the libraryreference into a different command-line slot. Both were the same mistake:
treating a hand-tuned link line as a two-field record.
module_extensionsis the knob, and everything follows it.ixxis not built in — the extension set is configuration, not a list mcppgrows one entry at a time. What that owes in return is that one declaration is
enough, and three places were not holding up their end.
The scanner accepted what it could not classify. An undeclared
.ixxproduced a compile edge whose object nothing links:
Two answers to "is this a module interface" and only one of them read.
The lib-root convention hard-coded
.cppm, so packing an.ixxlibrarystarted its closure at a file that does not exist — and the result was silently
wrong twice over:
An empty published set is precisely how the packer recognises a C surface, so
losing the interface also stopped the package constraining the C++ ABI, and
the compatibility gate stopped checking compiler and stdlib.
The generated manifest listed
.ixxsources without saying what an.ixxis.It now declares the extension — computed from the published files, so it cannot
disagree with
sources, and absent for a.cppmpackage whose manifest isunchanged.
Notes for review
sourcesentry mcpp cannot classify is now a harderror. A project with a
.mdinsourcesstarts failing.mcpp.manifest.toml, not beside itssibling in
types— measured: adding the extension-table import totypes,which nearly everything depends on, made GCC 16.1 ICE while compiling an
unrelated
src/main.cpp, with a cleared gcm.cache. The edge is not added;the function moved to where the edge already is.
22_doctor_cache_publish, which the previous RELEASE binary fails identicallyon this machine.
cl.execonsumption path end to end (needs a consumingcase in the msvc job), and the data-symbol
dllimportlimit is documented buthas no reproducing test.