Skip to content

feat(windows): MSVC DLLs export, packages link with cl.exe, and module_extensions is followed everywhere - #453

Merged
Sunrisepeak merged 9 commits into
mainfrom
feat/windows-shared-library
Aug 18, 2026
Merged

feat(windows): MSVC DLLs export, packages link with cl.exe, and module_extensions is followed everywhere#453
Sunrisepeak merged 9 commits into
mainfrom
feat/windows-shared-library

Conversation

@Sunrisepeak

Copy link
Copy Markdown
Member

Closes the three questions left open by 2026.8.18.1, from
.agents/docs/2026-08-18-windows-shared-library-and-module-extensions.md. They
shared a shape: mcpp treated "the toolchain does not do this for us" as "this
cannot be done", and each had an industry answer older than a decade.

kind = "shared" works on the MSVC ABI

MSVC exports nothing from a DLL without __declspec(dllexport) or a .def, so
the import library came out empty and consumers failed with unresolved externals
for symbols visibly in the objects. The reason was right; the conclusion
was not. CMake has shipped WINDOWS_EXPORT_ALL_SYMBOLS since 3.4, and its
bindexplib reads COFF directly — decisive here, because dumpbin lives in a
Visual Studio developer environment and mcpp's default Windows toolchain is
clang, so a plain mcpp build is not inside one.

mcpp.build.coff_exports is that reader, a pure function over bytes so it
can be tested on hosts that cannot produce COFF at all. 16 unit tests build
objects byte by byte — the only way to vary a storage class on demand — plus two
committed real mingw objects, because a reader fed only its own test's output
agrees with itself and nothing else. Past 65535 exports it refuses rather than
truncates
: a truncated export table links cleanly and fails at whichever
consumer needed the symbol that fell off the end.

Annotation wins. An object already carrying /EXPORT: directives — what
__declspec(dllexport) emits — makes mcpp stand down. Adding a list on top would
export the same names twice (LNK4197) and export everything else besides,
replacing a chosen public surface with all of it. Detected, not configured: a key
for "I annotated my exports" is a second place to say what the objects say.

Two limits survive that no tool removes, and they are CMake's documented ones for
the same mechanism — exported data still needs dllimport on the consumer,
and a class whose vtable is referenced must be marked whole. Both are in
docs/12 rather than left to be discovered.

A package can be consumed by native cl.exe

The generated manifest now also carries the dialect-neutral
[target.<pred>.runtime] pair, which mcpp renders as /LIBPATH: + <n>.lib or
-L + -l<n>. Not new vocabulary — the same two keys [runtime] has had at top
level, made per-target. Both spellings ship: an older mcpp reads only the
ldflags and silently ignores the new block, so dropping them would leave every
older client with no link line.

⚠️ One leg is deliberately excluded, and e2e 257 is why. A PE/MinGW shared
leg links with -L… -Wl,-Bdynamic -lmathkit, and -Wl,-Bdynamic only works
immediately before the -l it enables. Two attempts to route it through the
neutral channel both failed with have you installed the static version of the mathkit library? — first by clearing the ldflags, then by rendering the library
reference into a different command-line slot. Both were the same mistake:
treating a hand-tuned link line as a two-field record.

module_extensions is the knob, and everything follows it

.ixx is not built in — the extension set is configuration, not a list mcpp
grows one entry at a time. What that owes in return is that one declaration is
enough, and three places were not holding up their end.

The scanner accepted what it could not classify. An undeclared .ixx
produced a compile edge whose object nothing links:

build obj/mathkit.ixx.o | gcm.cache/mathkit.gcm : cxx_object …
  bmi_out = gcm.cache/mathkit.gcm      ← the BMI was produced
build bin/app : cxx_link obj/main.o    ← the object is not here
ld: undefined reference to `mk::answer@mathkit()'

Two answers to "is this a module interface" and only one of them read.

The lib-root convention hard-coded .cppm, so packing an .ixx library
started its closure at a file that does not exist — and the result was silently
wrong twice over:

$ mcpp pack mathkit
     Interface (headers only)      ← the module interface, gone
      Withheld (nothing)
  Packed …-x86_64-linux-gnu        ← the C-SURFACE tag

An empty published set is precisely how the packer recognises a C surface, so
losing the interface also stopped the package constraining the C++ ABI, and
the compatibility gate stopped checking compiler and stdlib.

The generated manifest listed .ixx sources without saying what an .ixx is.
It now declares the extension — computed from the published files, so it cannot
disagree with sources, and absent for a .cppm package whose manifest is
unchanged.

Notes for review

  • ⚠️ Behaviour change: a sources entry mcpp cannot classify is now a hard
    error. A project with a .md in sources starts failing.
  • ⚠️ The probing lib-root resolver lives in mcpp.manifest.toml, not beside its
    sibling in types — measured: adding the extension-table import to types,
    which nearly everything depends on, made GCC 16.1 ICE while compiling an
    unrelated src/main.cpp
    , with a cleared gcm.cache. The edge is not added;
    the function moved to where the edge already is.
  • Local: 239 passed, 1 failed, 14 skipped. The failure is
    22_doctor_cache_publish, which the previous RELEASE binary fails identically
    on this machine.
  • Still unverified: the cl.exe consumption path end to end (needs a consuming
    case in the msvc job), and the data-symbol dllimport limit is documented but
    has no reproducing test.

…thing follows it

`.ixx` is not built in and should not be: the extension set is configuration,
not a list mcpp grows one entry at a time. What that owes in return is that ONE
declaration is enough — and three places were not holding up their end.

**The scanner accepted what it could not classify.** An `.ixx` matched by
`sources` but declared nowhere produced a compile edge whose object nothing
links. Measured:

    build obj/mathkit.ixx.o | gcm.cache/mathkit.gcm : cxx_object …
      bmi_out = gcm.cache/mathkit.gcm      ← the BMI was produced
    build bin/app : cxx_link obj/main.o    ← the object is not here
    ld: undefined reference to `mk::answer@mathkit()'

Two answers to "is this a module interface" and only one of them read: the
scanner sees `export module` and records `provides`, which is why the edge got a
`bmi_out`; the classifier says `Other`, and the link set is built from the
classifier. Refused now, where the classification happens, naming the file, the
extension and the key.

**The lib-root convention hard-coded `.cppm`**, so packing an `.ixx` library
started its closure at a file that does not exist:

    $ mcpp pack mathkit
         Interface (headers only)      ← the module interface, gone
          Withheld (nothing)
      Packed …-x86_64-linux-gnu        ← the C-SURFACE tag

Silently wrong in both directions, and the second one is worse than the first:
an empty published set is precisely how the packer recognises a C surface, so
losing the interface also downgraded the compatibility tag and the gate stopped
checking compiler and stdlib. The convention now offers one candidate per
declared extension and the packer takes the one that exists.

**The generated manifest listed `.ixx` sources without saying what an `.ixx`
is**, so every consumer was refused by the classifier. The package now declares
it — computed from the published FILES, so it cannot disagree with `sources`,
and absent entirely for a `.cppm` package, whose manifest is unchanged.

⚠️ The probing resolver lives in `mcpp.manifest.toml`, not in
`mcpp.manifest.types` where its sibling is, and the reason is measured: probing
needs the extension table, and adding that import to `types` — a module nearly
everything depends on — made GCC 16.1 ICE while compiling `src/main.cpp`, an
unrelated file, with a cleared gcm.cache. Same module-poisoning shape this
project has hit before. The edge is not added; the function moves to where the
edge already is.
MSVC exports nothing from a DLL without `__declspec(dllexport)` or a `.def`, so
mcpp refused `kind = "shared"` on that ABI. The reason was right and the
conclusion was not: CMake has shipped `WINDOWS_EXPORT_ALL_SYMBOLS` since 3.4 and
its `bindexplib` reads COFF directly — which matters here, because `dumpbin`
lives in a Visual Studio developer environment and mcpp's default Windows
toolchain is clang, so a plain `mcpp build` is not inside one.

`mcpp.build.coff_exports` is that reader, as a pure function over bytes. Its
filter follows bindexplib's semantics — external storage class, defined section,
DATA for anything that is not code, and the skip list (destructor thunks, dotted
managed names, ARM64EC bridges) each entry of which has its own reason. Past
65535 exports it refuses rather than truncates: PE addresses exports by 16-bit
ordinal, and a truncated table links cleanly and then fails at whichever consumer
needed the symbol that fell off the end.

Pure over bytes so it can be tested everywhere: 16 unit tests build synthetic
objects byte by byte, which is the only way to vary a storage class on demand,
plus a committed real mingw-cross object — a reader fed only its own test's
output agrees with itself and with nothing else, and macOS and Windows CI cannot
produce COFF at all.

In the graph it is `mcpp coff-def`, an edge whose inputs are the same objects
the link consumes, so the exported surface cannot drift from what was compiled.
A subcommand rather than a shell fragment for the reason `bmi-equal` is one: a
generated POSIX-shell command is skipped entirely on Windows, the only platform
this edge exists for. No edge is emitted for MinGW — its linker auto-exports, and
a second source of truth for a DLL's surface is how the two come to disagree.

Two limits survive that no tool can remove, and they are CMake's documented ones
for the same mechanism: a consumer still needs `__declspec(dllimport)` to read
exported DATA, and a class whose vtable is referenced must be marked whole. 258
exercises a function across the boundary and says why.
…xe can consume a package

Two things the auto-`.def` needed to be usable rather than merely present.

**An author who annotates keeps their surface.** `__declspec(dllexport)` makes
the compiler write `/EXPORT:` directives into the object's `.drectve` section.
Generating a list on top of that exports the same names twice (LNK4197) and,
worse, exports everything else as well — replacing a chosen public surface with
all of it. So mcpp reads `.drectve` and stands down. Detected rather than
configured: a manifest key for "I annotated my exports" would be a second place
to say what the objects already say, and the two could disagree. Tested against a
real annotated object, because the point is recognising what a COMPILER emits.

**A package can now be linked by native cl.exe.** The generated manifest already
carried each leg's link line as `ldflags`, which is GNU spelling — cl rejects
`-L`. It now also carries the dialect-neutral pair `[target.<pred>.runtime]`
`link_library_dirs` / `libraries`, which mcpp renders as `/LIBPATH:` + `<n>.lib`
or `-L` + `-l<n>` from the target. Not new vocabulary: the same two keys
`[runtime]` has had at the top level, made per-target.

BOTH spellings are emitted, and that is the load-bearing part. An older mcpp
reads only the ldflags and silently ignores the runtime block (measured), so
dropping the ldflags would leave every older client with no link line at all. A
newer mcpp seeing the neutral form therefore IGNORES that leg's ldflags rather
than adding to them — adding would put `-L` back on a cl command line, which is
the whole thing being avoided. Scoped to distribution packages: a hand-written
manifest that states both may well mean both.
Records the three deviations and their evidence: .ixx is NOT built in (the
extension set is configuration); pack had to follow module_extensions too, which
the design did not cover; and the export-macro layer became detection of
`.drectve` rather than a new key, because a manifest key for 'I annotated my
exports' is a second place to say what the objects already say.

Also the two defects only measurement found — a .ixx library packed to a
silently C-surface-tagged package with no interface at all, and a new module
edge into mcpp.manifest.types making GCC 16.1 ICE on an unrelated file — and an
honest list of what is still unverified.
…thing

e2e 257 caught a regression the previous commit introduced, and the reason is
worth keeping: a leg's link line is not always just a library reference.

A PE/MinGW shared leg links with `-L… -Wl,-Bdynamic -lmathkit`, and
`-Wl,-Bdynamic` only works IMMEDIATELY BEFORE the `-l` it enables — mcpp gives PE
executables `-static`, which otherwise leaves ld in static-only mode where it
refuses an import library and says `have you installed the static version of the
mathkit library?`, naming neither the DLL nor `-static`.

The first attempt cleared the leg's ldflags wholesale and lost the flag. The
second kept it but rendered the library reference through the neutral channel,
which puts it in a different slot on the command line — so the flag and its
argument were separated and the same failure came back. Both are the same
mistake: treating a hand-tuned link line as if it were a two-field record.

So the neutral form is emitted only for legs where it is EQUIVALENT, and the
PE/MinGW shared leg keeps the spelling that works. It costs nothing — a PE/GNU
leg is not an MSVC-ABI leg, and cl.exe, the reason the neutral form exists, never
reads it. On the consuming side only the library references are replaced, never
the whole list.
… using

Two things a read-through caught.

`/bigobj` objects are a different container — machine 0 and `0xFFFF` where the
section count would be — and the reader fell through to "unsupported COFF machine
0x0000". True and useless: it blames the reader for a flag the project passed.
Named now, with the two ways out (drop /bigobj, or annotate the surface and need
no generated .def at all).

`kScnMemWrite` was left over from a first version whose DATA rule keyed on
writability. It does not: a `const` in `.rdata` is a variable and consumers read
it as one, so the rule keys on "not executable" instead. The constant was
documenting a decision that is no longer taken.
Windows CI reported

    error: build failed
    FAIL: shared pack failed off ELF

and nothing else, because `mcpp pack` printed `BuildError::message` and dropped
`BuildError::diagnosticOutput` — the field that carries what the compiler
actually said. `mcpp build` has printed it all along; the two pack pipelines
were the only callers that did not.

That is a defect on its own terms, not just an inconvenience for this
investigation: a packaging failure is exactly when a maintainer has least context,
and three words is not a report. Both pipelines now print it.
Windows CI, once the packer stopped swallowing the compiler's output:

    error: unsupported option '-fPIC' for target 'x86_64-pc-windows-msvc'

every MSVC-ABI shared build died in clang-scan-deps before compiling anything.
The comment beside the condition already said the right thing — "PE code is
position independent by design" — and the condition tested the DIALECT:
`!isMsvcDialect`. Windows' default toolchain is clang, which speaks the GNU
dialect while targeting the MSVC ABI, so the flag went out anyway.

Same shape as the shared-library guard this PR replaced: asking which COMPILER
when the question is which TARGET. It keys on the target format now.

Unreachable until this PR, because `kind = "shared"` was refused on that ABI —
the refusal was hiding an untested path, which is what refusals do. 257 pins the
absence on every Linux CI pass through mingw-cross rather than only on Windows:
the flag is equally meaningless for a PE target whichever compiler emits it, and
GCC merely ignores it where clang refuses.
Windows CI again, one layer further in:

    lld-link: warning: ignoring unknown argument '--out-implib'
    lld-link: error: could not open 'bin/mathkit-shared.lib': no such file

Clang targeting the MSVC ABI speaks the GNU DIALECT while driving lld-link, so a
dialect-keyed spelling handed the MSVC linker a MinGW flag. The `.def` had the
same problem from the other side: `/DEF:` was added only to the MSVC-dialect
rule, so a clang-driven MSVC-ABI link generated the file and never passed it.

Three flags in this PR made the identical mistake — `-fPIC`, the import library,
and `/DEF:` — and it is always the same one: asking which COMPILER when the
question is which TARGET. So the spelling leaves the dialect table entirely and
becomes `pe_link_flag`, which reads the target triple and wraps in `-Wl,` unless
the linker is invoked directly. The dialect table now says, where the entry used
to be, why it cannot answer this.

Verified in both directions locally: a MinGW target still gets
`-Wl,--out-implib,` and no def edge; ELF and Mach-O are untouched.
@Sunrisepeak
Sunrisepeak merged commit 33e4b34 into main Aug 18, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants