Menu

Guides

Reference

Internals

Manticore module system & distribution — design

Status: PARTLY SHIPPED. The .sig module interface and manticore build manticore.json (library + application targets, cross-target linking) are in the tree — user guide: docs/modules.md. Still unbuilt, and absent from src/ entirely: transitive dependency resolution, the global build cache (MANTICORE_HOME, ~/.manticore/cache), the compiler_abi field, and the composer packaging bootstrap.

⚠ The .sig example below is still partly aspirational. The emitter (Manticore\Sig) writes {"schema":2,"abi":N,"functions":[…],"classes":[…], "constants":[…],"libs":[…],"weak":[…]} — so classes and constants are real now (see docs/modules.md), but package, version and target are not, and the ABI is pinned by a bare abi integer rather than a compiler_abi string. Traits and generic classes are still not exportable.

Goal: cargo/go/swift-class module system. Seamless, config-level deps (composer.json-style). Easy to distribute the compiler, libraries, and compiled programs.

Mental model#

LayerRustSwiftGoManticore
manifestCargo.tomlPackage.swiftgo.modmanticore.json
module interfacecrate metadata.swiftinterfaceexport data.sig (JSON)
code artifact.rlib.o/.a.a.o
package managercargo/cratesSwiftPMgo/VCScomposer/packagist (fetch)

composer only fetches + resolves versions. manticore compiles + links. The runtime preamble is linkonce_odr in every .o, so there is NO separate Manticore runtime library to ship/link — a binary carries libc, PCRE2 and OpenSSL plus any FFI-bound library it names, and nothing of ours.

Locked decisions#

  1. Lib distribution: source + OPTIONAL prebuilt. Default ships PHP source + manticore.json (portable, cargo-style); a package MAY also ship prebuilt .o+.sig for popular target-triples (Swift binary-framework style). The compiler prefers a matching prebuilt, else builds from source into cache.
  2. .sig format: JSON (reuse json_encode/json_decode; readable, extensible).
  3. Build cache: global ~/.manticore/cache/, keyed by (source-hash, compiler_abi, target-triple) — Go-module-cache style, shared across projects, auto-rebuild on any key change. MANTICORE_HOME overrides.

.sig — module interface#

A .sig is the serialized EXPORTED symbol table of a library target — the same data LowerFromAst builds in fnDecls/fnAliasByBare, so a consumer hydrates it and resolves calls IDENTICALLY to having the source in-module.

{
  "schema": 1,
  "package": "manticorephp/stdlib",
  "version": "1.0.0",
  "compiler_abi": "mir-1",
  "target": "any",                       // or e.g. "arm64-apple-darwin" for a prebuilt
  "functions": [
    { "name": "str_starts_with", "symbol": "manticore_str_starts_with",
      "params": [ {"name":"haystack","type":"string"}, {"name":"needle","type":"string"} ],
      "ret": "bool" },
    { "name": "trim", "symbol": "manticore_trim",
      "params": [ {"name":"s","type":"string"},
                  {"name":"mask","type":"string","default":{"k":"str","v":" \t\n\r\\0"}} ],
      "ret": "string" },
    { "name": "array_keys", "symbol": "manticore_array_keys",
      "params": [ {"name":"arr","type":"array"} ],
      "ret": "vec<int|string>" },                       // element type → fixes typing loss
    { "name": "Runtime\\Libc\\strncmp", "symbol": "manticore_Runtime_Libc_strncmp",
      "params": [ {"name":"a","type":"string"}, {"name":"b","type":"string"}, {"name":"n","type":"int"} ],
      "ret": "int" }                                    // namespaced/FFI exported too
  ],
  "libs":  ["pcre2-8", "ssl", "crypto"],  // #[Ffi\Library] of the wrappers in here
  "weak":  ["signalfd", "__errno_location"], // extern_weak symbols it declared
  "classes": [],     // v2: cross-lib classes (layout + method sigs)
  "constants": []    // v2
}

Rules (these are exactly what unblocks the compiler self-build):

manticore.json#

{
  "name": "acme/app",
  "dependencies": { "manticorephp/stdlib": "^1.0", "acme/json-fast": "^2.1" },
  "libraries":   [ {"name":"foo","src":"lib","output":"...","exclude":[]} ],
  "applications":[ {"name":"app","src":"src","entry":"src/main.php","output":"bin/app","exclude":[]} ]
}

Versions/lock are composer's job (packagist constraints + composer.lock).

manticore build flow#

  1. Resolve deps transitively (composer.lock pins versions).
  2. For each dep: locate package (vendor/ or global), pick a matching prebuilt .o+.sig if present+compatible, else build from source into ~/.manticore/cache/<pkg>-<srchash>-<abi>-<triple>.{o,sig}.
  3. Application target: a. Load all dep .sig → merged imported symbol table. b. LowerFromAst hydrates fnDecls + fnAliasByBare from it and emits declare-only externs (FunctionDef::isExtern). c. Compile app sources → app.o. d. cc app.o <dep1>.o <dep2>.o … -o bin/app.
  4. linkonce_odr dedups the runtime preamble across all objects.

Sharp edge to fix: qualified call vs builtin short-name#

emitBuiltin strips the namespace, so a FQ call \Runtime\Libc\strlen is hijacked by the strlen codegen builtin. Rule: a call that resolves to an IMPORTED FQN symbol takes precedence over the builtin short-name. Disable builtin interception for qualified calls that have an import.

Symbol conflicts between deps#

Two libs exporting trim → exported lib symbols are weak, app definition (strong) wins; lib-vs-lib conflict resolves by manifest dep order with a warning (or hard error — TBD at impl).

Distribution#

On-disk layout#

~/.composer/vendor/manticorephp/compiler/   bin/manticore  lib/stdlib.o  lib/stdlib.sig  manticore.json
~/.manticore/cache/                          <pkg>-<srchash>-<abi>-<triple>.{o,sig}
<project>/  manticore.json  composer.json  composer.lock
            vendor/acme/json-fast/   (source + manticore.json)
            bin/app                  (static)

Implementation roadmap (incremental, gated)#

  1. Type↔string codec + .sig writer (lib build emits JSON .sig, incl. namespaced/FFI + const-folded defaults + element types).
  2. .sig reader → hydrate fnDecls/fnAliasByBare + extern FunctionDefs from JSON instead of re-parsing sources. Fix qualified-call-vs-builtin edge.
  3. weak linkage for exported lib symbols.
  4. Wire manticore build: per-target .sig emit/consume + global cache keyed by (srchash, abi, triple). Validate the COMPILER self-build via manticore.json.
  5. deps resolution: read dependencies, locate under vendor/ + global, transitive + composer.lock.
  6. composer packaging: manticorephp/compiler post-install bootstrap.
  7. Rewire bin/compile/bin/rebuild onto manticore build (done; bin/compile has since been removed); full gate (suite, difftest, stability, fixpoint).

Already done (commit 34418d9): stdlib bundling (prebuilt stdlib.o, declare externs, linkonce/internal preamble, link tail), native json_decode + mixed/cell consumption, manticore build for normal apps, str_replace O(n²) fix.

Edit this page on GitHub