Backend Lowering Architecture

Informative References

Commentary: For accessible explanation of the portable middle end and why a target commitment is deferred to the backend, see Why Clef Is A Natural Fit for MLIR in the Clef design documentation.

Artifact Class: The deployment-facing rationale for the sealed-image artifact class that the freestanding mode of §5 enables is developed in Getting to the Heart of Unikernels on the Clef blog.

For .NET Developers: Guidance on transitioning from CLR concepts to native compilation is available in the rationale documentation.

1. Overview

This chapter specifies how Clef lowers high-level constructs through a portable middle end to a target-specific backend, using MLIR’s multi-dialect architecture. Portable operation vocabulary is distinct from target-specific realization. Baker settles source semantics and the selected platform’s requirements; Alex witnesses those facts through the Huet zipper using Elements/Patterns/Witnesses appropriate to that platform and backend. The backend realizes the resulting expression with its required information preserved. Portable vocabulary does not imply target-blind witness selection.

2. Portable Middle End, Target-Committing Backend

Clef separates its intermediate representation by whether a construct has committed to a target:

Clef Source → CCS / Baker → settled PSG → Alex → admitted portable MLIR → Target pathway → Artifact
                                selected platform and correlated evidence ────────────►

The tier boundary is portable-versus-target-specific. The LLVM serializer handles CPU, WASM and MCU targets, the CIRCT path handles FPGA targets, the MLIR-AIE path handles NPU tile-array targets, and the JSIR serializer handles the JavaScript target. CIRCT, MLIR-AIE, and JSIR are also MLIR, but they are target-specific MLIR, so they live in the backend for the same reason the LLVM serializer does. What separates the tiers is the commitment, not the technology. The artifact class is part of the pathway’s commitment: a native binary from the LLVM pathway, a bitstream from the CIRCT pathway, an NPU binary from the MLIR-AIE pathway, a JavaScript module from the JSIR pathway.

Premature target-specific encoding can lose information required by later consumers. The middle end’s job is therefore information preservation: each admitted pathway receives the complete relevant expression and correlated graph facts, without reconstructing semantics from the operation stream. Selecting a target-aware portable form does not itself perform the backend’s target-specific encoding. Emitting an llvm.* operation above this boundary commits to LLVM; a CIRCT hardware operation likewise belongs to the declared FPGA realization stage. Neither may silently replace the common receiving contract. Target realization SHALL preserve the source observables and proof premises required by the applicable contracts.

2.1 Portable Dialects

The documented baseline vocabulary is:

Operations are admitted per expression, selected platform/backend profile and witness form under §2.1.1. Extensions to the baseline include math, affine, vector, tensor, async and cf, subject to the same admission requirements. Coverage is established per operation and pathway, including for the baseline dialects.

Structured scf and explicit-block cf forms can suit different profiles. Alex SHALL select an admitted form using Baker-settled relationships and the selected profile’s requirements. A pathway may receive structured control and lower it to cf later; direct cf witnessing requires its own admission. There is no universal requirement to give every backend an identical dialect subset.

2.1.1 Operation and Pathway Admission

An admission SHALL identify the source operation family, selected platform/backend profile and exact witness form, including operations, types, attributes, regions and successor relationships. It SHALL record:

1. The governing language contract and Baker nanopasses that establish the required graph facts, joint constraints, coeffects and proof premises.
2. The target capabilities and selection rule that make the form suitable, with declaration/quotation provenance and validity conditions. BAREWire, Fidelity.Platform, applicable Fidelity.UI expressions and program declarations contribute these facts through Baker.
3. Complete typed Elements/Patterns/Witnesses and the declared downstream consumer/pass sequence. Witnessing SHALL remain a passive observation through the Huet zipper; missing semantic prerequisites SHALL be reported, not reconstructed in Alex.
4. A carrier for every fact required downstream: an operation/type/attribute or an accessible correlated graph projection, retaining graph identity and proof correspondence through rewrites. Textual MLIR alone is sufficient only where it carries all facts required by that consumer.
5. Positive and negative source, graph, witness, verifier and backend acceptance evidence for the named scope, including relevant tool versions and target configuration. A verifier result alone SHALL NOT establish semantic preservation.

Numeric selection and arithmetic construction govern the choice and composition of arith, math and other admitted operations under Numeric Selection §§10.3–10.5. A representation’s availability alone does not establish operation eligibility. Witness forms and subsequent rewrites SHALL preserve required intermediate precision, scale, rounding, capacity, exceptional behavior and permitted decomposition. The accuracy-only selection objective remains unchanged; realization-cost comparison SHALL satisfy §10.4 of that chapter.

Parallel and suspended execution SHALL retain the blocking dependencies and classification of Synchronous RPC and Wait Classification, together with the target-specific progress, admission and assumption requirements of Scheduler Contract. Numeric equivalence and concurrency progress are separately required. An async or control-flow operation SHALL NOT substitute for those graph relationships or establish them by its presence.

The operation/profile contract SHALL identify missing or inconsistent prerequisites and preserve their source provenance for design-time projection. Changing a depended-upon target fact invalidates its form-selection and preservation evidence.

2.2 Constructs Whose Realization Is a Target Commitment

Some constructs are expressed portably in the middle end and take a target-specific form only in the pathway. Each has a portable carrier consumed by the pathway’s standard lowerings, without a deferred cast. Function values use the multi-value encoding of §4.

3. Where a Target Commitment Happens

3.1 The Middle End Emits Portable Dialects For

• Every operation with no representation dependency on the target
• Directly-called functions (func.func, called by name)
• Structural control flow (branches, loops, conditions)
• Arithmetic and comparison
• Function values and closures, as the two-value pair (fn, env) (§4.2)

3.2 The Target Pathway Commits For

• The memory form of a function address — at the extern boundary, where a C API receives a callback (FFI Boundary §3.2)
• Realizing heterogeneous structs (records, closures, unions) under their settled Clef layouts, or a separately declared native layout at an FFI crossing
• Operations with ABI implications (memory layout, alignment)
• Target intrinsics (syscalls, atomics, volatile MMIO)

None of these commitments appears in middle-end IR. Each is realized by the pathway’s standard lowerings for the portable dialects (§4.3); no pathway requires a resolution pass or plugin.

All source analysis, inference and semantic repair SHALL remain in CCS/Baker; they SHALL NOT be restored as Alex preprocessing or an MLIR semantic pass. Target-specific lowering belongs to Composer’s backend.

4. Function Values in the Interior

The middle end represents a function value as two SSA values — a function symbol and an environment buffer — and passes them as two parameters. This is the multi-value form of Closure Representation §6.3, and it is the whole of the closure calling convention above the witness boundary. Nothing about a function value is deferred to a target pathway.

4.1 Why No Commitment Is Deferred

The environment is a byte buffer of captured data; the code is a func.func symbol referenced by name. func.constant is a first-class SSA value in the portable dialect, func.call_indirect consumes it, and the pair (fn, env) crosses any function boundary as two parameters. This encoding does not require a memref of function type. The interior forms are enumerated in Closure Representation §7, all in func + memref + arith.

4.2 The Multi-Value Encoding

A closure value is (%fn : (memref<Exi8>, args...) -> ret, %env : memref<Exi8>), where E is the environment extent settled at saturation. Creation is func.constant @lifted plus the allocation the lifetime lattice selects (Closure Representation §3.3) and a memref.store of each capture at its literal offset through a static memref.view. Application is func.call_indirect %fn(%env, args...); where the callee is known at saturation the form degenerates to func.call @lifted(%env, args...) with no function value at all.

There is no cast. builtin.unrealized_conversion_cast SHALL NOT appear in the middle end’s output for any construct, and no target pathway resolves a closure form: each pathway receives standard func and memref operations that its standard lowerings already handle. A function held by an admitted aggregate resides as its callable component: its environment view where present and, where its family has several alternatives, a finite selector. The code is reconstructed through portable control flow together with its own environment; it is never stored as data or cast. The published component rows retain formation, contract, lifetime and tag dependencies. Alex witnesses that settled selection and does not infer a missing alternative, environment or contract.

4.3 Realization on Each Pathway

Every pathway consumes the multi-value form with its standard lowerings, and none adds a pass:

• CPU and MCU (LLVM). func.constant → llvm.mlir.addressof; func.call_indirect → llvm.call through the function value; the environment memref → the pathway’s memref lowering.
• WebAssembly. func.constant → a table index consumed by call_indirect; the environment lives in linear memory. The pair remains two values.
• JSIR. Under §4.5, the environment buffer is realized by the host closure and the pair collapses to the host function value.

A lazy thunk shows the interior at its simplest: Lazy<int> is an environment {state: i1, value: i32} with a known-callee body, so forcing is a direct func.call @thunk(%env) guarded by the state flag — no indirect call and no pointer anywhere. Escaping function values use call_indirect; the discriminating condition is the closure form, decided at saturation.

4.4 Function-Body Composition

Within a target pathway, portable and committed operations can coexist in one function body during lowering:

1. func.call inside llvm.func: valid; an llvm.func can call a func.func using func.call.
2. llvm.call target restriction: llvm.call can only call functions defined as llvm.func.
3. Portable ops in any function: arith.*, cf.*, scf.*, memref.* operations remain valid in a committed llvm.func body during lowering.

4.5 Carrier Realization on Pathways Without Linear Memory

The portable carriers of this chapter presume nothing about the target’s memory model. A memref of a storage block and an index-carried pointer are stand-ins. On a pathway whose target has linear memory (LLVM, CIRCT), they commit to byte layouts and machine pointers, and the layout figures of the representation chapters describe that commitment. On a pathway whose target has no linear memory, they commit to the pathway’s own value model. The JSIR pathway is the current instance: its target manipulates host objects and function values, not bytes and addresses.

Two provisions make this realizable without weakening the middle end’s target neutrality:

1. Access is realized per pathway. Structured storage is reached through the access forms the middle end emits: the generated functions of Discriminated Union Representation §10.2, field access over the storage carrier for records and tuples, capture reads for closures. A pathway realizes each access form in its own model: a byte-offset load on the LLVM pathway, a field select on the CIRCT pathway, a property access or a host-closure capture on the JSIR pathway.

2. A pathway MAY read the graph. The Program Semantic Graph carries the program’s type structure (field names, case identities, capture sets) as codata beside the portable IR, under the same discipline that carries blade support and escape classification (Grade Discipline §3.3.1): read during lowering, absent from what is emitted. A pathway whose value model needs structural identity reads it from the graph during realization; the JSIR pathway needs property names where the LLVM pathway needs byte offsets.

A realization SHALL preserve the structural requirements of the representation chapters: construction, elimination, initialization, capture modes, case identity, and every other observable the chapter states. The layout figures of those chapters (sizes, offsets, alignment, tag width) bind only pathways that realize memory layouts. Each representation chapter whose realization on the JSIR pathway requires a decision carries a JSIR-pathway realization section stating it (Discriminated Union Representation §10.2.2, Option Operations Representation, Closure Representation §6.4).

5. Entry Point Example

The entry point is where a pathway’s target commitment is most visible, because it is inherently target-specific: how a program receives control and how it exits are properties of the target, not of the program. The middle end emits main as an ordinary func.func; the pathway supplies the entry glue.

A pathway operates in one of two modes. A hosted pathway targets an environment with an OS runtime beneath the program (an x86-64 Linux pathway, say), which supplies process startup, syscall, and stack-passed argc/argv. A freestanding pathway targets an environment with no such runtime: the program is self-contained and receives control directly. The mode governs only the presence of a host runtime; it does not by itself fix word size, whether an allocator or C library is linked, or the core count — those are properties of the specific target, not of being freestanding. (A unikernel is one freestanding target: a self-contained image with no host OS. Not every freestanding target is a unikernel, and this mode distinction does not turn on that term.)

In freestanding mode the pathway emits an _start in its committed function form (its address is taken by the linker) that calls the portable main. The glue below is written for a hosted x86-64 pathway; another pathway supplies its own. On the Cortex-M33 freestanding pathway there is no syscall and no stack-passed argc/argv: _start is the reset entry, arguments do not exist, and exit is a halt, so the glue is entirely different while main is unchanged.

// Target pathway (x86-64 hosted) — committed dialect. NOT middle-end output.
llvm.func @_start() -> i32 {
    // Read argc/argv via inline asm — target-specific glue.
    %argc = llvm.inline_asm "mov (%rsp), $0", "=r" : () -> i64
    %argv = llvm.inline_asm "lea 8(%rsp), $0", "=r" : () -> !llvm.ptr

    // Call main — func.call, since main is the portable func.func.
    %result = func.call @main(%argc, %argv) : (i64, !llvm.ptr) -> i32

    // Exit syscall — target-specific glue.
    llvm.inline_asm has_side_effects "syscall", "..." %result : ...
    llvm.unreachable
}

The JSIR pathway has no _start analog. Its artifact is a module, and its entry commitment is the export glue for the host’s module convention: the pathway emits exported bindings, and the host invokes them (the entry-point modes of Program Structure and Execution). Entry glue remains per pathway in every case: a syscall preamble on the hosted x86-64 pathway, a reset vector on the M33 pathway, module exports on the JSIR pathway.

6. Platform Configuration

The project file specifies the target platform. The values below are one target; the M33 freestanding pathway sets target = "thumbv8m.main-none-eabi" and word_size = 32, and the same fields drive its layout and word size.

[compilation]
target = "x86_64-unknown-linux-gnu"

[platform]
word_size = 64
endianness = "little"

The word size is a platform property resolved from this configuration, not a constant baked into the middle end. Pointer and word width in every layout (type layouts) come from word_size for the selected target, so the same IR yields a four-byte word on the M33 and an eight-byte word on x86-64. This configuration flows through:

1. Fidelity.Platform selects the appropriate PlatformDescriptor
2. CCS uses platform info for type layouts and intrinsic typing
3. Alex emits portable IR parameterized by the platform word
4. The target pathway for the selected target commits and receives correctly-lowered IR

The JSIR pathway takes no word_size: it realizes no byte layouts (§4.5), and its integer realization is specified in Width Inference §8. Its platform configuration specifies a host environment in place of an architecture triple, described by the managed-substrate descriptor of Platform Bindings.

7. Normative Requirements

1. The middle end SHALL use the admitted portable vocabulary: func, scf, arith, memref and index form the baseline; extensions require operation/profile admission under §2.1.1. Alex SHALL use the selected platform’s settled facts to choose the appropriate admitted form, including structured or explicit-block control where admitted. The middle end SHALL NOT emit llvm.*, builtin.unrealized_conversion_cast or target-specific operations above the declared target-realization boundary.
2. A target commitment SHALL occur only in a target pathway: The memory form of a function address, ABI struct layout, and target intrinsics are committed by the selected pathway’s standard lowerings (e.g., func.constant → llvm.mlir.addressof on the LLVM pathway), never in middle-end IR.
3. Struct manipulation SHALL be carried portably and committed per pathway: Record, union, and closure struct access is emitted over memref in the middle end and realized by the target pathway under the settled Clef layout. A native structure crossing requires a separately declared and admitted projection (FFI Boundary §3.6); interior placement does not imply C ABI layout. A pathway without memory layouts uses its own value model (§4.5).
4. llvm.call target restriction: within the LLVM pathway, llvm.call SHALL only call functions defined as llvm.func; to call a func.func from llvm.func, use func.call
5. Function values are two SSA values: the middle end SHALL represent a closure as (fn, env) per §4.2 and SHALL NOT convert a function address to or from data; builtin.unrealized_conversion_cast SHALL NOT appear in middle-end output, and no pathway SHALL require a cast-resolution pass or plugin to consume a closure
6. Platform configuration flow: fidproj platform settings, including word_size, SHALL inform all lowering decisions; pointer and word width SHALL be taken from the selected target’s word_size and SHALL NOT be assumed to be 64-bit
7. Carrier realization: a target pathway SHALL realize the portable storage carriers and their access operations in its own value model, MAY read the Program Semantic Graph’s type structure during that realization, and SHALL preserve every structural requirement the representation chapters state; the layout figures of the representation chapters SHALL bind only pathways that realize memory layouts (§4.5)
8. Artifact class: the artifact class a pathway produces is part of its commitment: a native binary on the LLVM pathway, a bitstream on the CIRCT pathway, an NPU binary on the MLIR-AIE pathway, a JavaScript module on the JSIR pathway
9. Complete information handoff: each consumer SHALL have access to the Baker-settled expression and correlated semantic/proof facts its realization requires, under §2.1.1. Target-specific realization SHALL preserve those contracts; it SHALL NOT introduce replacement source semantics or silently weaken numeric, memory or progress requirements.