Binding C++ to Clef with Farscape

A C++ object’s address crosses into Clef through an opaque CHandle<'T>. Clef code cannot dereference that handle, offset it, inspect a virtual table through it, or convert it to an integer. It can pass the handle back to a binding that operates on the foreign object. This is the source-language boundary defined by FFI Boundary Semantics, and every C++ binding must preserve it.

This chapter describes the required architecture for Farscape’s proposed C++ integration. Plugify’s ABI knowledge is a proposed input to that integration. The chapter does not establish completed C++ generator support, verified foreign lifetimes, or measured performance. Executable binding examples belong here once the generator emits them and their boundary contracts have been validated.

Architectural Foundation

The Clef Surface

The foreign object’s representation stays outside Clef source. The types visible to a caller describe the operations admitted at the boundary:

These types serve different contracts. A CHandle<'T> supplies neither a buffer extent nor ownership information. A Ptr<'T, 'Region, 'Access> describes interior storage; substituting it for a foreign handle does not establish the foreign allocation’s lifetime. The specification defines no general conversion from a CHandle to an interior Ptr.

The flat closure is the interior mechanism for functions with captured state. A C callback uses FnPtr.ofFunction, which accepts only a reference to a module-level binding. It rejects both captured closures and captureless lambdas. That restriction removes a callback environment from the function value; it does not settle the lifetime of separately registered user data.

Foreign Implementation and Compiler Obligations

Virtual dispatch, C++ object layout, casts, and exception handling belong in a native shim or an explicitly admitted target-pathway implementation. Generated Clef declarations expose the sanctioned boundary types. Binding metadata must describe the native contract and provide the facts needed to justify each call.

  flowchart TD
    Headers["C++ declarations and target ABI facts"] --> Native["Native shim: dispatch, layout, exceptions"]
    Headers --> Contracts["Binding declarations and foreign contracts"]
    Contracts --> PSG["CCS / Baker: elaboration and obligations on the PSG"]
    PSG --> Alex["Alex: witness settled facts"]
    Alex --> Portable["Admitted portable MLIR"]
    Portable --> Target["Selected native target pathway"]
    Native --> Target

This is the proposed division of work. ABI information can describe how to call a function correctly. Establishing that its arguments remain live, have the required extent, and satisfy its ownership contract is additional work.

Boundary Contracts and Admission

The FFI chapter defines conservative nullability and explicit ownership, lifetime, extent and callback obligations. Farscape’s C generator implements the conservative pointer policy and refuses unsupported adapters before writing output. C++ object and method generation is refused until its storage, ownership and ABI contracts can be admitted. These requirements govern each supported binding; metadata alone does not discharge them.

1. Check Nullability Before Introducing a Handle

FFI §5.2 now requires unknown pointer nullability to remain optional. Farscape generates Option<CHandle<'T>> for unannotated data-pointer arguments and results, including incoming callback slots, output cells and value-record fields. Explicit native non-null evidence can narrow that carrier; contradictory evidence is diagnosed.

A declaration that promises a non-null result must identify its evidence. An annotation describes a foreign contract and does not verify the foreign implementation. The boundary must check the conversion or retain the declared foreign premise explicitly; it must never report foreign validity as a fact proved by the Clef type checker.

The native optional-pointer carrier uses one platform word. That does not establish a niche layout for generic interior Option<CHandle<'T>>: the current generic option layout is separate. An unavailable conversion adapter is a compiler admission failure, rather than permission to erase optionality.

2. Carry Ownership and Release Obligations

Non-null does not mean live. A handle can refer to an object that has already been destroyed, and its opaque type alone does not exclude a later call through a retained copy. The revised contract requires release and alias-retirement obligations; the former unchecked allocation/free example has been removed.

A foreign-resource contract must distinguish owned, borrowed, and static results. It must identify the release operation for an owned result, the owner whose lifetime bounds a borrowed result, and any retain operation that extends that lifetime. Destruction must consume the resource under an inferred affine discipline or be justified by a region obligation that excludes subsequent uses and escaping aliases. A destructor name in metadata is insufficient without this connection to the uses of the resource.

Clef’s lifetime discipline carries inferred coeffects and obligations on the Program Semantic Graph (PSG), without ownership or borrowing annotations in source. Where foreign acquisition, borrowing, retention or release lacks an admitted adapter, CCS refuses the boundary before native emission. General foreign-resource and destroy-hook adapters still require implementation; a passing source signature is insufficient evidence of a safe native call.

A scope wrapper that calls a destructor after an action returns does not establish this property: the action could retain a handle, and an exceptional exit could skip the release. Cleanup on every exit and exclusion of uses after release both need evidence.

3. Preserve Buffer Extents Across the Call

A bounded Clef buffer needs a boundary contract relating its storage, element count, byte extent, access permissions, and lifetime to the foreign call. An opaque handle alone carries none of those bounds. A C string operation also requires a terminator within the admitted extent.

The generator must identify which calls borrow a buffer for the duration of the call and which retain it. Checks can establish the extent supplied to C; they cannot prove that an arbitrary C implementation respects it. Foreign code’s compliance must be verified separately or recorded as an explicit trusted premise. The FFI chapter now requires these contracts; general buffer-bound adapters remain unavailable and must be refused.

4. Keep Callback User Data Typed

Callback registration must preserve the relationship between a context’s nominal type, its owner, the callback signature, and its release operation. A typed context handle, such as CHandle<CallbackContext>, can be passed back to context-access bindings; it cannot be opened or reinterpreted by Clef code. Nullable contexts require Option.

That is the required pattern, not an existing typed-user-data API. Giving an untyped foreign value a type parameter by hand does not validate its provenance. The generator and native shim must establish that registration and callback delivery refer to the same context contract. They must also establish when callbacks have stopped before releasing the context or unloading callback code. Rejecting closure callbacks alone does not discharge these obligations.

5. Validate Foreign Values Before Constructing Interior Values

A union discriminator, exception status, or returned count is foreign input. The conversion must validate it before constructing a Clef value. Unknown discriminators and violated return contracts produce an explicit Result error; they do not select a union field unchecked or raise an ad hoc exception inside the binding.

C++ exceptions must be caught in the native shim and converted into a documented status and error payload before control returns to Clef. That payload needs its own ownership and lifetime contract. Neither a status code nor a type-safe wrapper makes an unchecked foreign payload valid.

6. Discharge the Boundary Obligations on the PSG

The closure representation contract already requires elaboration of initialization and lifetime obligations, retention of unresolved premises, and preservation of evidence through lowering. A flat environment does not by itself discharge the ownership and lifetime obligations of storage referenced by its captures.

For a foreign call, the proposed integration must carry the relationships among the result conversion, resource owner, buffer extent, callback registration, and release site into the PSG. Baker’s elaboration and saturation must settle the applicable obligations or retain unresolved premises during analysis. Under the conformance requirements, unresolved required obligations must be diagnosed before the relevant commitment; trusted foreign premises remain explicit conditions of the safety claim. Alex witnesses the settled graph; it cannot infer a missing foreign lifetime contract during emission.

The compiler’s boundary checks reject raw source addresses, forged handles and forged function pointers, including values hidden in aggregates and higher-order factories. Generated-source conformance checks exercise real headers and complete Clef project manifests, requiring unsupported lifetime adapters to produce their owning boundary diagnostic. Applying the remaining foreign-resource contracts requires settled PSG facts before an adapter can be admitted.

Virtual Method Handling

A virtual method binding takes an opaque object handle and typed arguments. The native shim performs the C++ dispatch using the selected compiler and platform ABI. A target-pathway implementation is another possible realization, provided that its operation and contract are admitted explicitly.

Neither realization exposes a virtual-table address or slot-offset calculation in Clef. The ABI description must account for inheritance adjustments, calling conventions, and the concrete library build. Plugify’s proposed role is to contribute these facts; layout knowledge does not establish that the receiver is still live.

Template Instantiation Strategy

C++ templates require concrete instantiations for the supported library build. Each selected instantiation needs a native callable surface and a declared Clef contract. A foreign container is an opaque handle whose operations are bindings; its storage, size, and capacity are not exposed as a Clef record containing an address.

A Clef generic wrapper is justified only when every supported instantiation has a compatible operation, ownership, and layout contract. A generic fallback cannot manufacture missing C++ instantiations. Complex template patterns and multiple inheritance remain generator coverage questions.

Exception Boundary Management

RAII and Resource Management

C++ RAII describes native resource management. The Clef binding must connect acquisition and destruction to the ownership and release obligations above, including failure during construction, exceptional native exits, borrowed views, and asynchronous callbacks.

The public API can report acquisition and operation failures through Result and absence through Option. It must not publish a resource-safe wrapper until release is guaranteed on all admitted exits and no dependent use outlives the resource.

Memory Layout Compatibility

Structure Padding and Alignment

Layout descriptors must reflect the actual target, compiler settings, field offsets, alignment, and library build. A padding sketch for one ABI is not a portable layout contract. Where the contract admits a value conversion, the shim can return a validated value; where it admits shared storage, the binding needs the corresponding extent, access, and lifetime obligations.

Union and Variant Types

The native side interprets the active C++ union or variant representation. Only a validated case and payload cross into an interior discriminated union. A C++ union without a reliable discriminator needs an external protocol establishing its active member; the generator cannot derive that fact from its storage layout.

Integration with BAREWire

Zero-Copy Serialization

A BAREWire descriptor can contribute schema and layout facts. Sharing a packet buffer without copying additionally requires a bounded extent, compatible alignment and byte order, an established lifetime, and access permissions that exclude conflicting use. Retention by C++ adds a lifetime obligation beyond the call.

Clef operates on admitted interior values, bounded buffers, or region-typed storage. A packet available only as CHandle<NetworkPacket> is processed through native bindings. Matching its layout does not authorize an interior dereference or conversion to a Ptr. General zero-copy C++ buffer integration remains contingent on the missing buffer and lifetime contracts.

Optimization Metadata Generation

The backend lowering architecture requires the middle end to emit admitted portable dialects. External declarations and function values remain portable; realizing a function value as an address is a target-pathway commitment. An LLVM operation belongs to the LLVM pathway, not to Farscape’s public surface or the portable middle end.

On a compatible native pathway, static linking with LLVM LTO may enable inlining, devirtualization, or removal of conversions when the required definitions and evidence are available. Dynamic linking provides different opportunities. Neither configuration guarantees zero overhead, and optimization must preserve required boundary checks and lifetime premises. Performance claims need measurements for the actual binding and target.

Testing and Validation Strategy

The generated-source conformance suite checks conservative carriers, preserved descriptor identity, actual dependency paths and refusal before writes. The native callback runner records compiler failures, preserves logs and continues to later cases without an uncaught exception. Current native callback acceptance is still blocked by the missing source-published target ABI realization; listener-entry acceptance is blocked by unavailable lifetime and pointer adapters. Neither refusal is replaced with an address cast or fabricated proof.

The October 3, 2026 checkpoint passed all 649 Farscape tests, 17 generated-source conformance cases, 149 Calque tests and 111 focused compiler boundary tests. Complete compiler acceptance still has 61 failures matching the clean baseline’s remaining failures; the focused results do not establish complete language or native FFI acceptance. The Farscape checkpoint and regeneration guide records the admitted generator subset, refused contracts and checks required before adopting regenerated libraries. Linux remains the current desktop target; this checkpoint does not establish Windows or macOS bindings.

Publishing executable examples requires generated output that compiles against the sanctioned Clef surface and exercises the native implementation. Validation must cover both successful calls and rejected or invalid contracts:

• A foreign NULL result becomes None or a boundary error before a non-null handle is introduced.
• Release obligations exclude double release, uses through retained copies, and escapes of borrowed resources; cleanup covers every admitted exit.
• Callback conversion rejects lambdas and closures; context delivery and release preserve the declared type and lifetime contract.
• Buffer conversions establish counts, extents, terminators where required, and retention rules; malformed values return explicit errors.
• Native fixtures validate dispatch, template instantiations, exception conversion, and layouts for each supported compiler and target ABI.
• PSG and lowering checks preserve the boundary premises, keep the middle end portable, and retain required checks through optimization.

Native debugger support must come from the selected pathway’s debug information and the shim’s native types. Inspecting an object’s virtual table in a debugger does not require an address-inspection API in Clef.

Evolutionary Path

The conservative unannotated-pointer policy and fail-closed generator paths are implemented. The next architectural work is to admit foreign acquisition/release adapters and native function-address realization with their PSG evidence. Typed callback contexts and bounded buffer marshalling depend on those facts. C++ dispatch, templates, variants, and BAREWire integration can then be admitted per library and target with the obligations and generator fixtures needed to substantiate their guarantees.

Plugify integration remains part of that roadmap. It can contribute ABI facts; each supported binding still needs its foreign assumptions identified and its Clef obligations discharged before it is presented as safe.