TBCX

Tcl Bytecode eXchange tbcx 1.1 Apr 12, 2026

NAME

tbcx — serialize, load, and inspect precompiled Tcl 9.1 bytecode (procs, OO methods, and lambdas). Artifacts require an exact Tcl major/minor match at load time.

SYNOPSIS

tbcx::save in out
tbcx::load in
tbcx::dump filename
tbcx::gc

DESCRIPTION

The tbcx extension provides four commands that enable an efficient save → load → eval pipeline for Tcl 9.1 scripts.

The goal is to pay the cost of parsing/compiling at save time so that loading is as fast as reading a compact binary, while remaining functionally equivalent to source of the original script.

Artifacts store the compiled top-level, precompiled proc bodies, TclOO method/ctor/dtor bodies, and lambda literals for use with apply. Loading installs these into the current interpreter and executes the top-level block with TCL_EVAL_GLOBAL.

In safe interpreters, tbcx_SafeInit provides the package and type infrastructure but does not register any tbcx::* commands. A parent interpreter may selectively grant access with interp alias or interp expose.

COMMANDS

tbcx::save in out

Synopsis — Compile a script and write a .tbcx artifact.

Parameters

in — Resolved in this order:

  1. Open channel name — if the value names an existing open channel, it is read as text (the caller controls encoding; TBCX does not alter it).
  2. Readable file path — if the value is a path to a readable file, it is opened in text mode (default UTF-8), read, and closed by tbcx::save.
  3. Literal script text — otherwise the value is treated as inline Tcl script text. Consequently, a value that looks like a path but is not currently readable is compiled as script text, not reported as a file-open error.

out — One of:

  • Writable channel — an open channel; binary mode (-translation binary -eofchar {}) is enforced. The caller’s channel settings are mutated and not restored. The channel is not closed.
  • Writable path — TBCX writes a temporary file in the target directory and renames it into place only after serialization succeeds, so a failed save never leaves a truncated artifact at the final path.
Behavior
  • Performs a single-pass capture and rewrite of the script, extracting proc, namespace eval, oo::class create, oo::define/oo::objdefine, and self method definitions while producing a rewritten script with method bodies replaced by stubs.
  • Pre-compiles namespace eval body literals and other script-body patterns (try/on/trap/finally, foreach, lmap, while, for, catch, if/elseif/else, eval, uplevel, time, timerate, dict for/map/update/with, lsort -command, and self method bodies inside oo::define).
  • Performs two-phase instruction-level body detection: Phase 1 analyzes invokeStk operand stack patterns; Phase 2 identifies unpushed literals from inline-compiled foreach/lmap loops. Uses an O(1) opcode dispatch table covering all Tcl 9.1 instructions.
  • Strips startCommand debugging instructions from all compiled blocks, replacing them with nop bytes for ~15% leaner bytecode execution.
  • Detects bytearray literals (bytes ≥ 0x80) and emits them as TBCX_LIT_BYTEARR to prevent UTF-8 encoding corruption.
  • Body literals are emitted as TBCX_LIT_BYTESRC (source text + compiled bytecode) enabling cross-interpreter recompilation.
  • Serializes the header, top-level block, Procs table, Classes catalog (advisory), and Methods table (including kind 4 for self methods).
  • Lambda literals (apply forms) are compiled and serialized as lambda-bytecode literals.
  • Conflicting proc definitions across if/else branches are handled via indexed markers, enabling position-based matching at load time.
Returns
The output object: either the normalized path written, or the writable channel handle.
Errors
Typical errors include: unreadable input; unwritable output; unknown/unsupported AuxData; size limit exceeded; runaway serialization (too many literals, blocks, excessive recursion depth, or output too large); short read/write.
Notes
  • Input channels retain their encoding settings; output channels are set to binary mode (and not restored).
  • The produced artifact targets Tcl 9.1 (format version 91); other versions are rejected at load time.

Examples

# Save from path → path
set path [tbcx::save ./app.tcl ./app.tbcx]

# Save from string value → path
set script {proc hi {} {puts Hello}; hi}
tbcx::save $script ./hello.tbcx

# Save from channel → channel
set in  [open ./lib/foo.tcl r]
set out [open ./foo.tbcx w]
fconfigure $out -translation binary -eofchar {}
try {
    tbcx::save $in $out
} finally {
    close $in
    close $out
}

tbcx::load in

Synopsis — Load a .tbcx artifact, install precompiled entities, and execute the top-level block.

Parameters

in — One of:

  • Readable channel — an open channel positioned at the beginning of a .tbcx stream (binary).
  • Readable path — a filesystem path to a .tbcx file.
Behavior
  • Validates header and producer version (exact major.minor match required); deserializes sections.
  • Installs a temporary ProcShim that intercepts the proc command to substitute precompiled bodies when the name and argument signature match.
  • Installs a temporary OOShim that intercepts oo::define and oo::objdefine to substitute precompiled method/constructor/destructor bodies. Self methods (kind 4, TBCX_METH_SELF) are installed via oo::define { self method } to preserve metaclass inheritance for subclasses.
  • Registers precompiled lambdas in a persistent per-interpreter ApplyShim that recovers the compiled representation if type shimmer evicts it.
  • Executes the precompiled top-level block with TCL_EVAL_GLOBAL. A top-level return is handled identically to source.
  • Removes the ProcShim and OOShim after execution; the ApplyShim persists for the interpreter’s lifetime.
Returns
The result of the top-level block evaluation.
Errors
Typical errors include: unreadable input; bad header; incompatible Tcl version; malformed/unknown section; size limit exceeded; short read; class or namespace creation errors during top-level evaluation.
Notes
  • The top-level block executes with TCL_EVAL_GLOBAL; compiled locals for the global frame are linked to existing global variables.
  • Loading executes code; only load artifacts from trusted sources.

Examples

# Load into current interp
tbcx::load ./app.tbcx

# Load from an open channel
set ch [open ./app.tbcx r]
fconfigure $ch -translation binary -eofchar {}
try {
    tbcx::load $ch
} finally {
    close $ch
}

tbcx::dump filename

Synopsis — Disassemble and describe a .tbcx artifact in human-readable form.

Parameters
filename — A readable path to a .tbcx file.
Behavior
Reads the artifact and prints: header fields; section summaries; literals and AuxData; exception ranges; and a disassembly of the top-level, proc, method, and lambda bodies.
Returns
A string containing the formatted dump.
Errors
Unreadable file; bad header; malformed section; short read.

tbcx::gc

Synopsis — Purge stale entries from the per-interpreter lambda shimmer-recovery registry.

Behavior
The ApplyShim maintains a registry of precompiled lambda objects. Over time, entries for lambdas that are no longer referenced (except by the registry itself) accumulate. This command purges those stale entries immediately. Stale entries are also purged lazily on each tbcx::load call, so explicit use of tbcx::gc is typically unnecessary except in long-running interpreters.

tbcx::gc is a no-op if no ApplyShim has been installed yet (i.e. before any tbcx::load call), and it is safe to call multiple times.

Parameters
None.
Returns
Empty string.

FILE FORMAT (OVERVIEW)

This section summarizes the on-disk structure. Format version is 91 (Tcl 9.1). All integers are little-endian.

Header
Magic (0x58434254) + format version (91) + producing Tcl version; size/count metadata for the top-level block (code length, exception ranges, literal count, AuxData count, locals, max stack).
Sections (order)
(1) Top-level block (code, literals, AuxData, exceptions, locals epilogue); (2) Procs (FQN, ns, arg spec, compiled block); (3) Classes (advisory catalog of discovered class names; actual creation occurs at load time via the top-level script); (4) Methods (class, kind 0–4, name, args, compiled block).
Literal kinds
boolean, (wide)int/uint, double, bignum, string, bytearray, list, dict (insertion order preserved), bytesrc (bytecode + source text for cross-interp recompilation), and lambda-bytecode literals for use with apply.
AuxData families
jump tables (string or numeric), dictupdate, and NewForeachInfo.

LAMBDA SUPPORT

Literals in the script that represent lambdas for apply (lists of the form {args body ?ns?}) are compiled and serialized as lambda-bytecode literals at save time. On load, the compiled body, argument list, and optional namespace element are rehydrated into a Proc and registered in the ApplyShim so that the first call to apply does not trigger compilation. If type shimmer later evicts the lambdaExpr internal representation, the ApplyShim transparently re-installs it on the next apply call.

SEMANTICS AND PERFORMANCE

Artifacts are portable across little- and big-endian hosts. The loader detects host byte order once and applies a tight in-place swap over bulk sections that require conversion, avoiding per-field branching so cross-endian loads remain close to native performance.

Loading executes the precompiled top-level (with TCL_EVAL_GLOBAL), then installs precompiled proc/method bodies and rehydrates lambda literals. The intent is to be functionally indistinguishable from source of the original script, with the benefit of faster startup due to avoided parsing/compilation.

PRECOMPILATION BOUNDARY

TBCX precompiles bodies and lambdas only when they are present in statically identifiable literal positions (script-body arguments to commands like foreach, while, try, eval, etc., or lambda literals for apply). Strings assembled at runtime — for example with format, string interpolation, or list construction — still round-trip correctly, but they remain ordinary data and compile at execution time when Tcl evaluates them.

OO SUPPORT

TBCX preserves normal TclOO class/object construction semantics by executing the rewritten top-level script, while substituting precompiled bodies for recognized oo::define / oo::objdefine method forms. Tested scenarios include class methods, self methods, per-object methods, private methods, inheritance (including diamond), mixins, filters, forwards, abstract/singleton metaclasses, method rename/delete/export changes, metaclasses with self method, and next-based constructor chaining.

MULTI-INTERPRETER AND THREAD SUPPORT

Artifacts are designed to load into interpreters other than the originating one. The bundled tests cover child interpreters and threaded scenarios. Interpreter-specific state such as the ApplyShim lambda registry remains per-interpreter.

LIMITS

Sanity caps exist for code size (64 MiB), literal/AuxData/exception counts (1M each), string lengths (4 MiB), total serialized output (256 MB), serialization recursion depth (64), total WriteLiteral calls (2M), and total WriteCompiledBlock calls (256K). Exceeding any limit produces an error.

Nested or reentrant tbcx::load calls are capped at depth 8 per interpreter to prevent runaway recursive loading.

DIAGNOSTICS

Representative messages include: “bad header”, “incompatible Tcl version”, “short read/write”, “unsupported AuxData kind”, “input is neither an open channel nor a readable file”, “runaway serialization detected”, and Tcl errors from top-level evaluation.

SECURITY

Loading executes code. Only load artifacts you trust. Safe interpreters receive no tbcx::* commands by default; use interp alias or interp expose from a parent interpreter to grant selective access.

SEE ALSO

source(n), TclOO(n)

© 2025–2026 Miguel Banon

MIT License.

↑ Top