tbcx — serialize, load, and inspect precompiled Tcl 9.1 bytecode (procs, OO methods, and lambdas). Artifacts require an exact Tcl major/minor/patch/release-type match at load time.
tbcx::save in out ?-include-source?
tbcx::load in
tbcx::dump filename
tbcx::gcThe tbcx extension provides four commands that enable an efficient save → load → eval pipeline for Tcl 9.1 scripts.
The goal is to pay the cost of parsing/compiling at save time so that loading is as fast as reading a compact binary, while remaining functionally equivalent to source of the original script.
Artifacts store the compiled top-level, precompiled proc bodies, TclOO method/ctor/dtor bodies, and lambda literals for use with apply. Loading installs these into the current interpreter and executes the top-level block in the caller’s current namespace, with source-equivalent semantics for variable scope, frame identity, and info script.
In safe interpreters, tbcx_SafeInit provides the package and type infrastructure but does not register any tbcx::* commands. A parent interpreter may selectively grant access with interp alias or interp expose.
Synopsis — Compile a script and write a .tbcx artifact.
in — Resolved in this order:
out — One of:
-translation binary -eofchar {}) is enforced. The caller’s channel settings are mutated and not restored. The channel is not closed.-include-source — Optional flag. Selects include-source policy and embeds the exact authored bytes in every executable-source field: proc, TclOO method, TBCX_LIT_BYTESRC, and lambda body. Use it when consumers depend on info body, info class definition, TIP #280 line attribution, disassembly annotations, or introspection-based clone idioms. Retained text is introspection data; execution still uses precompiled bytecode.
Default behavior (no -include-source): Selects strict compiled-only policy. Every proc, method, BYTESRC, and lambda executable-source field is empty. Proc and method introspection receives the diagnostic sentinel described in SOURCE PRESERVATION; stripped lambdas receive a separate list-shaped diagnostic representation. Ordinary Tcl literals remain in bytecode, so source stripping is not encryption.
Examples
# Save from path → path (default: source stripped)
set path [tbcx::save ./app.tcl ./app.tbcx]
# Save from path → path with body source preserved for
# info body / info class definition / TIP #280 line attribution.
tbcx::save ./app.tcl ./app.tbcx -include-source
# Save from string value → path
set script {proc hi {} {puts Hello}; hi}
tbcx::save $script ./hello.tbcx
# Save from channel → channel
set in [open ./lib/foo.tcl r]
set out [open ./foo.tbcx w]
fconfigure $out -translation binary -eofchar {}
try {
tbcx::save $in $out
} finally {
close $in
close $out
}Synopsis — Load a .tbcx artifact, install precompiled entities, and execute the top-level block in the caller’s current namespace.
in — One of:
.tbcx stream (binary)..tbcx file.uplevel 1 [list tbcx::load $path] to pop to the outer frame — identical to the pattern already required for source in the same position..tcl source path when the artifact was built from a file, matching what source would have set. This supports both [file dirname [info script]] (for sibling asset lookup) and [info script] eq $::argv0 (self-invocation guards).Examples
# Load into current interp
tbcx::load ./app.tbcx
# Load from an open channel
set ch [open ./app.tbcx r]
fconfigure $ch -translation binary -eofchar {}
try {
tbcx::load $ch
} finally {
close $ch
}
# Drop-in replacement for source, inside a module loader proc.
# Both branches use uplevel 1 so they reach the caller’s frame.
proc moduleLoad {path} {
set tbcxPath [file join [file dirname $path] ../tbcx [file tail $path].tbcx]
if {[file exists $tbcxPath]} {
uplevel 1 [list tbcx::load $tbcxPath]
} else {
uplevel 1 [list source $path]
}
}Synopsis — Disassemble and describe a .tbcx artifact in human-readable form.
.tbcx file.source: <stripped at save time>. Each method record also shows its visibility scope and origin (class-definition vs object-definition). Literal operands in the disassembly are annotated using the instruction table’s operand types rather than instruction-name heuristics.Examples
% package require tbcx
% tbcx::save hello.tcl hello.tbcx -include-source
% puts [tbcx::dump hello.tbcx]
TBCX Header:
magic = 0x58434254 ('T''B''C''X')
format = 93
tcl_version = 9.1.0 (type 0)
top: code=12, except=0, lits=2, aux=0, locals=1, stack=2
source = /path/to/hello.tcl
Top-level block:
Disassembly (top-level):
...
Procs: 1
- proc hi (ns=::)
args:
source: (14 bytes)
puts Hello
Disassembly (hi):
...Synopsis — Purge stale entries from the per-interpreter lambda shimmer-recovery registry.
tbcx::gc is a no-op if no ApplyShim has been installed yet (i.e. before any tbcx::load call), and it is safe to call multiple times.
Without -include-source, every designated executable-source field (proc, method/constructor/destructor, BYTESRC, and lambda body) is empty on the wire. Ordinary Tcl literals remain in bytecode, so this is not encryption. For proc and method bodies the loader installs this two-line diagnostic sentinel:
# tbcx: body source stripped at save time; info body unavailable
error "tbcx: introspection-based cloning is not supported for this artifact"The first line is a Tcl comment visible to introspection and traces. The second line raises an error if code copies and evaluates the diagnostic body.
Stripped lambda records use a separate fixed list-shaped diagnostic string. Their registered lambdaExpr representation still holds the materialized Proc. A newly constructed object containing only the diagnostic has lost TBCX identity and fails instead of compiling an apparent authored body.
With -include-source, exact authored bytes are preserved in all four executable-source field kinds and attached for introspection as appropriate. The ByteCode internal representation is untouched and PRECOMPILED; execution still uses compiled material. info body, info class definition, info class constructor, and TIP #280 source attribution round-trip byte-for-byte.
This section summarizes the on-disk structure. Format version is 93 (Tcl 9.1). All integers are little-endian.
0x58434254) + format version (93) + producing Tcl version;
size/count metadata for the top-level block (code length, exception ranges,
literal count, AuxData count, locals, max stack); authored source path LPString
(empty for inline/channel inputs).
Literals in the script that represent lambdas for apply (lists of the form
{args body ?ns?}) are compiled and serialized as lambda-bytecode literals at
save time. A candidate is accepted only when its list value round-trips faithfully as a
Tcl lambda — rebuilt the way the loader rebuilds it, including validation that the
optional namespace element is absolute — so data lists that merely resemble lambdas
remain ordinary data. On load, the compiled body, argument list, and optional namespace element are
rehydrated into a Proc and registered in the ApplyShim so that the first
call to apply does not trigger compilation. If type shimmer later evicts the
lambdaExpr internal representation, the ApplyShim transparently re-installs it on the
next apply call.
Artifacts are portable across little- and big-endian hosts. The loader detects host byte order once and decodes the little-endian wire fields explicitly. Numeric jump tables use Tcl’s one-word key representation on every host; a serialized numeric key that is not representable by a 32-bit evaluator is rejected instead of being truncated or treated as a string key.
The loader requires an exact major, minor, patch, and release-type Tcl producer match. An interpreter-neutral image preflight validates source policy, source fields, section totals and tags, opcode and instruction boundaries, literal/local/AuxData indices, direct and jump-table targets, exception boundaries, nested policy words, and trailing bytes before any interpreter-owned artifact material is constructed. Reconstruction uses one final packed ByteCode allocation and one copy of each instruction stream, without compile-environment staging arrays.
Loading evaluates the precompiled top-level in the caller’s current namespace with iPtr->scriptFile set to the authored source path (if recorded), then installs precompiled proc/method bodies and rehydrates lambda literals. The intent is to be functionally indistinguishable from source of the original script, with the benefit of faster startup due to avoided parsing/compilation.
TBCX precompiles bodies and lambdas only when they are present in statically identifiable literal positions (script-body arguments to commands like foreach, while, try, eval, etc., or lambda literals for apply). Strings assembled at runtime — for example with format, string interpolation, or list construction — still round-trip correctly, but they remain ordinary data and compile at execution time when Tcl evaluates them.
TBCX preserves normal TclOO class/object construction semantics by executing the rewritten top-level script, while substituting precompiled bodies for recognized oo::define / oo::objdefine method forms. Tested scenarios include class methods, self methods, per-object methods, private methods, inheritance (including diamond), mixins, filters, forwards, abstract/singleton metaclasses, method rename/delete/export changes, metaclasses with self method, and next-based constructor chaining.
Method visibility survives the round-trip however it was expressed: definition options (-export, -unexport, -private), lexical private { … } blocks (TIP #500 true-private), and same-body export/unexport (and self export/self unexport) commands are all folded into the per-method scope byte at save time and re-applied at load. Class-instance and per-object methods sharing the same name coexist (distinct origin keys), and per-object methods — including true-private ones — are installed with precompiled bodies. A method the saver left verbatim (non-literal name, arguments, or body) always keeps its authored body; precompiled records are matched positionally, in definition order, and patch only stub-sentinel bodies.
With -include-source, info class definition, info class constructor, info class destructor, and info object method all return the authored body text byte-for-byte, enabling introspection-based clone and copy idioms to work identically to the source-based baseline.
TBCX follows Tcl’s standard threading model: only the thread that created an interpreter may call tbcx::save, tbcx::load, tbcx::dump, or tbcx::gc on that interpreter. Multi-thread support means multiple independent interpreters, each used by its owning thread — not sharing one interpreter across threads. Calling a TBCX command from a non-owning thread returns TCL_ERROR with a diagnostic message.
Artifacts are designed to load into interpreters other than the originating one.
Statically recognized interp eval literal crossings are rejected with
TBCX EVAL CROSSINTERP UNSUPPORTED; retained source is not compiled as a fallback.
Interpreter-specific state (ApplyShim lambda registry, load depth, OO shim state) is
consolidated in a single per-interpreter record and cleaned up automatically when the
interpreter is deleted. Shared process-wide state — the save-side opcode dispatch
table — is initialized exactly once, under a mutex, at package initialization, so
concurrent tbcx::save calls from multiple threads are safe. Debug builds (or builds
compiled with -DTBCX_THREAD_CHECKS) additionally assert interpreter-thread
ownership inside internal helpers.
Sanity caps exist for code size (64 MiB), literal/AuxData/exception counts (1M each), string lengths (4 MiB), total serialized output (256 MB), serialization recursion depth (64), total WriteLiteral calls (2M), and total WriteCompiledBlock calls (256K). Every raw instruction and metadata reference is validated before ByteCode construction, and unexplained trailing bytes are rejected. Exceeding any limit produces an error.
Nested or reentrant tbcx::load calls are capped at depth 8 per interpreter to prevent runaway recursive loading.
Representative messages include: “bad header”, “incompatible Tcl version”, “short read/write”, “unsupported AuxData kind”, “input is neither an open channel nor a readable file”, “runaway serialization detected”, “tbcx::save: unknown option …; expected -include-source”, “tbcx: called from non-owning thread”, and Tcl errors from top-level evaluation.
Loading executes code. Only load artifacts you trust.
Safe interpreters receive no tbcx::* commands by default; use interp alias or interp expose from a parent interpreter to grant selective access.
source(n), TclOO(n), info(n), apply(n), interp(n), tclcompiler and tbcload (Tcl 8.x bytecode tools)
© 2025–2026 Miguel Banon
MIT License.