honey/TODO.md

self-hosting

• milestone: Honey compiles its production compiler, then that compiler compiles itself again and builds/runs the example programs.

current boundary

directory packages support file-local import namespaces and postfix declaration selection through the interpreter and native backend. concrete 8/16/32/64-bit integers, checked numeric conversions, records/tuples, construction/defaults, enums, distinct/alias types, overlapping untagged/tagged unions, field reads/writes and exhaustive statement/value matching work through both execution paths. source data pointers/references support aliasing, mutation, calls and aggregate storage; explicit callback signatures, many-pointer addition, mutable/sentinel slices, optionals and layout/cast/memory builtins work through both execution paths. native scalar/pointer c_func declarations support allocator-required libc calls; executed foreign calls remain diagnostic traps in the interpreter. explicit fallible channels, composed error sets, try/catch and scoped LIFO defer/errdefer run through both paths; source errors are values, not diagnostic traps or compiler infrastructure failures. cached generic specialization and type-producing procedures support explicit/inferred compile-time arguments and generated record/array storage through both paths. ordinary procedures can compute scalar, nested record/array, tagged-union and fallible compile-time values using mutable locals, loops and recursive calls; explicit $() expressions use the same constant evaluation path. compile-time if/match selection and type comparisons specialize only the selected body, including owned by-value tagged payload captures in ordinary generic procedures. type-producing procedures also accept ordinary constant inputs, cached and erased like explicit compile-time arguments. mutable type/value locals, direct and aggregate field/index assignments, branches, scalar/enum/tagged-union matching and bounded integer-range/collection/while loops now generate types and storage. optional if/while captures and tagged payload matches select nested constructors, lengths, signatures and defaults; array/slice/string/pointer-to-array collection loops support exact copy/pointer captures and usize indices through both paths; type-producing bodies iterate materializable arrays/strings by value without retaining interpreter addresses. type-valued fixed arrays support nested copies, indexed/whole updates, checked projections and by-value collection captures in type-producing bodies; undefined type/value arrays track initialization per leaf while loops fill generated types/defaults. type-array parameters and results cross compile-time procedure boundaries with owned snapshots, cached nominal identity and erased arguments. initialized type-valued records/tuples support local schemas/defaults, nested field/index updates and owned copies/captures inside type-producing bodies. field! selects record/tuple/enum and sequence members using compile-time names, including runtime places and type-control updates without runtime reflection. general type-valued execution, metadata reflection and generic library acceptance remain open. the driver retains frontend storage and source provenance through @source/compilation, with compilation-wide AST node identities. file commands retain single-file behavior; directory commands compile the package's production files and load its dependencies in separate scopes.

completed milestone: scalar-function packages

verified: compile and run a small multi-file directory package importing a second package of scalar functions, through both the interpreter and native backend. using the existing standard library is a later acceptance gate: even std/mem needs records, pointers, generics and C calls.

• retain source/package identity and source ownership across the compilation; keep one identifier pool and unambiguous declaration IDs.
• load directory packages once per canonical path, with deterministic file discovery, package scopes and root-only entry selection. follow Bro's existing relative and project-root @ import rules; diagnose cycles rather than recurse indefinitely.
• finish postfix namespace selection and resolve file-local import aliases without injecting dependency names into the importing package; bare and aliased import declarations are already parsed for dependency discovery.
• reuse ordinary scalar signature checking, function values and IR calls for selected declarations; namespaces have no runtime representation.
• preserve originating filenames and spans in dependency diagnostics and native runtime traps; retain demand-driven diagnostic recovery.
• verify sibling source files, transitive/shared imports, same-spelled declarations in different packages, alias conflicts, missing packages/members, argument errors and dependency-file diagnostics. preserve existing single-file commands.

visibility follows Bro's public-by-default contract; unsupported visibility attributes must be diagnosed, never ignored. the scalar-function milestone did not include record fields or imported types; those now work for the record/tuple slice. test discovery and FFI remain outside these completed slices.

subsequent dependency gates

• concrete data models: required integer widths/conversions, records/tuples and construction, fields, enums, distinct/alias types, unions and matching. examples: source/lexer/token.hon, source/sema/resolve.hon, std/mem/mem.hon.
  • native integer widths (i8–i64, u8–u64, isize/usize) and checked explicit numeric conversions, including float truncation/range checks and integer-to-float rounding. preserve exact types through arrays, calls and range captures; retain strict implicit conversion rules.
  • records/tuples, typed/contextual/anonymous construction, field defaults and reads/writes, nested aggregate value copies, imported record types and constructors through interpreter/native execution.
  • nominal enums and distinct types, transparent declaration/type aliases, overlapping untagged/tagged unions, contextual payload construction, active-field checks and exhaustive scalar/enum/tagged-union matching through interpreter/native execution. pointer captures, inline specialization and yielding value blocks remain with their respective later gates; error-set composition is covered below.
• memory representation: source pointers/references, mutable and sentinel slices, optionals, layout/cast builtins and allocator-required libc calls. internal IR array addresses are not source pointers. examples: std/mem/mem.hon, std/arraylist/arraylist.hon.
  • source data pointers/references: @T/*T, mut, address-taking and dereference, pointer-aware fields/indexing, array .len/.ptr, readonly weakening, location equality, recursive pointer-linked records and shared pointees across interpreter/native calls and aggregate copies.
  • function-pointer signatures: explicit @proc(...) Result, structural signature checking, callbacks in parameters/results/records/arrays, vtable calls and imported signature aliases; many-pointer + usize arithmetic preserves pointee type, mutability, stride and location identity through interpreter/native execution.
  • mutable/sentinel slices with checked bounds, borrowed aliases and string views; optional values/pointers/callbacks, null checks, unwrap and conditional/loop captures; target layout, pointer/const casts and typed memory copy/set through interpreter/native execution. scalar/pointer C ABI declarations build/run allocator callbacks using malloc, realloc, free and posix_memalign; arbitrary interpreter FFI, generic standard-library compilation and native startup/IO remain with later gates.
• fallible results and cleanup: explicit T ! E, error-set composition, try/catch, defer/errdefer, error captures and matched error residuals through interpreter/native execution; infrastructure errors remain distinct from diagnostic traps. fallible void may complete on fallthrough; unhandled entry errors exit with status 1. inferred T! signatures and yielding catch blocks remain with inference/value-control-flow work. examples: source/main.hon, source/sema/check/check.hon.
• generic specialization and compile-time execution/reflection sufficient for existing ArrayList, hash maps, enummap, meta and io.print; include type/value parameters and compile-time generated storage.
  • specialization checkpoint: explicit/inferred type and integer/boolean/string/callback/null value parameters, cached concrete runtime calls, nominal type-producing procedures, generated record/array storage and recursive data pointers through interpreter/native execution. preserve declared value types, source diagnostics and repeatable compilation; runtime values cannot supply compile-time arguments.
  • compile-time execution checkpoint: ordinary procedure calls with constant inputs and integer/boolean/string results reuse checking, typed IR and bounded interpreter execution. mutable scalar locals, range/while loops, break/continue, nested/generic/recursive calls, exact numeric types, runtime traps and dependency-file provenance survive argument erasure through interpreter/native execution. runtime parameters and mutable global dependencies are rejected; recompilation retains no prior execution state.
  • aggregate/explicit-expression checkpoint: computed nested records/tuples and fixed arrays, contextual/dependent aggregate value parameters, normalized specialization caching and explicit $() expressions through interpreter/native execution. preserve by-value isolation, zero-cell shapes, integer widths, string ownership, enum/optional fields and exact IEEE float bits; reject runtime dependencies and values that would retain temporary interpreter storage. repeatable compilation restores erased expressions.
  • conditional specialization checkpoint: explicit compile-time if guards and side-effect-free constant guards in concrete generic procedures select a body before specialization/checking. type equality retains aliases, nominal identity and structural signatures; composed error-set identity/cache keys ignore ordering, grouping and repetition. guard checking, selected-body diagnostics, scoped cleanup, runtime guards and repeatable compilation work through interpreter/native execution; both arms remain parsed and name-resolved.
  • constant-input type-procedure checkpoint: type-producing procedures accept ordinary parameters alongside $ parameters, require every argument explicitly and evaluate/cache/erase all inputs. generated record/array storage, nominal cache identity, nested calls, dependent contextual aggregate inputs, exact erased-argument checking, source diagnostics, imported producers and repeatable compilation work through interpreter/native execution.
  • mutable type-execution checkpoint: type-producing bodies execute mutable type/value locals, direct binding assignments, scoped blocks, if/else and early returns, integer-range/while loops, break/continue and nested/recursive producers. generated nested types, scalar-computed lengths/defaults, nominal cache identity, exact value types, overflow/guard/assignment diagnostics, imported producers and repeatable compilation work through interpreter/native execution. type-control execution shares a 4096-step bound across nested producers; scalar operations reuse checked IR execution. aggregate field/index updates and type-valued arrays are covered below; cleanup and type-valued record storage remain open.
  • type-match checkpoint: type-producing bodies execute scalar/enum match statements and direct type-valued match expressions in initializers, assignments and returns, including nested selected arms. ordinary checking/IR execution select the arm and validate subject/pattern types, ranges, overlap, unreachable arms and exhaustiveness; only the selected body is elaborated. generated storage, scoped locals, loop transfers, early returns, nominal cache identity, selected-body diagnostics, dependency provenance and repeatable compilation work through interpreter/native execution. all bodies remain parsed/name-resolved and execution shares the type-control bound.
  • nested type-match checkpoint: matches select types inside constructors, array lengths, generic arguments, local annotations, callback signatures and field defaults in type-producing bodies. selection precedes ordinary expansion, preserves literal spelling/source provenance and loop snapshots, and caches nominal results before expanding recursive fields. selected-body and pattern diagnostics, failed deferred elaboration, imported producers and repeatable compilation work through interpreter/native execution; deferred selection shares the producer's type-control bound.
  • tagged/fallible constant checkpoint: explicit $() expressions, generic value parameters and type-procedure inputs materialize tagged unions and both fallible channels through interpreter/native execution. preserve active tags/payloads, nested raw aggregate projections, string ownership, void/zero-cell variants, composed/payload errors, declared channel types, normalized cache identity and repeatable compilation. catch computes generated lengths and runtime values. untagged unions and storage-bearing active payloads remain escape diagnostics rather than retaining temporary interpreter handles.
  • tagged type-match checkpoint: type-producing bodies execute tagged-union matches with immutable by-value payload captures in statement, expression and nested storage positions. ordinary match checking validates every arm's captures, including grouped variants; selected payloads retain exact scalar/aggregate types and owned strings. generated storage/defaults, loop snapshots/transfers, cache identity, selected-body diagnostics, imported producers, dependency provenance and repeatable compilation work through interpreter/native execution. storage-bearing payloads retain the constant-materialization escape boundary.
  • generic match-specialization checkpoint: side-effect-free constant scalar/enum/tagged-union subjects in concrete generic procedures and explicit match $(subject) expressions/statements select an arm before body specialization/checking. reuse ordinary checked IR for pattern types, ranges, overlap, exhaustiveness and every arm's capture validation; only the selected body is elaborated. exact by-value payload types/owned strings, runtime parameters, nested expression matches, cleanup, return/loop transfers, imported producers and repeatable compilation work through interpreter/native execution. ordinary call/runtime/mutable subjects stay runtime; contextual aggregates with possible runtime defaults are not implicitly evaluated. all bodies remain parsed/name-resolved; storage-bearing payloads retain the constant-materialization escape boundary.
  • optional type-control checkpoint: type-producing bodies execute optional if/while guards with immutable by-value payload captures. ordinary checked IR validates guards; selected scalar/aggregate payloads retain exact types and owned strings across nested captures, generated storage/defaults, loop updates/transfers and early returns. absent optionals skip unselected bodies; invalid guards, capture mutation/shadowing, overflow and execution bounds retain diagnostics. imported producers, dependency provenance, nominal cache identity and repeatable compilation work through interpreter/native execution. null data/function pointers materialize correctly; nonnull storage-bearing payloads retain the constant-materialization escape boundary.
  • aggregate type-update checkpoint: type-producing bodies update initialized, materializable aggregate locals through nested record fields and mutable fixed-array indices. ordinary checking validates assignments, retaining exact value types, mutability (including immutable payload captures), index types/bounds, unknown-field and narrow-overflow diagnostics. new owned constants preserve earlier by-value copies while later loops, generated storage and defaults observe updates. imported producers, dependency provenance, nominal cache identity and repeatable compilation work through interpreter/native execution. pointer/slice-backed mutation and partial record construction remain open; type-valued records/arrays and uninitialized value arrays are covered below.
  • collection-iteration checkpoint: arrays, slices, strings and single pointers-to-arrays support immutable exact-type copy captures, storage-qualified pointer captures and optional usize indices through interpreter/native execution. iterables/bounds evaluate once, array lvalues remain borrowed and later element updates are visible; temporary arrays, aggregate copy isolation, zero-cell/empty collections, nested loops and cleanup across transfers retain ordinary semantics. ordinary constant-input procedures reuse the same checked IR; type-producing bodies iterate materializable arrays/strings with by-value captures, fixed entry-time place indices, generated storage, nominal cache identity, dependency diagnostics, shared execution bounds and repeatable compilation. storage-bearing type-control captures retain their existing boundary; type-valued elements are covered below.
  • type-array storage checkpoint: type-producing bodies own initialized fixed arrays of type values, including explicit/inferred nested arrays, .len, checked usize indexing, whole/indexed updates and immutable by-value collection captures. persistent snapshots preserve copies and exact shape/mutability while borrowed array places observe later updates. selected types generate nested storage, constructors, signatures and defaults through interpreter/native execution, retaining selected-body and index/type/mutability diagnostics, imported provenance, nominal cache identity, shared bounds and repeatable compilation. uninitialized type arrays and local type-valued records are covered below; reflection remains open.
  • type-array procedure checkpoint: fixed arrays of type values cross compile-time procedure inputs/results, including nested/aliased results, dependent $N lengths, empty shapes and erased $types parameters in runtime generic procedures. all inputs to type/type-array producers are explicit constants; owned snapshots preserve by-value isolation, exact shape/mutability and canonical/nominal cache identity. projections generate signatures, constructors, storage and defaults through interpreter/native execution. malformed inputs/results, runtime dependencies, imported provenance, shared execution bounds and repeatable compilation retain diagnostics and isolation. runtime type-array storage, type-valued record procedure boundaries and reflection remain open.
  • field-selection reflection checkpoint: field!(base, name) evaluates/erases a constant string name and reuses ordinary selection through interpreter/native execution. records/tuples, enum/error-set members, tagged payloads, callbacks, pointer-qualified places and sequence .len/.ptr retain exact types, single base evaluation, address-taking, mutability and active-field checks. type-producing bodies snapshot changing selector names for nested aggregate updates without losing by-value isolation. runtime/nonstring names, malformed calls, missing members and noncanonical/overflowing tuple ordinals retain diagnostics; full metadata reflection and inline expansion remain open.
  • uninitialized type-array checkpoint: fixed type arrays in type/type-array producers start with undefined and fill through loop-driven indexed, nested and whole-array updates. persistent local copies retain independent initialization state; .len works before leaves initialize. premature leaf reads and incomplete procedure arguments/results diagnose read-before-initialization without caching invalid values. explicit nonnegative dimensions, empty shapes, ordinary mutability/shape checks, nominal identity and repeatable compilation work through interpreter/native execution. construction shares the type-control bound per expanded cell, including nested zero-leaf shapes. ordinary value-array construction and local type-valued records are covered below; runtime undefined storage and metadata reflection remain open.
  • uninitialized value-array checkpoint: ordinary fixed arrays in type/type-array producers start with undefined, fill nested indices or replace whole arrays/rows, and generate storage/defaults through interpreter/native execution. owned snapshots preserve independent partial initialization in local copies; initialized scalar/string/optional/record/tagged-union elements retain exact types and ordinary checked operations. premature leaf reads and incomplete procedure arguments diagnose read-before-initialization. empty shapes, .len before initialization, mutability/type/index/bounds/overflow diagnostics, per-cell construction limits and repeatable compilation are verified. runtime undefined storage, uninitialized arrays in ordinary value-returning procedures, partial record construction and storage-bearing constant materialization remain open.
  • type-record storage checkpoint: initialized records/tuples containing type values live inside type/type-array producers, including local schemas/defaults, typed/contextual/inferred construction, nested records/arrays, checked field/index reads and updates, whole-value replacement and immutable by-value collection captures. persistent snapshots isolate copies/captures while borrowed array places observe later updates; type leaves retain nominal identity and ordinary value leaves reuse checked IR. generated storage/defaults, invalid field/type/default diagnostics, readonly places, narrow overflow, imported producers/provenance, cached nominal results and repeatable compilation work through interpreter/native execution. field validation/construction shares the type-control bound. runtime type-valued records, type-record procedure inputs/results, partial record initialization, metadata reflection and inline expansion remain open.
  • remaining gate: general type-valued compile-time execution, inline match/loop expansion and metadata reflection (typeinfo!, struct_type!, generated type/default arrays), broader constant materialization (including untagged unions and storage-bearing values), plus existing-library acceptance. required expression-valued control flow, compound updates, formatting parsing and imported C-binding support may interleave with later gates.
• remaining source constructs: expression-valued control flow and yield, compound updates and multiline strings. examples: source/parser/parse.hon, source/llvm/runtime.hon.
• standard-library milestone: compile/run a multi-package program exercising existing allocation and IO, including error cleanup; no replacement miniature standard library.
• native driver integration: required C ABI declarations, process.Init startup arguments/IO, actual CLI exit statuses, file loading, temporary files and spawning Clang. Honey-generated programs currently accept only parameterless main and print its result; Bro's bootstrap startup support is not Honey backend support.
• self-hosting proof: bootstrap-built Honey builds the production compiler (excluding test-only imports); that executable rebuilds the compiler; the rebuilt compiler passes representative interpreter/native examples and CLI/error scenarios without Bro at runtime.

the gates may require interleaving (notably allocation, errors and generics). extend checking, IR and native emission together; accepting syntax alone does not complete a gate. additional targets, a package registry, a replacement linker and interpreter execution of arbitrary FFI are not prerequisites.