brolang/LANGUAGE.md

# language features

This file is the compact current-state ledger. `TODO.md` remains the detailed
roadmap and milestone history.

## IMPLEMENTED

### source, declarations, and packages

- newline-terminated statements and `#` comments
- immutable `::` bindings, typed mutable `=` locals/globals, and `_` sinks
- immutable package globals, mutable runtime globals, function-local mutable locals, and mutable declarations initialized with `undefined`
- package-level functions, globals, native type declarations, and `Name :: alias T`
- directory packages with merged declarations
- file-local relative imports, import aliases, and qualified member access
- transparent declaration aliases with `Name :: alias package.Member`; functions/type factories,
  named types, and globals retain their original declaration or storage identity
- `hide` makes any named top-level declaration package-local; declarations are public by default,
  leading underscores are ordinary identifier characters, and imports are always file-local
- relative `.h` imports as synthetic C header package namespaces
- native `name test { ... }` declarations with fallible-void results inferred from `testing.Error`
  and errors propagated by `try`, plus anonymous
  transitive `test import "..."` discovery used only by test builds
- root `main` validation with trap executable recovery for missing or unusable entry points

### scalar, aggregate, and pointer types

- exact-width integers, concrete pointer-sized `isize` / `usize`, `f32`, `f64`, `bool`, `void`, `noreturn`, and `anyopaque`; `noreturn` is a bottom type valid as a native function result and coerces to any expected value type; contextual `int` accepts the whole integer family, while `uint` accepts only unsigned native and target-classified C integers
- target-dependent C scalar primitives from `c_char` through `c_longdouble`, kept semantically distinct from native scalars
- contextual integer/float/character literals, backward type-demand inference through names and arithmetic/bitwise expressions, and typed compile-time evaluation for numeric constant expressions
- strict numeric conversion by default, widening where valid, C scalar coercions at C boundaries, and explicit scalar casts through keywords or transparent aliases, such as `i32(x)`, `c_float(x)`, or `StringId(x)`
- compile-time `minval!(T)` and `maxval!(T)` bounds for concrete native, C, and scalar-backed distinct integer types; the result retains `T`
- arrays `[N]T`, inferred-count arrays `[_]T`, sentinel arrays `[N;S]T`, compile-time expression array counts, slices `[]T` / `[;S]T`, single-item pointers `@T`, many-item pointers `*T`, and sentinel many-item pointers `[*;S]T`
- pointer mutability via `mut`, optional pointers as nullable pointers, pointer arithmetic for many-item pointers, postfix dereference `^`, and trapping optional unwrap `?`
- pointer-to-array `.len`, indexing, slicing, `.ptr` on slices and pointers-to-arrays, implicit address-taking for array-variable slices, and pointer/slice sentinel weakening
- `ptrcast!(T, ptr)` as a first-pass pointer-child retype that preserves optionality, pointer kind, mutability, and sentinel shape
- unsafe `constcast!(value)` for restoring mutability to pointers, optional pointers, and slices without changing their child type or shape
- UTF-8 string literals as immutable pointers to static zero-terminated byte arrays, plus raw backtick multiline strings
- narrow immutable zero-terminated byte pointer/slice conversion to `*c_char` / `?*c_char` without general `u8`/`c_char` interchange
- optionals with `null`, `orelse`, postfix `?`, conditional unwraps, guarded unwraps, and left-to-right short-circuiting multi-unwraps
- nominal distinct types with explicit scalar backing conversion during construction and explicit scalar backing extraction, native enums with optional explicit integer backing and explicit backing-to-scalar casts, contextual enum literals, and imported C enums as target-backed integer aliases
- source-order native structs, opaque nominal records with `Name :: opaque`, complete `c_struct { ... }`, keyed record literals, native untagged unions, and native tagged unions `union(Enum)` / `union(enum)`
- named native struct fields may declare defaults with `field T = expression`; keyed literals use defaults for omitted fields and explicit initializers override them
- void-payload tagged-union variants, anonymous struct payloads, contextual `.variant`, `.variant{payload}`, and `.variant{field = value}` construction
- native sum composition with `A | B` for unbacked enums and tagged unions, optionally grouped as `(A | B)`, using program-global `u16` variant ids
- fallible channel types `T ! E`, where `E` is a native enum/tagged union or supported sum composition; `void ! E` functions complete successfully on fallthrough, and void-success `catch` handlers may fall through without `yield`

#### distinct types

`Name :: distinct T` creates a nominal identity and reuses `T`'s runtime representation. When
`T` is a concrete numeric scalar, construction first applies the corresponding explicit scalar
cast, so `UserID(index)` is sufficient for `UserID :: distinct u32` even when `index` is `usize`.
There is still no implicit conversion in either direction. Construction with a non-scalar or
distinct immediate backing requires that exact backing type. An explicit scalar cast extracts one
layer: `u32(id)` works for `UserID`, while nested distinct values must be peeled one declared layer
at a time.

Scalar-backed distinct values support the operations of their representation while preserving the
nominal result type: checked integer `+`, `-`, `*`, unary `-`, bitwise operators, shifts,
comparisons, and compound assignments; float arithmetic, unary `-`, comparisons, and compound
assignments; and boolean equality/inequality. Integer literals and float literals are contextual.
Typed backing values and separate distinct identities remain incompatible in ordinary operations;
an explicit constructor is required to cross that boundary. Distinct integers also work as indices and slice bounds;
`minval!` / `maxval!` return the distinct type. Runtime and comptime behavior match.

`typeinfo!(Distinct).backing` reports the immediate declared backing. Standard formatting peels
distinct layers recursively, so all scalar format verbs behave like the final scalar backing.

#### native record constraint fields

A direct `int`, `uint`, `float`, or `range` field in a named native struct or union is a
program-wide constraint, not per-value polymorphism. Before record layout, all reachable keyed
constructors, field assignments, and concrete uses of field reads contribute demands and the field
resolves once to one concrete runtime type. Compatible scalar demands widen normally. Integer
literals remain provisional until inference settles, so a later `usize` use can resolve an `int`
field to `usize`; otherwise literal-only `int` fields use the widest smallest-signed type required,
literal-only `uint` fields use the widest smallest-unsigned type required, and literal-only `float`
fields use `f64`.

An undemanded field or incompatible demands are errors. This inference applies only to direct
fields of named native records. `c_struct` fields, nested constraints such as `[]int` / `[]uint`, and fields in
anonymous generated records still require concrete runtime types.

#### keyword member names

Reserved keywords are valid native enum members and tagged-union variants when used in an
unambiguous member context:

```bro
TokenKind :: enum {
    if
    else
    return
}

Token :: union(TokenKind) {
    if i32
    else void
    return i32
}

conditional func() TokenKind { return TokenKind.if }
fallback func() TokenKind { return .else }
token func() Token { return Token{ if = 1 } }
```

Keyword variants also work with field access and `.variant` match patterns; `.else:` remains
distinct from the `else:` catch-all arm. No escaping syntax is required. Keywords remain reserved
for ordinary declarations, struct fields, untagged-union fields, and anonymous payload-struct
fields. `_` is not a keyword member name.

### expressions and control flow

- checked integer `+ - *`, unary `-`, float-only `/`, IEEE float arithmetic, comparisons, `!`, `and`, and `or`
- Zig-style integer bitwise complement `~`, binary `&`, `|`, `xor`, shifts `<<` / `>>`, and saturating left shift `<<|`; postfix `^` remains pointer dereference
- assignments and compound assignments `+= -= *= /= &= |= xor= <<= >>= <<|=` with single evaluation of complex lvalues; `/=` is float-only and `xor=` is contiguous
- field access through struct values and pointers, index/slice bounds contextually coerced to `usize`, and unsigned narrower index support
- boolean `if` / `else if` / `else` and `for` loops with braceless single-statement bodies when the preceding expression is parenthesized or a function call
- `while` loops with optional post-iteration update clauses
- `for` loops over ranges, arrays, slices, and pointers-to-arrays with copy captures, pointer captures `|@item|`, and optional `usize` index captures; `inline for` specializes a comptime aggregate into one checked body per element
- `break`, `continue`, labeled `break :label`, labeled `continue :label`, and labeled plain blocks; `break :label` can cross nested scopes to exit a labeled block
- bare block scopes, `defer`, and fallible-function `errdefer` with optional error capture; cleanup is block-scoped and LIFO
- always-trapping `unreachable`, a `noreturn` expression that diagnoses use during comptime evaluation and terminates the current runtime path
- bare void `return`, same-line `return value`, value blocks, value `if` with implicit single-expression branches, value loops, value `match`, and strictly value-producing `yield value` / `yield :label value`
- `match` statements/expressions over enums, tagged unions, and scalars, including exhaustiveness checks, payload captures, pointer payload captures, multi-pattern arms, and scalar range patterns
- a final `inline |value|:` enum arm or `inline |payload[, tag]|:` tagged-union arm generates one specialized arm for each variant not covered earlier; enum values and optional tags are comptime-known, while union payloads keep their concrete variant type
- fallible `try` and uniform `catch [|e|] value_source` fallbacks; captures work with ordinary expressions, value blocks, and value-producing `if` / loops / `match`
- direct `return match ...` and `yield match ...` value-control-flow operands

#### bitwise operations

Bitwise operands must be concrete integers. `~` preserves its operand type. `&`, `xor`, and `|`
use the ordinary common-integer widening rules; incompatible fixed integer families remain errors.
Shifts preserve the left operand type and require a concrete unsigned count. `>>` is arithmetic for
signed integers and logical for unsigned integers.

Ordinary `<<` and `>>` reject compile-time-known counts at least as large as the left type's bit
width and trap for such runtime counts. `<<` discards shifted-out bits. Saturating `<<|` permits any
unsigned count: zero remains zero, unsigned nonzero values clamp to the type maximum, and signed
values clamp to the minimum or maximum according to their sign.

Binary precedence, from tightest to loosest, is:

```text
* /
+ -
<< >> <<|
& xor |
== != < > <= >=
and
or
```

Each level is left-associative. Because `|` also delimits `if` and `for` captures, a bitwise-OR
header expression must be parenthesized before a capture list, for example
`if (flags | mask) |value| { ... }`.

#### division

Compiler intrinsics use direct unqualified `name!(...)` syntax. The `!` marks the call as an
intrinsic; it is not part of the identifier. Bare and qualified calls without `!` resolve as
ordinary user functions, while qualified bang calls are rejected.

`/` and `/=` accept only floating-point operands. Integer division must state its rounding and
remainder convention with one of these intrinsics:

| Builtin | Result |
| --- | --- |
| `divtrunc!(a, b)` | quotient rounded toward zero |
| `divfloor!(a, b)` | quotient rounded toward negative infinity |
| `divexact!(a, b)` | truncated quotient; traps unless it divides exactly |
| `divceil!(a, b)` | quotient rounded toward positive infinity |
| `rem!(a, b)` | remainder paired with `divtrunc!`; sign follows `a` |
| `mod!(a, b)` | modulus paired with `divfloor!`; sign follows `b` |

The operands may be compatible concrete integer or float scalars. Existing literal coercion and
numeric widening rules apply, the result has the common operand type, and float quotients are
integral-valued floats. These identities hold when representable:

```bro
divtrunc!(a, b) * b + rem!(a, b) == a
divfloor!(a, b) * b + mod!(a, b) == a
```

Negative operands distinguish the operations:

```bro
divtrunc!(-5, 3) == -1
divfloor!(-5, 3) == -2
divceil!(-5, 3) == -1
rem!(-5, 3) == -2
mod!(-5, 3) == 1
mod!(5, -3) == -1
```

All six builtins diagnose a zero denominator at comptime and trap at runtime, including float
zero. Quotient operations also trap for signed `minval!(T), -1`; `rem!` and `mod!` return zero for
that pair. `divexact!` traps when `divtrunc!(a, b) * b == a` is false in the operand type, so float
exactness follows floating-point equality. Other float NaN and infinity behavior follows the
underlying IEEE operations. Ordinary float `/` remains unchecked and therefore preserves IEEE
infinity/NaN behavior.

Only these six division bang calls select integer-division behavior. Bare calls such as
`divfloor(a, b)` and qualified calls such as `math.divfloor(a, b)` resolve to ordinary functions.

#### typed memory operations

`memcopy!(destination, source)` and `memset!(destination, value)` are available at runtime and
comptime. A destination must be a mutable slice or mutable pointer-to-array. A `memcopy!` source
may be a slice or pointer-to-array; many-item pointers must first be sliced. Array pointers are
treated as regions containing their explicit logical elements, including a sentinel only when it
is part of that array region.

`memcopy!` requires the same element type after alias resolution and the same element count. Its
non-empty regions must not overlap. Comptime calls diagnose unequal lengths and overlap; runtime
calls trap for either condition or if the element count cannot be converted to a byte count.
Zero-sized elements still require equal counts. Empty copies are no-ops and may name the same
region.

`memset!` coerces `value` to the destination element type. Use `memset!(destination, 0)` to zero a
region; there is no separate `memzero!`, and `memset!` does not promise secure zeroing. Copying or
filling with `undefined` transfers undefined state without reading it. Each operand is evaluated
exactly once. Bare functions named `memcopy` or `memset` remain ordinary user functions.

### functions, C interop, and linking

- demand-monomorphized Brolang and C-ABI functions
- integer comptime value parameters such as `make_array func($N usize) [N]u8`, specialized by value and omitted from the runtime ABI
- explicit comptime type parameters such as `max func($T type, a, b T) T`, specialized by type and omitted from the runtime ABI
- later comptime value parameters may depend on earlier type parameters, as in `factory func($T type, $default T) type`
- comptime parameters may appear anywhere, are erased from the runtime ABI, and accept recursively stable booleans, integers, floats, types, immutable bytes, enums, fixed arrays, records/tuples, optionals, tagged unions, and bare function identities; equal structural values and aliases of one function declaration share specializations, while distinct declarations remain distinct and pointers, general slices, fallibles, ranges, untagged unions, and undefined values have no stable comptime identity
- comptime parameters may be omitted when uniquely recoverable from runtime arguments, the immediate expected result, or exact type-factory provenance; `_` is an explicit inference hole
- direct bodyful value calls with no runtime parameters, including parameterless and all-`$` functions, are evaluated at comptime when their resolved result can materialize; otherwise they retain their zero-argument runtime specialization, while a reached `compile_error!` remains a diagnostic
- forced typed comptime expressions such as `$sum(1, 2)`, `$Point { x = 1, y = 2 }`, and comptime value blocks such as `${ yield 4 }`
- comptime execution for bodyful Brolang functions with mutable locals, loops, `defer`, `match`, `try`/`catch`, exact type `==`/`!=`, pointer/slice storage mutation, pointer captures, and calls through comptime-known function values; `undefined` storage may be initialized at comptime, but remaining poison cannot be observed
- comptime type factories such as `Box func($T type) type { return struct { value T } }`; calls like `Box(i32)` are concrete nominal types and may appear anywhere a type is expected
- `struct_type!(layout, names, types, defaults)` constructs a nominal record type from comptime fixed arrays or tuples and is valid in every type position; layout is `.auto` or `.c`, bare `null` means a required field, and `some!(null)` installs an optional `null` default
- `some!(value)` explicitly constructs the present branch of an expected optional, including nested optionals where `some!(null)` differs from outer `null`
- tuple types are unnamed-field structs (`struct { i32, []u8 }`), tuple values use `{1, "bro"}` / `{1,}` / `{}`, and fields use canonical numeric names such as `.0`
- anonymous keyed records use `{x = 1, name = "bro"}`; without context their declaration-ordered names and inferred value types form a structurally interned record type, while a record context applies that type's coercions and field defaults; `{}` remains an empty tuple without context and constructs an empty contextual record when a record is expected
- `typeinfo!`, `field!`, `compile_error!`, and semantic `inline for` provide compile-time record and enum reflection and heterogeneous static expansion without runtime metadata; enum reflection exposes declaration-ordered fields, reflected aggregates remain persistent compile-time values, and inline-loop `break` / `continue` must be selected entirely at comptime
- `tag!(value)` reads a tagged union's active discriminant and folds when the value is comptime-known; `tagname!(enum_value)` requires a comptime-known enum value and returns its immutable declaration name
- bodyful `c_func` definitions and bodyless `c_func` declarations with exact external symbol names
- concrete-only C signatures, C variadic declarations/calls, and C default argument promotions
- bare `func(...) R` and `c_func(...) R` values are comptime-only declaration identities; arrays, native records, optionals, and tagged unions containing one are also comptime-only and cannot enter runtime storage, ordinary ABI parameters/results, runtime globals, or C-layout records
- native function pointer values and types use `@func(...) R`, fallible `@func(...) R ! E`, and optional `?@func(...) R`; bare native identities implicitly materialize compatible pointers when a runtime pointer context requires one, but pointers never convert back to bare identities
- statically known bare identities and comptime-known pointers lower calls directly; native indirect calls remain available for runtime-selected non-variadic pointers
- Apple Silicon C ABI lowering for scalars, pointers, fixed-signature plain records/unions, small aggregates, homogeneous float aggregates, and indirect aggregate returns
- imported C typedefs, scalar constants, enum constants, fixed arrays, complete plain structs/unions, C `void*` as nullable `anyopaque` pointers, and pointers to opaque records
- imported external C object variables, including mutable variables and immutable object globals
- object-like scalar and plain record/union macro constants
- supported static inline C functions through generated external wrappers
- C function pointer types, imported nullable callback typedefs, bare `c_func` identities with one-way pointer materialization, and postfix calls through non-null function pointers
- `brolang translate-c <header.h>... [--output-dir <dir>]` for native `.bro` bindings from supported C declarations, with package-wide declaration deduplication when writing multiple headers
- `brolang --translate-c stdio.h` for offline bindings from Zig-bundled standard C headers
- ordered linking of additional C sources, objects, archives, library paths, and libraries through compiler CLI options

### standard packages

- root `std` re-exports `ArrayList(T)` while its operations remain in `std/arraylist`
- `std/mem` generic slice equality, allocator contract with raw byte operations, typed `empty` / `alloc` / `realloc` / `free`, overflow checks, zero-sized-type support, and failure-preserving reallocation
- `std/arraylist` generic `ArrayList(T)` with direct `items` slice access, explicit capacity, allocator ownership, fallible reserve/append, clear, and deinit
- `std/meta` reflection records plus `EnumFieldStruct(E, Field, default ?Field)`, implemented with `struct_type!`; it produces a record with one field per native enum member in declaration order, where outer `null` means no field default
- `std/io` explicit `Io` capabilities, provider-bound `Reader`/`Writer` handles, existing-file open/close operations, allocation-free `write_all`, and comptime-expanded writer-first `print`; formatting supports natural `{}`, byte `{s}`, decimal `{d}`, integer `{b}` / `{o}` / `{x}` / `{X}`, byte-character `{c}`, scientific float `{e}`, recursively scalar-backed distinct values, and `{{` / `}}`, with malformed formats and incompatible tuple fields rejected at comptime
- entry points are either `main func() ...` or `main func(init process.Init) ...`; `std/process.Init` carries startup capabilities, currently only `io`, while the system provider remains hidden inside `std/io`
- `std/debug.print` is an allocation-free, failure-ignoring stderr escape hatch independent of `process.Init`
- `std/testing` supplies fallible `expect`, expected-first `expect_equal`, and exact compile-time `expect_type`; direct calls through
  an alias of exactly `@std/testing` receive compiler-injected source locations

### compiler behavior

- error-tolerant compilation with diagnostics and runtime traps where recovery is possible
- lazy semantic checking of demanded function specializations
- static, eager runtime, mutable runtime, and deferred problematic globals with cycle diagnostics
- demand-driven LLVM declarations for referenced foreign functions
- root `main` may be parameterless or accept canonical `@std/process Init`; the generated C entry point obtains the hidden system I/O provider and constructs the init value
- `brolang test [root]` reuses `build.bro`, discovers only explicit test-import edges, skips the
  application entry point, and runs tests sequentially while continuing after assertion failures
- replaceable dynamically loaded libclang C-import backend
- C-header import caching by canonical path, target, include paths, and defines

## PLANNED / DEFERRED

- exporting Brolang functions to C and broader target-specific C ABI lowering
- non-plain C record layouts such as bitfields, packed records, flexible arrays, qualified fields, and C variadic record arguments
- arenas, pools, build-mode heap policy, and escaping-allocation diagnostics
- recursive type factories, reflection payloads beyond records/enums, and type-producing unions/enums
- broader Zig-style pointer/result casts beyond V1 `ptrcast!(T, ptr)`
- sum-type ABI/layout polish, including dynamic tag-width shrinking, all-void channel collapse, and cross-module global-id determinism
- backed/C enum composition and must-consume fallible linting
- result-to-argument type-demand propagation through function call boundaries
- string concatenation operator