hl-valdemar/brolang
1
0
Fork 0
Code Issues Pull Requests Actions Packages Releases Wiki Activity

allocator interface (first pass)

Browse Source
This commit is contained in:
Valdemar Høgsvig Lorenzen
2026-07-06 21:24:49 +02:00
parent 7ce3917c15
commit 95c61311ca
24 changed files with 833 additions and 161 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TODO.md
+83 -59
View File
@@ -11,7 +11,7 @@
1. interop type foundation (implemented)
- unsigned integers, floats, and target-dependent c scalar types
- atomic `c_*` primitive types remain distinct until target-aware lowering
- `c_func` and pointer-only `c_struct`; `c` remains an ordinary identifier
- `c_func`, complete `c_struct`, and pointer-only `opaque`; `c` remains an ordinary identifier
- keep binding mutability (`::` / `=`) separate from element or pointee mutability (`mut`)
- arrays and indexing
- `[N]T`: array with `N` logical elements
@@ -28,9 +28,9 @@
- character literals
- optionals with trapping unwrap and fallback operations
- native structs with compiler-controlled layout
- pointer-only `c_struct` support with target c layout
- complete `c_struct` support with target c layout and pointer-only `opaque` records
- `Some :: c_struct { ... }`: defined c-layout struct
- `Some :: c_struct`: opaque c-layout struct
- `Some :: opaque`: incomplete nominal record
- passing c structs by value was deferred until milestone 4.1
2. restricted c header imports (implemented)
@@ -164,12 +164,19 @@
- added a native type-alias declaration `Name :: alias T` (parser/lexer/token surface; the
`types.define_alias` / `.Alias` machinery already existed) so C typedefs and callback
typedefs round-trip
- emits functions, complete/opaque structs (collapsing `typedef struct {...} Foo`),
- emits functions, complete structs and opaque records (collapsing `typedef struct {...} Foo`),
typedef aliases, and scalar/aggregate/enum-member constants
- C unions, external variables, static-inline functions, and unsupported declarations have
no hand-writable spelling and are emitted as `# unsupported in bindings:` comments
(functions that reference an un-spellable union therefore keep a dangling reference)
11.1. opaque, anyopaque, and pointer casts (implemented; v1)
- `Name :: opaque` is the incomplete nominal record spelling; bodyless `c_struct` is invalid
- `anyopaque` is the erased object type used behind pointers for C `void*` and allocator contexts
- C `void` function results remain `void`; C `void*` / `const void*` import and render as `?*mut anyopaque` / `?*anyopaque`
- `ptr_cast(T, ptr)` preserves pointer shape and only changes the child type in v1
- future direction: generalize toward Zig-style arbitrary pointer-result casts once casts have a broader result-type story
12. `undefined` as inspired by zig (implemented):
- allow mutable local declarations with `undefined`
- undefined values are assigned a poison value (0xaa...)
@@ -620,10 +627,11 @@
parameters
25. dynamic heap allocation (implemented; v1)
- `std/mem/heap` is a tiny relative-importable package over libc `malloc`/`free`
- `heap.alloc(size usize) ?*mut u8` returns nullable mutable byte memory; callers use existing optional unwraps
- `heap.free(ptr ?*mut u8) void` forwards to C `free`, including `none` / null
- typed allocation, allocator parameters, arenas/pools, build-mode heap policy, and escaping-allocation diagnostics remain deferred
- `std/mem` exposes a plain-data `Allocator` contract with `?*mut anyopaque` context, `alloc`, and `free` `@func` pointers
- `mem.heap` is the default libc-backed allocator; `mem.alloc(mem.heap, size, alignment)` returns nullable mutable byte memory
- `mem.free(mem.heap, ptr, size, alignment)` frees with the same allocator; size/alignment are part of the contract, though libc ignores them in v1
- `std/mem/heap` remains as legacy compatibility wrappers over `std/mem`
- typed allocation helpers, arenas/pools, build-mode heap policy, and escaping-allocation diagnostics remain deferred
26. import from project "root" (implemented)
- imports beginning with `@` resolve from the project root
@@ -705,7 +713,7 @@
- comptime storage pointers/slices cannot materialize as runtime memory; escaped
dead storage is rejected
- bare concrete non-comptime function names are values; native function pointer
types use `*func(...) R` and fallible `*func(...) R ! E`
types use `@func(...) R` and fallible `@func(...) R ! E`
- comptime-known native/bodyful `c_func` values can be called; bodyless/imported
callbacks remain runtime-only
- native function pointers are non-variadic v1; C variadic function pointers stay
@@ -1323,8 +1331,25 @@ total :: $sum_loop(4) # implemented: mutable locals/loops/def
## A word on memory allocation
(NOTE THAT SYNTAX MAY NOT MATCH BROLANG EXACTLY AND SHOULD BE TAKEN WITH A GRAIN OF SALT - INSPIRATION ONLY)
(FURTHER, EXAMPLES ASSUME ARGUMENTS WITH DEFAULT VALUES AND COMPTIME POLYMORPHISM IN THE FORM OF GENERIC TYPE PARAMETERS - MONOMORPHISED)
Current v1 is intentionally byte-oriented and plain data:
```
mem :: import "@std/mem"
bytes := mem.alloc(mem.heap, 128, 1)
defer mem.free(mem.heap, bytes, 128, 1)
```
`@std/mem` is the preferred allocator API. `@std/mem/heap` remains as legacy compatibility wrappers over the same heap allocator:
```
heap :: import "@std/mem/heap"
bytes := heap.alloc(128)
defer heap.free(bytes)
```
Typed allocation helpers, arenas, pools, build-mode heap policy, and escaping-allocation diagnostics are future work. Older examples below are design sketches where noted, not committed syntax.
Memory allocation in Brolang is designed to be **explicit but not verbose**. We reject the dogma that global state is inherently evil — allocators are a cross-cutting concern that nearly every function needs, making them a perfect candidate for sensible defaults.
@@ -1345,26 +1370,26 @@ Memory allocation in Brolang is designed to be **explicit but not verbose**. We
### The Default Heap Allocator
Brolang provides a **thread-local global heap allocator** that is:
Brolang provides a default heap allocator value:
* Determined at compile time by build mode
* Available as `mem.heap`
* Passed explicitly to `mem.alloc` and `mem.free`
* **Immutable at runtime** — cannot be reconfigured
```
import "std/mem/heap"
mem :: import "@std/mem"
process func(input []u8) u64 {
# heap used for internal temporary work — does not escape
temp := heap.alloc(u8, size: input.len * 2)
defer heap.free(temp)
temp ?*mut u8 = mem.alloc(mem.heap, input.len * 2, 1)
defer mem.free(mem.heap, temp, input.len * 2, 1)
# ... work with temp ...
return compute_hash(temp) # only the result escapes, not the allocation
return compute_hash(temp)
}
```
The behavior of `heap` depends on build mode:
Future build modes can choose different implementations behind `mem.heap` without changing the allocator contract:
| Build Mode | Allocator Behavior |
| -- | -- |
@@ -1372,7 +1397,7 @@ The behavior of `heap` depends on build mode:
| Release | Fast allocator, zero overhead |
| ReleaseSafe | Bounds-checking allocator |
This is configured at compile time. You cannot change which allocator `heap` uses at runtime. This is intentional — it prevents bugs where memory allocated with one allocator is freed with another.
That policy is configured at compile time. You cannot change which allocator `mem.heap` uses at runtime. This is intentional — it prevents bugs where memory allocated with one allocator is freed with another.
### The Escaping Allocation Rule
@@ -1381,42 +1406,41 @@ This is configured at compile time. You cannot change which allocator `heap` use
This rule makes ownership transfer visible at the function signature level. The caller never needs to read the function's implementation to know whether heap cleanup is involved:
```
import "std/mem"
import "std/mem/heap"
mem :: import "@std/mem"
# Allocation escapes via return value — requires allocator
duplicate func(input []u8, allocator @mem.Allocator) []u8 {
result := allocator.alloc(u8, size: input.len)
duplicate func(input []u8, allocator mem.Allocator) ?*mut u8 {
result ?*mut u8 = mem.alloc(allocator, input.len, 1)
mem.copy(result, input)
return result # caller manages this memory
}
# Allocation escapes via mutable parameter — requires allocator
init func(obj: @mut MyStruct, allocator: @mem.Allocator) void {
obj.buffer = allocator.alloc(u8, size: 100)
init func(obj @mut MyStruct, allocator mem.Allocator) void {
obj.buffer = mem.alloc(allocator, 100, 1)
# caller now knows heap memory was written into obj
}
# No allocation escapes — no allocator needed
process func(input: []u8) u64 {
temp := heap.alloc(u8, size: input.len)
defer heap.free(temp)
process func(input []u8) u64 {
temp ?*mut u8 = mem.alloc(mem.heap, input.len, 1)
defer mem.free(mem.heap, temp, input.len, 1)
# ... work with temp ...
return compute_hash(temp)
}
# No heap allocation at all — no allocator needed
reset func(obj: @mut MyStruct) void {
reset func(obj @mut MyStruct) void {
obj.count = 0
}
main func() void {
data := duplicate("hello", heap)
defer heap.free(data)
data := duplicate("hello", mem.heap)
defer mem.free(mem.heap, data, 5, 1)
mut obj := MyStruct{ ... }
init(&obj, heap)
defer heap.free(obj.buffer)
init(&obj, mem.heap)
defer mem.free(mem.heap, obj.buffer, 100, 1)
}
```
@@ -1424,9 +1448,9 @@ main func() void {
* Without the rule, a function like `init(obj: @mut MyStruct) void` is ambiguous — did it heap-allocate into `obj`, or just set some fields to stack/static data? The caller has no way to know without reading the implementation.
* With the rule, the allocator parameter is a clear signal: "this function produces heap memory that outlives its scope, and you are responsible for cleaning it up."
* Internal allocations (temporary buffers, scratch space) use `heap` directly and are freed before the function returns. No allocator parameter needed, no burden on the caller.
* Internal allocations (temporary buffers, scratch space) use `mem.heap` directly and are freed before the function returns. No allocator parameter needed, no burden on the caller.
**The compiler enforces this rule.** If a function heap-allocates memory that escapes without accepting an allocator parameter, the compiler emits an error.
The compiler should eventually enforce this rule. If a function heap-allocates memory that escapes without accepting an allocator parameter, the compiler should emit an error.
### Why Immutable Defaults?
@@ -1437,16 +1461,16 @@ Consider what would happen if you could reconfigure the default allocator:
mem.heap_set(my_custom_heap)
# Somewhere else in the codebase...
data := heap.alloc(u8, size: 100)
data := mem.alloc(mem.heap, 100, 1)
# Later, someone changes it again...
mem.heap_set(different_heap)
# Now who frees `data`? With which allocator?
heap.free(data) # 💥 Wrong allocator - undefined behavior!
mem.free(mem.heap, data, 100, 1) # wrong allocator - undefined behavior
```
This is "action at a distance" — the behavior of `heap.free()` depends on what some unrelated code did earlier. By making `heap` immutable, Brolang guarantees:
This is "action at a distance" — the behavior of `mem.free(mem.heap, ...)` depends on what some unrelated code did earlier. By making `mem.heap` immutable, Brolang guarantees:
**Whatever you allocate with, you free with.**
@@ -1454,15 +1478,16 @@ This is "action at a distance" — the behavior of `heap.free()` depends on what
For specialized needs, you create explicit allocator instances. These are not global — you manage their lifetime and pass them where needed.
The examples in this section are future typed API sketches. The v1 allocator contract is still `mem.Allocator` plus byte-oriented `mem.alloc`/`mem.free`.
**Arena Allocator**: Fast bump allocation, bulk deallocation:
```
import "std/mem"
import "std/mem/heap"
mem :: import "@std/mem"
process_file func(path: []u8, allocator: @mem.Allocator) !Data {
process_file func(path []u8, allocator mem.Allocator) !Data {
# arena manages its own backing memory via heap
arena := mem.Arena.init(heap, capacity: mem.megabytes(1))
arena := mem.Arena.init(mem.heap, capacity: mem.megabytes(1))
defer arena.deinit()
# all temporary allocations from arena (fast bump allocation)
@@ -1484,14 +1509,13 @@ process_file func(path: []u8, allocator: @mem.Allocator) !Data {
**Pool Allocator**: O(1) fixed-size allocation, no fragmentation:
```
import "std/mem"
import "std/mem/heap"
mem :: import "@std/mem"
EntitySystem :: struct {
pool: mem.Pool(Entity),
}
init_entities func(allocator: @mem.Allocator) EntitySystem {
init_entities func(allocator mem.Allocator) EntitySystem {
return EntitySystem{
pool = mem.Pool(Entity).init(allocator, capacity: 10_000),
}
@@ -1511,23 +1535,23 @@ despawn func(sys: @mut EntitySystem, entity: @Entity) void {
As described in the escaping allocation rule, when a function heap-allocates memory that escapes its scope, it must accept an allocator parameter. The caller decides which allocator to use:
```
import "std/mem"
mem :: import "@std/mem"
# Function that uses caller's allocator
parse func(input: []u8, allocator: @mem.Allocator) !ParseResult {
buffer := allocator.alloc(u8, size: input.len)
defer allocator.free(buffer)
parse func(input []u8, allocator mem.Allocator) !ParseResult {
buffer := mem.alloc(allocator, input.len, 1)
defer mem.free(allocator, buffer, input.len, 1)
# ... parse into buffer ...
result := allocator.alloc(ParseResult) # size defaults to 1
result := mem.alloc(allocator, parse_result_size, parse_result_alignment)
return result
}
# Caller decides which allocator to use
main func() void {
# use an arena for this parsing work
arena := mem.Arena.init(heap, capacity: mem.kilobytes(64))
arena := mem.Arena.init(mem.heap, capacity: mem.kilobytes(64))
defer arena.deinit()
result := parse(input, &arena) catch |err| {
# handle error
@@ -1546,15 +1570,15 @@ main func() void {
| What | How | When to Use |
| -- | -- | -- |
| `heap.alloc(T, size: n)` | Thread-local global | General purpose, 90% of cases |
| `heap.create(T)` | Thread-local global | Allocate single item |
| `allocator.alloc(T, size: n)` | Caller-provided | Escaping allocations (returned or written to caller's data) |
| `arena.alloc(T, size: n)` | Explicit instance | Temporary/scoped work, bulk free |
| `pool.alloc()` | Explicit instance | Many same-sized objects, O(1) |
| `mem.alloc(mem.heap, n, a)` | Preferred heap allocator | General purpose byte allocation |
| `mem.free(mem.heap, ptr, n, a)` | Preferred heap allocator | Free byte allocation with original size/alignment |
| `heap.alloc(n)` / `heap.free(ptr)` | Legacy wrapper | Compatibility with older `@std/mem/heap` code |
| `mem.alloc(allocator, n, a)` | Caller-provided allocator | Escaping allocations (returned or written to caller's data) |
| typed helpers / arenas / pools | Future APIs | Higher-level allocation patterns |
Note that `heap` satisfies the `Allocator` interface, so callers can pass `heap` as the allocator argument when they don't need a specialized allocator — which is most of the time.
Note that `mem.heap` is a `mem.Allocator`, so callers can pass it as the allocator argument when they don't need a specialized allocator — which is most of the time.
**The golden rule:** Allocate and free with the same allocator. The type system helps enforce this — memory from `heap` can only be freed with `heap`, memory from your arena can only be freed with that arena. The escaping allocation rule ensures the caller always knows which allocator was used.
**The golden rule:** Allocate and free with the same allocator. In v1 this is explicit in the call sites; future diagnostics should use the escaping allocation rule to ensure the caller always knows which allocator was used.
### Compared to Other Languages