comptime storage and function values

LANGUAGE.md

@@ -55,9 +55,10 @@ roadmap and milestone history.
 - integer comptime value parameters such as `make_array func($N usize) [N]u8`, specialized by value and omitted from the runtime ABI
 - explicit comptime type parameters such as `max func($T type, a, b T) T`, specialized by type and omitted from the runtime ABI
 - forced typed comptime expressions such as `$sum(1, 2)`, `$Point { x = 1, y = 2 }`, and comptime value blocks such as `${ yield 4 }`
-- comptime execution for bodyful Brolang functions with mutable locals, loops, `defer`, `match`, and `try`/`catch`
+- comptime execution for bodyful Brolang functions with mutable locals, loops, `defer`, `match`, `try`/`catch`, pointer/slice storage mutation, pointer captures, and calls through comptime-known function values
 - bodyful `c_func` definitions and bodyless `c_func` declarations with exact external symbol names
 - concrete-only C signatures, C variadic declarations/calls, and C default argument promotions
+- native function pointer values and types with `*func(...) R`, fallible `*func(...) R ! E`, optional `?*func(...) R`, and non-variadic native indirect calls
 - Apple Silicon C ABI lowering for scalars, pointers, fixed-signature plain records/unions, small aggregates, homogeneous float aggregates, and indirect aggregate returns
 - imported C typedefs, scalar constants, enum constants, fixed arrays, complete plain structs/unions, and pointers to opaque records
 - imported external C object variables, including mutable variables and immutable object globals
@@ -83,7 +84,7 @@ roadmap and milestone history.
 
 ## PLANNED / DEFERRED
 
-- comptime pointers, slices, aggregate comptime parameters, and calls through comptime-known function values/function pointers
+- aggregate comptime parameters and stable aggregate specialization keys
 - tuples and native Brolang variadic functions
 - exporting Brolang functions to C and broader target-specific C ABI lowering
 - non-plain C record layouts such as bitfields, packed records, flexible arrays, qualified fields, and C variadic record arguments

README.md

@@ -74,6 +74,15 @@ call_mapper func(mapper native.Imported_Mapper) c_int {
 }
 ```
 
+Native Brolang function pointer values use `*func(...) R`, with fallible
+channels written on the result:
+
+```bro
+call func(callback *func(value i32) i32, value i32) i32 {
+	return callback(value)
+}
+```
+
 Bodyless manual and imported C functions may be variadic:
 
 ```bro
@@ -138,7 +147,8 @@ Current prototype features:
 - Demand-monomorphized Brolang and C-ABI functions
 - Integer and type comptime parameters (`func($N usize) [N]u8`, `func($T type, value T) T`) specialized by comptime argument
 - Forced typed comptime expressions (`$sum(1, 2)`, `$Point { x = 1, y = 2 }`) and comptime value blocks (`${ yield 4 }`)
-- Comptime execution for bodyful Brolang functions with mutable locals, loops, `defer`, `match`, and `try`/`catch`
+- Comptime execution for bodyful Brolang functions with mutable locals, loops, `defer`, `match`, `try`/`catch`, pointer/slice storage mutation, pointer captures, and calls through comptime-known function values
+- Native function pointer values and types (`*func(...) R`, `*func(...) R ! E`, `?*func(...) R`)
 - Bodyless concrete C function declarations with exact external symbol names
 - Bodyless manual and imported C variadic declarations with default argument promotions
 - Ordered linking of additional C sources, objects, archives, and libraries

TODO.md

@@ -676,11 +676,9 @@
   - immutable locals/globals with comptime-known initializers may feed comptime
     evaluation; runtime-dependent values remain invalid in comptime contexts
   - runtime-only behavior is rejected in comptime: external/bodyless `c_func`,
-    writable globals, pointers/slices, address/deref storage APIs, pointer captures,
+    writable globals, and materializing comptime storage pointers/slices as runtime memory
-    and function-pointer calls
+  - v1 keeps integer-only `$N` specialization keys; aggregate comptime parameters
-  - v1 keeps integer-only `$N` specialization keys; aggregate comptime parameters,
+    and stable aggregate serialization are deferred
-    stable aggregate serialization, comptime pointers/slices, and calls through
-    comptime-known function values/function pointers are deferred
 
 27.8 source-defined mutable runtime globals (implemented)
   - allow mutable global declarations in Brolang source for process-global runtime
@@ -698,15 +696,19 @@
   - reject user-visible name shadowing across imports, named types, globals, functions,
     params, locals, comptime params, captures, and labels; `_` remains reusable
 
-27.9 comptime storage and function values
+27.9 comptime storage and function values (implemented; practical v1)
-  - add a comptime pointer/storage model for pointers, slices, address/deref,
+  - comptime locals, params, and immutable globals can own evaluator storage cells
-    pointer captures, lifetimes, aliasing, mutability, and escape rules
+    addressable through places instead of compiler-owned memory
-  - define what `$&value` and other comptime addresses can legally materialize into,
+  - comptime supports address/deref, mutable pointer and slice mutation, field/index
-    without exposing compiler-owned memory as runtime memory
+    places, slicing, `.len`, `.ptr`, pointer captures, and pointer-param aliasing
-  - add first-class comptime function values and calls through comptime-known function
+  - comptime storage pointers/slices cannot materialize as runtime memory; escaped
-    pointers
+    dead storage is rejected
-  - resolve function-pointer targets during comptime execution, apply ABI/runtime
+  - bare concrete non-comptime function names are values; native function pointer
-    restrictions, and reliably reject imported/runtime callbacks
+    types use `*func(...) R` and fallible `*func(...) R ! E`
+  - comptime-known native/bodyful `c_func` values can be called; bodyless/imported
+    callbacks remain runtime-only
+  - native function pointers are non-variadic v1; C variadic function pointers stay
+    under `*c_func(...) R`
 
 28. brolang build system (requires comptime execution)
 

compiler/checker/checker.odin

@@ -903,13 +903,16 @@ function_value_signature :: proc(
 		return nil, types.INVALID, false
 	}
 	function := checker.ast_module.functions[template]
-	if !function.c_abi || types.is_valid(function.error) {
-		return nil, types.INVALID, false
-	}
 	if function_has_comptime_params(function) {
 		return nil, types.INVALID, false
 	}
-	result = type_from_syntax(checker, function.result, function.pkg, function.file)
+	if function.c_abi && types.is_valid(function.error) {
+		return nil, types.INVALID, false
+	}
+	if !function.c_abi && (!function.has_body || function.variadic) {
+		return nil, types.INVALID, false
+	}
+	result = function_channel_type(checker, function)
 	if !types.is_void(result) && !is_runtime_type(checker, result) {
 		return nil, types.INVALID, false
 	}
@@ -929,6 +932,7 @@ function_pointer_type_for_template :: proc(
 	checker: ^Checker,
 	template: ast.Function_Id,
 	demanded: ^[dynamic]Spec_Id = nil,
+	demand_spec := true,
 ) -> (types.Type, Spec_Id, bool) {
 	params, result, ok := function_value_signature(checker, template)
 	if !ok {
@@ -936,16 +940,20 @@ function_pointer_type_for_template :: proc(
 	}
 	defer delete(params, checker.allocator)
 	function := checker.ast_module.functions[template]
-	function_type := types.function(&checker.module.types, params, result, true, function.variadic)
+	function_type := types.function(&checker.module.types, params, result, function.c_abi, function.variadic)
 	pointer_type := types.pointer(&checker.module.types, function_type, false, true)
 	spec := INVALID_SPEC
 	if demanded == nil {
-		spec = ensure_spec(checker, template, params)
+		if demand_spec {
+			spec = ensure_spec(checker, template, params)
+		} else {
+			spec = find_spec(checker, template, params)
+		}
 	} else {
 		spec = find_spec(checker, template, params)
 		mark_spec_demanded(checker, spec, demanded)
 	}
-	return pointer_type, spec, spec != INVALID_SPEC
+	return pointer_type, spec, spec != INVALID_SPEC || !demand_spec
 }
 
 contains_name :: proc(names: []symbol.Id, name: symbol.Id) -> bool {
@@ -1391,16 +1399,30 @@ validate_type_nodes :: proc(checker: ^Checker) {
 			}
 		}
 		if item.kind == .Function {
-			if !item.c_abi {
+			if item.c_abi {
-				source.add(checker.diagnostics, source.Span{}, "only c_func function pointer types are supported")
+				if types.kind(item.child, &checker.module.types) == .Fallible {
-			}
+					source.add(checker.diagnostics, source.Span{}, "c_func pointer results cannot be fallible")
-			for param in types.params_for(&checker.module.types, id) {
+				}
-				if types.is_void(param.type) || !types.is_c_signature_type(param.type, &checker.module.types) {
+				for param in types.params_for(&checker.module.types, id) {
-					source.add(checker.diagnostics, source.Span{}, "function pointer parameters must be concrete C signature types")
+					if types.is_void(param.type) || !types.is_c_signature_type(param.type, &checker.module.types) {
+						source.add(checker.diagnostics, source.Span{}, "c_func pointer parameters must be concrete C signature types")
+					}
+				}
+				if !types.is_c_signature_type(item.child, &checker.module.types, true) {
+					source.add(checker.diagnostics, source.Span{}, "c_func pointer results must be concrete C signature types or void")
+				}
+			} else {
+				if item.variadic {
+					source.add(checker.diagnostics, source.Span{}, "native function pointer types do not support variadic parameters")
+				}
+				for param in types.params_for(&checker.module.types, id) {
+					if types.is_void(param.type) || !is_runtime_type(checker, param.type) {
+						source.add(checker.diagnostics, source.Span{}, "native function pointer parameters must be concrete runtime types")
+					}
+				}
+				if !types.is_void(item.child) && !is_runtime_type(checker, item.child) {
+					source.add(checker.diagnostics, source.Span{}, "native function pointer results must be concrete runtime types or void")
 				}
-			}
-			if !types.is_c_signature_type(item.child, &checker.module.types, true) {
-				source.add(checker.diagnostics, source.Span{}, "function pointer results must be concrete C signature types or void")
 			}
 		}
 	}
@@ -1597,7 +1619,7 @@ infer_compound_expr :: proc(
 	store := &checker.module.types
 	#partial switch expr.kind {
 	case .Comptime:
-		return infer_comptime_expr_type(checker, expr, pkg, file)
+		return infer_comptime_expr_type(checker, expr, pkg, file, demanded)
 	case .Bool:
 		return types.BOOL
 	case .Not:
@@ -3535,13 +3557,13 @@ build_function_value :: proc(
 		id := source.addf(
 			checker.diagnostics,
 			span,
-			"function '%s' cannot be used as a C callback; expected a concrete c_func",
+			"function '%s' cannot be used as a function value; expected a concrete non-comptime signature",
 			symbol_text(checker, function.name),
 		)
 		return invalid_hir_expr(checker, span, id)
 	}
 	defer delete(params, checker.allocator)
-	function_type := types.function(&checker.module.types, params, result, true, function.variadic)
+	function_type := types.function(&checker.module.types, params, result, function.c_abi, function.variadic)
 	pointer_type := types.pointer(&checker.module.types, function_type, false, true)
 	spec := find_spec(checker, template, params)
 	if spec == INVALID_SPEC {
@@ -4444,7 +4466,7 @@ build_expr :: proc(
 						last = build_global_reference(checker, global, expr.span, global_reads)
 					} else {
 						template := find_template(checker, expr.name, target_pkg)
-						if template != ast.INVALID_FUNCTION && checker.ast_module.functions[template].c_abi {
+						if template != ast.INVALID_FUNCTION {
 							last = build_function_value(checker, template, expr.span, frame.expected)
 						} else {
 							id := add_unsupported_diagnostic(checker, expr.span, target_pkg, expr.name)

compiler/parser/parser.odin

@@ -113,7 +113,7 @@ is_type_token :: proc(kind: token.Kind) -> bool {
 	     .Keyword_C_Short, .Keyword_C_Ushort, .Keyword_C_Int, .Keyword_C_Uint,
 	     .Keyword_C_Long, .Keyword_C_Ulong, .Keyword_C_Longlong, .Keyword_C_Ulonglong,
 	     .Keyword_C_Float, .Keyword_C_Double, .Keyword_C_Longdouble,
-	     .Keyword_Void, .Keyword_Bool, .Keyword_C_Func, .Identifier, .Question, .At, .Star, .Left_Bracket:
+	     .Keyword_Void, .Keyword_Bool, .Keyword_Func, .Keyword_C_Func, .Identifier, .Question, .At, .Star, .Left_Bracket:
 		return true
 	}
 	return false
@@ -329,10 +329,11 @@ parse_type_atom :: proc(parser: ^Parser) -> ast.Type_Syntax {
 	case .Keyword_Bool:
 		advance(parser)
 		return types.BOOL
-	case .Keyword_C_Func:
+	case .Keyword_Func, .Keyword_C_Func:
+		c_abi := tok.kind == .Keyword_C_Func
 		advance(parser)
 		if _, ok := allow(parser, .Left_Paren); !ok {
-			source.add(parser.diagnostics, current(parser).span, "expected '(' after c_func type")
+			source.add(parser.diagnostics, current(parser).span, "expected '(' after function type")
 			return types.INVALID
 		}
 		params, variadic := parse_params(parser)
@@ -340,11 +341,19 @@ parse_type_atom :: proc(parser: ^Parser) -> ast.Type_Syntax {
 			source.add(parser.diagnostics, current(parser).span, "expected ')' after function type parameters")
 		}
 		result := parse_type(parser)
+		if _, ok := allow(parser, .Bang); ok {
+			error_type := parse_error_type(parser)
+			if c_abi {
+				source.add(parser.diagnostics, current(parser).span, "c_func pointer types cannot be fallible")
+			} else {
+				result = types.fallible(&parser.module.type_store, result, error_type)
+			}
+		}
 		param_types := make([]types.Type, len(params), parser.module.allocator)
 		for param, index in params {
 			param_types[index] = param.type
 		}
-		function_type := types.function(&parser.module.type_store, param_types, result, true, variadic)
+		function_type := types.function(&parser.module.type_store, param_types, result, c_abi, variadic)
 		delete(param_types, parser.module.allocator)
 		delete(params, parser.module.allocator)
 		return function_type

compiler_tests.odin

@@ -293,6 +293,43 @@ main func() void {}
 	testing.expect(t, params[0].type == types.C_INT)
 }
 
+@(test)
+parser_accepts_native_function_pointer_types :: proc(t: ^testing.T) {
+	text := `Error :: enum {
+	bad
+}
+take func(callback ?*func(value i32) i32, fallible *func() i32 ! Error) void
+main func() void {}
+`
+	source_file := source.Source{path="test.bro", text=text}
+	diagnostics := source.init_diagnostics(&source_file)
+	defer source.destroy_diagnostics(&diagnostics)
+	symbols := symbol.init_table()
+	defer symbol.destroy_table(&symbols)
+	stream := lexer.lex(&source_file, &diagnostics, &symbols)
+	defer delete(stream.items)
+	module := parser.parse(&stream, &source_file, &diagnostics)
+	defer ast.destroy_module(&module)
+
+	optional, optional_ok := types.node(&module.type_store, module.functions[0].params[0].type)
+	pointer, pointer_ok := types.node(&module.type_store, optional.child)
+	function, function_ok := types.node(&module.type_store, pointer.child)
+	params := types.params_for(&module.type_store, pointer.child)
+	fallible_pointer, fallible_pointer_ok := types.node(&module.type_store, module.functions[0].params[1].type)
+	fallible_function, fallible_function_ok := types.node(&module.type_store, fallible_pointer.child)
+	fallible, fallible_ok := types.node(&module.type_store, fallible_function.child)
+	testing.expect_value(t, len(diagnostics.items), 0)
+	testing.expect(t, optional_ok && optional.kind == .Optional)
+	testing.expect(t, pointer_ok && pointer.kind == .Pointer && pointer.many && !pointer.mutable)
+	testing.expect(t, function_ok && function.kind == .Function && !function.c_abi && !function.variadic)
+	testing.expect(t, function.child == types.I32)
+	testing.expect_value(t, len(params), 1)
+	testing.expect(t, params[0].type == types.I32)
+	testing.expect(t, fallible_pointer_ok && fallible_pointer.kind == .Pointer)
+	testing.expect(t, fallible_function_ok && fallible_function.kind == .Function && !fallible_function.c_abi)
+	testing.expect(t, fallible_ok && fallible.kind == .Fallible && fallible.child == types.I32)
+}
+
 @(test)
 parser_accepts_c_function_pointer_alias_types :: proc(t: ^testing.T) {
 	text := `callback_alias :: alias ?*c_func(value i32) i32
@@ -882,13 +919,15 @@ main func() void {
 	defer hir.destroy_module(&hir_module)
 
 	found_global := false
-	found_function := false
+	found_old_function_error := false
 	for diagnostic in diagnostics.items {
 		found_global = found_global || strings.contains(diagnostic.message, "'value' is a global, not a function")
-		found_function = found_function || strings.contains(diagnostic.message, "'give' is a function, not a global value")
+		found_old_function_error =
+			found_old_function_error ||
+			strings.contains(diagnostic.message, "'give' is a function, not a global value")
 	}
 	testing.expect(t, found_global)
-	testing.expect(t, found_function)
+	testing.expect(t, !found_old_function_error)
 }
 
 @(test)
@@ -2354,6 +2393,10 @@ main func() void {
 	runtime i32 = 1
 	_ = $runtime
 	_ = $native()
+	_ = ${
+		callback :: native
+		yield callback()
+	}
 	_ = $&GLOBAL
 	_ = $spin()
 	_ = $missing()
@@ -2375,7 +2418,7 @@ main func() void {
 	defer hir.destroy_module(&hir_module)
 
 	found_runtime := false
-	found_external := false
+	runtime_only_count := 0
 	found_pointer := false
 	found_quota := false
 	found_missing := false
@@ -2383,20 +2426,47 @@ main func() void {
 	for diagnostic in diagnostics.items {
 		message := diagnostic.message
 		found_runtime = found_runtime || strings.contains(message, "unresolved comptime value 'runtime'")
-		found_external = found_external || strings.contains(message, "runtime-only")
+		runtime_only_count += 1 if strings.contains(message, "runtime-only") else 0
-		found_pointer = found_pointer || strings.contains(message, "pointers and slices are not supported")
+		found_pointer = found_pointer || strings.contains(message, "comptime storage pointers and slices cannot materialize as runtime memory")
 		found_quota = found_quota || strings.contains(message, "comptime evaluation exceeded the step quota")
 		found_missing = found_missing || strings.contains(message, "did not return a value")
 		found_yield = found_yield || strings.contains(message, "comptime block must yield a value")
 	}
 	testing.expect(t, found_runtime)
-	testing.expect(t, found_external)
+	testing.expect(t, runtime_only_count >= 2)
 	testing.expect(t, found_pointer)
 	testing.expect(t, found_quota)
 	testing.expect(t, found_missing)
 	testing.expect(t, found_yield)
 }
 
+@(test)
+native_function_pointer_type_restrictions_are_diagnosed :: proc(t: ^testing.T) {
+	text := `main func() void {
+	callback *func(...) void = undefined
+}
+`
+	source_file := source.Source{path="test.bro", text=text}
+	diagnostics := source.init_diagnostics(&source_file)
+	defer source.destroy_diagnostics(&diagnostics)
+	symbols := symbol.init_table()
+	defer symbol.destroy_table(&symbols)
+	stream := lexer.lex(&source_file, &diagnostics, &symbols)
+	defer delete(stream.items)
+	ast_module := parser.parse(&stream, &source_file, &diagnostics)
+	defer ast.destroy_module(&ast_module)
+	hir_module := checker.check(&ast_module, &diagnostics, &symbols)
+	defer hir.destroy_module(&hir_module)
+
+	found_variadic := false
+	for diagnostic in diagnostics.items {
+		found_variadic =
+			found_variadic ||
+			strings.contains(diagnostic.message, "native function pointer types do not support variadic parameters")
+	}
+	testing.expect(t, found_variadic)
+}
+
 @(test)
 unused_function_signatures_are_validated_eagerly :: proc(t: ^testing.T) {
 	text := `broken func(value, value i8, nope void) void {}

examples/programs/comptime_v1/main.bro

@@ -74,6 +74,18 @@ may_fail func(flag bool) i32 ! Error {
 	return 7
 }
 
+increment func(value i32) i32 {
+	return value + 1
+}
+
+call_native func(callback *func(value i32) i32, value i32) i32 {
+	return callback(value)
+}
+
+call_fallible func(callback *func(flag bool) i32 ! Error, flag bool) i32 ! Error {
+	return try callback(flag)
+}
+
 use_try func() i32 ! Error {
 	value :: try may_fail(false)
 	return value + 1
@@ -87,6 +99,41 @@ recover func() i32 {
 	}
 }
 
+bump_ptr func(value @mut i32) void {
+	value^ += 1
+}
+
+alias_add func(left @mut i32, right @mut i32) void {
+	left^ += 2
+	right^ += 3
+}
+
+storage_mutation func() i32 {
+	values [3]mut i32 = [1, 2, 3]
+	values[0] += 1
+	bump_ptr(&values[1])
+
+	view []mut i32 = values[..]
+	for view |@item| {
+		item^ += 1
+	}
+
+	pointer *mut i32 = view.ptr
+	pointer[2] += 1
+	alias_add(&values[0], &view[0])
+
+	box Box = Box { point = Point { x = 2, y = 3 } }
+	match box {
+		.point |@p|: p.x += values[1]
+		.empty: values[0] = values[0]
+	}
+
+	if view.len != 3 {
+		return 0
+	}
+	return values[0] + view[1] + pointer[2] + box.point.x
+}
+
 GLOBAL :: $sum_loop(4)
 
 main func() i32 {
@@ -100,6 +147,15 @@ main func() i32 {
 	optional i32 :: $maybe(true)?
 	tried i32 :: $use_try() catch 0
 	recovered i32 :: $recover()
+	storage i32 :: $storage_mutation()
+	called i32 :: $call_native(increment, 11)
+	fallible_ok i32 :: $call_fallible(may_fail, false) catch 0
+	fallible_err i32 :: $call_fallible(may_fail, true) catch |e| {
+		result i32 :: match e {
+			.bad: 13
+		}
+		yield result
+	}
 
 	if point.x + point.y != 7 {
 		return 1
@@ -134,5 +190,20 @@ main func() i32 {
 	if GLOBAL != 8 {
 		return 11
 	}
+	if storage != 23 {
+		return 12
+	}
+	if called != 12 {
+		return 13
+	}
+	if call_native(increment, 20) != 21 {
+		return 14
+	}
+	if fallible_ok != 7 {
+		return 15
+	}
+	if fallible_err != 13 {
+		return 16
+	}
 	return 0
 }