Foundry/docs/architecture.md

Architecture

Overview of Foundry's design and architecture.

---

Design Goals

1. Zero Python Dependencies - Pure C, no Python runtime
2. Minimal External Dependencies - Only POSIX APIs
3. Memory Efficient - No unnecessary copies, mmap support
4. Embeddable - Static library, simple integration
5. Portable - x86_64 and ARM (Raspberry Pi)

---

Module Dependency Graph

                        ┌─────────┐
                        │ tensor  │ ◄── Foundation
                        └────┬────┘
                             │
         ┌───────────────────┼───────────────────┐
         │                   │                   │
    ┌────▼────┐        ┌────▼────┐        ┌────▼────┐
    │  optim  │        │attention│        │quantize │
    └────┬────┘        └────┬────┘        └─────────┘
         │                   │
         │              ┌────▼────┐
         │              │  lora   │
         │              └─────────┘
         │
    ┌────▼────┐
    │ swarm   │ ◄── Distributed training
    └─────────┘

         ┌─────────┐   ┌─────────┐   ┌─────────┐
         │tokenizer│   │safetens │   │  data   │ ◄── I/O modules
         └─────────┘   └─────────┘   └─────────┘

         ┌─────────┐   ┌─────────┐
         │progress │   │  eval   │ ◄── Utilities
         └─────────┘   └─────────┘

---

Backends

CPU is the baseline; every op works there first.

Backend	Directory	Reaches Foundry ops via	Notes
CPU	(core modules)	—	the reference path
CUDA	cuda/, cuda_kernels/	tensor.c / rwkvx.c dispatch	dlopen, embedded PTX, sm_37+
Metal	metal/	tensor.c / rwkvx.c dispatch	Apple Silicon, embedded MSL
Tenstorrent Blackhole	tt/	driven directly, no tensor dispatch	own userspace driver (tt-kmd optional), SFPI to rebuild kernels; kernels are reader/compute/writer triads on Tensix cores, embedded as ELF
Vulkan	—	—	planned

tt/ is the odd one out on purpose: there is no vendor runtime to hand a buffer to, so the backend owns the whole stack from device open to op dispatch. It is not yet wired into tensor/. See tenstorrent.md.

---

Memory Model

Tensor Memory

• Tensors own their data by default
• Reference counting for safe memory management
• Views share underlying data (no copy)
• Explicit tm_contiguous() forces copy when needed

tm_tensor* a = tm_randn(shape, 2, TM_F32);  // Owns data
tm_tensor* b = tm_transpose(a, 0, 1);       // View, shares data
tm_tensor* c = tm_contiguous(b);            // Copy, owns new data
tm_free(c);
tm_free(b);  // Only frees view struct, not data
tm_free(a);  // Frees data

Autograd Graph

• Operations create graph nodes
• Nodes hold references to inputs
• tm_backward() traverses graph in reverse
• Graph freed after backward (or via tm_detach())

---

Autograd Implementation

Forward:
    a ──┐
        ├── add_backward ── c
    b ──┘

c.grad_fn = {
    .backward = add_backward,
    .inputs = {a, b},
    .num_inputs = 2
}

Backward:
    tm_backward(c)
    ├── c.grad = 1.0
    └── c.grad_fn.backward(c)
        ├── a.grad += c.grad
        └── b.grad += c.grad

---

File Format: Safetensors

┌──────────────────────────────────────────┐
│ Header Length (8 bytes, little-endian)   │
├──────────────────────────────────────────┤
│ JSON Header (variable length)            │
│ {                                        │
│   "__metadata__": {...},                 │
│   "tensor_name": {                       │
│     "dtype": "F32",                      │
│     "shape": [1024, 1024],               │
│     "data_offsets": [0, 4194304]         │
│   },                                     │
│   ...                                    │
│ }                                        │
├──────────────────────────────────────────┤
│ Tensor Data (contiguous, aligned)        │
│ [tensor 1 data][tensor 2 data][...]      │
└──────────────────────────────────────────┘

---

Swarm Protocol

Wire Format

┌──────────────────────────────────────────┐
│ Magic (4 bytes): "SWRM"                  │
├──────────────────────────────────────────┤
│ Version (2 bytes)                        │
├──────────────────────────────────────────┤
│ Message Type (1 byte)                    │
├──────────────────────────────────────────┤
│ Flags (1 byte)                           │
├──────────────────────────────────────────┤
│ Payload Length (4 bytes)                 │
├──────────────────────────────────────────┤
│ Checksum (4 bytes, CRC32)                │
├──────────────────────────────────────────┤
│ Payload (variable length)                │
└──────────────────────────────────────────┘

Compression: TopK

Only send the K largest gradients by magnitude:

Original: [0.01, 5.0, 0.02, -3.0, 0.03, 2.0, 0.0, -1.0]
TopK (k=3):
  Indices: [1, 3, 5]
  Values:  [5.0, -3.0, 2.0]

Compression ratio: 3/8 = 37.5%
With k=10%: 90% bandwidth reduction

---

Threading Model

• Main thread: Coordinator event loop
• Worker threads: Forward/backward computation
• I/O threads: Network send/receive

Thread safety:

• Tensors are NOT thread-safe
• Coordinator uses mutex for worker list
• Worker uses mutex for state transitions

---

Build System

# Top-level Makefile structure
all: tensor optim tokenizer safetensors data progress \
     quantization attention lora distributed eval swarm

tensor:
    $(CC) -c tensor/tensor.c -o obj/tensor.o

# ... other modules ...

lib/libfoundry.a: $(ALL_OBJS)
    ar rcs $@ $^

---

Error Handling

• Functions return NULL or false on error
• No exceptions (C doesn't have them)
• Errors logged to stderr
• Assertions for programmer errors

tm_tensor* t = tm_zeros(shape, 2, TM_F32);
if (t == NULL) {
    fprintf(stderr, "Failed to allocate tensor\n");
    return 1;
}

---

Future Considerations

GPU Support

Planned abstraction:

typedef enum {
    TM_DEVICE_CPU,
    TM_DEVICE_CUDA,
    TM_DEVICE_METAL,
} tm_device;

tm_tensor* tm_zeros_on(shape, ndim, dtype, device);
tm_tensor* tm_to(tensor, device);

Python Bindings

If needed, via ctypes or cffi:

import ctypes

lib = ctypes.CDLL("libfoundry.so")
lib.tm_zeros.restype = ctypes.c_void_p
lib.tm_zeros.argtypes = [ctypes.POINTER(ctypes.c_int64), ctypes.c_int, ctypes.c_int]