Foundry/README.md

# Foundry

Pure C implementations of 11 core Python ML tools. CPU is the baseline; GPU backends exist for CUDA, Metal and Tenstorrent Blackhole (see [GPU Support](#gpu-support)). Vulkan is planned.

## Why Foundry?

The Python ML ecosystem is powerful but has limitations:

- **Dependency hell** - Complex Python environments, version conflicts

- **Not embeddable** - Can't easily integrate training into native apps

- **Performance overhead** - Python GIL, interpreter overhead

- **Deployment complexity** - Requires Python runtime everywhere

Foundry provides C implementations that can be:

- Embedded in any application

- Compiled to a single binary

- Used from any language via C FFI

- Deployed without Python

## The Tools

We identified 11 core Python tools that everything else builds on. Each has alternatives, but these are the essential functions:

| Foundry Module | Python counterpart | Function                               |
| -------------- | --------------- | ----------------------------------------- |
| tensor/        | PyTorch         | Tensor operations + autograd              |
| optim/         | PyTorch         | Optimizers (SGD, Adam) + LR schedulers    |
| distributed/   | DeepSpeed       | Gradient accumulation, data sharding (single-process; real distributed planned) |
| lora/          | PEFT            | LoRA and adapter training                 |
| tokenizer/     | tokenizers      | BPE tokenization (WordPiece planned)      |
| data/          | datasets        | Dataset loading and processing            |
| safetensors/   | safetensors     | Safe model weight serialization           |
| attention/     | PyTorch attention | Standard (unfused) multi-head attention, KV cache, RoPE, TurboQuant KV compression; no flash kernel yet |
| quantization/  | bitsandbytes    | Model quantization (int8, int4)           |
| eval/          | lm-eval-harness | Model evaluation and benchmarks           |
| progress/      | tqdm            | Progress reporting                        |

## Why These Tools?

Many Python ML tools are alternatives or convenience wrappers:

**Frameworks** - PyTorch vs TensorFlow vs JAX (pick one) **Distributed** - DeepSpeed vs Megatron vs FairScale (pick one) **LoRA** - PEFT vs Unsloth vs Axolotl (PEFT is the core, others wrap it) **Tokenizers** - tokenizers vs SentencePiece vs tiktoken (same function)

We chose the most fundamental tool in each category. Everything else can be built on top.

## Architecture

```
┌─────────────────────────────────────────────────────────────────┐
│                      Your Application                           │
├─────────────────────────────────────────────────────────────────┤
│                       Foundry C API                             │
├─────────────────────────────────────────────────────────────────┤
│  tensor/  │  lora/  │  tokenizer/  │  data/  │  eval/  │  ...  │
├─────────────────────────────────────────────────────────────────┤
│        Backends: CPU · CUDA · Metal · Tenstorrent Blackhole     │
│                       (Vulkan planned)                          │
└─────────────────────────────────────────────────────────────────┘
```

## Quick Start

**Prerequisites:** a C11 compiler and `make`. BLAS is optional (`USE_BLAS=1`: Accelerate on
macOS, OpenBLAS on Linux). The default flags include `-march=native`, so build on the machine
that will run the code, or add `PORTABLE=1` for a copyable build (`-march=x86-64-v2` on x86-64).

```
# Build all modules
make all

# Run all CPU tests (483 tests, 27 suites)
make test

# Run training example
make train
```

## Training Example

```
$ make train

===========================================
  Foundry Training Example: XOR Problem
===========================================

Model: 2-layer MLP (2 -> 8 -> 1)
Task: Learn XOR function
Learning rate: 1.00
Epochs: 1000

Epoch    1 | Loss: 0.265296
Epoch  500 | Loss: 0.001784
Epoch 1000 | Loss: 0.000652

Predictions vs Targets:
  0 XOR 0 = 0.042 (target: 0) [OK]
  0 XOR 1 = 0.980 (target: 1) [OK]
  1 XOR 0 = 0.986 (target: 1) [OK]
  1 XOR 1 = 0.015 (target: 0) [OK]

Accuracy: 4/4 (100%)
[SUCCESS] Model learned XOR perfectly!
```

## Status

| Module        | Status   | Tests | Description                                      |
| ------------- | -------- | ----- | ------------------------------------------------ |
| tensor/       | Complete | 141   | Tensor ops with autograd (PyTorch parity: 80/83) |
| optim/        | Complete | 15    | SGD, Adam, AdamW, LR schedulers                  |
| progress/     | Complete | 18    | Training progress bars & logging                 |
| tokenizer/    | Complete | 18    | BPE tokenization                                 |
| safetensors/  | Complete | 18    | Safe tensor serialization                        |
| lora/         | Complete | 15    | Low-rank adaptation                              |
| data/         | Complete | 19    | Dataset loading & batching                       |
| quantization/ | Complete | 57    | INT8/INT4 quantization, export (test_quantization 43 + test_export 14) |
| attention/    | Complete | 23    | Multi-head attention, KV cache, RoPE, TurboQuant KV compression (fa_tq_) (test_attention 12 + test_turboquant 11) |
| distributed/  | Complete | 18    | Gradient accumulation, data sharding             |
| eval/         | Complete | 18    | Perplexity, accuracy, F1 metrics                 |

**Core modules: 360 tests.** `make test` also runs the extension suites — hdc 18, hybridmem 20,
threadpool 7, rwkvx 26 (7 suites), rosa 11, barswap 14, pth 10, ternarize 17 — for **483 tests
across 27 suites, all passing** (macOS, 2026-10-09). The rwkvx suites need the RWKV vocabulary,
which `make test` downloads once via `rwkvx/data/fetch_rwkv_vocab.sh` (network, `curl`,
`sha256sum` or `shasum`). Hardware suites are separate: `make USE_CUDA=1 test-cuda`,
`make USE_METAL=1 test-metal`, and the Tenstorrent suites in `tt/tests/` (see
[`tt/README.md`](./tt/README.md)); each needs the hardware.

**PyTorch numerical parity: 80/83 comparison tests (tensor module)**

## Custom Extensions

| Module | Status   | Description                                  |
| ------ | -------- | -------------------------------------------- |
| swarm/ | Complete | Distributed CPU training across Docker Swarm |
| ternary_optim/ | Reference (CPU) | Memory-neutral **ternary** optimizer (SR4). Trains ternary weights (`{-1,0,+1}`) via a 4-bit signed latent accumulator + stochastic rounding — the *entire* per-weight training state is 4 bits (~24× less optimizer state than Adam), no FP32 master weight or momentum. In the research prototype it was extracted from, it reached 98.3% on an MNIST 784-256-10 MLP, matching that MLP's FP32-dense result. Standalone CPU reference library; not yet wired into `optim/`. See `ternary_optim/README.md`. |

These are project-specific additions built on top of the core Foundry modules. See `swarm/README.md` and `ternary_optim/README.md` for details.

## Other Modules

Modules in the tree that are not in the 11-tool table:

| Module | What it is |
| ------ | ---------- |
| rwkvx/ | RWKV-X hybrid LM (RWKV-7 + sparse attention): inference, training, backward passes |
| rwkvx/rfa.* | RWKV-7 + full-attention (Qwen2-style GQA) hybrid inference and training |
| convert/ | HuggingFace safetensors → `.btw`/`.btv` converter for Llama-family models |
| pth/ | PyTorch `.pth` (ZIP + pickle) state_dict reader, zero-copy via mmap |
| threadpool/ | Minimal pthread `parallel_for` (`USE_THREADS=1`) |
| ternarize/ | Post-training ternarization of FP16/BF16 models, layer by layer |
| rosa/ | Suffix automaton (binary and vocab-sized alphabets) |
| hdc/ | Hyperdimensional computing: binary hypervectors, Hamming-distance search |
| hybridmem/ | Disk-backed K/V memory tier for RWKV-X, indexed with hdc/ |
| barswap/ | K80 die time-multiplexing through one PCIe BAR slot on a specific board (BTC-S37) |
| swarm/ | Distributed CPU training across Docker Swarm (`make test-swarm`, not in `make test`) |
| ternary_optim/ | SR4 ternary optimizer, standalone CPU reference |
| dedup/ | MinHash/LSH deduplication — interface only (header + README), not implemented |
| retrieval/ | Hash-based document retrieval — interface only (header + README), not implemented |

## Build Options

```
# Build specific module
make tensor
make lora

# Run specific test suite
make test-tensor
make test-attention

# Debug build
make DEBUG=1 all

# Clean
make clean
```

## GPU Support

CPU is the baseline — every operation works on CPU first, and the GPU backends
extend them without changing the public API.

- **CUDA** (`cuda/`, `cuda_kernels/`) — loaded via `dlopen`, kernels compiled to
  PTX and embedded, so Foundry builds without nvcc present. Targets `sm_37`
  (Kepler) upward; that is deliberate, since most current runtimes have dropped
  below-Pascal support.
- **Metal** (`metal/`) — Apple Silicon.
- **Tenstorrent Blackhole** (`tt/`) — a p150a's Tensix cores, through Foundry's own firmware,
  kernels and userspace driver, plain C. The backend was rewritten from public documentation and
  measured hardware behaviour; no tt-metal, TT-NN, tt-umd or tt-llk source is compiled, linked
  or vendored. The only dependency is the
  SFPI compiler, plus `python3` for the `tt/tools/emit_tensix_*.py` generators, and only to
  rebuild the device images; a Foundry build needs neither. Kernels are hand-written reader/compute/writer triads compiled ahead of time
  and embedded, so Foundry builds without the toolchain present. 8 op families run on
  silicon, plus Gated DeltaNet; multi-core matmul exists.
  [Pantheon](https://git.teamide.dev/David/Pantheon.git) runs a 9B `qwen35` model end-to-end
  on the card through this backend. See
  [`tt/README.md`](./tt/README.md) and [`docs/tenstorrent.md`](./docs/tenstorrent.md).
- **Vulkan** — planned.

Contributions to the GPU backends are very welcome.

## License

Foundry is licensed under the **GNU AGPL-3.0** — see [`LICENSE`](./LICENSE).

Contributions are accepted under the CLA in [`CLA.md`](./CLA.md), with a DCO
sign-off on every commit — see [`CONTRIBUTING.md`](./CONTRIBUTING.md). You keep
your copyright; signing is one entry in
[`.cla/signatures.md`](./.cla/signatures.md), added in your first pull request.

Copyright © 2026 David (Corey) Herrera and the Foundry contributors. See
[`NOTICE`](./NOTICE).