David Herrera
775df8ab31
|
||
|---|---|---|
| .cla | ||
| .github | ||
| LICENSES | ||
| attention | ||
| barswap | ||
| convert | ||
| cuda | ||
| cuda_kernels | ||
| data | ||
| dedup | ||
| distributed | ||
| docs | ||
| eval | ||
| examples | ||
| hdc | ||
| hybridmem | ||
| lib | ||
| lora | ||
| metal | ||
| optim | ||
| progress | ||
| pth | ||
| quantization | ||
| retrieval | ||
| rosa | ||
| rwkvx | ||
| safetensors | ||
| scripts | ||
| swarm | ||
| tensor | ||
| ternarize | ||
| ternary_optim | ||
| threadpool | ||
| tokenizer | ||
| tools | ||
| tt | ||
| .gitignore | ||
| CHANGELOG.md | ||
| CLA-CORPORATE.md | ||
| CLA.md | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| Makefile | ||
| NOTICE | ||
| RADLADS.md | ||
| README.md | ||
| SECURITY.md | ||
| roadmap.md | ||
README.md
Foundry
Pure C implementations of 11 core Python ML tools. CPU is the baseline; GPU backends exist for CUDA, Metal and Tenstorrent Blackhole (see GPU Support). Vulkan is planned.
Why Foundry?
The Python ML ecosystem is powerful but has limitations:
-
Dependency hell - Complex Python environments, version conflicts
-
Not embeddable - Can't easily integrate training into native apps
-
Performance overhead - Python GIL, interpreter overhead
-
Deployment complexity - Requires Python runtime everywhere
Foundry provides C implementations that can be:
-
Embedded in any application
-
Compiled to a single binary
-
Used from any language via C FFI
-
Deployed without Python
The Tools
We identified 11 core Python tools that everything else builds on. Each has alternatives, but these are the essential functions:
| Foundry Module | Python counterpart | Function |
|---|---|---|
| tensor/ | PyTorch | Tensor operations + autograd |
| optim/ | PyTorch | Optimizers (SGD, Adam) + LR schedulers |
| distributed/ | DeepSpeed | Gradient accumulation, data sharding (single-process; real distributed planned) |
| lora/ | PEFT | LoRA and adapter training |
| tokenizer/ | tokenizers | BPE tokenization (WordPiece planned) |
| data/ | datasets | Dataset loading and processing |
| safetensors/ | safetensors | Safe model weight serialization |
| attention/ | PyTorch attention | Standard (unfused) multi-head attention, KV cache, RoPE, TurboQuant KV compression; no flash kernel yet |
| quantization/ | bitsandbytes | Model quantization (int8, int4) |
| eval/ | lm-eval-harness | Model evaluation and benchmarks |
| progress/ | tqdm | Progress reporting |
Why These Tools?
Many Python ML tools are alternatives or convenience wrappers:
Frameworks - PyTorch vs TensorFlow vs JAX (pick one) Distributed - DeepSpeed vs Megatron vs FairScale (pick one) LoRA - PEFT vs Unsloth vs Axolotl (PEFT is the core, others wrap it) Tokenizers - tokenizers vs SentencePiece vs tiktoken (same function)
We chose the most fundamental tool in each category. Everything else can be built on top.
Architecture
┌─────────────────────────────────────────────────────────────────┐
│ Your Application │
├─────────────────────────────────────────────────────────────────┤
│ Foundry C API │
├─────────────────────────────────────────────────────────────────┤
│ tensor/ │ lora/ │ tokenizer/ │ data/ │ eval/ │ ... │
├─────────────────────────────────────────────────────────────────┤
│ Backends: CPU · CUDA · Metal · Tenstorrent Blackhole │
│ (Vulkan planned) │
└─────────────────────────────────────────────────────────────────┘
Quick Start
Prerequisites: a C11 compiler and make. BLAS is optional (USE_BLAS=1: Accelerate on
macOS, OpenBLAS on Linux). The default flags include -march=native, so build on the machine
that will run the code, or add PORTABLE=1 for a copyable build (-march=x86-64-v2 on x86-64).
# Build all modules
make all
# Run all CPU tests (483 tests, 27 suites)
make test
# Run training example
make train
Training Example
$ make train
===========================================
Foundry Training Example: XOR Problem
===========================================
Model: 2-layer MLP (2 -> 8 -> 1)
Task: Learn XOR function
Learning rate: 1.00
Epochs: 1000
Epoch 1 | Loss: 0.265296
Epoch 500 | Loss: 0.001784
Epoch 1000 | Loss: 0.000652
Predictions vs Targets:
0 XOR 0 = 0.042 (target: 0) [OK]
0 XOR 1 = 0.980 (target: 1) [OK]
1 XOR 0 = 0.986 (target: 1) [OK]
1 XOR 1 = 0.015 (target: 0) [OK]
Accuracy: 4/4 (100%)
[SUCCESS] Model learned XOR perfectly!
Status
| Module | Status | Tests | Description |
|---|---|---|---|
| tensor/ | Complete | 141 | Tensor ops with autograd (PyTorch parity: 80/83) |
| optim/ | Complete | 15 | SGD, Adam, AdamW, LR schedulers |
| progress/ | Complete | 18 | Training progress bars & logging |
| tokenizer/ | Complete | 18 | BPE tokenization |
| safetensors/ | Complete | 18 | Safe tensor serialization |
| lora/ | Complete | 15 | Low-rank adaptation |
| data/ | Complete | 19 | Dataset loading & batching |
| quantization/ | Complete | 57 | INT8/INT4 quantization, export (test_quantization 43 + test_export 14) |
| attention/ | Complete | 23 | Multi-head attention, KV cache, RoPE, TurboQuant KV compression (fa_tq_) (test_attention 12 + test_turboquant 11) |
| distributed/ | Complete | 18 | Gradient accumulation, data sharding |
| eval/ | Complete | 18 | Perplexity, accuracy, F1 metrics |
Core modules: 360 tests. make test also runs the extension suites — hdc 18, hybridmem 20,
threadpool 7, rwkvx 26 (7 suites), rosa 11, barswap 14, pth 10, ternarize 17 — for 483 tests
across 27 suites, all passing (macOS, 2026-10-09). The rwkvx suites need the RWKV vocabulary,
which make test downloads once via rwkvx/data/fetch_rwkv_vocab.sh (network, curl,
sha256sum or shasum). Hardware suites are separate: make USE_CUDA=1 test-cuda,
make USE_METAL=1 test-metal, and the Tenstorrent suites in tt/tests/ (see
tt/README.md); each needs the hardware.
PyTorch numerical parity: 80/83 comparison tests (tensor module)
Custom Extensions
| Module | Status | Description |
|---|---|---|
| swarm/ | Complete | Distributed CPU training across Docker Swarm |
| ternary_optim/ | Reference (CPU) | Memory-neutral ternary optimizer (SR4). Trains ternary weights ({-1,0,+1}) via a 4-bit signed latent accumulator + stochastic rounding — the entire per-weight training state is 4 bits (~24× less optimizer state than Adam), no FP32 master weight or momentum. In the research prototype it was extracted from, it reached 98.3% on an MNIST 784-256-10 MLP, matching that MLP's FP32-dense result. Standalone CPU reference library; not yet wired into optim/. See ternary_optim/README.md. |
These are project-specific additions built on top of the core Foundry modules. See swarm/README.md and ternary_optim/README.md for details.
Other Modules
Modules in the tree that are not in the 11-tool table:
| Module | What it is |
|---|---|
| rwkvx/ | RWKV-X hybrid LM (RWKV-7 + sparse attention): inference, training, backward passes |
| rwkvx/rfa.* | RWKV-7 + full-attention (Qwen2-style GQA) hybrid inference and training |
| convert/ | HuggingFace safetensors → .btw/.btv converter for Llama-family models |
| pth/ | PyTorch .pth (ZIP + pickle) state_dict reader, zero-copy via mmap |
| threadpool/ | Minimal pthread parallel_for (USE_THREADS=1) |
| ternarize/ | Post-training ternarization of FP16/BF16 models, layer by layer |
| rosa/ | Suffix automaton (binary and vocab-sized alphabets) |
| hdc/ | Hyperdimensional computing: binary hypervectors, Hamming-distance search |
| hybridmem/ | Disk-backed K/V memory tier for RWKV-X, indexed with hdc/ |
| barswap/ | K80 die time-multiplexing through one PCIe BAR slot on a specific board (BTC-S37) |
| swarm/ | Distributed CPU training across Docker Swarm (make test-swarm, not in make test) |
| ternary_optim/ | SR4 ternary optimizer, standalone CPU reference |
| dedup/ | MinHash/LSH deduplication — interface only (header + README), not implemented |
| retrieval/ | Hash-based document retrieval — interface only (header + README), not implemented |
Build Options
# Build specific module
make tensor
make lora
# Run specific test suite
make test-tensor
make test-attention
# Debug build
make DEBUG=1 all
# Clean
make clean
GPU Support
CPU is the baseline — every operation works on CPU first, and the GPU backends extend them without changing the public API.
- CUDA (
cuda/,cuda_kernels/) — loaded viadlopen, kernels compiled to PTX and embedded, so Foundry builds without nvcc present. Targetssm_37(Kepler) upward; that is deliberate, since most current runtimes have dropped below-Pascal support. - Metal (
metal/) — Apple Silicon. - Tenstorrent Blackhole (
tt/) — a p150a's Tensix cores, through Foundry's own firmware, kernels and userspace driver, plain C. The backend was rewritten from public documentation and measured hardware behaviour; no tt-metal, TT-NN, tt-umd or tt-llk source is compiled, linked or vendored. The only dependency is the SFPI compiler, pluspython3for thett/tools/emit_tensix_*.pygenerators, and only to rebuild the device images; a Foundry build needs neither. Kernels are hand-written reader/compute/writer triads compiled ahead of time and embedded, so Foundry builds without the toolchain present. 8 op families run on silicon, plus Gated DeltaNet; multi-core matmul exists. Pantheon runs a 9Bqwen35model end-to-end on the card through this backend. Seett/README.mdanddocs/tenstorrent.md. - Vulkan — planned.
Contributions to the GPU backends are very welcome.
License
Foundry is licensed under the GNU AGPL-3.0 — see LICENSE.
Contributions are accepted under the CLA in CLA.md, with a DCO
sign-off on every commit — see CONTRIBUTING.md. You keep
your copyright; signing is one entry in
.cla/signatures.md, added in your first pull request.
Copyright © 2026 David (Corey) Herrera and the Foundry contributors. See
NOTICE.