Foundry/docs

Foundry Documentation

Pure C implementation of essential ML tools for training large language models on CPU clusters.

---

Quick Links

• Getting Started
• API Reference
• Examples
• Architecture

---

API Reference

Core Modules

Module	Description	Docs
tensor	Tensor operations with autograd	tensor.md
optim	Optimizers (SGD, Adam, AdamW)	optim.md
tokenizer	BPE tokenization	tokenizer.md
safetensors	Model serialization	safetensors.md
data	Dataset loading and batching	data.md

Training Modules

Module	Description	Docs
attention	Efficient attention with KV cache	attention.md
lora	Low-Rank Adaptation fine-tuning	lora.md
quantization	INT8/INT4/ternary quantization, training + export	quantization.md
distributed	Distributed training primitives	distributed.md
swarm	Docker Swarm distributed training	swarm.md

GPU Acceleration

Module	Description	Docs
metal	Apple Metal GPU compute backend	metal.md
tt	Tenstorrent Blackhole compute backend (p150a; own userspace driver, tt-kmd optional)	tenstorrent.md
barswap	K80 BAR reassignment (kernel module + userspace)	bar-swap.md

Utility Modules

Module	Description	Docs
progress	Progress bars and metrics	progress.md
eval	Model evaluation framework	eval.md

CLI Tools

Tool	Description	Docs
rwkvx_generate	RWKV-X text generation (INT8, HybridMem)	rwkvx-generate.md
rwkvx_train	RWKV-X LoRA training (moba attention blocks)	rwkvx-train.md
rfa_generate	RFA hybrid text generation (RWKV-7 + Qwen)	rfa-generate.md
rfa_train	RFA hybrid LoRA training (parallel, INT8)	rfa-train.md
rfa_lora_merge	LoRA adapter averaging (cross-machine merge)	rfa-lora-merge.md
quantize_model	Offline model quantization (INT8/INT4/ternary)	quantize-model.md

---

Building Foundry

Prerequisites

• GCC or Clang with C11 support
• Make
• POSIX-compatible system (Linux, macOS)

Build Commands

# Build all modules
make all

# Build specific module
make tensor
make swarm

# Run all tests
make test

# Clean build artifacts
make clean

Build Options

Option	Description
DEBUG=1	Build with debug symbols
SANITIZE=1	Build with address sanitizer
OPTIMIZE=1	Build with -O3 optimization

---

Project Structure

foundry/
├── tensor/          # Core tensor operations
│   ├── tensor.h
│   ├── tensor.c
│   └── tests/
├── optim/           # Optimizers
├── tokenizer/       # Tokenization
├── safetensors/     # Model I/O
├── data/            # Data loading
├── attention/       # Attention mechanisms
├── lora/            # LoRA adapters
├── quantization/    # Quantization
├── metal/           # Apple Metal GPU backend
├── tt/              # Tenstorrent Blackhole backend (host C + Tensix kernels + converter)
├── barswap/         # K80 BAR reassignment (kernel module + userspace lib)
├── distributed/     # Distributed primitives
├── swarm/           # Docker Swarm training
├── progress/        # Progress bars
├── eval/            # Evaluation
├── examples/        # Example programs
├── docs/            # Documentation (you are here)
├── Makefile
└── roadmap.md

---

Python Counterparts

Each module covers the core function of one Python library, in C:

Python Tool	Foundry Module	Notes
PyTorch	tensor, optim	tensor ops + autograd; PyTorch parity 80/83 comparison tests
HuggingFace Tokenizers	tokenizer	BPE (WordPiece planned)
HuggingFace Safetensors	safetensors	read/write, mmap
HuggingFace Datasets	data	JSONL, batching, sqlite chat loading
tqdm	progress
bitsandbytes	quantization	INT8/INT4, export
PyTorch attention	attention	standard attention, KV cache, RoPE; no fused flash kernel yet
PEFT	lora
Accelerate/DeepSpeed	distributed	single-process primitives; real multi-node planned
lm-eval-harness	eval	perplexity, accuracy, F1

---

Design Principles

1. Zero Python Dependencies - Pure C, no Python runtime required
2. Minimal External Dependencies - Only POSIX APIs
3. Memory Efficient - No unnecessary copies, mmap support
4. Embeddable - Designed to be embedded in other applications
5. Portable - Works on x86_64 and ARM (Raspberry Pi)

---

Target Use Case

Training 1B+ parameter RWKV-x models across heterogeneous CPU clusters:

• Raspberry Pi 4/5
• Laptops and desktops
• Small servers
• Connected via WiFi/Starlink

The swarm module provides distributed training with:

• Async gradient updates
• TopK gradient compression (90% bandwidth reduction)
• Hardware-adaptive batch sizes
• Fault tolerance

---

License

See ../LICENSE and the License section of the top-level README.md.

---

Contributing

See ../CONTRIBUTING.md.