186 lines
5.7 KiB
Markdown
186 lines
5.7 KiB
Markdown
# Foundry Documentation
|
|
|
|
Pure C implementation of essential ML tools for training large language models on CPU clusters.
|
|
|
|
---
|
|
|
|
## Quick Links
|
|
|
|
- [Getting Started](getting-started.md)
|
|
- [API Reference](#api-reference)
|
|
- [Examples](examples.md)
|
|
- [Architecture](architecture.md)
|
|
|
|
---
|
|
|
|
## API Reference
|
|
|
|
### Core Modules
|
|
|
|
| Module | Description | Docs |
|
|
|--------|-------------|------|
|
|
| **tensor** | Tensor operations with autograd | [tensor.md](tensor.md) |
|
|
| **optim** | Optimizers (SGD, Adam, AdamW) | [optim.md](optim.md) |
|
|
| **tokenizer** | BPE tokenization | [tokenizer.md](tokenizer.md) |
|
|
| **safetensors** | Model serialization | [safetensors.md](safetensors.md) |
|
|
| **data** | Dataset loading and batching | [data.md](data.md) |
|
|
|
|
### Training Modules
|
|
|
|
| Module | Description | Docs |
|
|
|--------|-------------|------|
|
|
| **attention** | Efficient attention with KV cache | [attention.md](attention.md) |
|
|
| **lora** | Low-Rank Adaptation fine-tuning | [lora.md](lora.md) |
|
|
| **quantization** | INT8/INT4/ternary quantization, training + export | [quantization.md](quantization.md) |
|
|
| **distributed** | Distributed training primitives | [distributed.md](distributed.md) |
|
|
| **swarm** | Docker Swarm distributed training | [swarm.md](swarm.md) |
|
|
|
|
### GPU Acceleration
|
|
|
|
| Module | Description | Docs |
|
|
|--------|-------------|------|
|
|
| **metal** | Apple Metal GPU compute backend | [metal.md](metal.md) |
|
|
| **tt** | Tenstorrent Blackhole compute backend (p150a; own userspace driver, tt-kmd optional) | [tenstorrent.md](tenstorrent.md) |
|
|
| **barswap** | K80 BAR reassignment (kernel module + userspace) | [bar-swap.md](bar-swap.md) |
|
|
|
|
### Utility Modules
|
|
|
|
| Module | Description | Docs |
|
|
|--------|-------------|------|
|
|
| **progress** | Progress bars and metrics | [progress.md](progress.md) |
|
|
| **eval** | Model evaluation framework | [eval.md](eval.md) |
|
|
|
|
### CLI Tools
|
|
|
|
| Tool | Description | Docs |
|
|
|------|-------------|------|
|
|
| **rwkvx_generate** | RWKV-X text generation (INT8, HybridMem) | [rwkvx-generate.md](rwkvx-generate.md) |
|
|
| **rwkvx_train** | RWKV-X LoRA training (moba attention blocks) | [rwkvx-train.md](rwkvx-train.md) |
|
|
| **rfa_generate** | RFA hybrid text generation (RWKV-7 + Qwen) | [rfa-generate.md](rfa-generate.md) |
|
|
| **rfa_train** | RFA hybrid LoRA training (parallel, INT8) | [rfa-train.md](rfa-train.md) |
|
|
| **rfa_lora_merge** | LoRA adapter averaging (cross-machine merge) | [rfa-lora-merge.md](rfa-lora-merge.md) |
|
|
| **quantize_model** | Offline model quantization (INT8/INT4/ternary) | [quantize-model.md](quantize-model.md) |
|
|
|
|
---
|
|
|
|
## Building Foundry
|
|
|
|
### Prerequisites
|
|
|
|
- GCC or Clang with C11 support
|
|
- Make
|
|
- POSIX-compatible system (Linux, macOS)
|
|
|
|
### Build Commands
|
|
|
|
```bash
|
|
# Build all modules
|
|
make all
|
|
|
|
# Build specific module
|
|
make tensor
|
|
make swarm
|
|
|
|
# Run all tests
|
|
make test
|
|
|
|
# Clean build artifacts
|
|
make clean
|
|
```
|
|
|
|
### Build Options
|
|
|
|
| Option | Description |
|
|
|--------|-------------|
|
|
| `DEBUG=1` | Build with debug symbols |
|
|
| `SANITIZE=1` | Build with address sanitizer |
|
|
| `OPTIMIZE=1` | Build with -O3 optimization |
|
|
|
|
---
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
foundry/
|
|
├── tensor/ # Core tensor operations
|
|
│ ├── tensor.h
|
|
│ ├── tensor.c
|
|
│ └── tests/
|
|
├── optim/ # Optimizers
|
|
├── tokenizer/ # Tokenization
|
|
├── safetensors/ # Model I/O
|
|
├── data/ # Data loading
|
|
├── attention/ # Attention mechanisms
|
|
├── lora/ # LoRA adapters
|
|
├── quantization/ # Quantization
|
|
├── metal/ # Apple Metal GPU backend
|
|
├── tt/ # Tenstorrent Blackhole backend (host C + Tensix kernels + converter)
|
|
├── barswap/ # K80 BAR reassignment (kernel module + userspace lib)
|
|
├── distributed/ # Distributed primitives
|
|
├── swarm/ # Docker Swarm training
|
|
├── progress/ # Progress bars
|
|
├── eval/ # Evaluation
|
|
├── examples/ # Example programs
|
|
├── docs/ # Documentation (you are here)
|
|
├── Makefile
|
|
└── roadmap.md
|
|
```
|
|
|
|
---
|
|
|
|
## Python Counterparts
|
|
|
|
Each module covers the core function of one Python library, in C:
|
|
|
|
| Python Tool | Foundry Module | Notes |
|
|
|-------------|----------------|-------|
|
|
| PyTorch | tensor, optim | tensor ops + autograd; PyTorch parity 80/83 comparison tests |
|
|
| HuggingFace Tokenizers | tokenizer | BPE (WordPiece planned) |
|
|
| HuggingFace Safetensors | safetensors | read/write, mmap |
|
|
| HuggingFace Datasets | data | JSONL, batching, sqlite chat loading |
|
|
| tqdm | progress | |
|
|
| bitsandbytes | quantization | INT8/INT4, export |
|
|
| PyTorch attention | attention | standard attention, KV cache, RoPE; no fused flash kernel yet |
|
|
| PEFT | lora | |
|
|
| Accelerate/DeepSpeed | distributed | single-process primitives; real multi-node planned |
|
|
| lm-eval-harness | eval | perplexity, accuracy, F1 |
|
|
|
|
---
|
|
|
|
## Design Principles
|
|
|
|
1. **Zero Python Dependencies** - Pure C, no Python runtime required
|
|
2. **Minimal External Dependencies** - Only POSIX APIs
|
|
3. **Memory Efficient** - No unnecessary copies, mmap support
|
|
4. **Embeddable** - Designed to be embedded in other applications
|
|
5. **Portable** - Works on x86_64 and ARM (Raspberry Pi)
|
|
|
|
---
|
|
|
|
## Target Use Case
|
|
|
|
Training 1B+ parameter RWKV-x models across heterogeneous CPU clusters:
|
|
- Raspberry Pi 4/5
|
|
- Laptops and desktops
|
|
- Small servers
|
|
- Connected via WiFi/Starlink
|
|
|
|
The swarm module provides distributed training with:
|
|
- Async gradient updates
|
|
- TopK gradient compression (90% bandwidth reduction)
|
|
- Hardware-adaptive batch sizes
|
|
- Fault tolerance
|
|
|
|
---
|
|
|
|
## License
|
|
|
|
See [`../LICENSE`](../LICENSE) and the
|
|
License section of the top-level [`README.md`](../README.md).
|
|
|
|
---
|
|
|
|
## Contributing
|
|
|
|
See [`../CONTRIBUTING.md`](../CONTRIBUTING.md).
|