Initial extract: templates + methodology guide from Ideation

Public, version-controlled source of truth for TeamIDE's project templates, coding conventions, and the build-plan → PM scaffolding rules. Mounted read-only by TeamIDE's planner agent at runtime. Extracted from the personal Ideation repo as a precondition for workspace-projects-agent-spec.md §9.4. Personal Ideation stays private; this repo holds only the publishable methodology.
2026-05-16 10:01:25 -05:00
commit 83c52bc011
95 changed files with 13727 additions and 0 deletions
@@ -0,0 +1,387 @@
+# Research Project Guidelines
+
+## Project Structure
+
+```
+ProjectName/
+├── README.md                    # Project overview (public-facing)
+├── LICENSE                      # MIT Transparency License
+├── CLAUDE.md                    # AI assistant guidelines + project config
+├── goals.md                     # Success criteria (what does "done" look like?)
+├── spec.md                      # Technical specification (optional)
+├── conclusions.md               # Living findings document
+├── open_questions.md            # Unresolved questions tracker
+├── .gitignore
+├── experiments/                 # Experiment code (numbered runs)
+│   ├── run1-baseline.ext
+│   ├── run2-variable-change.ext
+│   └── ...
+├── results/                     # Experiment results documentation
+│   ├── TEMPLATE-results.md      # Template for new results docs
+│   └── experiment_N_results.md  # One per experiment group
+├── data/                        # Datasets (gitignored — large files)
+├── literature/                  # Paper notes & prior art
+│   ├── TEMPLATE-paper.md        # Template for paper reviews
+│   └── paper-name.md
+├── src/                         # Implementation code (when research produces software)
+├── tests/                       # Test suite
+├── docs/                        # Technical documentation
+│   ├── README.md                # Docs index
+│   ├── TEMPLATE-experiment-plan.md
+│   ├── TEMPLATE-milestone.md
+│   ├── getting-started.md
+│   └── architecture.md
+└── screenshots/                 # Visuals for README / portfolio
+```
+
+## TeamIDE Project Management
+
+If this project was scaffolded from Ideation, `.teamide/` PM files (plans, tasks, project.yml) should already exist — generated from milestone definitions during scaffolding. See the Ideation workspace's `claude.md` for the PM generation workflow and format details.
+
+## Terminology
+
+| Term | Meaning |
+|------|---------|
+| **goals.md** | What success looks like — pass/fail criteria, anti-goals, kill gates |
+| **spec.md** | Technical specification for the system being built or studied (optional) |
+| **conclusions.md** | Living findings — updated after each experiment phase |
+| **open_questions.md** | Unresolved questions with context and leads |
+| **experiment** | A numbered run in `experiments/` with a hypothesis |
+| **results doc** | Structured write-up in `results/` documenting what happened |
+| **milestone** | A dependency-tracked unit of work with success criteria |
+
+## The Research Pipeline
+
+```
+Vision doc  -->  Math/tradeoff analysis  -->  Milestone plan  -->  Implementation  -->  Tests pass
+(speculative)    (constrained)                (prescriptive)       (code)               (done)
+```
+
+Each stage has a different document type and level of commitment. Ideas flow left to right and can be killed at any stage.
+
+## Document Layers
+
+### Layer 1: Vision Documents
+
+Thinking documents. Speculative, conversational, organized around questions and hypotheses. These live at the repo root or in topic-specific folders.
+
+**Purpose:** Capture the frontier of thinking. These are allowed to be wrong.
+
+**Examples:** Mathematical explorations, system architecture visions, training/data blueprints, goal documents.
+
+### Layer 2: Specs & Plans
+
+Prescriptive. Designed to be turned directly into implementation tasks. Live in `docs/` or as top-level files (`spec.md`).
+
+**Purpose:** Bridge between "what if" and "what to build." Every spec includes tasks, API surface, success criteria, and test cases.
+
+### Layer 3: Research Subprojects
+
+Active investigations with clear experiment plans and validation gates. Each gets its own directory or subfolder within `experiments/` with a README explaining the hypothesis, experiment design, and expected outcomes.
+
+### Layer 4: Architecture Docs
+
+Written after implementation. Describe how things actually work, not how they were planned. Live in `docs/architecture.md`.
+
+**Purpose:** Let a new developer understand the system without reading source code.
+
+## Key Patterns
+
+### 1. Hypothesis First (Non-Negotiable)
+
+**Every experiment requires a written hypothesis before any code runs.** No hypothesis, no experiment.
+
+The hypothesis states what you expect to happen and why — it forces you to think before you compute.
+
+```markdown
+## Hypothesis
+Increasing batch size from 1 to 32 will reduce gradient noise enough for
+convergence, because individual gradient samples have high variance but
+batch-averaged gradients should be stable.
+
+## Expected Outcome
+Loss should decrease monotonically with batch=32, vs oscillating with batch=1.
+If loss still oscillates with batch=32, the variance problem is fundamental,
+not just a sample-size issue.
+```
+
+Without a hypothesis, you don't know what the experiment is testing. You'll see a number and not know if it's good or bad.
+
+**The scientific method, applied:**
+1. **Observe** — notice something in your data, results, or literature
+2. **Question** — formulate a specific question about what you observed
+3. **Hypothesize** — predict an answer and state WHY you expect it
+4. **Experiment** — design a run that can confirm or reject the hypothesis
+5. **Analyze** — compare results to the prediction
+6. **Conclude** — state what was learned, update your mental model
+7. **Repeat** — the conclusion feeds the next observation
+
+### 2. Kill Gates
+
+Research questions get structured investigation with kill gates:
+
+```
+Phase 1: Literature review (what exists?)
+Phase 2: Build minimal viable module
+Phase 3: Integration experiments (defined matrix of configs)
+Phase 4: Investigate properties (hypothesis-based, only if Phase 3 passes)
+Phase 5: Optimization (only if Phase 4 is promising)
+```
+
+Each phase is a gate. If Phase 3 fails, Phase 4 doesn't happen. This prevents sunk-cost fallacy on dead-end research.
+
+### 3. Constraint-Driven Design
+
+Start with hard limits. Let them drive architecture.
+
+```markdown
+## Constraints
+- [Your hardware/budget/time limit here]
+- [Your performance requirement here]
+- [Your compatibility requirement here]
+
+## Design (driven by above)
+- [Architecture decision that follows from constraints]
+```
+
+Don't design the ideal system then optimize. Design around your actual constraints from day one.
+
+### 4. Explicit Deferral
+
+Every milestone says what it's NOT doing and why:
+
+```markdown
+**Deferred:** [Feature/capability].
+**Reason:** [Why it's not worth doing now].
+```
+
+Three categories for all work items:
+- **Active/Concrete** — being built now, with tests
+- **Outlined** — architectural sketch, no code yet
+- **Deferred** — recognized need, explicitly postponed with rationale
+
+This prevents scope creep while preserving ideas for later.
+
+### 5. Tradeoff Tables
+
+Every complex decision gets a table showing the option space:
+
+```markdown
+| Option    | Pros             | Cons                  | When to pick   |
+|-----------|------------------|-----------------------|----------------|
+| Option A  | Simple, fast     | Doesn't scale past N  | Prototyping    |
+| Option B  | Scales well      | Complex to implement  | Production     |
+| Option C  | Best performance | Vendor lock-in        | If budget > $X |
+```
+
+No "we chose X because it's better." Instead: "here's the tradeoff space, here's what we picked and why."
+
+### 6. Testable Success Criteria
+
+Every milestone ends with a concrete, falsifiable statement:
+
+```markdown
+**Success Criteria:**
+- [System] does [specific thing] in [specific time/resource bound]
+- [N] tests pass with zero warnings
+- [Specific scenario] produces [specific measurable result]
+```
+
+Not "it should work" — specific numbers, specific scenarios.
+
+### 7. Math First, Code Second
+
+Abstract ideas get mathematical or formal framing before any code is written:
+
+```
+Idea (what if we tried X?)
+  --> Formalization (what does X actually mean mathematically?)
+    --> Existing work (has anyone done this? what libraries exist?)
+      --> Experiment plan (how would we test this?)
+        --> Validation gate (what result would make us continue vs abandon?)
+```
+
+This filters out ideas that sound good but don't survive formalization.
+
+### 8. Documentation as Specification
+
+Write implementation-ready docs before coding:
+
+1. Write detailed spec in markdown (API signatures, wire formats, data structures)
+2. Implement against the spec
+3. Test against the spec's success criteria
+4. Update the spec if reality diverges
+
+The spec IS the source of truth. Code follows docs, not the other way around.
+
+## Experiment Tracking
+
+All experiments live in the `experiments/` folder and follow strict naming and documentation conventions.
+
+### Naming Convention
+
+Experiments are numbered sequentially with a descriptive kebab-case suffix:
+
+```
+experiments/
+├── run1-batch-gradient-baseline.c
+├── run2-per-sample-online.c
+├── run3-leaky-relu-activation.c
+├── run4-temperature-annealing.c
+└── ...
+```
+
+Rules:
+- Prefix: `run<N>-` where N is sequential (run1, run2, run3...)
+- Suffix: short kebab-case description of what this run changes vs the previous
+- One variable changed per run when possible (isolation)
+- If a run is a variant of another, use sub-numbering: `run9b-normalization-lower-temp`
+- Run numbering stays globally sequential across the whole project — don't restart at run1 inside subfolders
+
+When a single experiment direction spawns many runs, group them in a subfolder inside `experiments/`:
+
+```
+experiments/
+├── run1-baseline.ext
+├── run2-different-approach.ext
+├── scaling-study/              # Subfolder for related runs
+│   ├── run3-small-scale.ext
+│   ├── run4-medium-scale.ext
+│   └── run5-large-scale.ext
+└── ...
+```
+
+### Results Documentation
+
+Every experiment's results are documented in `results/`. One results file per experiment or per logical group:
+
+```
+results/
+├── experiment_1_results.md
+├── experiment_2_results.md
+└── ...
+```
+
+Use `results/TEMPLATE-results.md` as the starting point for each results file.
+
+### Key Rules for Results
+
+1. **Every run gets a row in the summary table** — even failed runs. Especially failed runs.
+2. **Link to source files** — reference experiment files as `experiments/run1-descriptive-name.ext`
+3. **Include raw output** — paste actual terminal output. Don't just summarize.
+4. **One finding per discovery** — each finding gets its own subsection with a bold **Verdict** line
+5. **Comparison tables for pivots** — when switching approaches, add a side-by-side comparison table
+6. **Strikethrough completed next steps** — mark done items with `~~strikethrough~~` and add **DONE**
+7. **Decision points with kill gates** — define upfront what results would make you continue, pivot, or abandon
+8. **Literature context** — compare your results to published work
+
+### Running Experiments
+
+**All experiments MUST be run in the background** so the user can continue chatting while they execute. Never run a build or training run in the foreground — it blocks the conversation. Use background execution and check on results later.
+
+## Milestone Format
+
+Every implementation milestone follows this structure:
+
+```markdown
+### Milestone X.Y: Title — [STATUS]
+
+**What:** One-sentence summary of the deliverable.
+
+**Tasks:**
+- [ ] Task with enough detail that someone else could do it
+- [ ] Another task
+
+**Completed.** [Summary paragraph when done.]
+
+**Deferred:** [What was intentionally left out and why.]
+
+**New Files:**
+
+| File | What |
+|------|------|
+| path/to/file | One-line description |
+
+**API Surface:**
+[Function signatures or interface definitions]
+
+**Depends on:** Milestone X.Y-1
+
+**Success Criteria:** [Falsifiable statement with numbers]
+
+**Test Cases:**
+1. [Specific scenario] — [expected result]
+2. [Specific scenario] — [expected result]
+```
+
+### Dependency Graphs
+
+Milestones form a dependency chain. Visualize it:
+
+```
+Phase 1:  1.1 --> 1.2 --> 1.3 --> 1.4 --> 1.5
+                    |                |
+Phase 2:            v                v
+              2.1 --> 2.2 --> 2.3 --> 2.4
+```
+
+This makes it obvious what can be parallelized and what blocks what.
+
+## Principles
+
+1. **Hypothesis before experiment** — no code runs without a written prediction of what will happen and why. This is the most important rule.
+2. **Speculation is welcome but quarantined** — vision docs are clearly marked as exploratory
+3. **Implementation follows specs** — not the reverse
+4. **Deferral is first-class** — saying "not now" with a reason is as important as saying "yes"
+5. **Constraints are features** — limits drive creative solutions rather than being obstacles
+6. **Tests are the definition of done** — not "it compiles" but "N tests pass with zero warnings"
+7. **Tradeoffs are documented** — future-you can understand why decisions were made
+8. **Kill gates protect your time** — research that fails a gate gets abandoned, not propped up
+9. **Every experiment is tracked** — numbered, named, with raw output in results files. Failed runs are documented just as thoroughly as successes.
+10. **One variable per run** — isolate changes so you know what caused the result. Sub-number variants (run9b) when testing minor tweaks.
+11. **Experiments run in the background** — never block the user's ability to chat. All builds, training runs, and long-running processes execute in the background.
+
+## Commit Rules
+
+- Never mention Claude, AI, or generated code in commits
+- Write as the developer, first person
+- Concise and descriptive
+
+## Customization Checklist
+
+When creating a new project from this template:
+
+- [ ] Update `README.md` with project name, problem, idea, approaches
+- [ ] Fill in `goals.md` with success criteria from the project plan
+- [ ] Fill in `spec.md` if the project has a technical specification (otherwise delete it)
+- [ ] Set up `open_questions.md` with initial research questions
+- [ ] Create initial experiment plans in `docs/` using `TEMPLATE-experiment-plan.md`
+- [ ] Add project-specific entries to `.gitignore` (language-specific, data formats, etc.)
+- [ ] Update the "Project-Specific Configuration" section below
+- [ ] Initialize git and create GitHub repo
+
+---
+
+# Project-Specific Configuration
+
+<!-- Everything below this line is customized per project -->
+
+## Project Name
+
+[PROJECT_NAME]
+
+## Description
+
+[Brief description of the research project]
+
+## Research Question
+
+[The core question this project investigates]
+
+## Tech Stack
+
+[Languages, frameworks, libraries being used]
+
+## TeamIDE Workspace ID
+
+[Workspace ID if deployed via TeamIDE]