# Research Project Guidelines ## Project Structure ``` ProjectName/ ├── README.md # Project overview (public-facing) ├── LICENSE # MIT Transparency License ├── CLAUDE.md # AI assistant guidelines + project config ├── goals.md # Success criteria (what does "done" look like?) ├── spec.md # Technical specification (optional) ├── conclusions.md # Living findings document ├── open_questions.md # Unresolved questions tracker ├── .gitignore ├── experiments/ # Experiment code (numbered runs) │ ├── run1-baseline.ext │ ├── run2-variable-change.ext │ └── ... ├── results/ # Experiment results documentation │ ├── TEMPLATE-results.md # Template for new results docs │ └── experiment_N_results.md # One per experiment group ├── data/ # Datasets (gitignored — large files) ├── literature/ # Paper notes & prior art │ ├── TEMPLATE-paper.md # Template for paper reviews │ └── paper-name.md ├── src/ # Implementation code (when research produces software) ├── tests/ # Test suite ├── docs/ # Technical documentation │ ├── README.md # Docs index │ ├── TEMPLATE-experiment-plan.md │ ├── TEMPLATE-milestone.md │ ├── getting-started.md │ └── architecture.md └── screenshots/ # Visuals for README / portfolio ``` ## TeamIDE Project Management If this project was scaffolded from Ideation, `.teamide/` PM files (plans, tasks, project.yml) should already exist — generated from milestone definitions during scaffolding. See the Ideation workspace's `claude.md` for the PM generation workflow and format details. ## Terminology | Term | Meaning | |------|---------| | **goals.md** | What success looks like — pass/fail criteria, anti-goals, kill gates | | **spec.md** | Technical specification for the system being built or studied (optional) | | **conclusions.md** | Living findings — updated after each experiment phase | | **open_questions.md** | Unresolved questions with context and leads | | **experiment** | A numbered run in `experiments/` with a hypothesis | | **results doc** | Structured write-up in `results/` documenting what happened | | **milestone** | A dependency-tracked unit of work with success criteria | ## The Research Pipeline ``` Vision doc --> Math/tradeoff analysis --> Milestone plan --> Implementation --> Tests pass (speculative) (constrained) (prescriptive) (code) (done) ``` Each stage has a different document type and level of commitment. Ideas flow left to right and can be killed at any stage. ## Document Layers ### Layer 1: Vision Documents Thinking documents. Speculative, conversational, organized around questions and hypotheses. These live at the repo root or in topic-specific folders. **Purpose:** Capture the frontier of thinking. These are allowed to be wrong. **Examples:** Mathematical explorations, system architecture visions, training/data blueprints, goal documents. ### Layer 2: Specs & Plans Prescriptive. Designed to be turned directly into implementation tasks. Live in `docs/` or as top-level files (`spec.md`). **Purpose:** Bridge between "what if" and "what to build." Every spec includes tasks, API surface, success criteria, and test cases. ### Layer 3: Research Subprojects Active investigations with clear experiment plans and validation gates. Each gets its own directory or subfolder within `experiments/` with a README explaining the hypothesis, experiment design, and expected outcomes. ### Layer 4: Architecture Docs Written after implementation. Describe how things actually work, not how they were planned. Live in `docs/architecture.md`. **Purpose:** Let a new developer understand the system without reading source code. ## Key Patterns ### 1. Hypothesis First (Non-Negotiable) **Every experiment requires a written hypothesis before any code runs.** No hypothesis, no experiment. The hypothesis states what you expect to happen and why — it forces you to think before you compute. ```markdown ## Hypothesis Increasing batch size from 1 to 32 will reduce gradient noise enough for convergence, because individual gradient samples have high variance but batch-averaged gradients should be stable. ## Expected Outcome Loss should decrease monotonically with batch=32, vs oscillating with batch=1. If loss still oscillates with batch=32, the variance problem is fundamental, not just a sample-size issue. ``` Without a hypothesis, you don't know what the experiment is testing. You'll see a number and not know if it's good or bad. **The scientific method, applied:** 1. **Observe** — notice something in your data, results, or literature 2. **Question** — formulate a specific question about what you observed 3. **Hypothesize** — predict an answer and state WHY you expect it 4. **Experiment** — design a run that can confirm or reject the hypothesis 5. **Analyze** — compare results to the prediction 6. **Conclude** — state what was learned, update your mental model 7. **Repeat** — the conclusion feeds the next observation ### 2. Kill Gates Research questions get structured investigation with kill gates: ``` Phase 1: Literature review (what exists?) Phase 2: Build minimal viable module Phase 3: Integration experiments (defined matrix of configs) Phase 4: Investigate properties (hypothesis-based, only if Phase 3 passes) Phase 5: Optimization (only if Phase 4 is promising) ``` Each phase is a gate. If Phase 3 fails, Phase 4 doesn't happen. This prevents sunk-cost fallacy on dead-end research. ### 3. Constraint-Driven Design Start with hard limits. Let them drive architecture. ```markdown ## Constraints - [Your hardware/budget/time limit here] - [Your performance requirement here] - [Your compatibility requirement here] ## Design (driven by above) - [Architecture decision that follows from constraints] ``` Don't design the ideal system then optimize. Design around your actual constraints from day one. ### 4. Explicit Deferral Every milestone says what it's NOT doing and why: ```markdown **Deferred:** [Feature/capability]. **Reason:** [Why it's not worth doing now]. ``` Three categories for all work items: - **Active/Concrete** — being built now, with tests - **Outlined** — architectural sketch, no code yet - **Deferred** — recognized need, explicitly postponed with rationale This prevents scope creep while preserving ideas for later. ### 5. Tradeoff Tables Every complex decision gets a table showing the option space: ```markdown | Option | Pros | Cons | When to pick | |-----------|------------------|-----------------------|----------------| | Option A | Simple, fast | Doesn't scale past N | Prototyping | | Option B | Scales well | Complex to implement | Production | | Option C | Best performance | Vendor lock-in | If budget > $X | ``` No "we chose X because it's better." Instead: "here's the tradeoff space, here's what we picked and why." ### 6. Testable Success Criteria Every milestone ends with a concrete, falsifiable statement: ```markdown **Success Criteria:** - [System] does [specific thing] in [specific time/resource bound] - [N] tests pass with zero warnings - [Specific scenario] produces [specific measurable result] ``` Not "it should work" — specific numbers, specific scenarios. ### 7. Math First, Code Second Abstract ideas get mathematical or formal framing before any code is written: ``` Idea (what if we tried X?) --> Formalization (what does X actually mean mathematically?) --> Existing work (has anyone done this? what libraries exist?) --> Experiment plan (how would we test this?) --> Validation gate (what result would make us continue vs abandon?) ``` This filters out ideas that sound good but don't survive formalization. ### 8. Documentation as Specification Write implementation-ready docs before coding: 1. Write detailed spec in markdown (API signatures, wire formats, data structures) 2. Implement against the spec 3. Test against the spec's success criteria 4. Update the spec if reality diverges The spec IS the source of truth. Code follows docs, not the other way around. ## Experiment Tracking All experiments live in the `experiments/` folder and follow strict naming and documentation conventions. ### Naming Convention Experiments are numbered sequentially with a descriptive kebab-case suffix: ``` experiments/ ├── run1-batch-gradient-baseline.c ├── run2-per-sample-online.c ├── run3-leaky-relu-activation.c ├── run4-temperature-annealing.c └── ... ``` Rules: - Prefix: `run-` where N is sequential (run1, run2, run3...) - Suffix: short kebab-case description of what this run changes vs the previous - One variable changed per run when possible (isolation) - If a run is a variant of another, use sub-numbering: `run9b-normalization-lower-temp` - Run numbering stays globally sequential across the whole project — don't restart at run1 inside subfolders When a single experiment direction spawns many runs, group them in a subfolder inside `experiments/`: ``` experiments/ ├── run1-baseline.ext ├── run2-different-approach.ext ├── scaling-study/ # Subfolder for related runs │ ├── run3-small-scale.ext │ ├── run4-medium-scale.ext │ └── run5-large-scale.ext └── ... ``` ### Results Documentation Every experiment's results are documented in `results/`. One results file per experiment or per logical group: ``` results/ ├── experiment_1_results.md ├── experiment_2_results.md └── ... ``` Use `results/TEMPLATE-results.md` as the starting point for each results file. ### Key Rules for Results 1. **Every run gets a row in the summary table** — even failed runs. Especially failed runs. 2. **Link to source files** — reference experiment files as `experiments/run1-descriptive-name.ext` 3. **Include raw output** — paste actual terminal output. Don't just summarize. 4. **One finding per discovery** — each finding gets its own subsection with a bold **Verdict** line 5. **Comparison tables for pivots** — when switching approaches, add a side-by-side comparison table 6. **Strikethrough completed next steps** — mark done items with `~~strikethrough~~` and add **DONE** 7. **Decision points with kill gates** — define upfront what results would make you continue, pivot, or abandon 8. **Literature context** — compare your results to published work ### Running Experiments **All experiments MUST be run in the background** so the user can continue chatting while they execute. Never run a build or training run in the foreground — it blocks the conversation. Use background execution and check on results later. ## Milestone Format Every implementation milestone follows this structure: ```markdown ### Milestone X.Y: Title — [STATUS] **What:** One-sentence summary of the deliverable. **Tasks:** - [ ] Task with enough detail that someone else could do it - [ ] Another task **Completed.** [Summary paragraph when done.] **Deferred:** [What was intentionally left out and why.] **New Files:** | File | What | |------|------| | path/to/file | One-line description | **API Surface:** [Function signatures or interface definitions] **Depends on:** Milestone X.Y-1 **Success Criteria:** [Falsifiable statement with numbers] **Test Cases:** 1. [Specific scenario] — [expected result] 2. [Specific scenario] — [expected result] ``` ### Dependency Graphs Milestones form a dependency chain. Visualize it: ``` Phase 1: 1.1 --> 1.2 --> 1.3 --> 1.4 --> 1.5 | | Phase 2: v v 2.1 --> 2.2 --> 2.3 --> 2.4 ``` This makes it obvious what can be parallelized and what blocks what. ## Principles 1. **Hypothesis before experiment** — no code runs without a written prediction of what will happen and why. This is the most important rule. 2. **Speculation is welcome but quarantined** — vision docs are clearly marked as exploratory 3. **Implementation follows specs** — not the reverse 4. **Deferral is first-class** — saying "not now" with a reason is as important as saying "yes" 5. **Constraints are features** — limits drive creative solutions rather than being obstacles 6. **Tests are the definition of done** — not "it compiles" but "N tests pass with zero warnings" 7. **Tradeoffs are documented** — future-you can understand why decisions were made 8. **Kill gates protect your time** — research that fails a gate gets abandoned, not propped up 9. **Every experiment is tracked** — numbered, named, with raw output in results files. Failed runs are documented just as thoroughly as successes. 10. **One variable per run** — isolate changes so you know what caused the result. Sub-number variants (run9b) when testing minor tweaks. 11. **Experiments run in the background** — never block the user's ability to chat. All builds, training runs, and long-running processes execute in the background. ## Commit Rules - Never mention Claude, AI, or generated code in commits - Write as the developer, first person - Concise and descriptive ## Customization Checklist When creating a new project from this template: - [ ] Update `README.md` with project name, problem, idea, approaches - [ ] Fill in `goals.md` with success criteria from the project plan - [ ] Fill in `spec.md` if the project has a technical specification (otherwise delete it) - [ ] Set up `open_questions.md` with initial research questions - [ ] Create initial experiment plans in `docs/` using `TEMPLATE-experiment-plan.md` - [ ] Add project-specific entries to `.gitignore` (language-specific, data formats, etc.) - [ ] Update the "Project-Specific Configuration" section below - [ ] Initialize git and create GitHub repo --- # Project-Specific Configuration ## Project Name [PROJECT_NAME] ## Description [Brief description of the research project] ## Research Question [The core question this project investigates] ## Tech Stack [Languages, frameworks, libraries being used] ## TeamIDE Workspace ID [Workspace ID if deployed via TeamIDE]