TeamIDE-methodology/templates/ResearchStarter/CLAUDE.md

Research Project Guidelines

Project Structure

ProjectName/
├── README.md                    # Project overview (public-facing)
├── LICENSE                      # MIT Transparency License
├── CLAUDE.md                    # AI assistant guidelines + project config
├── goals.md                     # Success criteria (what does "done" look like?)
├── spec.md                      # Technical specification (optional)
├── conclusions.md               # Living findings document
├── open_questions.md            # Unresolved questions tracker
├── .gitignore
├── experiments/                 # Experiment code (numbered runs)
│   ├── run1-baseline.ext
│   ├── run2-variable-change.ext
│   └── ...
├── results/                     # Experiment results documentation
│   ├── TEMPLATE-results.md      # Template for new results docs
│   └── experiment_N_results.md  # One per experiment group
├── data/                        # Datasets (gitignored — large files)
├── literature/                  # Paper notes & prior art
│   ├── TEMPLATE-paper.md        # Template for paper reviews
│   └── paper-name.md
├── src/                         # Implementation code (when research produces software)
├── tests/                       # Test suite
├── docs/                        # Technical documentation
│   ├── README.md                # Docs index
│   ├── TEMPLATE-experiment-plan.md
│   ├── TEMPLATE-milestone.md
│   ├── getting-started.md
│   └── architecture.md
└── screenshots/                 # Visuals for README / portfolio

TeamIDE Project Management

If this project was scaffolded from Ideation, .teamide/ PM files (plans, tasks, project.yml) should already exist — generated from milestone definitions during scaffolding. See the Ideation workspace's claude.md for the PM generation workflow and format details.

Terminology

Term	Meaning
goals.md	What success looks like — pass/fail criteria, anti-goals, kill gates
spec.md	Technical specification for the system being built or studied (optional)
conclusions.md	Living findings — updated after each experiment phase
open_questions.md	Unresolved questions with context and leads
experiment	A numbered run in experiments/ with a hypothesis
results doc	Structured write-up in results/ documenting what happened
milestone	A dependency-tracked unit of work with success criteria

The Research Pipeline

Vision doc  -->  Math/tradeoff analysis  -->  Milestone plan  -->  Implementation  -->  Tests pass
(speculative)    (constrained)                (prescriptive)       (code)               (done)

Each stage has a different document type and level of commitment. Ideas flow left to right and can be killed at any stage.

Document Layers

Layer 1: Vision Documents

Thinking documents. Speculative, conversational, organized around questions and hypotheses. These live at the repo root or in topic-specific folders.

Purpose: Capture the frontier of thinking. These are allowed to be wrong.

Examples: Mathematical explorations, system architecture visions, training/data blueprints, goal documents.

Layer 2: Specs & Plans

Prescriptive. Designed to be turned directly into implementation tasks. Live in docs/ or as top-level files (spec.md).

Purpose: Bridge between "what if" and "what to build." Every spec includes tasks, API surface, success criteria, and test cases.

Layer 3: Research Subprojects

Active investigations with clear experiment plans and validation gates. Each gets its own directory or subfolder within experiments/ with a README explaining the hypothesis, experiment design, and expected outcomes.

Layer 4: Architecture Docs

Written after implementation. Describe how things actually work, not how they were planned. Live in docs/architecture.md.

Purpose: Let a new developer understand the system without reading source code.

Key Patterns

1. Hypothesis First (Non-Negotiable)

Every experiment requires a written hypothesis before any code runs. No hypothesis, no experiment.

The hypothesis states what you expect to happen and why — it forces you to think before you compute.

## Hypothesis
Increasing batch size from 1 to 32 will reduce gradient noise enough for
convergence, because individual gradient samples have high variance but
batch-averaged gradients should be stable.

## Expected Outcome
Loss should decrease monotonically with batch=32, vs oscillating with batch=1.
If loss still oscillates with batch=32, the variance problem is fundamental,
not just a sample-size issue.

Without a hypothesis, you don't know what the experiment is testing. You'll see a number and not know if it's good or bad.

The scientific method, applied:

1. Observe — notice something in your data, results, or literature
2. Question — formulate a specific question about what you observed
3. Hypothesize — predict an answer and state WHY you expect it
4. Experiment — design a run that can confirm or reject the hypothesis
5. Analyze — compare results to the prediction
6. Conclude — state what was learned, update your mental model
7. Repeat — the conclusion feeds the next observation

2. Kill Gates

Research questions get structured investigation with kill gates:

Phase 1: Literature review (what exists?)
Phase 2: Build minimal viable module
Phase 3: Integration experiments (defined matrix of configs)
Phase 4: Investigate properties (hypothesis-based, only if Phase 3 passes)
Phase 5: Optimization (only if Phase 4 is promising)

Each phase is a gate. If Phase 3 fails, Phase 4 doesn't happen. This prevents sunk-cost fallacy on dead-end research.

3. Constraint-Driven Design

Start with hard limits. Let them drive architecture.

## Constraints
- [Your hardware/budget/time limit here]
- [Your performance requirement here]
- [Your compatibility requirement here]

## Design (driven by above)
- [Architecture decision that follows from constraints]

Don't design the ideal system then optimize. Design around your actual constraints from day one.

4. Explicit Deferral

Every milestone says what it's NOT doing and why:

**Deferred:** [Feature/capability].
**Reason:** [Why it's not worth doing now].

Three categories for all work items:

• Active/Concrete — being built now, with tests
• Outlined — architectural sketch, no code yet
• Deferred — recognized need, explicitly postponed with rationale

This prevents scope creep while preserving ideas for later.

5. Tradeoff Tables

Every complex decision gets a table showing the option space:

| Option    | Pros             | Cons                  | When to pick   |
|-----------|------------------|-----------------------|----------------|
| Option A  | Simple, fast     | Doesn't scale past N  | Prototyping    |
| Option B  | Scales well      | Complex to implement  | Production     |
| Option C  | Best performance | Vendor lock-in        | If budget > $X |

No "we chose X because it's better." Instead: "here's the tradeoff space, here's what we picked and why."

6. Testable Success Criteria

Every milestone ends with a concrete, falsifiable statement:

**Success Criteria:**
- [System] does [specific thing] in [specific time/resource bound]
- [N] tests pass with zero warnings
- [Specific scenario] produces [specific measurable result]

Not "it should work" — specific numbers, specific scenarios.

7. Math First, Code Second

Abstract ideas get mathematical or formal framing before any code is written:

Idea (what if we tried X?)
  --> Formalization (what does X actually mean mathematically?)
    --> Existing work (has anyone done this? what libraries exist?)
      --> Experiment plan (how would we test this?)
        --> Validation gate (what result would make us continue vs abandon?)

This filters out ideas that sound good but don't survive formalization.

8. Documentation as Specification

Write implementation-ready docs before coding:

1. Write detailed spec in markdown (API signatures, wire formats, data structures)
2. Implement against the spec
3. Test against the spec's success criteria
4. Update the spec if reality diverges

The spec IS the source of truth. Code follows docs, not the other way around.

Experiment Tracking

All experiments live in the experiments/ folder and follow strict naming and documentation conventions.

Naming Convention

Experiments are numbered sequentially with a descriptive kebab-case suffix:

experiments/
├── run1-batch-gradient-baseline.c
├── run2-per-sample-online.c
├── run3-leaky-relu-activation.c
├── run4-temperature-annealing.c
└── ...

Rules:

• Prefix: run<N>- where N is sequential (run1, run2, run3...)
• Suffix: short kebab-case description of what this run changes vs the previous
• One variable changed per run when possible (isolation)
• If a run is a variant of another, use sub-numbering: run9b-normalization-lower-temp
• Run numbering stays globally sequential across the whole project — don't restart at run1 inside subfolders

When a single experiment direction spawns many runs, group them in a subfolder inside experiments/:

experiments/
├── run1-baseline.ext
├── run2-different-approach.ext
├── scaling-study/              # Subfolder for related runs
│   ├── run3-small-scale.ext
│   ├── run4-medium-scale.ext
│   └── run5-large-scale.ext
└── ...

Results Documentation

Every experiment's results are documented in results/. One results file per experiment or per logical group:

results/
├── experiment_1_results.md
├── experiment_2_results.md
└── ...

Use results/TEMPLATE-results.md as the starting point for each results file.

Key Rules for Results

1. Every run gets a row in the summary table — even failed runs. Especially failed runs.
2. Link to source files — reference experiment files as experiments/run1-descriptive-name.ext
3. Include raw output — paste actual terminal output. Don't just summarize.
4. One finding per discovery — each finding gets its own subsection with a bold Verdict line
5. Comparison tables for pivots — when switching approaches, add a side-by-side comparison table
6. Strikethrough completed next steps — mark done items with ~~strikethrough~~ and add DONE
7. Decision points with kill gates — define upfront what results would make you continue, pivot, or abandon
8. Literature context — compare your results to published work

Running Experiments

All experiments MUST be run in the background so the user can continue chatting while they execute. Never run a build or training run in the foreground — it blocks the conversation. Use background execution and check on results later.

Milestone Format

Every implementation milestone follows this structure:

### Milestone X.Y: Title — [STATUS]

**What:** One-sentence summary of the deliverable.

**Tasks:**
- [ ] Task with enough detail that someone else could do it
- [ ] Another task

**Completed.** [Summary paragraph when done.]

**Deferred:** [What was intentionally left out and why.]

**New Files:**

| File | What |
|------|------|
| path/to/file | One-line description |

**API Surface:**
[Function signatures or interface definitions]

**Depends on:** Milestone X.Y-1

**Success Criteria:** [Falsifiable statement with numbers]

**Test Cases:**
1. [Specific scenario] — [expected result]
2. [Specific scenario] — [expected result]

Dependency Graphs

Milestones form a dependency chain. Visualize it:

Phase 1:  1.1 --> 1.2 --> 1.3 --> 1.4 --> 1.5
                    |                |
Phase 2:            v                v
              2.1 --> 2.2 --> 2.3 --> 2.4

This makes it obvious what can be parallelized and what blocks what.

Principles

1. Hypothesis before experiment — no code runs without a written prediction of what will happen and why. This is the most important rule.
2. Speculation is welcome but quarantined — vision docs are clearly marked as exploratory
3. Implementation follows specs — not the reverse
4. Deferral is first-class — saying "not now" with a reason is as important as saying "yes"
5. Constraints are features — limits drive creative solutions rather than being obstacles
6. Tests are the definition of done — not "it compiles" but "N tests pass with zero warnings"
7. Tradeoffs are documented — future-you can understand why decisions were made
8. Kill gates protect your time — research that fails a gate gets abandoned, not propped up
9. Every experiment is tracked — numbered, named, with raw output in results files. Failed runs are documented just as thoroughly as successes.
10. One variable per run — isolate changes so you know what caused the result. Sub-number variants (run9b) when testing minor tweaks.
11. Experiments run in the background — never block the user's ability to chat. All builds, training runs, and long-running processes execute in the background.

Commit Rules

• Never mention Claude, AI, or generated code in commits
• Write as the developer, first person
• Concise and descriptive

Customization Checklist

When creating a new project from this template:

• Update README.md with project name, problem, idea, approaches
• Fill in goals.md with success criteria from the project plan
• Fill in spec.md if the project has a technical specification (otherwise delete it)
• Set up open_questions.md with initial research questions
• Create initial experiment plans in docs/ using TEMPLATE-experiment-plan.md
• Add project-specific entries to .gitignore (language-specific, data formats, etc.)
• Update the "Project-Specific Configuration" section below
• Initialize git and create GitHub repo

---

Project-Specific Configuration

Project Name

[PROJECT_NAME]

Description

[Brief description of the research project]

Research Question

[The core question this project investigates]

Tech Stack

[Languages, frameworks, libraries being used]

TeamIDE Workspace ID

[Workspace ID if deployed via TeamIDE]