Packaging a Reproducible Capstone Project

Hello. Your capstone now has the essential scientific pieces: a validated cohort, a version-pinned connectivity analysis, visual summaries, and an uncertainty/sensitivity record. This final lesson turns those pieces into a reproducible project release: a package another person can inspect, run in a clean environment, and understand without reconstructing decisions from notebook history.

The goal is not merely to upload a notebook. A good package makes clear which FlyWire snapshot was queried, which annotation release informed membership, which files are immutable inputs, how results were generated, and where the conclusions stop. Plan for about 40 minutes: define the reproducibility contract, organize the repository, add machine-readable provenance, and perform one clean execution test.

---

Reproducibility means preserving a computational claim

For this capstone, your main finding is conditional: it applies to a stated cohort, a stated FlyWire materialization, stated query filters, and stated validation decisions. Packaging should preserve all four.

A practical project release has two complementary guarantees:

1. Artifact reproduction: someone can regenerate your tables and figures from the exact frozen inputs you saved.
2. Data re-query: someone with appropriate FlyWire access can rerun the remote query using the recorded materialization and parameters, provided that snapshot remains available.

The first is the stronger immediate guarantee. Remote connectomics data are versioned and may have expiry constraints; an analysis should not become impossible to inspect merely because a materialization is no longer queryable. The second is still valuable because it documents how the canonical data were obtained.

The distinction is especially important in FlyWire. A root ID is meaningful in relation to a segmentation state, and a materialization is the coherent data snapshot in which root mappings, synapses, and annotation-table queries are interpreted.

The figure below gives a compact model of this idea. At time $T_0$, segmentation and annotations define an analysis snapshot. Later edits alter them, but the lineage graph and time-aware queries make it possible to interpret what changed rather than treating the new state as if it had always been there.

For your project, use this rule:

Never overwrite a result’s provenance with a later FlyWire state. A later analysis may supersede it, but it should be a new recorded run.

---

Design the project around immutable inputs and regenerated outputs

Exploratory notebooks often mix downloaded data, manual decisions, plots, API calls, and temporary outputs in one directory. That is efficient while investigating, but difficult to audit. Your release should separate these concerns.

Use a repository layout such as this:

flywire-cohort-capstone/
├── README.md
├── requirements.txt
├── requirements-lock.txt
├── .gitignore
├── config/
│   └── project_config.json
├── notebooks/
│   └── 01_cohort_connectivity_capstone.ipynb
├── data/
│   ├── frozen/
│   │   ├── cohort_connectivity_edges.csv
│   │   ├── cohort_validation.csv
│   │   ├── cohort_uncertainty_ledger.csv
│   │   └── cohort_connectivity_provenance.json
│   └── derived/
│       ├── cohort_sensitivity_overview.csv
│       ├── cohort_sensitivity_edge_comparison.csv
│       └── ...
├── figures/
│   ├── strongest_connections.png
│   └── ...
└── provenance/
    ├── project_manifest.json
    ├── input_checksums.csv
    └── environment.json

The key boundary is:

• data/frozen/ contains the canonical inputs used to reproduce the reported capstone. Do not let the notebook modify these files.
• data/derived/ contains tables regenerated from frozen inputs.
• figures/ contains regenerated presentation artifacts.
• config/ contains the declared analysis choices.
• provenance/ records versions, environment information, checksums, and the Git commit.

This separation is similar to a small production data pipeline: configuration and inputs are explicit; computation is repeatable; outputs can be rebuilt.

Keep a frozen input set

Copy the capstone’s essential analysis inputs into data/frozen/ once you have completed the analysis. At minimum, preserve:

• the canonical directed connectivity edge table;
• the cohort validation table;
• the uncertainty ledger;
• the original query provenance JSON;
• any small annotation subset that was necessary for cohort membership or labeling.

Do not silently replace these with a newer query result. If you later repeat the study against another materialization, create a separate dated or tagged release.

For a small capstone dataset, it may be appropriate to commit frozen CSV and JSON files to Git. If an input is too large or cannot be redistributed, do not commit it just because it makes the repository convenient. Instead:

1. Add it to .gitignore.
2. Record its filename, checksum, source, access conditions, and retrieval method in the README and manifest.
3. Ensure the notebook fails clearly when the missing input is needed.

Never commit FlyWire credentials, API tokens, browser cookies, or a .env file.

A minimal .gitignore for this project could be:

# Secrets and local settings
.env
.env.*
*.pem

# Jupyter and Python transient files
.ipynb_checkpoints/
__pycache__/
*.py[cod]

# Large or non-redistributable local inputs
data/private/
data/raw/

The FlyWire annotation repository is a useful real-world model: it separates code from supplemental files and treats releases as meaningful versions rather than as an undifferentiated “latest” dataset.

---

Make provenance machine-readable

Your notebook’s Markdown narrative is useful for a reader, but a machine-readable configuration is better for checking that the notebook, frozen data, and stated findings agree.

Create config/project_config.json. Replace every placeholder before packaging:

{
  "project_title": "FlyWire cohort connectivity capstone",
  "analysis_question": "<FILL_IN_A_NARROW_CONNECTOMICS_QUESTION>",
  "flywire": {
    "dataset": "<FILL_FROM_QUERY_PROVENANCE>",
    "materialization_version": "<FILL_FROM_QUERY_PROVENANCE>",
    "coordinate_system": "<FILL_FROM_MODULE_3_NOTEBOOK>"
  },
  "cohort": {
    "selection_criteria": "<FILL_IN>",
    "primary_member_file": "data/frozen/cohort_validation.csv",
    "validation_rule": "<FILL_IN>"
  },
  "annotations": {
    "source": "<FILL_IN_SOURCE>",
    "release_or_commit": "<FILL_IN_RELEASE_OR_COMMIT>",
    "retrieved_utc": "<FILL_IN_UTC_TIMESTAMP>"
  },
  "synapse_query": {
    "synapse_table": "<FILL_IN>",
    "directions": "incoming and outgoing",
    "filters": "<FILL_IN_ALL_QUERY_FILTERS>",
    "autapse_handling": "retained in canonical table; excluded from strongest-edge display"
  },
  "analysis": {
    "strong_edge_rule": "<FILL_IN_TOP_K_AND_TIE_RULE>",
    "default_run_mode": "frozen"
  }
}

There are two version identifiers that should remain distinct:

Version	What it identifies	Why it matters
FlyWire materialization version	A coherent snapshot of segmentation-linked tables	Determines which root IDs and synapse mappings your query used
Annotation release or commit	The state of annotation labels and metadata	Determines how labels were interpreted and, if labels selected the cohort, who was included
Git commit or release tag	Your code, configuration, frozen files, and documentation	Identifies the exact project package behind your reported findings

A materialization such as 783 is not automatically an annotation release. Conversely, an annotation repository can update labels while retaining the same underlying materialization. Record both.

Add a project manifest and checksums

The following notebook cell creates an environment record and checksums for the inputs. Run it after you have copied canonical files into data/frozen/.

from __future__ import annotations

import hashlib
import json
import platform
import subprocess
import sys
from datetime import datetime, timezone
from importlib.metadata import PackageNotFoundError, version
from pathlib import Path

import pandas as pd

PROJECT_ROOT = Path.cwd()
FROZEN_DIR = PROJECT_ROOT / "data" / "frozen"
PROVENANCE_DIR = PROJECT_ROOT / "provenance"
CONFIG_PATH = PROJECT_ROOT / "config" / "project_config.json"

PROVENANCE_DIR.mkdir(parents=True, exist_ok=True)

required_frozen_files = [
    "cohort_connectivity_edges.csv",
    "cohort_validation.csv",
    "cohort_uncertainty_ledger.csv",
    "cohort_connectivity_provenance.json",
]

missing_files = [
    filename
    for filename in required_frozen_files
    if not (FROZEN_DIR / filename).exists()
]

if missing_files:
    raise FileNotFoundError(
        "Missing canonical frozen inputs: "
        + ", ".join(missing_files)
    )


def sha256_file(path: Path) -> str:
    digest = hashlib.sha256()
    with path.open("rb") as file:
        for block in iter(lambda: file.read(1024 * 1024), b""):
            digest.update(block)
    return digest.hexdigest()


def installed_version(package_name: str) -> str | None:
    try:
        return version(package_name)
    except PackageNotFoundError:
        return None


def git_commit() -> str | None:
    result = subprocess.run(
        ["git", "rev-parse", "HEAD"],
        cwd=PROJECT_ROOT,
        capture_output=True,
        text=True,
        check=False,
    )
    return result.stdout.strip() if result.returncode == 0 else None


checksums = pd.DataFrame(
    [
        {
            "path": str(path.relative_to(PROJECT_ROOT)),
            "bytes": path.stat().st_size,
            "sha256": sha256_file(path),
        }
        for path in sorted(FROZEN_DIR.iterdir())
        if path.is_file()
    ]
)

checksums.to_csv(
    PROVENANCE_DIR / "input_checksums.csv",
    index=False,
)

environment = {
    "created_utc": datetime.now(timezone.utc).isoformat(),
    "python_version": sys.version,
    "platform": platform.platform(),
    "packages": {
        package: installed_version(package)
        for package in [
            "pandas",
            "matplotlib",
            "seaborn",
            "fafbseg",
            "caveclient",
            "jupyter",
        ]
    },
}

(PROVENANCE_DIR / "environment.json").write_text(
    json.dumps(environment, indent=2, sort_keys=True)
)

print(checksums)
print(environment)

Then build provenance/project_manifest.json by combining your configuration with the prior query provenance:

config = json.loads(CONFIG_PATH.read_text())

query_provenance_path = (
    FROZEN_DIR / "cohort_connectivity_provenance.json"
)
query_provenance = json.loads(query_provenance_path.read_text())

placeholder_values = []


def find_placeholders(value, location="config"):
    if isinstance(value, dict):
        for key, nested_value in value.items():
            find_placeholders(nested_value, f"{location}.{key}")
    elif isinstance(value, list):
        for index, nested_value in enumerate(value):
            find_placeholders(nested_value, f"{location}[{index}]")
    elif isinstance(value, str) and "<FILL_IN" in value:
        placeholder_values.append(location)


find_placeholders(config)

if placeholder_values:
    raise ValueError(
        "Replace all configuration placeholders before release: "
        + ", ".join(placeholder_values)
    )

if config["flywire"]["dataset"] != query_provenance["dataset"]:
    raise ValueError("Dataset differs between project config and query provenance.")

if str(config["flywire"]["materialization_version"]) != str(
    query_provenance["materialization_version"]
):
    raise ValueError(
        "Materialization differs between project config and query provenance."
    )

manifest = {
    "manifest_schema_version": "1.0",
    "created_utc": datetime.now(timezone.utc).isoformat(),
    "git_commit": git_commit(),
    "project_config": config,
    "query_provenance": query_provenance,
    "frozen_input_checksums": checksums.to_dict(orient="records"),
    "environment_file": "provenance/environment.json",
}

(PROVENANCE_DIR / "project_manifest.json").write_text(
    json.dumps(manifest, indent=2, sort_keys=True)
)

print("Wrote provenance/project_manifest.json")

The manifest deliberately does not contain a password, token, or personally identifying browser session. Authentication should come from an untracked local environment variable or login flow.

---

Turn the notebook into a clean, standalone document

Your capstone notebook should have one default execution path: reproduce the reported artifacts from frozen inputs. This makes the project inspectable even for a reader who cannot access FlyWire services.

Near the top of the notebook, make the run mode visible:

RUN_MODE = "frozen"  # "frozen" for the released analysis; "remote" for a re-query

For the release, RUN_MODE = "frozen" should:

• load data/frozen/cohort_connectivity_edges.csv;
• load validation and uncertainty records;
• regenerate summaries, sensitivity tables, and figures;
• write outputs only to data/derived/ and figures/;
• verify that the dataset and materialization recorded in the manifest match the frozen query provenance.

A future RUN_MODE = "remote" can contain the authenticated FlyWire query path, but it must use the same pinned materialization and should never replace frozen inputs automatically. If it produces a genuinely updated analysis, save it as a new run or release.

Organize notebook cells in this order:

1. Question and scope
   State the cohort, the directional connectivity question, and the claim boundaries.

2. Configuration and provenance
   Load project_config.json, the query-provenance JSON, and the frozen-input checksum table.

3. Imports and paths
   Put imports in one cell and define project paths once.

4. Load and validate inputs
   Read frozen tables. Check required columns, root-ID types, and pinned materialization consistency.

5. Reproduce analysis
   Calculate cohort edge counts, degree summaries, strongest edges, and sensitivity results.

6. Generate figures
   Save each figure with a stable filename and a descriptive title.

7. Findings and limitations
   State findings in prose, then connect each limitation to the uncertainty ledger and sensitivity output.

8. Release manifest
   Build or verify checksums and print the Git commit used for the run.

The short video below demonstrates two habits worth adopting: do not place unmanageably large data directly in Git history, and refactor exploratory cells until the notebook can run as an independent document.

---

Write a README for a new analyst, not for your future memory

A strong README allows someone to understand the project before opening the notebook. It should report your actual result but avoid overstating it.

Use this compact structure:

# FlyWire cohort connectivity capstone

## Research question
[Your narrowly scoped question.]

## Main finding
[Your observed result, qualified by cohort, materialization, and filters.]

## Scope
This analysis uses FlyWire dataset [dataset], materialization
[version], and annotation release [release or commit].

## Repository contents
- `notebooks/`: executable capstone notebook
- `data/frozen/`: canonical inputs used for reported results
- `data/derived/`: regenerated tables
- `figures/`: regenerated visualizations
- `config/`: analysis choices
- `provenance/`: manifest, checksums, environment record

## Quick start
1. Create a Python environment.
2. Install `requirements.txt`.
3. Open and run `notebooks/01_cohort_connectivity_capstone.ipynb`.
4. Use the default `RUN_MODE = "frozen"`.

## Data and authentication
Frozen inputs are [included / not included because ...].
Remote FlyWire re-querying requires appropriate authentication.
Credentials are not included in this repository.

## Reproducibility checks
The notebook verifies frozen-input checksums and records package versions
in `provenance/environment.json`.

## Limitations
[List the specific reconstruction, annotation, cohort, and version limits
identified in the uncertainty ledger.]

## Citation and attribution
[Record the FlyWire, annotation, and software citations appropriate to
your actual data sources.]

Treat requirements.txt and requirements-lock.txt differently:

• requirements.txt contains the direct runtime dependencies you intend another analyst to install.
• requirements-lock.txt is the exact pip freeze output from your successful environment. It is an audit record and may be more platform-specific.

From the activated environment that successfully runs the notebook, create the lock file:

python -m pip freeze > requirements-lock.txt

Before committing, inspect it briefly. A lock file is useful evidence, but it does not replace a README that explains what the code does.

---

Perform a release-quality execution test

A notebook that contains correct-looking output is not necessarily reproducible. Its cells may have been run out of order, may depend on hidden variables, or may rely on a local file not represented in the project.

Before declaring the capstone complete:

1. Commit the current project state or copy it to a clean test directory.
2. Restart the Jupyter kernel.
3. Run all cells from top to bottom with RUN_MODE = "frozen".
4. Confirm that the notebook recreates its tables and figures without manual intervention.
5. Confirm that input_checksums.csv matches the frozen inputs.
6. Read the final findings and limitations as if you had not performed the analysis yourself.
7. Check that no credentials, machine-specific absolute paths, or unexplained manual edits remain.
8. Commit the successful run and apply a release tag.

A typical Git release sequence is:

git status
git add README.md config/ notebooks/ data/frozen/ data/derived/ figures/ provenance/
git commit -m "Release reproducible FlyWire cohort capstone"
git tag -a v1.0.0 -m "Reproducible capstone release"
git push origin main --tags

Use a private repository if any project files cannot be shared publicly. Public availability is useful, but it is not a substitute for respecting data-access conditions or contributor policies.

---

Final checklist

Your capstone is ready to package when all of the following are true:

• The research question and scope are stated in the README.
• Dataset, materialization, coordinate system, annotation release, cohort rule, and query filters are recorded.
• Frozen inputs are either included with checksums or documented with a clear retrieval/access policy.
• The notebook’s default path regenerates outputs from frozen inputs.
• Derived tables and figures are separated from immutable inputs.
• The uncertainty ledger and sensitivity outputs are included.
• Limitations state exactly which claims are robust, conditional, or unresolved.
• Environment details and Git commit are captured in provenance files.
• A restart-and-run-all test succeeds without hidden state.
• Secrets and local-only files are excluded from Git.
• The final Git commit is tagged as a release.

---

Wrap-up

You have now completed the course’s final deliverable: a reproducible FlyWire cohort-connectomics project.

The central habits are straightforward but consequential:

• Pin the FlyWire materialization and record annotation provenance separately.
• Preserve canonical frozen inputs so the reported results remain inspectable.
• Treat the notebook as an executable document, not as a record of interactive history.
• Keep configuration, data, code, figures, findings, and limitations distinct but linked through a manifest.
• Test the whole project from a clean kernel before releasing it.

Your finished package should let a reviewer answer four questions quickly: What was asked? What exact data and decisions were used? Can the reported artifacts be regenerated? Which conclusions remain conditional? That is the standard your capstone now meets.