Building a Reproducible Single-Neuron Profile Notebook

Hello again. Your notebook now has the core evidence for one FlyWire neuron: a root ID resolved at a chosen materialization, a directional synapse table, and separate rankings of its incoming and outgoing partners.

This final lesson in the module turns those pieces into a single-neuron profile notebook that another person, or future you, can inspect and rerun. The goal is not merely to produce an attractive plot. It is to package a versioned claim: for this root ID, at this materialization, using these query settings, these were the observed strongest connectivity partners.

Plan for roughly 40 minutes: a short reading, then notebook assembly, validation, visualization, and export.

---

A profile is a small reproducible analysis package

A FlyWire root ID is not an eternal biological identifier. Edits can change roots, and a connectivity query maps synapse-associated supervoxels to roots through a materialization. Therefore, a plot labelled only “top partners of neuron X” is incomplete.

A defensible single-neuron profile needs four layers:

Layer	What it answers	Example contents
Identity	Which neuron was analyzed?	Target root ID, display label, annotation snapshot
Query	Which data snapshot and settings produced the evidence?	Dataset, materialization, filters, coordinate convention
Evidence	What did the query return?	Directional synapse rows and partner rankings
Interpretation	What compact statement does the evidence support?	Top incoming and outgoing partners, with limitations

The materialization is especially important: it freezes the mapping from synapse locations to root IDs at a particular snapshot. If the same biological structure is later edited, a newer query may resolve to a different root and produce a different partner table.

For this lesson, treat your notebook much like a small data product in a software repository. Its outputs should not depend on hidden notebook state, manually remembered options, or a currently selected FlyWire UI view.

---

Establish the notebook’s contract

Create a new notebook, for example:

flywire_single_neuron_profile.ipynb

Begin with a Markdown cell stating the scope. Keep it factual and bounded:

# FlyWire single-neuron connectivity profile

**Question.** Which root IDs are the strongest observed presynaptic and
postsynaptic partners of the selected FlyWire neuron?

This notebook reports partner rankings separately for incoming and outgoing
synapse rows. All results are tied to the dataset and materialization recorded
below. Synapse-row counts are observed connectome data, not validated measures
of physiological strength.

Your first code cell should import libraries and define an output directory. It is good practice to use a clean directory rather than scattering CSV and image files beside the notebook.

from datetime import datetime, timezone
from pathlib import Path
import hashlib
import importlib.metadata as importlib_metadata
import json
import platform
import sys

import matplotlib.pyplot as plt
import pandas as pd

OUTPUT_DIR = Path("outputs/single_neuron_profile")
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)

The next cell is the notebook’s configuration record. It should refer to the variables already created in the earlier lessons:

• DATASET
• MAT_VERSION
• TARGET_ROOT_ID
• directional_synapses
• incoming_partner_summary
• outgoing_partner_summary

You also need an annotation or metadata snapshot retrieved earlier in this module. Below, it is called annotation_snapshot. If your notebook used a different variable name, assign it here. An empty annotation table can still be meaningful, but it should be saved together with a record of how it was queried.

# Rename this assignment if your earlier notebook used another variable name.
# It should contain the metadata or annotation rows retrieved for this target.
annotation_snapshot = neuron_annotations.copy()

TOP_K = 12

PROFILE_IDENTITY = {
    "target_root_id": str(TARGET_ROOT_ID),
    "display_label": "unclassified target neuron",
    "annotation_interpretation": (
        "Use the saved annotation snapshot as the source of any cell-type "
        "or classification statement in this notebook."
    ),
}

COORDINATE_SYSTEM = {
    "axes": "x, y, z",
    "units": "nanometres",
    "source": (
        "FlyWire/CAVE coordinate convention used in the earlier synapse "
        "and annotation queries."
    ),
}

Set display_label to an annotation-supported label if you have one. Do not assign a confident biological name merely because a neuron has a suggestive morphology or partner distribution.

Record the actual query, not a reconstructed memory

The values in SYNAPSE_QUERY_PARAMETERS must match the query that produced directional_synapses. Copy the exact semantic options from the earlier notebook cell. In particular, record the filtering choices. If an option was deliberately omitted, write "omitted"; do not leave it as an unknown placeholder.

SYNAPSE_QUERY_PARAMETERS = {
    "function": "fafbseg.flywire.get_synapses",
    "dataset": str(DATASET),
    "materialization_version": int(MAT_VERSION),
    "target_root_id": str(TARGET_ROOT_ID),

    # Replace these values with the settings actually used earlier.
    "filtered": "RECORD_ACTUAL_VALUE",
    "min_score": "RECORD_ACTUAL_VALUE_OR_OMITTED",
    "transmitters": "RECORD_ACTUAL_VALUE_OR_OMITTED",
    "neuropils": "RECORD_ACTUAL_VALUE_OR_OMITTED",
    "clean": "RECORD_ACTUAL_VALUE_OR_OMITTED",
}

ANNOTATION_QUERY_PARAMETERS = {
    "source": "RECORD_ANNOTATION_TABLE_OR_METHOD",
    "dataset": str(DATASET),
    "materialization_version": int(MAT_VERSION),
    "target_root_id": str(TARGET_ROOT_ID),
    "selection_or_query": "RECORD_THE_QUERY_OR_SELECTION_CRITERION",
}

Add a guard that prevents you from exporting a notebook with unfinished provenance. This is deliberately strict: an analysis with an unknown filter is not fully reproducible.

def find_unrecorded_values(value, path=""):
    """Return configuration paths that still contain a RECORD placeholder."""
    missing = []

    if isinstance(value, dict):
        for key, nested_value in value.items():
            nested_path = f"{path}.{key}" if path else key
            missing.extend(find_unrecorded_values(nested_value, nested_path))

    elif isinstance(value, str) and "RECORD_" in value:
        missing.append(path)

    return missing


unrecorded = (
    find_unrecorded_values(SYNAPSE_QUERY_PARAMETERS)
    + find_unrecorded_values(ANNOTATION_QUERY_PARAMETERS)
)

if unrecorded:
    raise RuntimeError(
        "Fill in the actual query settings before exporting provenance:\n- "
        + "\n- ".join(unrecorded)
    )

This configuration is not busywork. It protects you from a common error in connectomics: comparing a current annotation or live root ID to a partner distribution derived from an older materialization without saying so.

---

Validate the profile inputs before plotting

A visualization can make bad data look authoritative. Before producing one, re-check the invariants from the previous lesson:

1. Incoming rows have the target in post.
2. Outgoing rows have the target in pre.
3. Partner-summary counts reconcile with the directional raw rows.
4. Every summary row has a partner root ID.

def validate_profile_inputs(
    directional_df,
    incoming_summary,
    outgoing_summary,
    target_root_id,
):
    required_columns = {
        "direction",
        "pre",
        "post",
        "partner_root_id",
    }
    missing_columns = required_columns.difference(directional_df.columns)

    if missing_columns:
        raise RuntimeError(
            f"directional_synapses is missing columns: {sorted(missing_columns)}"
        )

    target_root_id = int(target_root_id)

    incoming_rows = directional_df.loc[
        directional_df["direction"].eq("incoming")
    ]
    outgoing_rows = directional_df.loc[
        directional_df["direction"].eq("outgoing")
    ]

    if incoming_rows["post"].ne(target_root_id).any():
        raise RuntimeError(
            "At least one incoming row does not have the target in post."
        )

    if outgoing_rows["pre"].ne(target_root_id).any():
        raise RuntimeError(
            "At least one outgoing row does not have the target in pre."
        )

    if directional_df["partner_root_id"].isna().any():
        raise RuntimeError("At least one row has a missing partner root ID.")

    expected_incoming = len(incoming_rows)
    expected_outgoing = len(outgoing_rows)

    observed_incoming = int(incoming_summary["synapse_count"].sum())
    observed_outgoing = int(outgoing_summary["synapse_count"].sum())

    if expected_incoming != observed_incoming:
        raise RuntimeError(
            "Incoming summary does not reconcile with directional synapse rows."
        )

    if expected_outgoing != observed_outgoing:
        raise RuntimeError(
            "Outgoing summary does not reconcile with directional synapse rows."
        )

    print("Profile-input validation passed.")
    print(
        f"Incoming: {expected_incoming} synapse rows, "
        f"{len(incoming_summary)} unique partners."
    )
    print(
        f"Outgoing: {expected_outgoing} synapse rows, "
        f"{len(outgoing_summary)} unique partners."
    )


validate_profile_inputs(
    directional_synapses,
    incoming_partner_summary,
    outgoing_partner_summary,
    TARGET_ROOT_ID,
)

A successful check establishes that the plotted bars have a clear interpretation:

• An incoming bar means detected synapse rows from that partner onto the target.
• An outgoing bar means detected synapse rows from the target onto that partner.

The two panels must remain separate. Combining them would hide directionality and could turn a reciprocal relationship into a misleading single number.

---

Create a partner-distribution visualization

For a single-neuron profile, a pair of horizontal bar charts is usually more useful than a network diagram. It makes ranking readable, keeps root IDs visible, and preserves the distinction between inputs and outputs.

The function below plots the top $k$ partners in each direction. Each bar label contains both the synapse-row count and its fraction of that direction’s total. The fraction helps distinguish a sharply concentrated partner distribution from a long, diffuse one.

def plot_partner_distribution(
    incoming_summary,
    outgoing_summary,
    top_k,
    target_root_id,
):
    fig, axes = plt.subplots(
        nrows=2,
        ncols=1,
        figsize=(13, 10),
        constrained_layout=True,
    )

    plot_specs = [
        (
            axes[0],
            incoming_summary,
            "Top presynaptic partners: inputs to target",
            "#3b82c4",
        ),
        (
            axes[1],
            outgoing_summary,
            "Top postsynaptic partners: outputs from target",
            "#d97706",
        ),
    ]

    for axis, summary, title, color in plot_specs:
        plot_table = (
            summary.head(top_k)
            .sort_values("synapse_count", ascending=True)
            .copy()
        )

        if plot_table.empty:
            axis.text(
                0.5,
                0.5,
                "No synapse rows returned for this direction.",
                ha="center",
                va="center",
                transform=axis.transAxes,
            )
            axis.set_axis_off()
            continue

        root_labels = plot_table["partner_root_id"].map(str)
        bars = axis.barh(
            root_labels,
            plot_table["synapse_count"],
            color=color,
            alpha=0.9,
        )

        axis.set_title(title, loc="left", fontweight="bold")
        axis.set_xlabel("Detected synapse rows")
        axis.set_ylabel("Partner root ID")
        axis.grid(axis="x", alpha=0.25)
        axis.set_axisbelow(True)

        max_count = int(plot_table["synapse_count"].max())
        axis.set_xlim(0, max_count * 1.28)

        for bar, (_, row) in zip(bars, plot_table.iterrows()):
            label = (
                f'{int(row["synapse_count"])} '
                f'({row["share_of_direction"]:.1%})'
            )
            axis.text(
                bar.get_width() + max_count * 0.02,
                bar.get_y() + bar.get_height() / 2,
                label,
                va="center",
                fontsize=9,
            )

    fig.suptitle(
        (
            f"FlyWire partner distribution for root {TARGET_ROOT_ID}\n"
            f"Dataset: {DATASET} | Materialization: {MAT_VERSION}"
        ),
        fontweight="bold",
    )

    return fig


partner_figure = plot_partner_distribution(
    incoming_partner_summary,
    outgoing_partner_summary,
    top_k=TOP_K,
    target_root_id=TARGET_ROOT_ID,
)

figure_path = OUTPUT_DIR / "partner_distribution.png"
partner_figure.savefig(figure_path, dpi=200, bbox_inches="tight")
plt.show()

print(f"Saved figure: {figure_path.resolve()}")

Read the result with appropriate restraint:

• A tall incoming bar identifies a partner root with many detected synapses onto the target in this materialization.
• A tall outgoing bar identifies a partner root receiving many detected synapses from the target.
• The bars are counts, not measurements of synaptic efficacy, behavioral importance, or cell-type identity.
• The full rankings matter. A top-$12$ plot is a reporting choice, not evidence that all lower-ranked partners are negligible.

---

Export evidence and provenance together

The exported outputs should support both lightweight review and later re-analysis. Save:

1. The directional synapse rows that underlie aggregation.
2. The complete incoming and outgoing ranking tables.
3. The annotation snapshot.
4. A JSON provenance record.
5. The partner-distribution figure.

When exporting CSV files, root IDs should be strings. This avoids spreadsheet applications converting large integer identifiers into imprecise scientific notation.

def prepare_for_csv(table):
    """Copy a DataFrame and serialize common FlyWire identifier columns as text."""
    export_table = table.copy()

    id_columns = [
        column
        for column in [
            "id",
            "pre",
            "post",
            "partner_root_id",
            "root_id",
        ]
        if column in export_table.columns
    ]

    for column in id_columns:
        export_table[column] = export_table[column].astype("string")

    return export_table


def dataframe_sha256(table):
    """Fingerprint the exact exported table content for integrity checking."""
    csv_bytes = table.to_csv(
        index=False,
        na_rep="<NA>",
        lineterminator="\n",
    ).encode("utf-8")

    return hashlib.sha256(csv_bytes).hexdigest()


def save_csv(table, filename):
    export_table = prepare_for_csv(table)
    path = OUTPUT_DIR / filename
    export_table.to_csv(path, index=False)
    return path, dataframe_sha256(export_table)


directional_path, directional_hash = save_csv(
    directional_synapses,
    "directional_synapses.csv",
)

incoming_path, incoming_hash = save_csv(
    incoming_partner_summary,
    "incoming_partner_ranking.csv",
)

outgoing_path, outgoing_hash = save_csv(
    outgoing_partner_summary,
    "outgoing_partner_ranking.csv",
)

annotations_path, annotations_hash = save_csv(
    annotation_snapshot,
    "annotation_snapshot.csv",
)

print("Saved evidence tables.")

Now create one machine-readable provenance record. Notice the difference between the two timestamps:

• materialization_version identifies the scientific data snapshot.
• profile_generated_at_utc records when you executed the notebook.

The latter is useful operational metadata; it does not replace the materialization version.

def distribution_summary(summary):
    return {
        "synapse_rows": int(summary["synapse_count"].sum()),
        "unique_partners": int(len(summary)),
        "top_partner_root_id": (
            str(summary.iloc[0]["partner_root_id"])
            if not summary.empty
            else None
        ),
        "top_partner_synapse_count": (
            int(summary.iloc[0]["synapse_count"])
            if not summary.empty
            else None
        ),
    }


def package_version(package_name):
    try:
        return importlib_metadata.version(package_name)
    except importlib_metadata.PackageNotFoundError:
        return "not installed"


provenance = {
    "profile_generated_at_utc": datetime.now(timezone.utc).isoformat(),
    "profile_identity": PROFILE_IDENTITY,
    "coordinate_system": COORDINATE_SYSTEM,
    "synapse_query_parameters": SYNAPSE_QUERY_PARAMETERS,
    "annotation_query_parameters": ANNOTATION_QUERY_PARAMETERS,
    "partner_aggregation": {
        "input_table": "directional_synapses.csv",
        "incoming_rule": (
            "direction == incoming; post == target_root_id; "
            "partner_root_id == pre"
        ),
        "outgoing_rule": (
            "direction == outgoing; pre == target_root_id; "
            "partner_root_id == post"
        ),
        "aggregation_method": "groupby(partner_root_id).size()",
        "ranking": "descending synapse_count; stable root-ID tie order",
        "top_k_shown_in_figure": TOP_K,
    },
    "distribution_summary": {
        "incoming": distribution_summary(incoming_partner_summary),
        "outgoing": distribution_summary(outgoing_partner_summary),
    },
    "outputs": {
        "directional_synapses_csv": directional_path.name,
        "incoming_ranking_csv": incoming_path.name,
        "outgoing_ranking_csv": outgoing_path.name,
        "annotation_snapshot_csv": annotations_path.name,
        "partner_distribution_png": figure_path.name,
    },
    "sha256_of_exported_csv_content": {
        "directional_synapses": directional_hash,
        "incoming_partner_ranking": incoming_hash,
        "outgoing_partner_ranking": outgoing_hash,
        "annotation_snapshot": annotations_hash,
    },
    "software_environment": {
        "python": sys.version,
        "platform": platform.platform(),
        "pandas": package_version("pandas"),
        "matplotlib": package_version("matplotlib"),
        "fafbseg": package_version("fafbseg"),
    },
    "limitations": [
        (
            "Synapse-row counts are based on the recorded query settings and "
            "materialization; they may change with data versions, filtering, "
            "or reconstruction updates."
        ),
        (
            "Root IDs refer to the recorded materialization and should not be "
            "assumed to identify the same current root after later edits."
        ),
        (
            "Partner rank is an observed structural-connectivity summary, not "
            "a direct measurement of synaptic efficacy or functional role."
        ),
        (
            "Annotations are reported from the saved snapshot and may be "
            "missing, provisional, or revised in later releases."
        ),
    ],
}

provenance_path = OUTPUT_DIR / "provenance.json"

with provenance_path.open("w", encoding="utf-8") as file:
    json.dump(provenance, file, indent=2, default=str)

print(f"Saved provenance: {provenance_path.resolve()}")

A hash is not a scientific validation of the data. It is an integrity check: if someone receives your CSV and computes a different hash, they know they do not have precisely the exported table that produced your profile.

---

Add a concise human-readable result statement

End the notebook with a Markdown cell that reports what was actually observed while preserving its limits. You can generate values from the summaries rather than typing them by hand.

incoming_info = distribution_summary(incoming_partner_summary)
outgoing_info = distribution_summary(outgoing_partner_summary)

profile_statement = f"""
## Result summary

At materialization **{MAT_VERSION}** of the **{DATASET}** dataset, root
**{TARGET_ROOT_ID}** had **{incoming_info["synapse_rows"]}** incoming synapse
rows across **{incoming_info["unique_partners"]}** partner roots and
**{outgoing_info["synapse_rows"]}** outgoing synapse rows across
**{outgoing_info["unique_partners"]}** partner roots.

The highest-ranked observed incoming partner was root
**{incoming_info["top_partner_root_id"]}** with
**{incoming_info["top_partner_synapse_count"]}** synapse rows. The
highest-ranked observed outgoing partner was root
**{outgoing_info["top_partner_root_id"]}** with
**{outgoing_info["top_partner_synapse_count"]}** synapse rows.

These are materialization-specific structural-connectivity summaries. They
should be interpreted alongside annotation provenance, reconstruction quality,
and the stated synapse-query filters.
"""

print(profile_statement)

Copy the rendered text into a final Markdown cell, or use it as a compact README-style summary beside your exported files.

Before considering the profile complete, use this review checklist:

• Dataset and materialization are visible near the notebook start and in the figure title.
• Target root ID is serialized as text in every CSV and JSON record.
• Synapse-query filters are actual values, not placeholders.
• Incoming and outgoing partners are ranked and plotted separately.
• Raw directional rows reconcile with both ranking tables.
• Annotation output and its retrieval method are saved.
• The figure, data tables, and provenance JSON are all present under outputs/single_neuron_profile/.
• The conclusion describes observed data without claiming verified functional significance.

---

Key takeaways

You have assembled the core form of a reproducible FlyWire single-neuron profile:

• A partner-distribution figure communicates the strongest observed input and output partners without collapsing directionality.
• A query record ties every result to a dataset, materialization, root ID, coordinate convention, and filtering choices.
• Evidence exports retain the directional synapse rows and complete rankings behind the top-$k$ visual summary.
• A provenance JSON records methods, software versions, output hashes, and interpretive limitations.
• Root IDs and connectivity claims are always materialization-specific; a newer FlyWire state may legitimately yield different results.

In the next module, you will step behind the connectome interface and build a hands-on proxy for the segmentation-learning pipeline: from labelled EM image patches to a lightweight baseline segmenter and interpretable error analysis.