Benchmark Inputs

Truth table

The tracked benchmark input is:

  • data/manuscript/manuscript_primary.csv

This table defines the standard human full cohort analysis. The evaluation code uses the guide sequence, off target sequence, chromosome, truth label, and observed editing signal to construct the benchmark site set.

Staging the external data archive

The larger files are deposited outside GitHub. The expected archive contents are listed in config/zenodo_artifacts.yml. Once the Zenodo record is available, stage the archive with:

python scripts/download_zenodo_artifacts.py --archive-url https://zenodo.org/records/20627722/files/offtarget_prediction_benchmark_zenodo_artifacts_v1.zip?download=1

The GitHub Pages workflow uses the same script. Set the repository variable OFFTARGET_BENCHMARK_ZENODO_ARCHIVE_URL to the direct Zenodo zip URL before running the workflow on GitHub. The archive is extracted into data/zenodo/, so its top level should contain figure2_score_distributions/, standard_tool_predictions/, and no_bulge_ml_tool_predictions/.

Standardized prediction contracts

The public rerun expects one stored contract file per tool:

  • data/zenodo/standard_tool_predictions/prediction_contract_<tool>.csv

Each contract file is already normalized to a common site level schema. This removes format differences across tools before evaluation begins.

Matching fields

The benchmark matches predicted sites to benchmark sites on the normalized guide sequence, off target sequence, and chromosome. Ranks and raw scores are then used for guide level recall summaries, precision recall curves, and combination analyses.

Relation to the figures

  • Figure 1 is derived directly from the truth table.
  • Figure 2 is derived from staged scored-candidate manuscript asset tables.
  • Figures 3 to 6 are derived from benchmark outputs written after evaluation.