Skip to content
Home / Catalog / Privacy-Curve Accountant Forensics

Privacy-Curve Accountant Forensics

  • Task ID: computer_science.dp_accountant_forensics
  • Domain: computer_science
  • Subdomain: privacy_accounting
  • Status: final
  • Benchmark set: seed42 (60 tasks)
  • Tags: differential_privacy, privacy_accounting, numerical_composition, forensic_audit, privacy_curves

Runtime and requirements

  • Estimated time: 90-180 minutes
  • Python: >=3.10
  • Packages: numpy>=1.24, matplotlib>=3.7
  • GPU required: no
  • Network required: no

Public input and output contract

Inputs

  • data/audit_manifest.json (data): Agent-facing case metadata, anonymous accountant ids, target deltas, failure-label vocabulary, required output schemas, and the public diagnostic recipe.
  • data/mechanism_trace.csv (data): Mechanism settings by case, including composition count, sampling rate, noise multiplier, sensitivity scale, group size, and probe family.
  • data/mechanism_panels.npz (data): Public mechanism-side discrete support-grid evidence panels; benchmark answer tables are not exposed.
  • data/accountant_candidate_panels.npz (data): Anonymous candidate accountant curves, inverse epsilon panels, and diagnostic response panels.
  • data/probe_manifest.json (data): Neutral probe family names, array shapes, diagnostic column names, and a mirrored public diagnostic recipe.

Outputs

  • analysis.py (code): End-to-end analysis script that reads data/ and writes all requested outputs under results/.
  • results/accountant_audit.csv (data): One row per anonymous accountant with aggregate curve, inverse, tail, monotonicity, sampling, group, and reliability diagnostics.
  • results/privacy_curve.csv (data): Long-form per-case privacy-curve audit for all anonymous accountants.
  • results/delta_inverse_audit.csv (data): Long-form epsilon-at-delta inverse audit for all anonymous accountants.
  • results/mechanism_panel_diagnostics.csv (data): Per-case and per-accountant diagnostic panel agreement metrics.
  • results/failure_modes.json (data): Selected accountant id and one failure-mode label for every anonymous accountant.
  • results/selected_accountant.json (data): Selected reliable accountant id, selected label, and selected accountant score.
  • results/reliability_report.json (data): Structured report with selected accountant, aggregate diagnostics, ranking, and case warnings.
  • results/accountant_overview.png (figure): Non-empty diagnostic figure summarizing the anonymous accountant audit; low scoring weight.

Public repository files

View this task in ASI-Bench

Formal benchmark task directories in the public repository are metadata-only. Versioned prompts and inputs are distributed through the pinned benchmark dataset.

Provenance

This task is included in the current seed42 benchmark set at dataset revision f11a199f71fb4b854ac43a1bf548d5df519141a9.

This page is generated only from files tracked in the public ASI-Bench repository at commit c9c1ba960854. Submission bundles, run logs, private scoring configuration, and private reference answers are not read by this page generator.