Skip to content

Status and Release Policy

This page explains how the public website interprets task status.

The public site applies both a source boundary and a status boundary. This page explains how a task becomes eligible for the website.

Current Public Rule

The public website shows only Tasks whose approved frozen Revision was merged into the private benchmark through an administrator-reviewed Draft PR, whose public-safe files are tracked on public ASI-Bench main with status: final, and which have been added to a manually refreshed website manifest.

This prevents an internal Task, a public working branch, or an approved-but-not-published Revision from appearing automatically.

Why the Website Uses a Narrower Subset

The public task repository and the website serve different purposes:

  • the public repository is the canonical source for publishable task metadata and files
  • the website presents the release-ready subset and links every task back to an exact source commit

Internal repositories, submitted drafts, run logs, and private scoring material are outside this publication path.

Required Release State

All four conditions are required:

  1. Approved — an assigned reviewer approved the exact frozen Revision.
  2. Accepted internally — an administrator reviewed and merged the private benchmark Draft PR.
  3. Final and public-safe — an administrator placed only the approved public files on public main with status: final.
  4. Website released — an administrator regenerated and reviewed the checked-in website manifest.

test, in_development, needs_further_development, and abandoned Tasks remain outside the website.

Manual Publication

Task catalog and leaderboard updates are deliberately manual. The website build reads only the checked-in public snapshot and never discovers new private or public Tasks automatically.

Approval does not mean published, and creating a Draft PR does not mean published.