Submit Benchmark Results¶
Use the ASI-Bench CLI to package and upload a produce-only run for integrity checks, confirmation, maintainer scoring, and possible publication on the official leaderboard.
Current Workflow¶
- Produce public benchmark outputs with
asibench run --no-score. - Run the canonical submit command. It builds a schema-v1
.tar.gzbundle and, when a Portal endpoint is configured, uploads it as a private draft. - Open the confirmation link printed by the CLI, review the integrity and completeness report, then click Confirm.
- Maintainers score the confirmed bundle against the private reference set.
- Approved scores may be published to the official leaderboard.
Do not manually construct, rename, or repackage a raw results folder, and do not upload a ZIP file. The CLI records the current schema, checksums, output bytes, instance identity, and run provenance required for official evaluation.
When no upload endpoint is configured
The CLI still creates a local bundle, but prints NOT SUBMITTED — nothing was uploaded. Configure ASIBENCH_SUBMIT_ENDPOINT or pass --endpoint <url>, then run the command again to upload.
Uploading is not the last step
A successful upload creates a private draft. Open the confirmation link returned by the CLI and click Confirm before the run enters the maintainer scoring queue.
Required Properties¶
| Requirement | Details |
|---|---|
| Archive | .tar.gz with regular files only; no links, devices, or parent paths |
| Schema | submission_schema_version must be 1 |
| Integrity | Every included file is indexed with its relative path, byte size, and SHA-256 |
| Instances | At least one instance with identity, prompt level, provenance, and outputs |
| Sandbox | Effective sandbox mode is recorded in instance provenance |
| Privacy | Reference answers and private scoring configuration are not included |