Result Bundle Format

`asibench submit` generates a gzip-compressed tar archive containing one `manifest.json` and the files declared by that manifest. This page documents the generated format; runners should not assemble or edit it by hand.

Archive Layout

submission_<timestamp>.tar.gz
└── submission_<timestamp>/
    ├── manifest.json
    ├── run_metadata.json                         # optional
    └── instances/
        └── <instance-base-name>/
            ├── result.json
            ├── framework_task_info.json          # optional
            └── outputs/
                └── <agent artifacts>

The archive may use a single enclosing directory. Exactly one manifest.json must exist. All archive members must be regular files or directories, and paths must be relative without .. components.

Manifest Contract

{
  "submission_schema_version": 1,
  "created_at": "2026-08-04T03:00:00Z",
  "framework_version": "<runner version>",
  "benchmark_repo": "Apexintelligence-AI/ASI-Bench-seed42",
  "note": "<optional submitter note>",
  "run_metadata_included": false,
  "instance_count": 1,
  "instances_missing_outputs": [],
  "instances": [
    {
      "base_name": "physics.example__seed42__b1",
      "instance_id": "physics.example__seed42",
      "task_id": "physics.example",
      "prompt_level": "b1",
      "attempt": 1,
      "agent_name": "direct_llm",
      "status": "completed",
      "result_file": "instances/physics.example__seed42__b1/result.json",
      "framework_task_info_file": null,
      "output_files": [
        {
          "path": "outputs/result.csv",
          "sha256": "<64 lowercase hexadecimal characters>",
          "bytes": 1234
        }
      ],
      "missing_outputs": [],
      "provenance": {
        "agent": {
          "agent_name": "direct_llm",
          "config": {"model": "<model>", "effort": "medium"}
        },
        "sandbox": {"effective_mode": "os"}
      }
    }
  ],
  "files": [
    {
      "path": "instances/physics.example__seed42__b1/result.json",
      "bytes": 1234,
      "sha256": "<64 lowercase hexadecimal characters>"
    },
    {
      "path": "instances/physics.example__seed42__b1/outputs/result.csv",
      "bytes": 1234,
      "sha256": "<64 lowercase hexadecimal characters>"
    }
  ]
}

Manifest file paths are relative to the directory containing manifest.json. The top-level files array indexes every non-manifest file actually written to the bundle, including optional metadata and task-info snapshots. Every listed file must exist with the declared byte size and SHA-256 digest. benchmark_repo is the optional Hugging Face repository ID supplied for provenance, not a GitHub URL.

For a produced output, output_files records path, bytes, and sha256. If an agent did not produce a declared output, the record instead contains its original relative path and "missing": true; the same path appears in missing_outputs, and the instance base name appears in instances_missing_outputs. Missing outputs are reported as incomplete rather than represented with "missing": false.

Unexpected files are reported during confirmation, while missing or mismatched files make the integrity report fail.

Portal Safety Checks

• Rejects absolute paths, parent traversal, symlinks, hardlinks, devices, and FIFOs.
• Caps compressed size, uncompressed size, and archive member count.
• Rejects missing, multiple, malformed, or unsupported manifests.
• Verifies declared file sizes and SHA-256 hashes.
• Reports missing instance outputs before confirmation.

Generate the bundle with asibench submit --results-dir <results-dir>. Do not edit or repackage it: any byte change invalidates the manifest digest and prevents a clean confirmation.