Installation¶

Install the public ASI-Bench runner before downloading task instances or connecting an agent.

Requirements¶

Python 3.11 or newer
enough local storage for task workspaces and result artifacts
Linux namespace support for isolated custom-command runs, or Docker for compatible built-in os adapters
credentials required by your chosen agent or model provider

Install¶

pip install asibench
asibench --help

The installed command is asibench. The legacy ai4sci-bench command remains an alias, but new instructions use asibench consistently. Only the asibench Python distribution is published; Python integrations continue to import ai4sci_bench.

The package contains the framework, not benchmark instances. Download public prompts and input data separately with asibench task pull.

Public command surface

The retired internal commands batch-run, eval, quickeval, fullrun, rerun-flagged, and pipeline are not part of the public CLI. Use run, report, batch-report, review, and the Portal submission flow.

Verify the Environment¶

asibench task pull --output-dir hf_instances/

If this succeeds, continue to Run Your First Task.

Task-specific dependencies

Individual tasks may declare additional runtime packages. The selected sandbox installs or isolates those dependencies during execution.