Command line

Check the connection, download the benchmark, and run it from the shell or CI.

The generalcontext command comes with the SDK. It reads GENERALCONTEXT_API_KEY like the SDK does, prints one plain line per result, and exits 0 when everything worked, 1 when a check failed, and 2 for a usage or setup problem, so a coding agent or a CI job can act on it.

pip install generalcontext
generalcontext verify

#Check the connection

verify shows which workspace the key belongs to, what the key can do, and when traffic last arrived and from where. With --session, it checks that one conversation arrived, with its number of calls and models. It never prints conversation content. Add --json for the raw answer.

generalcontext verify --session ticket-4821
# Conversation ticket-4821: arrived, 6 calls (gpt-6.1-terra)

#Download the benchmark

benchmark pull writes your ready tasks to a file and prints how many are in each split. It needs a key that can run the benchmark: on API keys, create one with Run the benchmark checked. Keys that only send traffic are refused, so a leaked production key cannot read your tasks.

generalcontext benchmark pull --out benchmark.json

#Run it and upload the results

evals run runs every task with your agent or a reference model, as described in Tasks and environments. --upload sends the results to your workspace, where Tasks shows how each agent scores. Running needs Python; in Node, verify and benchmark pull work and the rest point you to Python.

generalcontext evals run benchmark.json --agent my_agent:handle --trials 3 --upload