API reference · Research

Run a bench

POST/v1/research/bench

One built-in strategy over every replayable event the filters keep, with a row per event and intervals across events.

On this page

Auth. An API token: Authorization: Bearer lb_… (create one). Signed-in web sessions work too.

Limits. One Sim run (20 a month on Free, 100 on Builder, unlimited on Standard).

Parameters

Body

Name	Type	Required	Description
`strategy`	string	no	A built-in strategy name. Default `hold_none`.
`params`	object	no	The strategy's parameters.
`latency_ms`	array of integer	no	Up to 6 latencies in ms; default `[250, 2250, 5250]`.
`starting_cash`	number	no	Paper cash in USD per event. Up to 10,000,000. Default `1000.0`.
`maker_fee_rate`	number	no	The maker fee rate. 0 to 0.5. Default `0.0175`.
`queue_position`	boolean	no	Make resting orders wait behind the size recorded at their price. Default `false`.
`taker_delay_ms`	integer	no	A venue speed bump on taking orders, in ms. 0 to 60,000. Default `0`.
`test_fraction`	number	no	Share of events, by start time, held out as the later split. Default `0.3`.
`speaker`	string	no	Only events of this speaker key.
`series`	string	no	Only events with markets in this series.
`since`	integer	no	Events that started at or after this ms epoch.
`until`	integer	no	Events that started before this ms epoch.
`event_ids`	array of string	no	Exactly these events. Up to 200 items.
`limit`	integer	no	At most this many events. 1 to 200. Default `50`.
`code`	string	no	Refused. Bench your own file locally with `langbot research bench -s my_bot.py`.

Request

Terminal

curl -X POST "https://api.langbot.ai/v1/research/bench" \
  -H "Authorization: Bearer $LANGBOT_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"strategy": "quote_both_sides", "latency_ms": [250, 2250], "limit": 20}'

Python

from langbot import LangbotClient

async with LangbotClient() as lb:  # reads LANGBOT_API_TOKEN
    result = await lb.bench("quote_both_sides", latency_ms=[250, 2250], limit=20)
    print(result["aggregate"]["by_latency"])

TypeScript

const res = await fetch("https://api.langbot.ai/v1/research/bench", {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.LANGBOT_API_TOKEN}`, "Content-Type": "application/json" },
  body: JSON.stringify({"strategy": "quote_both_sides", "latency_ms": [250, 2250], "limit": 20}),
});
console.log(await res.json());

Response

201 Created

JSON

{
  "v": 1,
  "strategy": { "name": "quote_both_sides", "params": {} },
  "latency_ms": [250, 2250],
  "events": [ { "event_id": "ev_kxearningsmentiongs_26oct13", "markets_traded": 3, "by_latency": [ { "latency_ms": 250, "net": 12.4, "fees": 1.9, "fills": 14 } ] } ],
  "aggregate": {
    "n_events": 20,
    "by_latency": [ { "latency_ms": 250, "n": 20, "net_total": 61.0, "net_mean": 3.05, "net_ci90": [-1.2, 7.4], "share_positive": 0.6, "p_value_mean": 0.18 } ]
  },
  "skipped": [],
  "caveats": ["20 events"]
}

Each event replayed is one Sim run; past your allowance the rest are listed in skipped, not refused. Results describe the record, not what real orders would get.

Errors

Status	`code`	When
401	`auth_required`	No token was sent.
401	`token_invalid`	The token was not recognised (mistyped, revoked or expired).
422	`validation_error`	A parameter failed validation; `detail[].loc` names it.
429	`quota_exceeded`	This month's Sim runs are spent; `quota.resets_at` says when they reset.
429	`rate_limited`	Over your request budget; wait `Retry-After` seconds.

Every error body is {"detail": …, "code": "…"}; branch on code. All codes: Errors.