API reference · Monitoring

Compare live with the test

GET/v1/bots/{bot_id}/vs-test

Live against the Sim run the bot is linked to: fill rate, fee per contract, P&L per contract and slippage against tested prices, each with its sample sizes.

On this page

Auth. An account token (Authorization: Bearer lb_…) or a signed-in web session. A report-only token (lbr_…) is refused here with 403 scope_denied.

Limits. Counts against your plan's API requests (60, 600 or 3,000 a minute on Free, Builder and Standard).

Parameters

Path parameters

Name	Type	Required	Description
`bot_id`	string	yes	The bot id (`bot_…`).

Query parameters

Name	Type	Required	Description
`since`	integer	no	Start of the live window, ms epoch (default: when the bot was created).

Request

Terminal

curl "https://api.langbot.ai/v1/bots/bot_01M6G72RFQRQ8JGNG8T1447GGP/vs-test" \
  -H "Authorization: Bearer $LANGBOT_API_TOKEN"

Python

from langbot import LangbotClient

async with LangbotClient() as lb:  # reads LANGBOT_API_TOKEN
    v = await lb.bots.vs_test("bot_01M6G72RFQRQ8JGNG8T1447GGP")
    if v["available"]:
        m = v["metrics"]["fill_rate"]
        print(m["live"], m["tested"], m["live_n"], m["tested_n"])

TypeScript

import { LangbotClient } from "@langbotai/sdk";

const lb = new LangbotClient(); // reads LANGBOT_API_TOKEN
const v = await lb.bots.vsTest("bot_01M6G72RFQRQ8JGNG8T1447GGP");

Response

200 OK

JSON

{
  "test_run_id": "run_01M6F9\u2026",
  "available": true,
  "reason": null,
  "window": {
    "since": 1793690000000,
    "until": 1793693150782
  },
  "metrics": {
    "fill_rate": {
      "live": 0.8,
      "tested": 0.5,
      "diff": 0.3,
      "live_n": 20.0,
      "tested_n": 100.0,
      "unit": "contracts ordered"
    },
    "fee_per_contract": {
      "live": 0.01,
      "tested": 0.02,
      "diff": -0.01,
      "live_n": 16.0,
      "tested_n": 50.0,
      "unit": "contracts filled (with a known fee)"
    },
    "pnl_per_contract": {
      "live": 0.031,
      "tested": 0.1,
      "diff": -0.069,
      "live_n": 16.0,
      "tested_n": 50.0,
      "unit": "contracts filled"
    },
    "slippage": {
      "cents": 2.0,
      "markets_compared": 1,
      "live_fills": 1,
      "tested_fills": 1,
      "by_market": [
        {
          "ticker": "KX-T1",
          "side": "yes",
          "action": "buy",
          "live_avg_price": 0.4,
          "tested_avg_price": 0.38,
          "cents": 2.0,
          "live_contracts": 16.0,
          "tested_contracts": 50.0,
          "live_fills": 1,
          "tested_fills": 1
        }
      ]
    }
  },
  "slippage_reason": null
}

Each number comes with the sample behind it (live_n, tested_n, in unit); no significance test is claimed, so read small samples as small. slippage is in cents, weighted by live contracts, positive when live bought higher or sold lower than the test, over markets present in both; it is usually null (slippage_reason), because a test replays a past event. available: false with a reason when the bot has no linked run, or the run has no scorecard.

Errors

Status	`code`	When
404	`not_found`	No bot with that id on your account (or it was deleted).
401	`auth_required`	No token was sent.
401	`token_invalid`	The token was not recognised (mistyped, revoked or expired).
422	`validation_error`	A parameter failed validation; `detail[].loc` names it.
429	`rate_limited`	Over your request budget; wait `Retry-After` seconds.

Every error body is {"detail": …, "code": "…"}; branch on code. All codes: Errors.