Docs / guides / observability

Observability

Observability in Runra Sandbox — event types, exporters (Axiom, OpenTelemetry, JSONL, Postgres), and monitoring patterns for AI agent sandboxes

Runra Sandbox emits structured observability events for every action in the sandbox lifecycle. You can export these events to your existing monitoring stack — Axiom, OpenTelemetry, JSONL files, or Postgres.

Event model

Every sandbox action generates an event. Events include:

  • Timestamp: When the event occurred (ISO 8601)
  • Sandbox ID: Which sandbox generated the event
  • Event type: What happened
  • Event data: Type-specific payload
  • Trace ID: For distributed tracing across services

Event types

Lifecycle events

EventDescriptionKey fields
sandbox.createdSandbox provisionedimage, resources
sandbox.startedSandbox ready for usestartupDurationMs
sandbox.pausedSandbox pausedruntimeDurationMs
sandbox.resumedSandbox resumedpausedDurationMs
sandbox.terminatedSandbox destroyedtotalRuntimeMs, reason

Execution events

EventDescriptionKey fields
exec.startedCommand execution begancommand, cwd
exec.completedCommand finishedexitCode, durationMs, stdout (truncated), stderr (truncated)
exec.timed_outCommand hit timeouttimeoutMs, signal
exec.failedCommand infrastructure errorerror

File events

EventDescriptionKey fields
file.createdFile writtenpath, sizeBytes
file.modifiedFile updatedpath, sizeBytes
file.deletedFile removedpath
file.readFile readpath, sizeBytes

Tool events

EventDescriptionKey fields
tool.invokedAgent tool calledtoolName, toolArgs, agentId
tool.completedAgent tool finishedtoolName, result, durationMs
tool.failedAgent tool erroredtoolName, error

Network events

EventDescriptionKey fields
port.exposedPort exposedport, url
port.closedPort closedport
network.egressOutbound connectionhost, port, bytesSent

Resource events

EventDescriptionKey fields
resource.cpuCPU metrics samplecpuPercent, cores
resource.memoryMemory metrics samplememoryUsedMb, memoryLimitMb
resource.diskDisk metrics samplediskUsedMb, diskLimitMb

Cost signals

EventDescriptionKey fields
cost.runtimeActive runtime costdurationMs, rate, cost
cost.storagePaused storage costdurationMs, rate, cost
cost.totalAggregated costsandboxCost, period

Configuration

Enable observability when creating the Runra client:

import { Runra } from "@runra/sdk";

const runra = new Runra({
  sandbox: {
    provider: "runra-sandbox",
    apiKey: process.env.RUNRA_API_KEY,
  },
  observability: {
    provider: "axiom", // or "otel" | "jsonl" | "postgres"
    // Provider-specific configuration follows
  },
});

Exporters

Axiom

Stream events to Axiom for real-time log analysis and dashboards:

const runra = new Runra({
  sandbox: { provider: "runra-sandbox", apiKey: process.env.RUNRA_API_KEY },
  observability: {
    provider: "axiom",
    token: process.env.AXIOM_API_TOKEN,
    dataset: "runra-sandbox-events",
    // Optional: filter which events to send
    eventTypes: ["sandbox.*", "exec.*", "tool.*", "cost.*"],
  },
});

Axiom is the recommended exporter for production. It provides:

  • Full-text search across all events
  • Pre-built dashboards for sandbox health and costs
  • Real-time alerting on error rates, timeouts, and cost anomalies
  • Unlimited retention with query-based pricing

OpenTelemetry

Export events via OpenTelemetry to any OTLP-compatible backend (Datadog, Honeycomb, Grafana, Jaeger, etc.):

const runra = new Runra({
  sandbox: { provider: "runra-sandbox", apiKey: process.env.RUNRA_API_KEY },
  observability: {
    provider: "otel",
    endpoint: "https://otlp.example.com/v1/traces",
    headers: {
      "api-key": process.env.OTEL_API_KEY,
    },
    // OpenTelemetry resource attributes
    resourceAttributes: {
      "service.name": "my-agent-platform",
      "service.version": "1.0.0",
      "deployment.environment": "production",
    },
  },
});

Events are sent as OpenTelemetry spans, with event types mapped to span names and event data mapped to span attributes. This enables distributed tracing across your entire stack.

JSONL

Write events to a local JSONL file — useful for development, debugging, or offline analysis:

const runra = new Runra({
  sandbox: { provider: "runra-sandbox", apiKey: process.env.RUNRA_API_KEY },
  observability: {
    provider: "jsonl",
    filePath: "./runra-events.jsonl",
    // Rotate files
    maxFileSizeMb: 100,
    maxFiles: 10,
  },
});

Each event is written as a single JSON line:

{"timestamp":"2026-06-15T10:30:00.000Z","sandboxId":"sb_abc123","type":"exec.completed","data":{"command":"npm test","exitCode":0,"durationMs":4521}}
{"timestamp":"2026-06-15T10:30:05.000Z","sandboxId":"sb_abc123","type":"sandbox.paused","data":{"runtimeDurationMs":120000}}

Postgres

Write events directly to a Postgres database:

const runra = new Runra({
  sandbox: { provider: "runra-sandbox", apiKey: process.env.RUNRA_API_KEY },
  observability: {
    provider: "postgres",
    connectionString: process.env.DATABASE_URL,
    tableName: "sandbox_events",
    // Batch writes for performance
    batchSize: 100,
    flushIntervalMs: 5000,
  },
});

Events are stored in a table with a JSONB column for flexible querying:

-- Find slow executions
SELECT sandbox_id, data->>'command', data->>'durationMs'
FROM sandbox_events
WHERE event_type = 'exec.completed'
  AND (data->>'durationMs')::int > 5000
ORDER BY created_at DESC
LIMIT 20;

-- Calculate costs per sandbox
SELECT sandbox_id, SUM((data->>'cost')::numeric) as total_cost
FROM sandbox_events
WHERE event_type = 'cost.runtime'
  AND created_at > NOW() - INTERVAL '24 hours'
GROUP BY sandbox_id
ORDER BY total_cost DESC;

Custom event handlers

Register custom handlers for specific event types:

runra.on("exec.completed", (event) => {
  if (event.data.exitCode !== 0) {
    // Alert on failed commands
    slack.send({
      channel: "#sandbox-alerts",
      text: `Command failed in sandbox ${event.sandboxId}: ${event.data.command}`,
    });
  }
});

runra.on("sandbox.terminated", (event) => {
  // Log total cost
  billing.record({
    sandboxId: event.sandboxId,
    cost: event.data.totalCost,
  });
});

runra.on("resource.memory", (event) => {
  // Alert on high memory usage
  const usedPercent = (event.data.memoryUsedMb / event.data.memoryLimitMb) * 100;
  if (usedPercent > 90) {
    console.warn(`Sandbox ${event.sandboxId} at ${usedPercent}% memory`);
  }
});

Monitoring patterns

Cost tracking dashboard

// Track costs across all sandboxes
const costs = new Map<string, number>();

runra.on("cost.runtime", (event) => {
  const current = costs.get(event.sandboxId) ?? 0;
  costs.set(event.sandboxId, current + event.data.cost);
});

// Periodic reporting
setInterval(() => {
  const total = Array.from(costs.values()).reduce((a, b) => a + b, 0);
  console.log(`Current period cost: $${total.toFixed(4)}`);
}, 60_000);

Error rate monitoring

let successCount = 0;
let failureCount = 0;

runra.on("exec.completed", (event) => {
  if (event.data.exitCode === 0) successCount++;
  else failureCount++;
});

// Alert if error rate exceeds threshold
setInterval(() => {
  const total = successCount + failureCount;
  const rate = total > 0 ? failureCount / total : 0;
  if (rate > 0.05) {
    console.error(`Error rate at ${(rate * 100).toFixed(1)}%!`);
  }
  successCount = 0;
  failureCount = 0;
}, 300_000); // Every 5 minutes

Agent activity audit

// Track every action an agent takes for audit purposes
const agentActions: any[] = [];

runra.on("tool.invoked", (event) => {
  agentActions.push({
    sandboxId: event.sandboxId,
    agentId: event.data.agentId,
    tool: event.data.toolName,
    args: event.data.toolArgs,
    timestamp: event.timestamp,
  });
});

// Export audit log on sandbox termination
runra.on("sandbox.terminated", async (event) => {
  const sandboxActions = agentActions.filter(
    (a) => a.sandboxId === event.sandboxId
  );
  await auditStore.save(sandboxActions);
});

Next steps