Connect LLMs

LLMs are good at generating code, but that code should run in an isolated environment. This quickstart shows the common pattern: expose a run_python tool to the model, execute tool calls in Runra, and return stdout/stderr back to the model.

Prerequisites

• Completed the Quickstart.
• RUNRA_API_KEY set in your environment.
• An LLM provider API key, such as OPENAI_API_KEY.

1. Install dependencies

npm install runra openai
# or
bun add runra openai

2. Define a sandbox-backed tool

import OpenAI from "openai";
import { Sandbox } from "runra";

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const sbx = await Sandbox.create({ timeout: 600 });

async function runPython(code: string) {
  const execution = await sbx.runCode(code, { timeoutMs: 60_000 });

  return JSON.stringify({
    stdout: execution.logs.stdout,
    stderr: execution.logs.stderr,
    result: execution.text,
    error: execution.error
      ? `${execution.error.name}: ${execution.error.value}`
      : null,
  });
}

3. Register the tool with the LLM

const tools = [
  {
    type: "function" as const,
    function: {
      name: "run_python",
      description: "Run Python code in an isolated Runra sandbox.",
      parameters: {
        type: "object",
        properties: {
          code: {
            type: "string",
            description: "Python code to execute.",
          },
        },
        required: ["code"],
      },
    },
  },
];

4. Handle tool calls

const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
  {
    role: "user",
    content: "Use Python to calculate the mean of [3, 5, 8, 13].",
  },
];

try {
  const first = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages,
    tools,
  });

  const assistantMessage = first.choices[0]?.message;
  if (!assistantMessage) throw new Error("No assistant message returned");
  messages.push(assistantMessage);

  for (const call of assistantMessage.tool_calls ?? []) {
    if (call.function.name !== "run_python") continue;

    const args = JSON.parse(call.function.arguments) as { code: string };
    const output = await runPython(args.code);

    messages.push({
      role: "tool",
      tool_call_id: call.id,
      content: output,
    });
  }

  const final = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages,
  });

  console.log(final.choices[0]?.message.content);
} finally {
  await sbx.kill();
}

Tool design tips

• Keep tool inputs small and explicit. A code string plus optional timeoutMs is usually enough.
• Return structured JSON with stdout, stderr, result, and error so the model can recover from failures.
• Set execution timeouts. Do not let arbitrary code run forever.
• Reuse one sandbox for a multi-step conversation when you want variables and files to persist between tool calls.
• Kill the sandbox at the end of the request, job, or conversation.

Reusing state across calls

Variables from runCode() persist in the default execution context:

await sbx.runCode("x = 41");
const result = await sbx.runCode("x += 1; x");
console.log(result.text); // 42

For longer-lived agents, store generated files under /home/user and use file upload/download for durable inputs and outputs.