AI Actions - Upstash Documentation

Beyond reading pages, a tab can act. A DOM-aware browser agent runs inside the box and resolves natural-language instructions against the live page. It can find elements, execute single actions, or complete multi-step tasks on its own.

AI actions use an LLM and are metered. They need an API key for the model's provider (Anthropic, OpenAI, OpenRouter, Vercel, or OpenCode) on the box or your account. Every method accepts a provider-prefixed model override such as "openai/gpt-4o". Without an override, the call uses the model the box was configured with. If the box has no model, it falls back to anthropic/claude-sonnet-4-5.

Observe

observe() finds actionable elements matching an instruction. Use it to check the page before acting, or to build your own action loop:

box.ts

const { elements } = await tab.observe("find the login and signup buttons")for (const el of elements) {  console.log(el.description, el.selector)}

box.py

result = tab.observe("find the login and signup buttons")for el in result.elements:    print(el.description, el.selector)

Act

act() resolves and executes exactly one action described in natural language:

box.ts

const action = await tab.act("click the primary call-to-action")console.log(action.success, action.actionDescription)console.log(action.inputTokens, action.outputTokens)

box.py

action = tab.act("click the primary call-to-action")print(action.success, action.action_description)print(action.input_tokens, action.output_tokens)

The result reports what was done (actions with the resolved selectors), whether it succeeded, and the token usage of the call.

Run

run() is the autonomous mode. The agent reads the page, acts, and repeats until the task is complete or it hits the step limit. Pass a schema to get structured data back at the end:

box.ts

import { z } from "zod"const { data, completed, steps } = await tab.run(  "Find the pricing page and summarize the free tier",  {    schema: z.object({ summary: z.string() }),    maxSteps: 15,    // model: "openai/gpt-4o", // any provider you hold a key for  },)console.log(completed, data.summary)for (const step of steps) {  console.log(step.step, step.action, step.url)}

box.py

from pydantic import BaseModelclass Summary(BaseModel):    summary: strresult = tab.run(    "Find the pricing page and summarize the free tier",    schema=Summary,    max_steps=15,    # model="openai/gpt-4o",  # any provider you hold a key for)print(result.completed, result.data.summary)for step in result.steps:    print(step.step, step.action, step.url)

maxSteps: defaults to 15, capped at 30.
schema: optional. Without it, run returns its findings as text in result.
The result includes completed, a step-by-step trace in steps (each with the action taken, its reasoning, and the URL), and total token usage.

Which one to use

Method	Does	Best for
`observe`	Finds elements, executes nothing	Inspecting a page, building custom loops
`act`	Executes one action	Flows where your code decides each step
`run`	Executes a whole task autonomously	Open-ended or navigation-heavy tasks

For fully scripted control with no LLM in the loop, connect over CDP with Playwright or Puppeteer instead. Both drive the same tabs, so you can mix scripted steps with AI steps. To watch or replay what the agent did, see Live View and Recordings.