BrowseComp-Plus

Use this sample to study an agent that searches a document corpus, reads passages, and submits an answer to a research question. SearchEnv owns the browsing task; make_agent() combines it with a policy and context management.

What you supply

Provide plaintext question records, reference answers, a retrieval client connected to the corpus, and a policy that generates tool calls. RetrievalArchive retains retrieved evidence. The grade() helper uses normalized exact match; it does not reproduce the benchmark’s semantic judge.

What to adapt

The search, open, and find tools illustrate document retrieval by corpus ID. Adapt their observations, call budgets, and grading for your own research task. For a step-by-step agent with TRACE-style result IDs and page cursors, follow BrowseComp-Plus: build a research agent; that tutorial defines its browser interface in the example code.

The reference below covers the sample’s tools, task state, retrieval archive, and assembly helpers.

BrowserFind

Find text within a corpus document and return matching passages.

from focus.environments.browsercomp_plus import BrowserFind

Definition

class BrowserFind(DocumentTool)

BrowserFind.call

async def call(self, params, **kwargs)

BrowserOpen

Read a range of lines from a document identified by its corpus ID.

from focus.environments.browsercomp_plus import BrowserOpen

Definition

class BrowserOpen(DocumentTool)

BrowserOpen.call

async def call(self, params, **kwargs)

BrowserSearch

Search the document corpus through the retriever client and return ranked document IDs with snippets.

from focus.environments.browsercomp_plus import BrowserSearch

Definition

class BrowserSearch(BaseTool)

BrowserSearch.init

def __init__(self, archive, top_k=10, snippet_chars=700)

BrowserSearch.call

async def call(self, params, **kwargs)

Case

Hold a BrowseComp-Plus question, reference answer, and dataset metadata.

from focus.environments.browsercomp_plus import Case

Definition

@dataclass(frozen=True)
class Case

Fields

id: str
question: str
answers: tuple[str, ...]

RetrievalArchive

Cache the exact observations returned by retrieval calls. An offline archive replays recorded observations and raises an error for a missing call.

from focus.environments.browsercomp_plus import RetrievalArchive

Definition

class RetrievalArchive

RetrievalArchive.init

def __init__(self, client=None, entries=None)

RetrievalArchive.observe

async def observe(self, name, args, fetch)

RetrievalArchive.save

def save(self, path)

RetrievalArchive.load

@classmethod
def load(cls, path)

SearchEnv

Apply browser actions, track retrieval progress, and grade the final answer against the case’s reference answer.

from focus.environments.browsercomp_plus import SearchEnv

Definition

class SearchEnv(ToolEnv[State])

SearchEnv.init

def __init__(self, archive, *, call_budget=10, top_k=10, snippet_chars=700, read_chars=6000, task_limit=96)

SearchEnv.initial_state

async def initial_state(self)

SearchEnv.should_finish

def should_finish(self, rollout)

Answer when the next task turn would exhaust the task limit.

SearchEnv.finish_tools

def finish_tools(self, tools)

Offer only the submit tool on the final-answer turn.

SearchEnv.on_compaction_failure

def on_compaction_failure(self, state, reasons)

SearchEnv.step

async def step(self, state, turn, *, workspace=None)

State

Hold the retrieval task’s branch-local progress, answer, tool-call counts, and remaining steps.

from focus.environments.browsercomp_plus import State

Definition

@dataclass(frozen=True)
class State(SubmitFields, RoundBudget)

Fields

task_limit: int = 96
compaction_failure: tuple[str, ...] = ()

DocumentTool

Share document retrieval and line formatting between tools that inspect a corpus document.

from focus.environments.browsercomp_plus.browser import DocumentTool

Definition

class DocumentTool(BaseTool)

DocumentTool.init

def __init__(self, archive, read_chars=6000)

DocumentTool.document

async def document(self, client, docid)

grade

Compare an answer with its reference using normalized exact match. Official BrowseComp-Plus evaluation uses a separate semantic judge.

from focus.environments.browsercomp_plus import grade

Definition

def grade(answer, case)

load_cases

Load BrowseComp-Plus cases from local data or the configured dataset source, with optional case selection.

from focus.environments.browsercomp_plus import load_cases

Definition

def load_cases(path)

make_agent

Assemble the browser environment, tools, context manager, and agent for BrowseComp-Plus. Run the returned agent on a case’s question.

from focus.environments.browsercomp_plus import make_agent

Definition

def make_agent(policy: Policy[ToolTurn], prompt_limit=14336, call_budget=10, *, archive, top_k=10, snippet_chars=700, read_chars=6000, max_steps=96, no_compact=False, compactor_max_tokens=16384, summary_target_tokens=2048, compactor_temperature=0.2, compactor_top_p=0.95, context_limit=98304, compaction_failure: Literal['raise', 'end']='raise') -> Agent[State, ToolTurn]