Values

Compute each checkpoint’s critic prior, direct-branch backup, and refined value once per tree. Estimators and critic targets read these values.

RewardConfig

Define how outcome rewards, step rewards, discounting, and critic predictions are interpreted consistently throughout one collection batch.

from focus import RewardConfig

rewards = RewardConfig(gamma=1.0, value_prior_strength=1.0)

This configuration uses undiscounted task returns and weights the critic prediction as one observation, the paper’s $\kappa = 1$. beta_step scales step rewards against the outcome reward, and component_weights weights named reward components. Pass it to Collector(reward=...); search and preparation then read rewards the same way.

Definition

@dataclass(frozen=True, kw_only=True)
class RewardConfig

Fields

gamma: float = 1.0
value_prior_strength: float = 1.0
beta_step: float = 1.0
component_weights: dict[str, float] = field(default_factory=dict)

RewardConfig.train_reward

def train_reward(self, transition: StepResult[object]) -> float

Return the outcome reward plus beta_step times the step reward for one environment transition.

RewardConfig.shaping

def shaping(self, turn: Turn[object]) -> float

Compute weighted reward components without advancing the task discount.

RewardConfig.configured_return

def configured_return(self, turns: Sequence[Turn[object]], start: int=0) -> float

Compute the discounted return over a sequence of turns, optionally beginning at start. Discounting advances once per environment transition. Compaction turns add any configured compaction penalty without adding a discount step.

NodeValues

One checkpoint’s values: the critic prior $V_\phi$, the number of direct branches $n$, the direct-branch backup $\bar G$, and the refined value $\tilde V$. Observed outcome counts live in the search’s evidence ledger.

from focus.extend import NodeValues

Definition

@dataclass(frozen=True)
class NodeValues

Fields

prior: float | None
branches: int
backup: float | None
refined: float | None

TreeValues

Values for every checkpoint of one tree, keyed by node id, with the tree and reward settings they were computed from.

from focus.extend import TreeValues

Definition

@dataclass(frozen=True)
class TreeValues(Mapping[int, NodeValues])

Fields

tree: SchedulerView[AnyRollout]
reward: RewardConfig
_values: Mapping[int, NodeValues]

TreeValues.getitem

def __getitem__(self, node_id: int) -> NodeValues

TreeValues.iter

def __iter__(self) -> Iterator[int]

TreeValues.len

def __len__(self) -> int

tree_values

Compute every checkpoint’s values once, deepest first: the direct-branch backup from its children’s returns, and the refined value that blends it with the critic prior.

from focus import RewardConfig

from focus.extend import tree_values

values = tree_values(tree, RewardConfig())
root = values[tree.root_id]

root.backup is the root’s critic target and root.refined the value actor advantages are measured against. A checkpoint without a critic prediction has no refined value.

Definition

def tree_values(tree: SchedulerState[AnyRollout] | SchedulerView[AnyRollout], reward: RewardConfig) -> TreeValues