Replay token meter service
回放式 token 计量服务
Context pressure is useful outside compaction. A compaction backend, an overflow guard, or a future request-policy plugin can all need the same answer: how many tokens does the durable request consume? Keeping that fold inside `dsh-compaction-basic` duplicates replay logic, makes measurement unavailable without compaction, and encourages callers to reuse stale accounting. Provider usage is not a complete answer. It d
English
Problem
Context pressure is useful outside compaction. A compaction backend, an overflow guard, or a future request-policy plugin can all need the same answer: how many tokens does the durable request consume? Keeping that fold inside dsh-compaction-basic duplicates replay logic, makes measurement unavailable without compaction, and encourages callers to reuse stale accounting.
Provider usage is not a complete answer. It describes one successful call under one exact request envelope, while the current surface can grow, shrink, or be replaced afterward. Sessions also switch providers and models, old logs can omit the chunk seqs behind an assistant message, and usage fields separate input, cache-read, cache-write, output, and reasoning counts. A useful service therefore combines the latest exact anchor with conservative heuristic repricing and exposes the log revision consumed by each result.
Decision
One concrete LLM-family service
@deepseek-ai/dsh-token-meter is one concrete package under packages/llm/ and registers ctx.tokenMeter. It is not split into an interface and backend before a second implementation exists. TokenMeter itself exposes measure(session, requestHeader?) and estimateMessage(message); consumers call the singleton service directly.
The service has no configuration. Estimation uses a fixed four-characters-per-token heuristic plus structural overhead. There are no model profiles, capacity settings, density settings, tokenizer backends, or language-specific strategies. Exact provider/model capacity is a separate adapter-owned query, as specified by the routed model context and compaction policy Agent Note.
Per-session replay folds
Each session owns one isolated incremental fold. Active folds advance from session/event; every read catches up through the durable tail, so listener ordering, seeded sessions, and service reload do not change the answer. The fold tracks canonical full request-header snapshots, step boundaries, surface appends and replacements, assistant usage, and the chunk seqs cited by each assistant message. A malformed next event fails transactionally and remains unread rather than partially mutating state.
measure(session, requestHeader?) synchronizes the fold once and returns scalar pressure together with positional per-node prices. totalTokens remains request-and-response pressure; surfaceTokens is the surface-only heuristic total and equals the sum of nodes[].tokens. A requestHeader override changes pressure pricing only, while the surface fields always describe the current session. estimateMessage(message) applies the fixed heuristic without session state. Each result is one detached, deeply immutable snapshot carrying one logRevision. Every measurement clones the current nodes and is therefore O(surface).
Provider usage is reused only when the measured canonical request envelope equals the latest successful-call anchor. Any provider, model, system, prefix, tool, or call-config change causes complete heuristic repricing. Surface changes remain a signed delta from a matching anchor, including negative values after a shrinking replacement. A later successful request replaces the earlier anchor, including across provider or model switches.
Usage sums the disjoint input, cache-read, cache-write, and output buckets. Reasoning is not added a second time. Every successful model call records an assistant/message, including content-less and max-token calls, with its exact earlier chunk seqs. An explicit empty sourceEventSeqs list means a known empty provider stream; an absent legacy list conservatively treats the durable assistant output as provider output.
Compact-basic consumes, but does not own, measurement
dsh-compaction-basic requires ctx.tokenMeter; CompactionEngine gains no token methods or types. Configuration, the region transaction, and summarization stay in separate modules; the service registers automatic listeners itself, while summarize() remains its sole subclass hook. The singleton meter consistently prices pressure, retention, shadowed content, cited source events, and non-shrinking-summary rejection.
Automatic compaction uses one unified measurement for each threshold-and-retention decision. The region transaction measures after appending its durable compaction/start lock and again after asynchronous summarization, then compares the detached surface-node vectors. An intervening surface mutation prevents replacement; logRevision may advance for unrelated log-only facts without invalidating an unchanged selected span.
Compact policy has service-wide defaults: threshold ratio 0.8, retained-tail ratio 0.16, summarizationProvider: '', summarizationModel: '', maxTokens: 8192, compactionRetries: 1, maxOverflowRetries: 1, and auto: true. Top-level fields apply to every routed target; exact provider/model entries in modelPolicies partially override them. Pressure scales ratios against capacity resolved from the owning adapter, and retainTokens may replace retainRatio; retention must remain below the resulting threshold. The summarization provider and model must both be set or both be empty; an empty pair resolves the latest logged request target, then the AgentOptions pair.
Automatic pressure runs at agent/pre-step before request derivation and measures the canonical durable envelope produced under the provider/model actually selected by the preceding agent/request. A headerless session has no completed routed request to assess and produces no work; any routed target can use the singleton estimator. Canonical overflow recovery uses the same measurement for forced range selection and retries only after a proven surface replacement.
Testing
Unit tests cover fixed estimation, envelope invalidation and anchor replacement, replay boundaries, immutable snapshots, routed pressure, convergence, overflow generation proof, and rollback. A real Loader/Include fixture verifies the zero-config token-meter and compaction-basic load path in dependency order.
Alternatives considered
- Keep estimation inside
CompactionEngine— rejected because measurement has consumers and replay semantics independent of compaction; it would also force every compactor to expose the same unrelated API. - Split a token-meter interface from a heuristic backend immediately — rejected because only one implementation exists. One concrete service preserves the future seam without speculative packages or configuration.
- Put model-keyed windows and density profiles in the meter — rejected because replay estimation does not own model routing or capacity facts. The route-owning adapter exposes capacity, while compaction-basic owns the consumer-specific threshold and retention policy.
- Keep separate scalar and surface measurements — rejected because callers would need two reads and revision matching for one decision. A scalar-only read could avoid cloning nodes below threshold, but the split API introduces a caller-side race window; the unified snapshot accepts O(surface) cloning in exchange for coherence.
- Treat provider usage as portable between envelopes — rejected because model, tools, prefixes, and call config are request facts. Mismatch reprices the whole current request.
Consequences
- Token pressure has one replay-aware owner that compaction and future plugins can share.
- The default makes the meter a zero-config composition entry; deployments configure capacity on each route-owning adapter and optional policy overrides on compaction-basic.
- Fixed heuristic pricing remains an estimate of provider behavior and is not an exact tokenizer or request serializer.
- Every measurement clones the current positional surface and therefore costs O(surface), including pressure checks that finish below threshold.
- Measurements fail loudly on malformed durable boundaries. This turns corrupted replay into a named integration failure instead of silently drifting pressure.
- Post-step pressure reads the exact logged routing/tools/prefix boundary; provider overflow classification remains the adapter-maintained backstop for requests rejected before a successful usage anchor.
中文
问题
上下文压力并不只对压缩（compaction）有用。压缩后端、溢出保护或未来的请求策略插件都可能需要回答同一个问题：持久请求消耗了多少 token？如果把该折叠逻辑留在 dsh-compaction-basic 内部，就会重复实现回放逻辑，使未加载压缩的调用方无法使用计量，并诱使调用方复用陈旧的核算结果。
提供方 usage 也不是完整答案。它只描述某个精确请求信封下的一次成功调用，而当前表层之后还可能增长、缩小或被替换。会话也可能切换提供方与模型，旧日志可能缺少构成 assistant 消息的分片 seq，usage 字段还会分别报告输入、缓存读取、缓存写入、输出与推理计数。因此，可用的服务必须把最新精确锚点与保守的启发式重新定价结合起来，并公开每个结果已经消费的日志修订号。
决策
一个具体的 LLM（大语言模型）家族服务
@deepseek-ai/dsh-token-meter 是 packages/llm/ 下的单个具体包，并注册 ctx.tokenMeter。在第二种实现出现之前，它不会被拆成接口与后端。TokenMeter 本身公开 measure(session, requestHeader?) 与 estimateMessage(message)；消费方直接调用这个单例服务。
服务没有配置。估算采用固定的每 token 四个字符启发式规则，并加上结构开销。服务不提供模型 profile、容量设置、密度设置、分词器后端或语言专用策略。对精确提供方/模型容量的查询由适配器单独负责，具体见路由模型上下文与压缩策略 Agent Note。
逐会话回放折叠
每个会话都有一个隔离的增量折叠。活跃折叠通过 session/event 前进；每次读取都会追到持久日志尾部，因此监听器顺序、种子会话与服务重载不会改变答案。折叠跟踪规范的完整请求头快照、步骤边界、表层追加与替换、assistant usage，以及每条 assistant 消息引用的分片 seq。下一个畸形事件会以事务方式失败并保持未读，不会让状态只修改一半。
measure(session, requestHeader?) 只同步一次折叠，并在返回标量压力的同时给出逐位置节点价格。totalTokens 仍表示请求与响应压力；surfaceTokens 是仅针对表层的启发式总量，并等于 nodes[].tokens 之和。requestHeader 覆盖只改变压力定价，表层字段始终描述当前会话。estimateMessage(message) 不依赖会话状态，直接应用固定启发式规则。每个结果都是一个分离且深度不可变的快照，只携带一个 logRevision。每次计量都会复制当前节点，因此成本为 O(surface)。
只有当待计量的规范请求信封等于最近一次成功调用的锚点时，服务才复用提供方 usage。提供方、模型、系统提示词、前缀、工具或调用配置任一变化都会触发完整的启发式重新定价。表层变化相对匹配锚点保留有符号增量，包括缩小替换后的负值。后续成功请求会替换先前锚点，提供方或模型切换时也一样。
Usage 会对互不重叠的输入、缓存读取、缓存写入与输出 bucket 求和，不会再次加入推理计数。每次成功模型调用都会记录 assistant/message，包括无内容调用与达到 token 上限的调用，并带上精确的更早分片 seq。显式的空 sourceEventSeqs 列表表示已知为空的提供方流；旧日志中缺失的列表则保守地把持久 assistant 输出视为提供方输出。
compaction-basic 消费计量，但不拥有计量
dsh-compaction-basic 要求 ctx.tokenMeter；CompactionEngine 不增加 token 方法或类型。配置、区域事务与摘要分别保留在独立模块中，服务自身注册自动监听器，而 summarize() 仍是唯一的子类钩子。单例计量器一致用于压力、保留、被遮蔽内容、引用的源事件以及非缩小摘要拒绝的定价。
自动压缩的每次阈值与保留联合决策只使用一次统一计量。区域事务会在追加持久 compaction/start 锁后执行计量，在异步摘要完成后再次计量，随后比较分离的表层节点向量。期间发生的表层变更会阻止替换；logRevision 可以因无关的纯日志事实而推进，而不会使未变的选定范围失效。
压缩策略采用服务级默认值：阈值比例 0.8、保留尾部比例 0.16、summarizationProvider: ''、summarizationModel: ''、maxTokens: 8192、compactionRetries: 1、maxOverflowRetries: 1 与 auto: true。顶层字段适用于每个路由目标；modelPolicies 中的精确提供方/模型项可以部分覆盖这些字段。压力检查以所属适配器解析的容量为基准换算这些比例，retainTokens 可以替代 retainRatio；保留值必须小于最终阈值。摘要提供方与模型必须同时设置或同时为空；空组合先解析最近记录的请求目标，再使用 AgentOptions 中的组合。
自动压力检查在请求派生前运行于 agent/pre-step，并计量前一个 agent/request 实际所选提供方/模型产生的规范持久信封。没有请求头的会话尚无已完成的路由请求可供判断，因此不执行工作；任意路由目标都可使用这个单例估算器。规范的溢出恢复流程使用同一计量结果强制选择范围，并且只有在表层替换得到证明后才重试。
测试
单元测试覆盖固定估算、信封失效与锚点替换、回放边界、不可变快照、已路由压力、收敛、溢出 generation 证明与回滚。真实 Loader/Include fixture（测试前置数据）验证零配置 token-meter 与 compaction-basic 按依赖顺序加载的路径。
考虑过的替代方案
- 把估算保留在
CompactionEngine内——不予采纳，因为计量拥有独立于压缩的消费方与回放语义；它还会强迫每个压缩器暴露同一套无关 API。 - 立即把 token meter 拆成接口与启发式后端——不予采纳，因为目前只有一种实现。单个具体服务保留未来的 seam，同时避免推测性的包与配置。
- 把模型键控窗口与密度 profile 放进 meter——不予采纳，因为回放估算不拥有模型路由或容量事实。路由所属适配器公开容量，compaction-basic 则拥有消费方专用的阈值与保留策略。
- 保留独立的标量与表层计量——不予采纳，因为消费方必须为一次决策执行两次读取并匹配修订号。仅读取标量可以避免在低于阈值时复制节点，但拆分 API 会在消费方引入竞态窗口；统一快照接受 O(surface) 复制成本，以换取结果一致性。
- 在不同信封之间移用提供方 usage——不予采纳，因为模型、工具、前缀与调用配置都是请求事实。不匹配时会重新定价完整当前请求。
后果
- Token 压力拥有一个可供压缩与未来插件共享的回放感知所有者。
- 默认值让 meter 成为零配置组合项；部署时在各个路由所属适配器上配置容量，并在 compaction-basic 上配置可选策略覆盖。
- 固定启发式定价仍然只是提供方行为的估计，并不是精确分词器或请求序列化器。
- 每次计量都会复制当前带位置信息的表层，因此成本为 O(surface)，低于阈值即可结束的压力检查也不例外。
- 遇到畸形持久边界时，计量会明确失败。这会把损坏的回放转化为具名集成错误，而不是让压力静默漂移。
- 步骤后压力检查读取精确记录的路由、工具与前缀边界；对于在成功 usage 锚点出现前就被拒绝的请求，提供方溢出分类仍是由适配器维护的兜底路径。