Partial / non-blocking compaction #107

New issue

Open

opened 2026-05-23 16:35:40 -04:00 by jasoncouture · 0 comments

jasoncouture commented

2026-05-23 16:35:40 -04:00

(Migrated from github.com)

Compactor takes a high-water-mark argument ("compact up to message X") and runs concurrently with the agent's main loop instead of stalling it. While compaction is in flight, the agent continues serving turns; new turns past X aren't touched by the in-flight pass. When compaction completes, the result is spliced in: entries ≤ X replaced by the summary, entries > X kept verbatim.

Why: full compaction today stalls the loop. With agents responding to channel input, a multi-second compaction window is a visible hang.

Considerations:

Splice contract: how to detect that entries between [X..now] reference earlier entries the summary now replaces (tool-call/result pairs that straddle the cut — likely refuse to cut between them, push X backwards).
Failure / cancellation: in-flight compaction must be safe to abandon if the agent state shifts under it.
Concurrency control: compaction itself calls a model. Two concurrent passes on the same agent shouldn't happen — first locks, second is no-op.

Depends on concurrency limits per model/provider (sibling) so that compaction doesn't fight the main turn loop for the same provider slot.

Pairs with: #11 (tool-pair preservation in cuts), compactor heuristics (sibling), compactor tool (sibling).

Compactor takes a high-water-mark argument (\"compact up to message X\") and runs concurrently with the agent's main loop instead of stalling it. While compaction is in flight, the agent continues serving turns; new turns past X aren't touched by the in-flight pass. When compaction completes, the result is spliced in: entries ≤ X replaced by the summary, entries > X kept verbatim. Why: full compaction today stalls the loop. With agents responding to channel input, a multi-second compaction window is a visible hang. Considerations: - Splice contract: how to detect that entries between [X..now] reference earlier entries the summary now replaces (tool-call/result pairs that straddle the cut — likely refuse to cut between them, push X backwards). - Failure / cancellation: in-flight compaction must be safe to abandon if the agent state shifts under it. - Concurrency control: compaction itself calls a model. Two concurrent passes on the same agent shouldn't happen — first locks, second is no-op. Depends on concurrency limits per model/provider (sibling) so that compaction doesn't fight the main turn loop for the same provider slot. Pairs with: #11 (tool-pair preservation in cuts), compactor heuristics (sibling), compactor tool (sibling).