Skip to content

perf: take general-purpose parsing and query costs off the interactive hot paths - #93

Merged
tfcace merged 8 commits into
masterfrom
claude/lockfile-parser-performance-rswz02
Aug 1, 2026
Merged

tfcace merged 8 commits into
masterfrom
claude/lockfile-parser-performance-rswz02

Conversation

@tfcace

@tfcace tfcace commented Aug 1, 2026 •

Copy link
Copy Markdown
Owner

Applies the idea behind uv's custom lockfile parser (astral-sh/uv#20648) — stop paying a general-purpose engine on a hot path with a known input shape — to hash's interactive paths. Four independent changes, one commit each, with benchmarks committed first (8687faa) so before/after run identical benchmark code.

Changes

  1. perf(history) — SearchByPrefix runs on every keystroke for ghost-text suggestions, backed by a GLOB + GROUP BY + MAX(timestamp) query that aggregates every historical occurrence of every matching command; with the default unlimited retention that cost grows with history size. Distinct successful commands now load once in the background at store open into a text-sorted in-memory slice; a lookup is a binary search plus a range scan, Store.Add updates the index in place, and SQL remains the fallback until the load lands (a load failure only costs speed). Side fix: in-memory SQLite DSNs cap the pool at one connection, since each pooled :memory: connection gets its own empty database and the background load is the first concurrent pool user.
  2. perf(completion) — the built-in docker/hash-builtins TOML specs were re-parsed on every handler-table rebuild (startup and each completions reload); they're compile-time constants, now parsed once per process under sync.OnceValue.
  3. perf(config) — Load parsed config.toml twice in full (struct + whole-file raw map just for [agent.<name>] tables) and the recovery path a third time. The happy path now decodes only the agent key; recovery reuses the raw map it already holds.
  4. perf(parser) — Parse and the shell dispatch site built map[string]any trace payloads on every line for Emit to discard when tracing is off. Payload construction now sits behind trace.Enabled guards; emitted events are unchanged when tracing is on.

Benchmarks

Linux amd64, Intel Xeon 2.80GHz, go1.25.0, -benchmem. Reproduce with:

go test -run '^$' -bench . -benchmem ./internal/history/ ./internal/completion/ ./internal/config/ ./internal/parser/

History: SearchByPrefix (per keystroke)

Case Before After Speedup
10k rows, dense prefix 845 µs 6.2 µs 136x
10k rows, narrow prefix 453 µs 3.8 µs 120x
10k rows, miss 32.9 µs 86 ns 383x
100k rows, dense prefix 15.2 ms 57 µs 268x
100k rows, narrow prefix 7.9 ms 40 µs 196x
100k rows, miss 32.7 µs 107 ns 307x

The before numbers scale with table size (0.85 ms → 15.2 ms from 10k to 100k rows — visible typing lag); the after numbers scale only with the number of distinct matching commands.

Completion: built-in plugin specs (startup, completions reload)

Benchmark Before After Speedup
builtinPluginSpecs 138 µs, 552 allocs 63 ns, 1 alloc ~2200x
NewPluginCompleter 137 µs, 561 allocs 1.6 µs, 10 allocs 86x

Config: Load (startup)

Benchmark Before After Speedup
clean load 44.0 µs, 8.8 KB, 133 allocs 39.2 µs, 5.1 KB, 72 allocs 1.12x (allocs −46%)
recovery load 154 µs, 30.4 KB, 438 allocs 131 µs, 25.4 KB, 347 allocs 1.18x (allocs −21%)

The modest one by design: it runs once at startup, so this is mostly about not parsing the same file two-to-three times.

Parser: Parse (per dispatched line)

Case Before After Speedup
regular command 699 ns, 720 B, 7 allocs 59 ns, 0 allocs 11.8x
?? prefix 854 ns, 1056 B, 9 allocs 18 ns, 0 allocs 46.7x
pipe | ?? 1295 ns, 1072 B, 10 allocs 31 ns, 0 allocs 42.0x
inline ?? 1074 ns, 1088 B, 11 allocs 75 ns, 0 allocs 14.4x

Testing

  • go test ./... passes; touched packages also pass with -race (three consecutive runs for internal/history).
  • New unit tests cover the prefix index: live-vs-loaded recency merge, top-k ordering, literal (non-glob) prefix matching, failed-command exclusion, SQL fallback before load, and index serving with the database closed.
  • golangci-lint run (v2.8.0, CI config): 0 issues.
  • Not covered: the e2e package doesn't compile on the base commit either (ScenarioMockTransport predates agent.Transport's AvailableModels method), so it's excluded from verification here.

🤖 Generated with Claude Code

https://claude.ai/code/session_012axhLiAUtKxRS6TaMux5UM

claude added 6 commits August 1, 2026 00:51
Cover the four hot paths about to be optimized: per-keystroke history
prefix search, built-in completion spec parsing, config load (clean and
recovery), and per-line command parsing. Committed before the changes so
before/after comparisons run the same benchmark code.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012axhLiAUtKxRS6TaMux5UM
The ghost-text suggestion function runs SearchByPrefix on every
keystroke, and the backing SQL (GLOB + GROUP BY + MAX(timestamp)) has to
aggregate every historical occurrence of every matching command. Under
the default unlimited-retention config that cost grows with history size:
~0.9ms per keystroke at 10k rows and ~15ms at 100k rows in benchmarks.

Load the distinct successful commands once in the background at store
open, keep them in a slice sorted by command text, and answer prefix
lookups with a binary search plus a scan of the matching range. Recency
is a session-local sequence number, so no timestamp parsing is needed and
commands run this session always outrank loaded ones. Store.Add updates
the index in place; until the initial load lands, SearchByPrefix keeps
answering from SQL, and a load failure only ever costs speed.

In-memory databases now cap the connection pool at one: a plain :memory:
DSN gives each pooled connection its own empty database, and the
background load is the first thing to make the pool open a second one.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012axhLiAUtKxRS6TaMux5UM
builtinPluginSpecs re-parsed the docker and hash-builtins TOML through
the full decoder on every handler-table rebuild: shell startup and every
completions reload. The specs are compile-time constants and immutable
once compiled, so parse them under sync.OnceValue and hand rebuilds a
cloned slice sharing the same spec instances.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012axhLiAUtKxRS6TaMux5UM
Every Load parsed config.toml twice in full: once into the Config struct
and once into a raw map just to pull out the [agent.<name>] tables, and
the recovery path parsed it a third time even though it already held a
raw map. Decode only the [agent] key on the happy path so the decoder
skips every other section, and hand the recovery path's existing raw map
straight to loadNamedAgents.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012axhLiAUtKxRS6TaMux5UM
Parse and the shell dispatch site allocated map[string]any trace
payloads on every dispatched line just for Emit to drop them, since
tracing is off in normal sessions. Guard the payload construction behind
trace.Enabled, the pattern the completion plugin code already uses.
Emitted events are unchanged when tracing is enabled.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012axhLiAUtKxRS6TaMux5UM
gocyclo: the per-branch trace guards pushed Parse past the complexity
limit; move emission into traceDetect/traceResult helpers that carry the
enabled check themselves, keeping Parse's control flow to the actual
parsing. Event payloads per command type are unchanged, and Parse stays
allocation-free when tracing is off.

gofmt: realign inline comments in prefix_index_test.go.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012axhLiAUtKxRS6TaMux5UM
@tfcace tfcace changed the title bench: add benchmarks for interactive-path parsing and lookup costs perf: take general-purpose parsing and query costs off the interactive hot paths Aug 1, 2026
claude added 2 commits August 1, 2026 04:57
Store.Close returned while the background prefix-index load could still
hold a live SQLite connection: db.Close only reaps a busy connection
once its query finishes, so the load's WAL activity kept touching the
database files after Close. Under go test this raced t.TempDir cleanup
into 'directory not empty' failures (reproducible locally within ~300
runs of TestStore_CreateAndAdd).

Run the load under a context that Close cancels, and have Close wait for
the goroutine before closing the pool. Also rename a local that the new
context import shadowed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012axhLiAUtKxRS6TaMux5UM
Ghost-text suggestions now come from a per-session in-memory index, so
commands run in another concurrently running shell only appear after a
restart. Plan is an optional refresh through the upcoming hook system
(after-execution hook re-running the index load); install already merges
a reload with live session entries, so only the trigger is missing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012axhLiAUtKxRS6TaMux5UM
@tfcace
tfcace merged commit 745ad59 into master Aug 1, 2026
14 checks passed
tfcace added a commit that referenced this pull request Aug 1, 2026
…e hot paths (#93)

Applies the idea behind uv's custom lockfile parser (astral-sh/uv#20648):
stop paying a general-purpose engine on a hot path with a known input shape.

- history: SearchByPrefix ran a GLOB + GROUP BY + MAX(timestamp) query on
  every keystroke, aggregating every historical occurrence of every match.
  Distinct successful commands now load once in the background at store
  open into a text-sorted in-memory slice; lookup is a binary search plus
  range scan, Store.Add updates the index in place, SQL stays the fallback
  until the load lands. In-memory SQLite DSNs cap the pool at one conn,
  since each pooled :memory: conn gets its own empty database.
- completion: built-in docker/hash-builtins TOML specs are compile-time
  constants; parse once per process under sync.OnceValue instead of on
  every handler-table rebuild.
- config: Load parsed config.toml twice in full, three times on recovery.
  Happy path now decodes only the agent key; recovery reuses its raw map.
- parser: map[string]any trace payloads were built per line for Emit to
  discard when tracing is off; now behind trace.Enabled guards. Emitted
  events are unchanged when tracing is on.

Benchmarks: linux/amd64, Intel Xeon 2.80GHz, go1.25.0, -benchmem. Committed
first in 8687faa so before/after run identical benchmark code.

  go test -run '^$' -bench . -benchmem \
    ./internal/history/ ./internal/completion/ \
    ./internal/config/ ./internal/parser/

  history.SearchByPrefix — per keystroke
  -----------------------------------------------------
  case                         before     after  speedup
  10k rows, dense prefix       845 µs    6.2 µs     136x
  10k rows, narrow prefix      453 µs    3.8 µs     120x
  10k rows, miss              32.9 µs     86 ns     383x
  100k rows, dense prefix     15.2 ms     57 µs     268x
  100k rows, narrow prefix     7.9 ms     40 µs     196x
  100k rows, miss             32.7 µs    107 ns     307x

  completion — startup, `completions reload`
  ----------------------------------------------------------------
  benchmark                    before     after  speedup     allocs
  builtinPluginSpecs           138 µs     63 ns   ~2200x   552 -> 1
  NewPluginCompleter           137 µs    1.6 µs      86x  561 -> 10

  config.Load — startup
  ----------------------------------------------------------------
  case                         before     after  speedup     allocs
  clean                       44.0 µs   39.2 µs    1.12x       -46%
  recovery                     154 µs    131 µs    1.18x       -21%

  parser.Parse — per dispatched line
  ----------------------------------------------------------------
  case                         before     after  speedup     allocs
  regular command              699 ns     59 ns    11.8x     7 -> 0
  ?? prefix                    854 ns     18 ns    46.7x     9 -> 0
  pipe | ??                   1295 ns     31 ns    42.0x    10 -> 0
  inline ??                   1074 ns     75 ns    14.4x    11 -> 0

The history before-numbers scale with table size (0.85 ms -> 15.2 ms from
10k to 100k rows, i.e. visible typing lag); the after-numbers scale only
with the number of distinct matching commands. config.Load is modest by
design — it runs once at startup, so the win is not parsing the same file
two-to-three times.

New tests cover the prefix index: live-vs-loaded recency merge, top-k
ordering, literal prefix matching, failed-command exclusion, SQL fallback
before load, and serving with the database closed. go test ./... passes;
touched packages pass with -race. golangci-lint v2.8.0: 0 issues. e2e is
excluded — it doesn't compile on the base commit either
(ScenarioMockTransport predates agent.Transport's AvailableModels).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants