Repository navigation
Fix response format race in AI binding's run(). - #8204
Open
nick-tsiones wants to merge 1 commit into
Open
nick-tsiones wants to merge 1 commit into
nick-tsiones wants to merge 1 commit into
Conversation
run() stored its per-call options in a single shared #options field, and after awaiting the fetch it read that field again to decide whether to return the raw Response or a parsed object/stream. When another run() call started while the first was still in flight, the first call could format its response according to the second call's options, e.g. a call with returnRawResponse set could get a parsed object instead of the Response it needs to stream. Decide the response format from the call's own options instead. The remaining reads of #options all happen synchronously before the fetch is started, so they cannot interleave with another call. No compatibility flag: only callers issuing concurrent run() calls observe a difference, and for them the old behavior was a race whose result depended on timing, not something to preserve. Adds a delayedModel to ai-mock.js and two regression tests covering both interleavings of concurrent plain and returnRawResponse runs.
|
All contributors have signed the CLA ✍️ ✅ |
|
This pull request was not reviewed automatically because its author does not have write access to this repository. A maintainer with write access can start a review by commenting |
Author
|
I have read the CLA Document and I hereby sign the CLA |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #8203
What was wrong
Ai.run()insrc/cloudflare/internal/ai-api.tsstored its per-call options in a single shared#optionsfield, and after awaiting the upstream fetch it read that field again to decide whether to return the rawResponseor a parsed object/stream. When anotherrun()call started while the first was still in flight, the first call formatted its response according to the second call's options: areturnRawResponse: truecall could receive a parsed object (breaking access tores.bodyfor streaming), and a plain call could receive an unexpectedResponse.All other reads of
#options(extra headers,prefix,signal, and thewebsocketbranch) happen synchronously before the fetch is started, so the post-await format decision was the only read that could interleave with another call.What changed
run()now decides the response format from its ownoptionsargument instead of the shared slot. One functional line, plus a comment explaining why the shared slot must not be read there, so it doesn't get "simplified" back.Compatibility
No compatibility flag: only callers issuing concurrent
run()calls observe a difference, and for them the previous behavior depended on timing — the old outcome is not something code could sanely depend on. This decision is recorded in the commit message as well.Tests
delayedModelinai-mock.jsholds its response for 100 ms so anotherrun()call can start and finish in the meantime (modeled on the existinghangingModel).ai-api-test.js, one per interleaving: the raw-response run starting first and finishing last, and starting last and finishing first.Both fail on
mainand pass with this change. Verified by executing the realai-api.tsunder Node with a stubbed fetcher: pre-fix, thereturnRawResponsecall resolved to a parsedObject; post-fix, each concurrent call gets its own format.prettier --check(repo config) passes on all three changed files.I prepared this PR with an AI coding agent (OpenCode). I am responsible for it and will answer review comments.