fix(cloud-agent-next): trust the intercept CA in the wrapper, not a DO exec by eshurakov · Pull Request #6673 · Kilo-Org/cloud

eshurakov · 2026-09-24T06:42:59Z

Problem

The intercept CA trust step ran as a control-plane container.exec() inside SandboxContainers, gated by a 5 s CONTAINER_CALL_TIMEOUT_MS. container.exec() resolves with an ExecProcess handle — completion is the separate exitCode promise — so the timeout only covered exec acceptance on a cold boot. When it elapsed, the effect reported launch_failed / container_CA_trust_timed_out and the session wedged on "Waiting for the sandbox to become available…" (incident workspace_8ce564ac-7648-441f-affe-dfe4fd3c536b).

Checked against the Cloudflare Sandbox SDK (packages/sandbox-container/src/server.ts, cert.ts): the SDK trusts the CA in the container's own process at startup, appends the PEM to the first existing system bundle, and points SSL_CERT_FILE/CURL_CA_BUNDLE/REQUESTS_CA_BUNDLE/GIT_SSL_CAINFO at the complete bundle.

Change

Trust moves into the wrapper, which owns the container's own startup:

SandboxContainers.trustInterceptCa and both call sites are deleted; installContainmentProxy stays.
Dockerfile.containers entrypoint is now only exec sleep infinity (no more update-ca-certificates).
New wrapper/src/control/cert.ts: polls for the CA (5000 ms budget, 100 ms poll), exit(1) on missing/unreadable/append failure, appends the PEM to the first existing bundle, and exports the four bundle env vars. main.ts installs trust before telemetry and main.
NODE_EXTRA_CA_CERTS stays owned by containedProcessEnv at exec time: Bun reads it at process start, so the wrapper must not assign it. One writer per decision.
New src/shared/container-intercept.ts holds the single source for SANDBOX_INTERCEPT_HTTPS and the CA path.

Verification

Fails without the change: reverting the pre-change SandboxContainers/Dockerfile.containers fails 3 sandbox-container tests and the entrypoint test; disabling the cert.ts guards fails 3 exit tests; removing the awaiting trust call fails the ordering test. All sources sha256-verified after restore.
vitest run src/sandbox-containers src/sandbox-control - 36 files, 1200 tests pass. Wrapper bun test src - 1695 pass; wrapper tsgo exit 0.
pnpm run lint 0 errors, format:check clean, typecheck exit 0.
Reviews: independent reviewer approved (single bundle writer, single NODE_EXTRA_CA_CERTS writer, no trust exec/timeout, non-returning exit(1) paths); simplicity reviewer found the current shape simpler than the plan.
Not verified: stage 9 E2E needs a deploy. The wrapper's own TLS under real interception (CA readable at Bun process start), real cold-boot timing, and the system-store append on a real image are unproven until then.

Follow-up

The unbounded unknown observe ladder that turned this into a wedge is fixed in the sibling PR (fail-fast-wedged-allocation).

…O exec Container launches failed with container_CA_trust_timed_out because the Durable Object bounded the CA-trust container.exec with the 5s call timeout, which covers exec acceptance on a cold boot, not command completion. Move trust into the control wrapper process: - Delete trustInterceptCa and both call sites; keep installContainmentProxy. - Remove the image entrypoint's cp + update-ca-certificates; it now only keeps PID 1 alive. Exactly one writer mutates the system bundle. - The wrapper installs trust at startup, before any agent tool spawns: poll for the injected CA, exit non-zero if absent/unreadable/append fails, append the PEM to the first existing system bundle, and point SSL_CERT_FILE/CURL_CA_BUNDLE/REQUESTS_CA_BUNDLE/GIT_SSL_CAINFO at the full bundle. - NODE_EXTRA_CA_CERTS stays owned by containedProcessEnv at exec time so Bun trusts the CA from process start; the wrapper never assigns it. - Marker name/value and CA path live once in src/shared/container-intercept.ts.

kilo-code-bot · 2026-09-24T06:47:44Z

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Incremental review of the follow-up commit 6179213a94 (testable CA-path injection replacing a process-global node:fs mock) found no new issues.

Files Reviewed (7 files)

services/cloud-agent-next/Dockerfile.containers
services/cloud-agent-next/src/sandbox-containers/SandboxContainers.ts
services/cloud-agent-next/src/sandbox-containers/sandbox-containers.test.ts
services/cloud-agent-next/src/shared/container-intercept.ts
services/cloud-agent-next/wrapper/src/control/cert.test.ts
services/cloud-agent-next/wrapper/src/control/cert.ts
services/cloud-agent-next/wrapper/src/control/main.ts

Incremental scope: wrapper/src/control/cert.ts and wrapper/src/control/cert.test.ts (changed in 6179213a94). The new optional RuntimeCertPaths parameter defaults to the real container paths, so production behavior is unchanged, and the tests now exercise real temp files instead of a process-global node:fs mock, removing the cross-file mock leak that failed CI.

Previous Review Summary (commit 7bb7bf4)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit `7bb7bf4`)

Status: No Issues Found | Recommendation: Merge

Files Reviewed (7 files)

services/cloud-agent-next/Dockerfile.containers
services/cloud-agent-next/src/sandbox-containers/SandboxContainers.ts
services/cloud-agent-next/src/sandbox-containers/sandbox-containers.test.ts
services/cloud-agent-next/src/shared/container-intercept.ts
services/cloud-agent-next/wrapper/src/control/cert.test.ts
services/cloud-agent-next/wrapper/src/control/cert.ts
services/cloud-agent-next/wrapper/src/control/main.ts

_{Reviewed by deepseek-v4.1-flash · Input: 0 · Output: 0 · Cached: 0}

_{Review guidance: REVIEW.md from base branch main}

… step CI failed the `cloud-agent-next` job: 30 wrapper tests failed in workload-cgroup, owned-processes and file-log-uploader with `Unexpected readFileSync path: ...` thrown from cert.test.ts. Cause: `cert.test.ts` installed a process-global `mock.module('node:fs')` and never restored it. Bun mocks a module for the lifetime of the process, so every test file whose `node:fs` import resolved after cert.test.ts got the mock. Whether that happens depends on bun's file order, which is why the suite passed locally and failed on the runner; the CI log shows the mock active in the files that ran after cert.test.ts. Fix: `trustRuntimeCert` and `installInterceptTrustIfEnabled` take an optional `RuntimeCertPaths` (cert path plus candidate bundle paths, defaulting to the real container paths), so the tests exercise the real filesystem against a temp directory instead of mocking the module. No `mock.module` remains in the wrapper. Verified: 11 cert tests pass; removing the bundle env-var assignments fails 4 of them and removing the append-failure exit fails 1; the full wrapper suite passes apart from timing flakes in process-spawning tests under load (both files pass in isolation); service lint 0 errors, format:check clean, typecheck exit 0.

eshurakov · 2026-09-24T07:19:13Z

CI follow-up pushed as 6179213a94.

The first run of this PR failed the cloud-agent-next job: 30 wrapper tests failed in workload-cgroup, owned-processes and file-log-uploader with Unexpected readFileSync path: ... thrown from cert.test.ts.

Cause: cert.test.ts installed a process-global mock.module('node:fs') and never restored it. Bun mocks a module for the lifetime of the process, so every test file whose node:fs import resolved after cert.test.ts received the mock; the CI log shows the mock active in exactly the files that ran after it. Whether that happens depends on bun's file order, which is why the suite passed locally and failed on the runner.

Fix: trustRuntimeCert and installInterceptTrustIfEnabled now take an optional RuntimeCertPaths (cert path plus candidate bundle paths, defaulting to the real container paths). The tests run against a real temp directory instead of mocking the module, so no mock.module remains anywhere in the wrapper.

Verified: 11 cert tests pass; removing the bundle env-var assignments fails 4 of them and removing the append-failure exit fails 1; service lint 0 errors, format:check clean, typecheck exit 0. The full wrapper suite passes locally apart from timing flakes in process-spawning tests under load (both files pass in isolation).

@iscekic

…4554) ## Automated docs sync — 2026-09-25 This PR keeps kilo.ai/docs in sync with features merged to [Kilo-Org/cloud](https://github.com/Kilo-Org/cloud) and [Kilo-Org/kilocode](https://github.com/Kilo-Org/kilocode). Every change below links to the merged PR it documents. - Window: `2026-09-24T07:08:33.639Z` → `2026-09-25T07:05:29.302Z` - Verification (docs build + tests): **passing** ### Surface: `cloud-mobile` - Assignees / requested reviewers: @iscekic and @eshurakov - Derivation: Derived from the repository layout. A product surface is a package under packages/ that ships a distinct client, plugin, backend, or hosted service: cli = packages/opencode/ + packages/tui/ + packages/server/ + packages/sdk/ + packages/plugin/; vscode = packages/kilo-vscode/ + packages/kilo-web-ui/ + packages/kilo-ui/; jetbrains = packages/kilo-jetbrains/; gateway = packages/kilo-gateway/; web = packages/kilo-console/ + packages/kilo-indexing/ + packages/kilo-memory/ + packages/kilo-sandbox/. Docs route from the IA tree packages/kilo-docs/pages/ plus docs/jetbrains-vscode-settings-parity.md: each surface lists the pages sections that document it, and the per-platform pages under packages/kilo-docs/pages/code-with-ai/platforms/ map to the matching extension surface (the vscode/ directory to vscode, jetbrains.md to jetbrains). A doc path belongs to the surface with the longest matching prefix; a path that matches none of those prefixes falls to `other` (the explicit other prefixes are listed under other.docs). The cloud surfaces are derived the same way from the Kilo-Org/cloud layout: cloud-mobile = apps/mobile/, cloud-web = apps/web/, cloud-extension = apps/extension/, and cloud-agent = the cloud-agent packages under packages/ (packages/cloud-agent-sdk/ + packages/cloud-agent-profile/). A cloud source names its repository while a bare string still means this repository. The pages under packages/kilo-docs/pages/collaborate/ document the cloud web app (app.kilo.ai: teams dashboard, billing, SSO, adoption dashboard), so they route to cloud-web. No page under packages/kilo-docs/pages/ documents the browser side-panel extension yet, so cloud-extension lists no docs prefix. - Map: `.github/docs-sync/surfaces.json` - Surface map: `cli`, `vscode`, `jetbrains`, `gateway`, `web`, `cloud-mobile`, `cloud-web`, `cloud-extension`, `cloud-agent`, `other` - Source prefixes: `apps/mobile/` (Kilo-Org/cloud) - Doc prefixes: `packages/kilo-docs/pages/code-with-ai/platforms/mobile.md` - Paths that fall to `other`: `packages/kilo-docs/pages/community/`, `packages/kilo-docs/pages/kiloclaw/`, `packages/kilo-docs/pages/contributing/`, `packages/kilo-docs/LEARNINGS.md`, `docs/` - Reviewers are ranked from `Kilo-Org/cloud`; the workflow needs a token with `contents: read` on that repository (repository secret `CROSS_REPO_ACCESS_TOKEN`, exposed to the upsert step as `CLOUD_REPO_TOKEN`). - How the two were computed: Reviewers for `cloud-mobile` are ranked from `Kilo-Org/cloud` git history over `apps/mobile/` (a commit 180 days old counts half as much, half-life 180 days). Bots (author type "Bot" or a login matching /\[bot\]$/i) and people without admin, write, or maintain permission are excluded. ### Changes  | Docs change | Source | | --- | --- | | updated pages/code-with-ai/platforms/mobile.md | [Kilo-Org/cloud#6386](Kilo-Org/cloud#6386) | | updated pages/ai-providers/openai-chatgpt-plus-pro.md | [Kilo-Org/cloud#6702](Kilo-Org/cloud#6702) | | updated pages/code-with-ai/platforms/cloud-agent.md | [Kilo-Org/cloud#6683](Kilo-Org/cloud#6683) | | updated pages/getting-started/byok.md | [Kilo-Org/cloud#6692](Kilo-Org/cloud#6692) |  ### Pending — will retry  _None._  ### Considered, no docs change needed  | PR | Reason | | --- | --- | | [Kilo-Org/cloud#6658](Kilo-Org/cloud#6658) | Internal sandbox lifecycle fix with no user-visible workflow or setting. | | [Kilo-Org/cloud#6673](Kilo-Org/cloud#6673) | Internal container CA trust plumbing, no user-facing behavior. | | [Kilo-Org/cloud#6672](Kilo-Org/cloud#6672) | Internal sandbox launch/recovery fix with no documented workflow change. | | [Kilo-Org/cloud#6660](Kilo-Org/cloud#6660) | Internal cloud-agent queue delivery fix; no new command, setting, or workflow for users. | | [Kilo-Org/cloud#6226](Kilo-Org/cloud#6226) | Internal gateway alias-routing change, not user-visible. | | [#14490](#14490) | Tool-call animation and streaming UI polish; users do not need to learn a new workflow. | | [#14530](#14530) | Bug fix restoring intended worktree-pool behavior, no doc change needed. | | [#14529](#14529) | Bug fix restoring tab/panel state across project switches. | | [#14531](#14531) | Reconnect recovery bug fix, restores already-documented behavior. | | [#14532](#14532) | Bug fix keeping session tab title in sync on rename. | | [Kilo-Org/cloud#6088](Kilo-Org/cloud#6088) | Removes internal/admin model-experiment surfaces, not public product docs. | | [Kilo-Org/cloud#6682](Kilo-Org/cloud#6682) | Internal control-socket reconnect race fix, no user-facing change. | | [#14534](#14534) | Transcript re-render performance bug fix. | | [#14535](#14535) | Bug fix preserving the loaded browser page across context switches. | | [Kilo-Org/cloud#6684](Kilo-Org/cloud#6684) | Reverted by Kilo-Org/cloud#6685. | | [Kilo-Org/cloud#6678](Kilo-Org/cloud#6678) | Dead-code constant removal, no user-visible effect. | | [Kilo-Org/cloud#6687](Kilo-Org/cloud#6687) | Removes internal model-experiment maintenance and retains tables, no user-facing change. | | [#14515](#14515) | JetBrains plugin unload crash fix, no documented behavior change. | | [#14520](#14520) | JetBrains transcript/list rendering performance work. | | [Kilo-Org/cloud#6614](Kilo-Org/cloud#6614) | Mobile PR Review header and session title bug fix, no doc change needed. | | [Kilo-Org/cloud#6625](Kilo-Org/cloud#6625) | Internal mobile secure-store error-handling refactor. | | [Kilo-Org/cloud#6624](Kilo-Org/cloud#6624) | Mobile auth bug fix that stops a retry loop; restores expected sign-in behavior with no new setting or workflow. | | [#14310](#14310) | Contributor/CI fix making the kilo-v2 checkout installable; not user-visible product behavior. | | [Kilo-Org/cloud#6611](Kilo-Org/cloud#6611) | Mobile notification-tap fix that selects the session's organization; restores correct behavior rather than adding a learnable feature. | | [Kilo-Org/cloud#6644](Kilo-Org/cloud#6644) | Mobile sign-in layout/alignment polish; no change to what a user must do. | | [Kilo-Org/cloud#6601](Kilo-Org/cloud#6601) | Mobile layout fix keeping empty states clear of the tab bar; purely visual. | | [#14543](#14543) | CI/release infrastructure adding Windows binary code signing; no public docs impact. | | [Kilo-Org/cloud#6616](Kilo-Org/cloud#6616) | Mobile visual defect fixes and a session-title fallback; no new user workflow or setting. | | [Kilo-Org/cloud#6630](Kilo-Org/cloud#6630) | Reports an edge-case partial worktree restore; failure-path plumbing with no new user-facing workflow, target setting, or config. | | [Kilo-Org/cloud#6699](Kilo-Org/cloud#6699) | Cloud Agent e2e stabilization plus internal idle-sandbox capacity handling; not user-visible. | | [#14545](#14545) | Automated JetBrains release/changelog PR; underlying user-facing changes are triaged from their own PRs. | | [Kilo-Org/cloud#6708](Kilo-Org/cloud#6708) | Internal AI-gateway request-logging policy change in the admin panel; no existing public docs surface and no change to how users run Kilo Code. | | [#14533](#14533) | Documentation already shipped with the merged PR. The experimental.task_model_selection flag is gone from the current source, and pages/code-with-ai/agents/model-selection.md, pages/code-with-ai/agents/context-mentions.md, and pages/getting-started/settings/index.md already describe per-task selection as default-on with no stale experiment references. | | [#14510](#14510) | Documentation already shipped with the merged PR. Marketplace companion-skill support is present in the current source (packages/opencode/src/kilocode/marketplace/companions.ts and installer), and pages/customize/marketplace.md already documents installing, publishing, and removing MCP servers with companion skills. |  --- (bot) Generated by the docs-sync workflow. Humans review and merge; while this PR stays open, the next daily run appends new changes here. Branch: `docs/auto-sync-2026-09-25`.

marius-kilocode approved these changes Sep 24, 2026

View reviewed changes

eshurakov merged commit 79ad7b6 into main Sep 24, 2026
26 checks passed

eshurakov deleted the eshurakov/container-ca-trust-in-wrapper branch September 24, 2026 07:49

Navigation Menu

fix(cloud-agent-next): trust the intercept CA in the wrapper, not a DO exec - #6673

fix(cloud-agent-next): trust the intercept CA in the wrapper, not a DO exec#6673
eshurakov merged 2 commits into
mainfrom
eshurakov/container-ca-trust-in-wrapper

eshurakov commented Sep 24, 2026

Uh oh!

kilo-code-bot Bot commented Sep 24, 2026 •

edited

Loading

Previous review (commit `7bb7bf4`)

Uh oh!

eshurakov commented Sep 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

eshurakov commented Sep 24, 2026

Problem

Change

Verification

Follow-up

Uh oh!

kilo-code-bot Bot commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Summary

Executive Summary

Previous review (commit 7bb7bf4)

Uh oh!

eshurakov commented Sep 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kilo-code-bot Bot commented Sep 24, 2026 •

edited

Loading

Previous review (commit `7bb7bf4`)