Repository navigation
allocator: Disable jemalloc at build-time when any sanitizers is enabled - #49065
Merged
Merged
Conversation
mrobinson
force-pushed
the
cargo-disable-jemalloc-dynamically
branch
from
October 19, 2026 21:24
e55aa2d to
8f64e41
Compare
sagudev
approved these changes
Oct 20, 2026
| let address_sanitizer_enabled = std::env::var("CARGO_CFG_SANITIZE") | ||
| .unwrap_or_default() | ||
| .split(',') | ||
| .any(|part| part == "address"); |
Member
There was a problem hiding this comment.
I believe at least memory sanitizer also need this so maybe we should just detect the presence of any sanitizer and disable jemalloc?
Member
Author
There was a problem hiding this comment.
That didn't happen before, but that makes sense to me as well. I've done this.
mrobinson
force-pushed
the
cargo-disable-jemalloc-dynamically
branch
from
October 21, 2026 06:18
8f64e41 to
f40c49c
Compare
mrobinson
enabled auto-merge
October 21, 2026 06:19
sagudev
reviewed
Oct 21, 2026
| [target.'cfg(not(any(windows, target_env = "ohos")))'.dependencies] | ||
| libc = { workspace = true } | ||
|
|
||
| [target.'cfg(all(not(sanitize = "address"), not(any(windows, target_env = "ohos"))))'.dependencies] |
Member
Author
There was a problem hiding this comment.
Okay. I've listed the three sanitizers people are most likely to use ("memory", "thread", and "address"). I don't think it makes sense to list every single Rust-supported sanitizer here. This line is harmless is any other sanitizer is used though: it just results in the compilation of the jemalloc crates. They are still disabled at runtime.
sagudev
disabled auto-merge
October 21, 2026 06:21
This is another commit on the path to making it so that `cargo build` does the same thing as `mach build`. This makes it so that jemalloc is disabled dynamically via a `build.rs` script rather than having this handled by `mach`. In addition, jemalloc is disabled by the build script for platforms it's not supported on (Windows and OHOS). This was handled via code configuration before, but is handled via `build.rs` now. Signed-off-by: Martin Robinson <mrobinson@abandonedwig.info>
mrobinson
force-pushed
the
cargo-disable-jemalloc-dynamically
branch
from
October 21, 2026 07:27
f40c49c to
2c9fc23
Compare
mrobinson
enabled auto-merge
October 21, 2026 07:31
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Oct 21, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Oct 22, 2026
jschwe
pushed a commit
to openharmony-rs/servo-ohos
that referenced
this pull request
Oct 22, 2026
…ndexeddb
Add an OpenHarmony RDB backend behind the `ohos-rdb-backend` feature: OHOS
builds can use the system database through ohos-rdb-sys C FFI instead of the
bundled sqlite3. The sqlite modules keep their upstream logic; the RDB
implementation lives in new files with its own copy of the schema and CRUD, a
deliberate duplication that keeps rebases trivial, wired in through a small set
of cfg hooks. The FFI wrappers free their native handles on drop. A cursor
borrows its transaction, so the compiler rejects reads after commit. Strings
with interior NUL bytes are stored as blobs and read back unchanged.
Backend selection is a feature choice rather than a target choice, so a plain
`./mach build --ohos` keeps the upstream SQLite behaviour and no configuration
fails to compile:
./mach build --ohos # SQLite, as before
./mach build --ohos --features ohos-rdb-backend
# both, chosen at runtime
./mach build --ohos --no-default-features \
--features "base,ohos-rdb-backend"
# RDB only
Dropping sqlite is what makes the third configuration worth having: when
first measured (base 79e2c7a) the stripped aarch64 release
`libservoshell.so` went from 127,736,736 to 125,736,080 bytes, 1.91 MiB
less. Since the 2026-08-29 sync this no longer holds: upstream fb04f8f
(servo#43819, on-disk HTTP cache) makes rusqlite an unconditional dependency of
components/net, so the RDB-only configuration still links sqlite. The saving
needs remeasuring, and the configuration is only worth keeping if servo#43819's
sqlite use becomes optional.
With both backends compiled in, `storage_ohos_rdb_backend_enabled` (default
true) picks one per store as it is opened, so a single binary can A/B them
without reinstalling. The two backends keep disjoint on-disk stores — sqlite
files versus RDB's nested `rdb/<store>` — so flipping the pref switches to the
other backend's data, which starts out empty. That is intended for A/B
comparison; there is no migration between them.
Deliberate difference from the SQLite twin: RDB delivers the callbacks of a
batch after its commit, so the whole batch lands or errors together (SQLite
reports each operation in autocommit). Schema operations join that batch
transaction and follow the same rule. A second deliberate difference: after a
failed webstorage load on OHOS the session's final save is skipped, so a
partially loaded state cannot overwrite the store.
On a HarmonyOS phone the two legs of the pref agree exactly. One signed HAP,
`storage_ohos_rdb_backend_enabled` flipped between runs, over the ten WPT
storage tests the backend touches: 1188 subtests pass and 37 fail on each
side, test for test, and the failures are the ones tests/wpt/meta already
expects. The stores confirm the switch rather than assuming it — with the pref
on the app writes only `rdb/reg.rdb` and `rdb/webstorage.rdb`, with it off only
`reg.sqlite` and `webstorage.sqlite`.
One RDB limitation is worth recording. Opening a second client_storage registry
fails (14800001 / 14800011) about two runs in three, on DAYU200 and PLR-AL00
alike, when the process is not an application: `new_storage_threads` with no
`config_dir` falls back to `tempfile::tempdir()`, which outside an application
sandbox is `/data/local/tmp`. Neither half alone is enough — a single registry
under `/data/local/tmp` is reliable, and so are two registries inside the
application, which a phone confirmed over ten launches of the default
configuration and five of `--temporary-storage`, the latter being the same
two-registry shape that fails in a bare process. Servo always passes the
application's cache dir, so it never meets the condition;
`test_new_storage_threads_create_independent_groups`, which does, is ignored
when the RDB backend is compiled in.
The RDB copies of the IndexedDB and client-storage schemas are checked
against the SQLite ones by `tests/schema_parity.rs`, a host test that builds
both in an in-memory SQLite database and compares tables, columns, keys,
foreign keys and indexes, so an upstream schema change that is not mirrored
fails on the host instead of diverging on devices. The webstorage `data`
tables differ on purpose and are not compared.
Validation: Parallel twin of the sqlite backend — revalidate on every rebase even when it merges cleanly, since upstream storage changes never conflict with the RDB engine files. (1) `./mach check --ohos -- -p servo-storage --all-targets --features ohos-rdb-backend` must pass warning-free, as must the plain and the `--no-default-features --features ohos-rdb-backend` configurations; a new `AsyncOperation`/engine-trait variant fails here loudly and must be implemented in the RDB twins, not wildcarded. (2) Diff upstream changes since the previous base with `git log -p <old-base>..<new-base> -- components/storage` and mirror any sqlite-engine semantics changes into `engines/ohos_rdb.rs` / `webstorage/engines/ohos_rdb.rs` / `client_storage_ohos_rdb.rs`. (3) Host `./mach test-unit -p servo-storage` pins the sqlite reference behaviour. (4) On a device, run the same suite against both backends with `CARGO_TARGET_AARCH64_UNKNOWN_LINUX_OHOS_RUNNER=ohos-test-runner cargo nextest run -p servo-storage --target aarch64-unknown-linux-ohos [--no-default-features --features ohos-rdb-backend]` (raise nextest's slow-timeout: every test is pushed to the device), then the same-HAP A/B: build servoshell with `--features ohos-rdb-backend` (flavor and signing per the device: `harmonyos`/`hos` for a phone whose UDID is in the profile, default/OpenHarmony for the DAYU200), install it, and run `./mach test-wpt --ohos --release --no-headless --ohos-device <serial> --pref dom_indexeddb_enabled=true --pref storage_ohos_rdb_backend_enabled=<bool> --log-wptreport <leg>.json <subset>` (the `servo_ohos` wptrunner product of the `wpt-ohos` runner branch, cherry-picked onto the stack) once per pref value with `bm clean -n org.servo.servo -d` before each leg. Subset = all of `webstorage/` plus `IndexedDB/{idbobjectstore_put,idbobjectstore_get,idbobjectstore_delete,idbobjectstore_createIndex,idbindex_get,idbcursor_continue_objectstore,keypath,key_valid,transaction-abort-generator-revert,upgrade-transaction-lifecycle-committed,idbfactory_open,idbfactory_deleteDatabase}.any.{html,worker.html}` (78 tests). Compare the two wptreports subtest by subtest: no subtest may differ (crash/timeout-truncated tests are re-run in isolation on both legs), and the app data dir must contain only `*.rdb` stores after the true leg and only `*.sqlite` after the false leg (a lone `reg.rdb` from the runner's pref-less first launch and the HTTP cache's `cache.sqlite3` are expected). (5) Host `cargo nextest run -p servo-storage schema_parity` must pass; it fails when an upstream schema change is not mirrored into `indexeddb/engines/shared.rs` or `client_storage_shared.rs`. `./mach clippy --locked -- -- --deny warnings` must pass.
AI-assisted: Yes (agent session; implementation and on-device testing)
AI-assisted: Yes (agent session; 2026-08-29 rebase conflict resolution -- components/servo/Cargo.toml and components/storage/Cargo.toml resolved to the upstream side with only the fork's `ohos-rdb-backend` feature and optional-dependency markers re-added: upstream's `no-wgl = ["dep:mozangle", ...]` (servo#47198) is kept and `ohos-rdb-backend = ["storage/ohos-rdb-backend"]` re-added after it, and servo#43819 moved rusqlite's `bundled` feature onto the workspace dependency, so components/storage now carries only `rusqlite = { workspace = true, optional = true }`. Validation step (2): of the three upstream commits touching components/storage, only 7bea9ed (`ImmutableOrigin::ascii_serialization` now returns `Cow<'_, str>`) reaches the RDB twins, and it is a type change with no behaviour change; mirrored into `client_storage_shared.rs`, whose `create_a_storage_shelf` now hands the `Cow` to `push_text` by deref instead of the gone `as_str()`; `indexeddb/engines/shared.rs` already coerced, and no other twin site serializes an origin. fb04f8f touches only the Cargo.toml line above and 39f05ac only sqlite-side destructuring, so no semantics were mirrored into `engines/ohos_rdb.rs`, `webstorage/engines/ohos_rdb.rs` or `client_storage_ohos_rdb.rs`. Note: fb04f8f gave components/net an unconditional rusqlite dependency, so the 1.91 MiB saving quoted above for the RDB-only configuration no longer holds and needs remeasuring)
AI-assisted: Yes (agent session; 2026-10-08 sync -- conflict resolution in Cargo.toml, components/servo/Cargo.toml and components/storage/Cargo.toml: upstream's 0.7.0 versions with the fork's `default-features = false` re-added, upstream's new `multiprocess` default kept in servo's `default` and servoshell's `base` group, the fork's `[lints.rust]` table dropped for upstream's `[lints] workspace = true`; mirrored servo#47936 (index names unique per object store) into the RDB schema copy, adapted the tests to servo#48037's `GenericCallback::new`, added tests/schema_parity.rs and Validation step (5), updated step (4) for the current WPT runner, folded the clippy `duplicate_mod` squash commit; Cargo.lock regenerated from upstream's so that `cargo metadata --locked` passes at this commit)
AI-assisted: Yes (agent session; 2026-10-13 sync -- upstream servo#47255 added the default_tooling feature group: kept it in servo's default set and servoshell's base group next to the storage backend features)
AI-assisted: Yes (agent session; 2026-10-22 sync -- upstream servo#48934 and servo#49065 renamed servoshell's `default_without_allocator` group to `default_base` (which mach selects with `--no-default-features` for dummy-media builds) and moved jemalloc into it: `default_base` = `base` + `sqlite-backend`, and `base` takes upstream's list including `jemalloc`, so the RDB-only command above no longer names the allocator; servo's `default` merged next to upstream's blank line)
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
jschwe
pushed a commit
to openharmony-rs/servo-ohos
that referenced
this pull request
Oct 30, 2026
…ndexeddb
Add an OpenHarmony RDB backend behind the `ohos-rdb-backend` feature: OHOS
builds can use the system database through ohos-rdb-sys C FFI instead of the
bundled sqlite3. The sqlite modules keep their upstream logic; the RDB
implementation lives in new files with its own copy of the schema and CRUD, a
deliberate duplication that keeps rebases trivial, wired in through a small set
of cfg hooks. The FFI wrappers free their native handles on drop. A cursor
borrows its transaction, so the compiler rejects reads after commit. Strings
with interior NUL bytes are stored as blobs and read back unchanged.
Backend selection is a feature choice rather than a target choice, so a plain
`./mach build --ohos` keeps the upstream SQLite behaviour and no configuration
fails to compile:
./mach build --ohos # SQLite, as before
./mach build --ohos --features ohos-rdb-backend
# both, chosen at runtime
./mach build --ohos --no-default-features \
--features "base,ohos-rdb-backend"
# RDB only
Dropping sqlite is what makes the third configuration worth having: when
first measured (base 79e2c7a) the stripped aarch64 release
`libservoshell.so` went from 127,736,736 to 125,736,080 bytes, 1.91 MiB
less. Since the 2026-08-29 sync this no longer holds: upstream fb04f8f
(servo#43819, on-disk HTTP cache) makes rusqlite an unconditional dependency of
components/net, so the RDB-only configuration still links sqlite. The saving
needs remeasuring, and the configuration is only worth keeping if servo#43819's
sqlite use becomes optional.
With both backends compiled in, `storage_ohos_rdb_backend_enabled` (default
true) picks one per store as it is opened, so a single binary can A/B them
without reinstalling. The two backends keep disjoint on-disk stores — sqlite
files versus RDB's nested `rdb/<store>` — so flipping the pref switches to the
other backend's data, which starts out empty. That is intended for A/B
comparison; there is no migration between them.
Deliberate difference from the SQLite twin: RDB delivers the callbacks of a
batch after its commit, so the whole batch lands or errors together (SQLite
reports each operation in autocommit). Schema operations join that batch
transaction and follow the same rule. A second deliberate difference: after a
failed webstorage load on OHOS the session's final save is skipped, so a
partially loaded state cannot overwrite the store.
On a HarmonyOS phone the two legs of the pref agree exactly. One signed HAP,
`storage_ohos_rdb_backend_enabled` flipped between runs, over the ten WPT
storage tests the backend touches: 1188 subtests pass and 37 fail on each
side, test for test, and the failures are the ones tests/wpt/meta already
expects. The stores confirm the switch rather than assuming it — with the pref
on the app writes only `rdb/reg.rdb` and `rdb/webstorage.rdb`, with it off only
`reg.sqlite` and `webstorage.sqlite`.
One RDB limitation is worth recording. Opening a second client_storage registry
fails (14800001 / 14800011) about two runs in three, on DAYU200 and PLR-AL00
alike, when the process is not an application: `new_storage_threads` with no
`config_dir` falls back to `tempfile::tempdir()`, which outside an application
sandbox is `/data/local/tmp`. Neither half alone is enough — a single registry
under `/data/local/tmp` is reliable, and so are two registries inside the
application, which a phone confirmed over ten launches of the default
configuration and five of `--temporary-storage`, the latter being the same
two-registry shape that fails in a bare process. Servo always passes the
application's cache dir, so it never meets the condition;
`test_new_storage_threads_create_independent_groups`, which does, is ignored
when the RDB backend is compiled in.
The RDB copies of the IndexedDB and client-storage schemas are checked
against the SQLite ones by `tests/schema_parity.rs`, a host test that builds
both in an in-memory SQLite database and compares tables, columns, keys,
foreign keys and indexes, so an upstream schema change that is not mirrored
fails on the host instead of diverging on devices. The webstorage `data`
tables differ on purpose and are not compared.
Validation: Parallel twin of the sqlite backend — revalidate on every rebase even when it merges cleanly, since upstream storage changes never conflict with the RDB engine files. (1) `./mach check --ohos -- -p servo-storage --all-targets --features ohos-rdb-backend` must pass warning-free, as must the plain and the `--no-default-features --features ohos-rdb-backend` configurations; a new `AsyncOperation`/engine-trait variant fails here loudly and must be implemented in the RDB twins, not wildcarded. (2) Diff upstream changes since the previous base with `git log -p <old-base>..<new-base> -- components/storage` and mirror any sqlite-engine semantics changes into `engines/ohos_rdb.rs` / `webstorage/engines/ohos_rdb.rs` / `client_storage_ohos_rdb.rs`. (3) Host `./mach test-unit -p servo-storage` pins the sqlite reference behaviour. (4) On a device, run the same suite against both backends with `CARGO_TARGET_AARCH64_UNKNOWN_LINUX_OHOS_RUNNER=ohos-test-runner cargo nextest run -p servo-storage --target aarch64-unknown-linux-ohos [--no-default-features --features ohos-rdb-backend]` (raise nextest's slow-timeout: every test is pushed to the device), then the same-HAP A/B: build servoshell with `--features ohos-rdb-backend` (flavor and signing per the device: `harmonyos`/`hos` for a phone whose UDID is in the profile, default/OpenHarmony for the DAYU200), install it, and run `./mach test-wpt --ohos --release --no-headless --ohos-device <serial> --pref dom_indexeddb_enabled=true --pref storage_ohos_rdb_backend_enabled=<bool> --log-wptreport <leg>.json <subset>` (the `servo_ohos` wptrunner product of the `wpt-ohos` runner branch, cherry-picked onto the stack) once per pref value with `bm clean -n org.servo.servo -d` before each leg. Subset = all of `webstorage/` plus `IndexedDB/{idbobjectstore_put,idbobjectstore_get,idbobjectstore_delete,idbobjectstore_createIndex,idbindex_get,idbcursor_continue_objectstore,keypath,key_valid,transaction-abort-generator-revert,upgrade-transaction-lifecycle-committed,idbfactory_open,idbfactory_deleteDatabase}.any.{html,worker.html}` (78 tests). Compare the two wptreports subtest by subtest: no subtest may differ (crash/timeout-truncated tests are re-run in isolation on both legs), and the app data dir must contain only `*.rdb` stores after the true leg and only `*.sqlite` after the false leg (a lone `reg.rdb` from the runner's pref-less first launch and the HTTP cache's `cache.sqlite3` are expected). (5) Host `cargo nextest run -p servo-storage schema_parity` must pass; it fails when an upstream schema change is not mirrored into `indexeddb/engines/shared.rs` or `client_storage_shared.rs`. `./mach clippy --locked -- -- --deny warnings` must pass.
AI-assisted: Yes (agent session; implementation and on-device testing)
AI-assisted: Yes (agent session; 2026-08-29 rebase conflict resolution -- components/servo/Cargo.toml and components/storage/Cargo.toml resolved to the upstream side with only the fork's `ohos-rdb-backend` feature and optional-dependency markers re-added: upstream's `no-wgl = ["dep:mozangle", ...]` (servo#47198) is kept and `ohos-rdb-backend = ["storage/ohos-rdb-backend"]` re-added after it, and servo#43819 moved rusqlite's `bundled` feature onto the workspace dependency, so components/storage now carries only `rusqlite = { workspace = true, optional = true }`. Validation step (2): of the three upstream commits touching components/storage, only 7bea9ed (`ImmutableOrigin::ascii_serialization` now returns `Cow<'_, str>`) reaches the RDB twins, and it is a type change with no behaviour change; mirrored into `client_storage_shared.rs`, whose `create_a_storage_shelf` now hands the `Cow` to `push_text` by deref instead of the gone `as_str()`; `indexeddb/engines/shared.rs` already coerced, and no other twin site serializes an origin. fb04f8f touches only the Cargo.toml line above and 39f05ac only sqlite-side destructuring, so no semantics were mirrored into `engines/ohos_rdb.rs`, `webstorage/engines/ohos_rdb.rs` or `client_storage_ohos_rdb.rs`. Note: fb04f8f gave components/net an unconditional rusqlite dependency, so the 1.91 MiB saving quoted above for the RDB-only configuration no longer holds and needs remeasuring)
AI-assisted: Yes (agent session; 2026-10-08 sync -- conflict resolution in Cargo.toml, components/servo/Cargo.toml and components/storage/Cargo.toml: upstream's 0.7.0 versions with the fork's `default-features = false` re-added, upstream's new `multiprocess` default kept in servo's `default` and servoshell's `base` group, the fork's `[lints.rust]` table dropped for upstream's `[lints] workspace = true`; mirrored servo#47936 (index names unique per object store) into the RDB schema copy, adapted the tests to servo#48037's `GenericCallback::new`, added tests/schema_parity.rs and Validation step (5), updated step (4) for the current WPT runner, folded the clippy `duplicate_mod` squash commit; Cargo.lock regenerated from upstream's so that `cargo metadata --locked` passes at this commit)
AI-assisted: Yes (agent session; 2026-10-13 sync -- upstream servo#47255 added the default_tooling feature group: kept it in servo's default set and servoshell's base group next to the storage backend features)
AI-assisted: Yes (agent session; 2026-10-22 sync -- upstream servo#48934 and servo#49065 renamed servoshell's `default_without_allocator` group to `default_base` (which mach selects with `--no-default-features` for dummy-media builds) and moved jemalloc into it: `default_base` = `base` + `sqlite-backend`, and `base` takes upstream's list including `jemalloc`, so the RDB-only command above no longer names the allocator; servo's `default` merged next to upstream's blank line)
Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is another commit on the path to making it so that
cargo builddoes the same thing as
mach build. This makes it so that jemalloc isdisabled dynamically via a
build.rsscript rather than having thishandled by
mach. In addition, jemalloc is disabled by the build scriptfor platforms it's not supported on (Windows and OHOS). This was handled
via code configuration before, but is handled via
build.rsnow.Testing: This should not have an effect on builds that aren't run with
cargo buildso existing tests suffice.