Skip to content

Query supported devices before graph compilation #815

Closed

Description

@anssiko

[ Spun off from PR #809 review comment by @fdwr and a discussion with @inexorabletash, @reillyeon, @RafaelCintron, @philloooo -- thanks! ]

(Emphasis mine:)

This afternoon we were talking with {@inexorabletash, @reillyeon, @RafaelCintron, Phillis} about the likely need for a caller to know whether a particular device is supported or not, because an app may want to (if say GPU is not supported) use a different more performant fallback than for WebNN to silently fall back to CPU. For example, if GPU was unavailable (even though you preferred high performance), then it might be faster to execute the model with WebGPU shaders than WebNN CPU, or it might be okay to use CPU, but the app could load a different model that's more CPU-friendly, if it knew that was the case.

This need is the reverse of what we currently have, querying the context for what kind of devices it actually supports rather than prescribing a preferred device type. Now, preferred device type prescription may return someday as a weaker hint (or list of hints, or list of antihints...) as we learn more about usage/needs, but for either case prescription or post-query, having device enums could be useful. 🤔

To kick off this discussion, I'll start with a simple and naive API further abstracted from the GPUDevice.features setlike interface:

// only MLPowerPreference supported at context creation time
const context = await navigator.ml.createContext({ powerPreference: 'high-performance' });

// query the newly created context
if (context.devices.has('cpu') && context.devices.has('npu')) {
   // context supports both CPU and NPU, choose the model accordingly
   // how the work is split across CPU and NPU is an implementation detail
} else if (!context.devices.has('gpu')) {
   //  no GPU available, perhaps try WebGPU?
} else {
   // ...
}

To help tease out more informed proposals, I'd like folks to test this imaginary API against the following:

@mwyrzykowski would this work with CoreML MLComputeUnits? A subset of possible MLDeviceType combinations is supported but there's also a catch-all all?

How about other backends?

This issue welcomes further discussion and proposals on the topic of post-context creation query mechanism for supported devices.

Activity

  1. mwyrzykowski commented on Feb 4, 2025

    @mwyrzykowski

    @mwyrzykowski would this work with CoreML MLComputeUnits? A subset of possible MLDeviceType combinations is supported but there's also a catch-all all?

    It would work but perhaps of limited usefulness? context.devices.has(...) would return true but there is no indication which device a particular model will run on. E.g., usage of float32 arithmetic will prevent the model from running on Apple's NPUs. Whereas on possibly another computing device, usage of float16 arithmetic could prevent running on a certain processor unit.

    I am not sure how we answer the question from #809 (comment) without providing some fine grained limits / capabilities as noted in #749. The concern there, mentioned in #749 (comment) was the privacy / security aspect from my understanding.

  2. reillyeon commented on Feb 4, 2025

    @reillyeon
    Contributor

    Fine grained signals are not necessary to answer the question from #809 (comment). A developer who has developed GPU- and CPU-optimized models wants to implement this flowchart:

    flowchart TD
        A[Device has GPU?] -->|Yes| B(Load GPU model)
        A -->|No| C(Load CPU model)
        B --> D(Can execute on GPU?)
        D -->|Yes| E(Execute on GPU)
        D -->|No| C
        C --> F(Execute on CPU)
    
    Loading

    This only requires knowing whether the GPU is an option for a given context (to avoid loading a GPU model if it has no chance of working) and whether a graph can run (or has run) on the GPU or not.

    The actual flowchart may be more complex, as it could include non-WebNN fallbacks such as WASM and WebGPU as well as models with different system performance requirements.

  3. zolkis commented on Feb 5, 2025

    @zolkis
    Collaborator

    If we remove MLDeviceType (as in #809), the only way to do the flowchart above is trying to create a WebGPU context, and if that fails, then create a generic context - and then this issue argues for adding query support to those contexts.

    Just to note, earlier Chai (?) mentioned we need a way to easily create a GPU context without a WebGPU device. We are losing that with #809. Querying context capabilities of generic contexts will not solve that.

    It seems there is still a strong use case to query the platform capabilities and match that with the requirements of the model at hand. Moreover, I could imagine a more complex flowchart, where also want to query other context specific constraints.

    We could go and create a generic context using the current options and then query for capabilities. But this does not guarantee getting a GPU context, even when one would be available.

    I propose we walk a middle path, e.g. by saying:

    const context = await navigator.ml.createContext({ 
        powerPreference: 'high-performance', 
        devicePreference: 'gpu'
    });

    This would only guarantee that GPU will be involved (e.g. a combination of GPU and NPU/CPU), otherwise it will fail.

    Note that passing "npu" alone might be too little information. So I do I see the reason in passing other context creation options/constraints, too, as suggested in #749 (comment)

    const context = await navigator.ml.createContext({ 
        powerPreference: 'low-power', 
        devicePreference: 'npu',
        supportLimits: { output: [ "float16" ] }
    });

    I am not sure do we also need the query function proposed there (the one that might make fingerprinting easier):

    partial interface ML {
      record<USVString, MLOpSupportLimits> opSupportLimitsPerAdapter();
    };

    If possible, I'd try doing the options above without this query function, at the first attempt.
    @mwyrzykowski would that work for you? @reillyeon?

    If yes, we could do the following:

    • I close Remove MLDeviceType #809 (i.e. cancel the PR) -- or modify it as follows.
    • keep the MLDeviceType enum
    • support a new context creation option as MLOpSupportLimits supportLimits, as proposed in #749 (comment)
    • write the related algorithmic steps.

    WDYT?

  4. anssiko commented on Feb 5, 2025

    @anssiko
    MemberAuthor

    Thanks for your swift responses and suggestions, everyone. (Markdown-friendly Mermaid is a great tool for quick diagrams, we should use it more often in the spec-land.)

    @zolkis, I see consensus emerging on PR #809, so I'd like to see that through and work on the query mechanism on top of it. If we need to revive some bits from that PR later, that is OK.

  5. mwyrzykowski commented on Feb 6, 2025

    @mwyrzykowski

    If we remove MLDeviceType (as in #809), the only way to do the flowchart above is trying to create a WebGPU context, and if that fails, then create a generic context - and then this issue argues for adding query support to those contexts.

    Technically, there is nothing in the WebGPU specification which requires a GPU (it was intentionally designed this way).

    const context = await navigator.ml.createContext({
    powerPreference: 'low-power',
    devicePreference: 'npu',
    supportLimits: { output: [ "float16" ] }
    });

    If possible, I'd try doing the options above without this query function, at the first attempt. @mwyrzykowski would that work for you? @reillyeon?

    The concept that an npu device has certain operations and a gpu device has others, or a cpu device can run all operations is something WebKit thinks should remain out of the WebNN specification and leave these details to the UA.

    Foreseeably, one vendor's npu device could have the capabilities of another vendor's gpu device. Or one vendor's gpu has all the same operations / capabilities as the cpu with different performance characteristics.

    I understand there are privacy concerns, but achieving this use case seems better suited to something like WebGPU's supportedLimits / supportedFeatures:
    https://www.w3.org/TR/webgpu/#gpusupportedlimits
    https://www.w3.org/TR/webgpu/#gpusupportedfeatures

    Perhaps we could have a similar approach with allowances to maintain privacy?

  6. reillyeon commented on Feb 6, 2025

    @reillyeon
    Contributor

    I imagine the "Device has GPU?" check would be implemented by running,

    const context = await navigator.ml.createContext({
        powerPreference: 'high-performance'
    });
    if (context.devices.has('gpu')) {
      // Yes, load GPU model.
    } else {
      // No, load CPU model.
    }

    Given the constraints of the Core ML framework the "Can execute on GPU?" check needs to actually invoke the graph,

    const [test_input, test_output] = await Promise.all([
        context.createTensor({...}),
        context.createTensor({...}),
    ]);
    context.dispatch(graph, { input: test_input }, { output: test_output });
    await context.readTensor(test_output);
    if (graph.devices.has('gpu')) {
      // Yes.
    } else {
      // No, load CPU model.
    }

    The naming and how exactly graph.devices gets populated is TBD but would be based on the history of previous inference runs.

  7. zolkis commented on Feb 6, 2025

    @zolkis
    Collaborator

    OK, so there seems to be a preference for creating a device-agnostic context, leave it to the UA to select the device(s) based on context options (currently only powerPreference) (any other hints to add to context options about device selection?), then apps would query the context for more detailed capabilities.

    @mwyrzykowski
    I understand there are privacy concerns, but achieving this use case seems better suited to something like WebGPU's supportedLimits / supportedFeatures:
    https://www.w3.org/TR/webgpu/#gpusupportedlimits
    https://www.w3.org/TR/webgpu/#gpusupportedfeatures

    Sounds good, especially the latter, for our use case. So what about the following.

    enum MLDeviceType {
      "cpu",
      "gpu",
      "npu"
    };
    
    interface MLDevices {
        readonly setlike<MLDeviceType>;
    };
    
    partial interface MLContext {
        [SameObject] readonly attribute MLDevices devices;
    }

    So then the examples by @anssiko and @reillyeon would work:

    if (context.devices.has('gpu')) {

  8. huningxin commented on Feb 8, 2025

    @huningxin
    Contributor

    @fdwr

    This afternoon we were talking with @inexorabletash, @reillyeon, @RafaelCintron, @philloooo about the likely need for a caller to know whether a particular device is supported or not,

    Thanks for the discussion. I'd like to understand more about the use case.

    because an app may want to (if say GPU is not supported) use a different more performant fallback than for WebNN to silently fall back to CPU. For example, if GPU was unavailable (even though you preferred high performance), then it might be faster to execute the model with WebGPU shaders than WebNN CPU,

    As I commented, WebNN implementation could use WebGPU shaders to execute graph on GPU for some devices where IHV-optimized GPU kernels are not available. Native ML frameworks are doing similar thing, for example ONNX Runtime native WebGPU execution provider. I suppose when high performance execution is preferred, silently falling back to sub-optimal CPU execution is an implementation issue.

    or it might be okay to use CPU, but the app could load a different model that's more CPU-friendly, if it knew that was the case.

    May I know what aspects make a model more CPU-friendly? Does it use CPU-friendly operators or data types? Are there any gaps indicating a WebNN implementation to select CPU device when the app loads a CPU-friendly model?

  9. reillyeon commented on Feb 10, 2025

    @reillyeon
    Contributor

    May I know what aspects make a model more CPU-friendly? Does it use CPU-friendly operators or data types?

    I am not an expert here but my understanding is that a CPU-friendly model may use different data types or simply be smaller so that it can meet the application's performance needs while providing acceptably reduced inference quality.

    Are there any gaps indicating a WebNN implementation to select CPU device when the app loads a CPU-friendly model?

    I think there was some discussion of adding a specific "cpu-only" flag to context creation if the current power preference flags are insufficient.

  10. mwyrzykowski commented on Feb 10, 2025

    @mwyrzykowski

    Sounds good, especially the latter, for our use case. So what about the following.

    enum MLDeviceType {
    "cpu",
    "gpu",
    "npu"
    };

    interface MLDevices {
    readonly setlike;
    };

    partial interface MLContext {
    [SameObject] readonly attribute MLDevices devices;
    }
    So then the examples by @anssiko and @reillyeon would work:

    if (context.devices.has('gpu')) {

    @zolkis What information does the 'cpu', 'gpu', 'npu' label really provide? The concern is it does not provide information over the chips capabilities.

    Instead, we could have a mechanism for saying that a given UA can, for example:

    • support the following MLOperandDataType values (e.g., float16, float32, int64, float64)
    • create tensors of maximum shape = K
    • other distinguishing limits?

    whether or not the UA is running on a device with an npu or gpu is not really meaningful. One vendor's NPU could be equal in ML capabilities to another vendor's GPU for instance.

  11. zolkis commented on Feb 11, 2025

    @zolkis
    Collaborator

    @mwyrzykowski yes, the idea for exposing capabilities has been recorded. I was thinking to remove device types (PR #809), then experiment with how to expose the capabilities, and/or other mappings between context options/properties and device selection.

    But it seems that #809 made people re-thinking what we'd lose and what we'd need, spanning this issue.

    IIUC, the capabilities approach does not necessarily conflict with having another mechanism to poll whether a given type (as standardized string label) of accelerator/device is associated with the context just created on the given platform/environment.
    In my current understanding, in most use cases CPU/GPU/NPU still have meanings (as well as symbolic mappings like low-power and high-performance), though it might be blurry and conflated with capabilities, and NPU alone is a rather diverse category.

    I agree that knowing the current device types is not enough information, but I don't think this is useless information, at least today.

    If you have a strong view on avoiding CPU/GPU/NPU device names altogether and focus only on capabilities, we need to discuss this during the next call (@anssiko please check). The problem with capabilities is that they are not well defined ATM, and appear to be (far) more complex to use. We need time to figure that out. Yet, I'd agree this might be the most flexible approach, and also the most exact approach.

  12. inexorabletash commented on Feb 13, 2025

    @inexorabletash
    Contributor

    Given discussion today on the telecon and @mwyrzykowski's comments above, WDYT about something like this:

    const support = await context.querySupport({
      dataTypes: ['float16', 'int8'],
      maximumRank: 6,
      operators: ['lstm', 'hardSwish'],
    });
    console.log(support); // "optimized" or "fallback" 
  13. handellm commented on Feb 14, 2025

    @handellm

    Thanks for the discussion yesterday on the telcon!

    Just to repeat the clarification of the usecase that we have for realtime video processing:

    1. If the user selects to use functionality like background blur, we want to offer the best quality the device can offer. So the product has a small set of candidate models and technologies (WebNN, WebGPU, WASM) that it has to choose between. Accelerated technologies come with allowance for beefier models.
    2. The model/tech choser algorithm needs to be fast, and we need to avoid spending seconds or even hundreds of milliseconds to figure out if a given model should be able to run accelerated. So for example downloading the entirety (could be large things..), compiling & try-running a model seems infeasible.

    @inexorabletash, @mwyrzykowski - that looks really interesting, and close to how MediaCapabilities works! I'd imagine querySupport would potentially be almost instant. It's easy to keep that kind of small metadata with each model and being a small download.

    In terms of naming here, perhaps "accelerated" is a term to consider to mean execution or NPU or GPU. It's widely used inside Chrome and elsewhere in other contexts.

  14. handellm commented on Feb 14, 2025

    @handellm

    Another clarification about our use case is that we'd strongly recommend the API to not do internal fallbacks from "accelerated" to CPU in runtime. We'd prefer the API errors out in that case to enable logic in the app to choose what to try next which could include using other tech than WebNN.

  15. 40 remaining items

  16. zolkis commented on Aug 14, 2025

    @zolkis
    Collaborator

    Using a fallback flag is fine as a mechanism to signal the implementation could not fulfill the hints.
    But we need to specify compared to what is this considered a fallback, i.e. we need more exact definitions for the hints/options.

    On the other hand, since there are multiple options listed in the Pre-download or pre-build hints and constraints
    section, we might need multiple context flags to explain what exactly are the reported as error paths.
    (Of course we could also limit the options to make a lone "fallback" flag meaningful.)

    In general, I've been thinking about how WebGPU adapters could be used in WebNN, but it would be quite a complication, in the light of what use cases it would be meant to solve in WebNN.

    The main objection seems to be to not use the terms "npu" or "gpu" as accelerators -- but (for hints at least) are there any computational patterns we could use for creating a simple naming mapping to current NPU/GPU capabilities that would be acceptable @mwyrzykowski?

    My position is that NPU/GPU terms and controversy is quite well understood among developers. We'd just need to add clarifying remarks in the WebNN spec / algorithms that would avoid misunderstandings. Would that work for the long term?

    The problem I see using only limits the complicated mapping between the use cases and the API. But if you could prove with examples that it would be easy, it could be an option.

  17. mwyrzykowski commented on Aug 15, 2025

    @mwyrzykowski

    are there any computational patterns we could use for creating a simple naming mapping to current NPU/GPU capabilities that would be acceptable

    To clarify @zolkis, do you mean mapping to either the NPU or GPU (or both)? I.e., the developer chooses a model which is expected to run somewhere which is not the CPU? Or is the developer expected to make a decision to run a model on the NPU instead of the GPU, or the inverse? The latter we think is not possible due to the current fragmentation amongst NPUs in additional to multiprocessing nature of most operating systems. Finding a solution to the former seems reasonable.

  18. zolkis commented on Aug 18, 2025

    @zolkis
    Collaborator

    Based on the comments in this issue, the main desired distinction is between CPU and GPU[/NPU], i.e. "massive parallel acceleration" ((GPU, NPU or both) or not. Then hints like "low-power" ["low-latency"] and "high-performance" provide enough information to the implementations to choose the right (combination of) accelerators.

    After reading through this issue again, I propose we could do the following to satisfy the both the use cases and the constraints listed in this issue:

    1. Expose a context property to tell whether CPU fallback is active (or likely) (true, false or undefined)
    2. Add a context creation option/hint for telling app preference for "massively parallel" acceleration (GPU/NPU/TPU/etc. or find a better name -- @handellm proposed "accelerated" for the reason of being used elsewhere too).
    3. Expose a context property to tell
      • whether "massive parallel" acceleration is available to the context (subject to system conditions) -- not necessarily guaranteed, but used with a very good chance, unless there is a higher priority exception.
      • Or, report that that massively parallel acceleration is supported in principle, but the underlying platform has full control over it (best effort).
        For instance, a context property named acceleration with values "massively-parallel", or "platform-controlled" (we could also include "cpu-fallback" here but I think it would be more useful if separated). All names can be changed.

    Fine-grained limits/capabilities introspection could be added later on need, but they would be in parallel to this.

  19. mwyrzykowski commented on Aug 20, 2025

    @mwyrzykowski

    The above proposal sounds reasonable to us @zolkis 👍

  20. mwyrzykowski commented on Aug 28, 2025

    @mwyrzykowski

    I can not attend the meeting today but I think we can move forward with #815 (comment) or some variant. My only feedback on the name "massively-parallel" is 'parallel' is an implementation detail of GPUs / NPUs, my preference would be something like accelerated

  21. handellm commented on Aug 29, 2025

    @handellm

    It seems clear from yesterday's discussion that underlying frameworks cannot support the querySupport without reverse engineering. If this is true it seems like the next best option is @zolkis proposal since it'll allow mitigation measures faster than detecting poor performance in runtime. So, 👍 .

    I'm a bit hesitant if bullet 3 in the proposal adds anything. Wouldn't presence of a GPU (probably close to 100% of all real clients) give an "accelerated" response in nearly all situations here?

  22. zolkis commented on Sep 1, 2025

    @zolkis
    Collaborator

    @handellm

    I'm a bit hesitant if bullet 3 in the proposal adds anything.

    Indeed, if we have a property/event telling CPU fallback is being used, then we don't really need to expose an "accelerated" property on the context -- if massive parallel acceleration (NPU, GPU) is indeed available at all on the platform. I think this still might be useful, for the sake of completeness. An example below.

    Wouldn't presence of a GPU (probably close to 100% of all real clients) give an "accelerated" response in nearly all situations here?

    My impression from the discussion here was that the sheer presence of a GPU doesn't guarantee an "accelerated" context.
    Also, there might not be a GPU at all on the platform.
    (There was also an observation that some might consider also CPUs to be able to handle "accelerated" contexts -- that is why I thought to specify "massively parallel" when talking about acceleration.)

    The reason the proposal included "massively parallel" or "accelerated" (the latter apparently being preferred) was to create a clean/explicit mapping from context options (power options such as "low-power" and "high-performance", plus "accelerated" which replaced the former device options such as "npu", "gpu", "cpu").

    For instance, in most typical cases, the mapping would be the following:

    • "low-power" + "accelerated" context will try to execute on NPU.
      Fall back on CPU (and set the CPU fallback flag) -- or, depending on the platform and system conditions, could also execute on GPU.
      (Note that since we don't return errors (to be discussed still!) but assume fall-backs, adding "accelerated" as a context option might not be too relevant, since the implementation could always try first the massive parallel acceleration if available.)
      EDIT: if implementations may return an error of they detect there is only CPU and "accelerated" was requested (TBD). This would allow to detect lack of massive parallel processing capability before creating a context.
    • "high-performance" + "accelerated" context will try to execute on GPU (or a super performant NPU).
      Fall back to NPU if available, or otherwise to CPU (and set the CPU fallback flag).
    • if the "accelerated" option is missing from above, the context might still execute on NPU/CPU ("low-power") or GPU/NPU/CPU ("high-performance"): it's up to the platform to choose from what's available.
      In that case, the implementation could expose the property "accelerated" on the context. This actually might make the "accelerated" context option redundant and the "bullet 3" "accelerated" property useful.

    In addition, I think that it would be more useful to expose CPU fallback as an event, rather than a property. @huningxin @reillyeon WDYT?

  23. handellm commented on Sep 2, 2025

    @handellm

    @zolkis are the context properties in the proposal allowed to change value after graph compilation?

  24. zolkis commented on Sep 2, 2025

    @zolkis
    Collaborator

    @handellm

    are the context properties in the proposal allowed to change value after graph compilation?

    Valid viewpoint. Based on the current assumptions, yes.
    But we should discuss the algorithms for context wrt. graph changes.

    Basically a Web ML context is now imagined to be device-agnostic, and should be able handle multiple accelerators on the system, as well as changing system conditions, hopefully. In that sense they are different from WebGPU adapters/devices. But how these will affect graph lifecycle is not yet mature.

    Any thoughts / inputs welcome.

    We could write the algorithms so that a context is mainly set when created, i.e. if it has massively parallel processing (MPP), graphs created in that context should assume MPP availability throughout its existence. A (CPU) fallback MAY happen, though. The exact handling is TBD.

    We could add that a context may change based on system conditions. For instance let's suppose it's created with MPP ("accelerated" is "probably"), but then a CPU fallback happens -- the question is whether should the impl. change the "accelerated" property to "best-effort" or "no". For simplicity, I would go with unchanged "accelerated" property, as the fallback is enough information, when the platform can tell the capacity loss is likely temporary. But it could be changed (to "best-effort" or even "no") when the platform detects a more permanent problem. It might be useful for applications to know about such conditions. However, we should check how uniformly that could be implemented.
    @mwyrzykowski, @huningxin WDYT?

  25. mwyrzykowski commented on Sep 11, 2025

    @mwyrzykowski

    We could add that a context may change based on system conditions. For instance let's suppose it's created with MPP ("accelerated" is "probably"), but then a CPU fallback happens -- the question is whether should the impl. change the "accelerated" property to "best-effort" or "no". For simplicity, I would go with unchanged "accelerated" property, as the fallback is enough information, when the platform can tell the capacity loss is likely temporary.

    Yes, I think the simple approach sounds best to start with

    But it could be changed (to "best-effort" or even "no") when the platform detects a more permanent problem. It might be useful for applications to know about such conditions. However, we should check how uniformly that could be implemented.
    @mwyrzykowski, @huningxin WDYT?

    I'm not sure this is reliably implementable on macOS unless a backend was not using CoreML. I.e., presumably a UA could have an entire path using only MPSGraph and then that ensures this.

  26. zolkis commented on Sep 11, 2025

    @zolkis
    Collaborator

    Based on the discussion on the WG call today, I record here the consensus that instead of vague/fuzzy terms, the planned "accelerated" property should be true or false. When true, the platform reports massively parallel processing capability, but the actual usage might be limited by real-time system conditions. When false, then GPU/NPU is not available.

    For those cases when the system conditions cannot allow "accelerated" processing, we'll signal CPU fallback on the context. We could start simple, with a getter (of an internal slot which is kept up to date by implementations). Needs implementability checks.

    These should still be able to solve the use case for Meet.

    At a later point, we could add an interface to request/query op limits, capabilities etc, as discussed earlier in this issue.

  27. zolkis commented on Sep 24, 2025

    @zolkis
    Collaborator

    Quoting the relevant passages from #884 here, for better overview.

    The following proposal gained support for a simple accelerator mapping solution (before using the previously discussed fine grained constraints):

    • Expose a context property (or event) to tell whether CPU fallback is active (or likely active).
    • Add a context creation option/hint (e.g. accelerated: true) for telling app preference for NPU and/or GPU accelerated "massively parallel" processing (MPP).
      • Note. This context option makes sense when an error is returned when the implementation overrides the option. Otherwise, if instead of returning an error a silent fallback is implemented (which seems the more generic behaviour), then applications could query the following proposed property on the context (albeit after context creation). If implementations could detect a CPU fallback, then they could also return an error. Whether to expose an error in this case is to be discussed, as it would allow detecting lack of massively parallel acceleration before creating a context.
    • Add a context property named "accelerated" with possible values: false (for likely no support for neither GPU nor NPU), and true (e.g. fully controlled by the underlying platform which makes a best effort for MPP, yet CPU fallback may occur).

    The following changes are proposed:

    partial dictionary MLContextOptions {
      boolean accelerated;  // TBD: could be true by default?
    };
    
    partial interface MLContext {
      boolean cpuFallbackActive;
    };

    The behavior of createContext() is proposed to follow this policy (TBD):

    • return an error in step 4 if the context option accelerated has been set to true, but the platform cannot provide massive parallel processing at all,
    • and set the accelerated property to false when the platform could in principle provide massive parallel processing which may or may not be available at the moment.

    Examples

    // create a context that should use massive parallel processing (e.g. GPU/NPU)
    context = await navigator.ml.createContext({accelerated: true});
    if (context.accelerated) {
        // the context will mostly use GPU/NPU, but CPU fallback may happen
    } else {
        // the platform tells it likely cannot provide NPU or GPU, so try something else
    }
    
    // create a context that should preferably use NPU
    context = await navigator.ml.createContext({accelerated: true, powerPreference: 'low-power'});
    if (context.accelerated) {
        // NPU is likely used -- further requirements could be set by opSupportLimitsPerDevice
    } else {
        // NPU is likely not available, and since GPU needs high power, it is not used
    }
  28. zolkis commented on Sep 24, 2025

    @zolkis
    Collaborator

    Then, in addition to the previous, the next step could be warming up the discussion on this proposal from @mwyrzykowski -- which could even replace the current context.opSupportLimits()?

    partial interface MLContext {
      record<DOMString, MLOpSupportLimits> perDeviceSupportedLimits();
    };

    where the DOMString is a UA supplied unique identifier of its choosing to denote the devices supported in the context.
    Note that this is relevant also for #836, which could use the same convention.

  29. added theissue type on Nov 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions