Repository navigation
Make zeroPoint optional for quantizeLinear and dequantizeLinear operations #943
Description
Summary
quantizeLinear and dequantizeLinear require a zeroPoint operand. Because the argument is mandatory, WebNN has to materialize a zero-point tensor even for a symmetrically quantized model(zero point is omitted), which introduces unnecessary memory consumption. And this makes symmetric quantization inexpressible in WebNN.
Additionally, some backends select more efficient kernels for symmetric quantization by observing that no zero point is attached. Since WebNN always requires one, those kernels are unreachable.
Current behavior
Operation definition:
MLOperand dequantizeLinear(MLOperand input, MLOperand scale, MLOperand zeroPoint,
optional MLOperatorOptions options = {});
MLOperand quantizeLinear(MLOperand input, MLOperand scale, MLOperand zeroPoint,
optional MLOperatorOptions options = {});zeroPoint's shape is redundant in both operators: method steps 7-8 require it to equal scale's. Its data type is redundant in dequantizeLinear, but quantizeLiear's output data type comes from the data type of zeroPoint (method steps 11).
Why this matters
Measured on Qwen3-4B (4-bit weight-compressed, symmetric) running through WebNN on the ONNX Runtime backend, through WebNN the EP synthesizes 252 zero-point constants that exist in no source file, costing:
- ~13.5 MiB of memory. Each is separately allocated, passed to
MLGraphBuilder.constant(), and copied across the renderer → service boundary. - 756 nodes that exist only to subtract zero.
- The symmetric kernel becomes unreachable. With a zero point attached, the kernel will takes the asymmetric path. Removing the zero point input will save ~3% kernel time.
Backends status
- ONNX —
DequantizeLinearandQuantizeLinearboth take 2–3 inputs with an optional zero point.QuantizeLinearcarries anoutput_dtypeattribute. - CoreML —
dequantizeandquantizedeclarezero_pointasoptional=True.quantizehas a separate, requiredoutput_dtype. - TFLite —
QuantizationParametersalways carries a zero-point vector; the format has no "absent" state, only all-zeros. Neutral: it would synthesize zeros, is not blocked, gains nothing directly.
Proposal
Make zeroPoint optional on both operators, and give quantizeLinear an explicit output data type.
dictionary MLQuantizeDequantizeLinearOptions : MLOperatorOptions {
MLOperand zeroPoint; // if omitted, the zero point is 0
};
dictionary MLQuantizeLinearOptions : MLQuantizeDequantizeLinearOptions {
MLOperandDataType outputDataType; // quantizeLinear only
};
partial interface MLGraphBuilder {
MLOperand dequantizeLinear(MLOperand input, MLOperand scale,
optional MLQuantizeDequantizeLinearOptions options = {});
MLOperand quantizeLinear(MLOperand input, MLOperand scale,
optional MLQuantizeLinearOptions options = {});
};Output type for quantizeLinear: zeroPoint's data type if present; otherwise outputDataType; if both, they must match, else TypeError; if neither, TypeError.
dequantizeLinear does not need an outputDataType; scale remains required.
Activity
fdwr commentedon Aug 11, 2026
That's substantial.
I forget the reasoning by now for wanting the zero-point tensor to be present - maybe it was because certain backends always require it (like TF), or for slightly reduced complexity (optional parameters add conditional logic), or for explicitness's sake if there was a difference in the defaults for the tensor (e.g. like uint8 would default to 128 for symmetric quantization in one backend but 0 in another). In any case, we are evidently losing notable performance for CoreML and ORT, and TF is no worse. So, I'm not opposed to optionality of the zero-point tensors.