Repository navigation
Parse data URLs (and their MIME types) according to the Fetch Standard #2909
Description
Spec is at https://fetch.spec.whatwg.org/#data-urls
Status as of November 2019: https://crates.io/crates/data-url has an initial implementation, but integration with Servo is blocked on figuring out the interactions with https://crates.io/crates/mime. CC hyperium/mime#78, servo/rust-url#567.
Initial title: Figure out the details of data URL parsing
Initial description:
For example, what should happen when a base64 sequence contains a (0x20 ASCII) space? Another non-alphabet character?
Since RFC 2397 does no define error handling, this might involve reverse-engineering other implementations.
https://www.w3.org/Bugs/Public/show_bug.cgi?id=19494
http://greenbytes.de/tech/tc/datauri/
Activity
frewsxcv commentedon Aug 19, 2015
Should this be moved to rust-url?
SimonSapin commentedon Aug 19, 2015
The corresponding code is currently in
components/net/data_loader.rsin this repository, not in rust-url. I might move that code to rust-url once all the details are figured out and I’m confident to tell the world "this is how you do it" (i.e. when this issue can be closed) but in the mean time I’m not planning to.nox commentedon Nov 8, 2018
The spec bug trail brought me to whatwg/fetch#234.
pshaughn commentedon Nov 25, 2019
https://infra.spec.whatwg.org/#forgiving-base64-decode might be the thing to do here?
jdm commentedon Nov 25, 2019
Indeed, step 11.2 of https://fetch.spec.whatwg.org/#data-urls uses the forgiving base64 decode algorithm, so that looks like a change we will need to make in our implementation.
pshaughn commentedon Nov 25, 2019
I am doing that now; we're already forgiving spaces, just need to add forgiveness of a few other kinds of byte.
pshaughn commentedon Nov 25, 2019
There is one remaining problem: "ab=" is invalid base64, but we're responding to this particular invalid base64 in a different way from all the other invalid base64. I don't know where this "Reached unreachable code" string is even being generated yet; it doesn't seem to be a string in Servo or in the base64 crate.
` ▶ Unexpected subtest result in /fetch/data-urls/base64.any.worker.html:
│ FAIL [expected PASS] data: URL base64 handling: "ab="
└ → assert_unreached: Should have rejected: undefined Reached unreachable code
▶ Unexpected subtest result in /fetch/data-urls/base64.any.html:
│ FAIL [expected PASS] data: URL base64 handling: "ab="
└ → assert_unreached: Should have rejected: undefined Reached unreachable code
`
jdm commentedon Nov 25, 2019
Reached unreachable codecomes from https://github.com/web-platform-tests/wpt/blob/783959beb60fc589e897cc11d08c23f621259f62/resources/testharness.js#L1852 (orservo/tests/wpt/web-platform-tests/resources/testharness.js
Line 1852 in f1aa5d8
SimonSapin commentedon Nov 25, 2019
If someone is looking at this, please consider using https://crates.io/crates/data-url or filing issues in https://github.com/servo/rust-url/issues about why it’s not ideal :)
pshaughn commentedon Nov 25, 2019
@SimonSapin do you have any advice about how to use the data-url crate in a project that's using https://crates.io/crates/mime elsewhere? I don't see an obvious interconversion strategy between the two Mime classes, even though they expose almost identical getters. I am new to Rust's object model and it's possible there is some obvious answer here.
pshaughn commentedon Nov 26, 2019
One possibility here is to keep the servo code that's parsing some of the URL to make a Mime object from the mime crate, call the data-url crate to get the data, and just ignore the data-url crate's own mime type parsing. That seems like it'd get correct behavior, but it feels a little odd to have to parse the mime type twice and throw one out. Still, I'll try it tomorrow and see if it at least passes all the tests.
[-]Figure out the details of data URL parsing[/-][+]Implement data URLs according to the Fetch Standard[/+][-]Implement data URLs according to the Fetch Standard[/-][+]Parse data URLs (and their MIME types) according to the Fetch Standard[/+]SimonSapin commentedon Nov 26, 2019
This is a very relevant question (that was out of my mind when I suggested using
data-url, and) that unfortunately doesn’t really have an answer yet. See hyperium/mime#78Parsing twice not only feels odd, but might cause subtle bugs when the two parsers don’t behave the same. So let’s not use the parser in
data-urlyet, keep this issue open, and accept #24865 as an incremental improvement.Would using https://docs.rs/data-url/0.1.0/data_url/forgiving_base64/ (without the rest of the crate) help with the
ab=case?pshaughn commentedon Nov 26, 2019
Yes, that saves net/data_loader.rs from having to even know about the definition of ASCII whitespace, and it fixes that one last test case! Thank you!
Fix servo#2909 as far as WPT is concerned, not full data-url class in…
Auto merge of #24870 - pshaughn:2909, r=jdm
Auto merge of #24870 - pshaughn:2909, r=jdm
SimonSapin commentedon Nov 27, 2019
As discussed above, this issue should not be closed yet.
@pshaughn Please note that some keywords in commit messages and issue descriptions are meaningful to GitHub: https://help.github.com/en/github/managing-your-work-on-github/closing-issues-using-keywords