Repository navigation
URLPattern use of URL record is ambiguous #252
Description
What is the issue with the URL Pattern Standard?
URLPattern is using URL records, and passing some URL records to URL algorithms. This might lead to ambiguous behaviour.
For instance https://urlpattern.spec.whatwg.org/#canonicalize-a-hostname is passing the URL record to parse hostname (via https://url.spec.whatwg.org/#hostname-state).
It is unclear whether an URL record is a special URL or not, so it is unclear whether parsing of the hostname (https://url.spec.whatwg.org/#concept-host-parser) will lead to https://url.spec.whatwg.org/#concept-opaque-host-parser.
Looking at WPT tests and Chrome results, the expected behaviour is to consider that the url is special and hostname is not considered opaque.
The same pattern is done in other places of the spec (canonicalisation algorithms of port, pathname, search and hash for instance), though it is not clear to me whether this has side effects.
Discussing with @annevk, it is unexpected to pass URL records directly like done in https://urlpattern.spec.whatwg.org/#canonicalize-a-hostname. A future url spec refactoring might make that clear.
Ideally, the url given to the basic URL Parser should be an URL built by a URL parser.
Maybe that is what the spec should do.
Activity
sisidovski commentedon Mar 12, 2025
Thank you for filing this. I agree, the current spec doesn't specify the given URL is special or not and this should be explicit. Or even the current spec may be wrong because the default value of scheme is empty string, which is not a special scheme so the URL wouldn't be special.
Yes.
Are you referring URL parser? I see there is a comment in URL parser, saying "Non-web-browser implementations only need to implement the basic URL parser". I assume some non-web-browser implementers don't implement this algorithm. Do you have any concern about just using the result of basic URL parser, given "http://dummy.test" as an input and pass it as dummyURL in https://urlpattern.spec.whatwg.org/#canonicalize-a-hostname?
annevk commentedon Mar 12, 2025
I think that generally works, but note that you cannot distinguish between someone passing in a hostname of
:and a hostname ofdummy.test. So how exactly do you know whether it succeeded?annevk commentedon Mar 12, 2025
I suppose we could maybe standardize upon a magic host value, such as
urlpattern.invalid, but it doesn't seem great.annevk commentedon Mar 13, 2025
Perhaps we can make https://url.spec.whatwg.org/#hostname-state return failure in more cases to make it more suitable for this caller. As all other callers with hostname state as override state ignore the return value. Which does make this API stand out, but so be it. I'll investigate.
Report all hostname state failures for URLPattern
URLPattern: correct port and hostname-related test
annevk commentedon Mar 13, 2025
@sisidovski I think whatwg/url#863 coupled with your suggested change here to create a proper dummy URL this should be good. When implementing this I found one impacted test for which I put up a patch at web-platform-tests/wpt#51316.
sisidovski commentedon Mar 14, 2025
Thank you so much. I'll check your PRs and then work on the URLPattern side.
URLPattern: correct port and hostname-related test
Report all hostname state failures for URLPattern
Bug 1953756 [wpt PR 51316] - URLPattern: correct port and hostname-re…
sisidovski commentedon Mar 24, 2025
I looked at #255 and #220.
Basically we'll have a dummy URL with a special scheme, but what if a non-special scheme is specified? e.g.
{protocol: 'git', hostname: 'café.com'}? In that case the encoded string should becaf%C3%A9.com, rather thanxn--caf-dma.com? If so, I feel #255 is a reasonable approach.annevk commentedon Mar 24, 2025
#255 still has the problem of dummyURL not being fully defined. It might also have the wrong default (not defaulting to a special URL) when protocol is not given or cannot be given. Overall though that seems like a reasonable direction.
rubycon commentedon Mar 25, 2025
I agree my PR feels a bit hackish, however the canonicalize a domain name method (with
httpsas default scheme) is only called for special URLs. All non special URLs (which including URLs without explicit protocol) are still processed through the canonicalize a hostname method. I don't know if its the right behaviour, but I think its consistent the specs.annevk commentedon Mar 25, 2025
I think what we want is that URLs without explicit protocol also use the special URL code path. At least I think that's what the current tests end up requiring and what is implemented.
Make dummyURL from the basic URL parser with a special scheme
Bug 1953756 [wpt PR 51316] - URLPattern: correct port and hostname-re…
Bug 1953756 [wpt PR 51316] - URLPattern: correct port and hostname-re…