HTML parser changes for stylable <select>

[josepharhar]

What is the issue with the HTML Standard?

In order to support the stylable select aka appearance:base-select proposal, we need to make changes to the HTML parser to allow more tags inside <select>, because the current HTML parser basically throws away all tags besides <option>, <optgroup>, and <hr>.

Here are options for how we can extend the HTML parser with varying levels of web compatibility:

1. Allow any tags within <select>
2. Allow <button> and <datalist> tags in <select>, and allow any tags within <button> and <datalist>
3. Allow any tags within a <select>’s <option>, <button>, or <datalist>

1. Allow any tags within <select>

Allowing any tags within <select> would be good because it is more flexible for developers since they won’t necessarily have to add a <datalist>, but it is the most breaking change of the options I listed. I added a use counter for tags which get dropped in select mode, and it is at 0.3%, which is quite high. I also looked at dozens of the websites with an experimental patch to allow any tags, and while most of them seemed unaffected by allowing anything, some of them were broken:

• http://tx.7ma.cn/
  This website has a <select> tag without a closing </select> tag, and it causes a bunch of the HTML to get put inside the <select> instead of being rendered/parsed after the <select>.
• http://top.ge/
  This website has a <select> with additional tags inside the <option>s, but it doesn’t actually use the <select> element to render and instead has some other custom thing which appears to be reading out the DOM contents of the <select>. The dropdown in the website now has a bunch of newlines and odd content in the options.

There’s good reason to believe other sites will be affected in a similar way.

2. Allow <button> and <datalist> tags in <select>

A more web-compatible option would be to make the parser allow <button> and <datalist> in <select>, and then make the parser allow anything inside <button>/<datalist>. These tags correspond to the two visual parts of the <select> as per the explainer. I have use counters for <button> and <datalist> tags inside <select> here, and they have very low usage (0.001% and 0.0001% respectively):
https://chromestatus.com/metrics/feature/timeline/popularity/4771
https://chromestatus.com/metrics/feature/timeline/popularity/4772

These usage numbers are low enough that we would be willing to ship in chrome.

3. Allow any tags within a <select>’s <option>, <button>, or <datalist>

This is like the first option, but doesn’t allow other tags in between <option>s. Based on the sites which broke, I don’t think this would be significantly more compatible than just allowing all tags. I also think this might be confusing to developers when arbitrary content can be added inside options but not in between them unless they are all wrapped by a datalist tag.

I also looked through the commit logs of both the webkit/chromium implementation and spec (and here) in order to find out what the justification was for dropping tags inside <select>, and I didn’t find out anything useful. The implementation had minimal context here. When going through the commit log of the HTML spec, I got back to the initial commit of the git repo, which didn’t explain either.

---

My preference is option 2 because it has the lowest risk of breaking websites, but I have gotten feedback that requiring developers to write <datalist> is not great. Nevertheless, given this compat data I think that’s our best option for moving forward.

[aardrian]

When you say "any tags" do you genuinely believe any element can and should be allowed in an <option> and/or <select>? As in, would this construct be legal?

<select>
 <main>
  <h1>sandwiches</h1>
  <ul>
   <li><option>Meat</option></li>
   <li><option>Not Meat</option></li>
  </ul>
 </main>
</select>

I know that only addresses your option 1, but pretend I am doing similar for options 2 and 3.

And if you do mean that, how do you propose exposing that through accessibility APIs? Using my example, if I want to navigate by landmarks, is the main region hidden until I expand the <select>? What happens when the <select> has focus and I just want to arrow through options using only my keyboard?

The scope of this suggestion feels more like using a howitzer to swat a fly (that fly being devs who fail to close their select menus). Or at least I am not understanding the benefit for users as outlined above.

[josepharhar]

When you say "any tags" do you genuinely believe any element can and should be allowed in an <option> and/or <select>? As in, would this construct be legal?

We discussed this in OpenUI here, and the conclusion as I understand it was to allow anything to be parsed/rendered like normal content, but emit console messages showing why it is not correct for accessibility: openui/open-ui#540

@scottaohara do you have any additional thoughts on this topic?

[aardrian]

That's a lot of reading, but this appears to be the resolution:

RESOLVED: Allow interactive content outside of <options>, but issue strong console errors or warnings in this case.

Which, ok, the console will tell devs to maybe not do that. However, you are making an issue to allow it. And your take on that resolution is:

the conclusion as I understand it was to allow anything to be parsed/rendered like normal content, but emit console messages showing why it is not correct for accessibility

Given that...

• In your words, what is "normal content"?
• Using my example above, how do you envision that working for users?

These aren't questions for Scott as he didn't file this issue. I would like to know your take.

[annevk]

The parser changes themselves don't have a11y impact. Web developers can already construct arbitrary node trees using API calls and browsers already have to handle those. This is purely about what trees can be constructed using syntax instead of API calls.

Now there is a separate question as to what the new content model of the select element is going to be and there we should discuss any new kind of a11y mapping that goes with that and how to determine the fallback for platforms that won't enable the richer rendering, etc.

Let's not conflate them however.

[zcorpan]

cc @whatwg/html-parser

[zcorpan]

I agree that option 2 seems most workable and easier to understand than option 3.

I'm guessing that the content model (i.e. what is allowed for authors to use, not what does the parser do) for button can stay as-is? The content model around datalist, option and optgroup probably needs changing to allow other elements in between.

[aardrian]

The parser changes themselves don't have a11y impact.

I think you are conflating the accessibility tree with tangible accessibility impact on users. For example, a keyboard user is not impacted by how my construct is represented in the accessibility tree -- they still need to open the select menu to get to the content. Users of Windows High Contrast Mode / Contrast Themes could be impacted based on how this content is mapped to native controls when it assigns user-chosen colors.

Authors will absolutely do things like nest interactive controls, conflicting roles, and more. So whether the changes impact the accessibility tree or the content model feels dismissive of my broader question.

So forgetting the accessibility tree, what if my construct is:

<select>
 <main>
  <h1>sandwiches</h1>
  <ul>
   <li><option>Meat</option></li>
   <li><option>Not Meat</option></li>
  </ul>
   <select>
    <button>Do</button>
    <option>This</option>
   </select>
 </main>
</select>

Is the idea that in that scenario a keyboard user has to expand the <select> to get to the other controls? How do arrow keys work in that case? What about when the user needs to scroll to see all the content? Etc.

Yeah, that's a very specific example, but this proposal is making a very broad suggestion without any equally broad statements how things like keyboard navigation would work.

Also:

My preference is option 2 because it has the lowest risk of breaking websites, but I have gotten feedback that requiring developers to write is not great.

Is this feedback a function of lack of existing support for <datalist> (because browser makers can fix that)? Is this feedback anchored on anything specific?

[annevk]

All I'm saying is that this is not the best issue to discuss that broader question. I also wasn't talking about the accessibility tree.

I'm saying that you can create such a construct today. Example: https://software.hixie.ch/utilities/js/live-dom-viewer/saved/12696. The result is a select with no options. This issue doesn't propose changing any of that. That requires bigger changes we'll indeed need to discuss in depth.

[aardrian]

All I'm saying is that this is not the best issue to discuss that broader question.

Fair enough. I don't see an issue for discussing the accessibility impacts I raised, though. @josepharhar have you filed that and I can't find it?

Also, still curious on the <datalist> justification you cited.

[josepharhar]

I don't see an issue for discussing the accessibility impacts I raised, though. @josepharhar have you filed that and I can't find it?

Looks like scott filed one here: #10317

Also, still curious on the datalist justification you cited.

I wrote that "I have gotten feedback that requiring developers to write datalist is not great" because developers said that it would be better if they could use the new parser mode without needing to also write <datalist>. It's not great to require <datalist> to opt in to new parsing behavior, but not a huge deal to have to write it.

[aardrian]

Looks like scott filed one here: #10317

Indeed he did. And I very much appreciate that Scott did so. But I note it was not filed alongside this issue and he was the one who stepped in to file it after I asked.

I wrote that "I have gotten feedback that requiring developers to write datalist is not great" because developers said…

Again, what developers? If it was a couple friends of yours, that's fine because it's still feedback but it also doesn't carry a lot of weight. But if it was hundreds of devs as a function of a survey as part of discovery for filing this issue, that feedback carries a lot more weight IMO.

[annevk]

It seems somewhat self-evident that requiring <datalist> is not great? If you could leave it out and get the same results how would that not be better?

[zcorpan]

Existing sites might have other tags in option and expect the current behavior. The parser currently drops most tags but has special handling for <select>, <input>, <keygen>, <textarea>, and for "in select in table" also special handling of table-related tags.

Maybe we could keep those special-cases and accept other tags in option? Maybe even in select? Keeping the "break out of select" tags as-is seems especially relevant for web compat.

[zcorpan]

https://ablefast.com/ has <div><select></div><option>. If we go with a variant of option 1, this should continue to work.

<div class="input-field">
<label for>Past Pools Results:</label>
<select class name="date" onChange="if (!window.__cfRLUnblockHandlers) return false; window.location.href=this.value" data-cf-modified-5aeb4253f0e4e65f25b139af->
<option disabled selected>Select Date</option>
</div>
<div class="table-wrapper table-responsive">
<option value="https://ablefast.com/results/2024-05-04">04-May-2024</option>

Also, https://html.spec.whatwg.org/#parsing-main-inselect:has-an-element-in-select-scope would need to be changed to correctly handle <select> and </select>, since it currently assumes there won't be other elements on the stack (except in the fragment case). Probably it should be more like table scope (with with select instead of table).

[josepharhar]

Maybe we could keep those special-cases and accept other tags in option? Maybe even in select? Keeping the "break out of select" tags as-is seems especially relevant for web compat.

Keeping some special cases sound reasonable, but allowing other tags within option without the datalist opt-in would break top.ge (mentioned in the issue description) because it has <a> and <b> tags inside <option>s, then pulls the DOM out and renders it outside of the <select>.

[zcorpan]

top.ge is affected by both option 1 and option 3. But it would still be usable, right? I tested inserting the a and b elements in devtools and the widget still worked.

[josepharhar]

top.ge is affected by both option 1 and option 3. But it would still be usable, right? I tested inserting the a and b elements in devtools and the widget still worked.

Thanks for taking a look, I really hope I'm wrong about this!

I'm really not sure exactly what the site is doing, but with my experimental patch to make the parser allow anything, this is what the site looks like:

The HTML of each item in that dropdown list before my patch looks like this:

<li data-original-index="0" class="selected">
  <a tabindex="0" class="" data-tokens="null" role="option" aria-disabled="false" aria-selected="true">
    <span class="text">  ყველა  (19576) </span>
    <span class="glyphicon glyphicon-ok check-mark"></span>
  </a>
</li>

And with the patch, it looks like this (I manually edited the whitespace/newlines to make it readable):

<li data-original-index="0" class="selected">
  <a tabindex="0" class="" data-tokens="null" role="option" aria-disabled="false" aria-selected="true">
    <span class="text"> </span>
  </a>
  <a tabindex="0" aria-disabled="false" aria-selected="true"> ყველა </a>
  (<b>19576</b>)
  <span class="glyphicon glyphicon-ok check-mark"></span>
</li>

[zcorpan]

OK, yeah that looks pretty broken, though still functional.

Regarding "break out of select" tags, is it a requirement to be able to use input in select? Is it a requirement to be able to use table in select in table (maybe a flight seat picker or so)? I don't see such examples in https://open-ui.org/components/selectlist/#use-cases or https://microsoftedge.github.io/Demos/selectlist/index.html

[LeaVerou]

Writing this with my TAG hat on, and is the same feedback I would give during a design review.

Requiring an opt-in so that HTML content can work in the predictable, obvious way is author-hostile. Elements being dropped within <select> and <option> has historically been the source of a lot of author pain, even outside styling use cases (e.g. in scripting). Fixing it by introducing more syntax means the default behavior continues to be unintuitive, while authors need to learn yet another trick so their HTML works in the expected way. Opting to hold up such a confusing behavior to avoid changing how unclosed <select>s render in a tiny fraction of obscure websites is …the wrong weighing of priorities IMO.

Not to mention it makes it impossible to use this feature via progressive enhancement, since it completely breaks <select> in browsers that don’t support this. This is not necessarily a showstopper by itself, but it will slow down adoption considerably which also means that by the time we have enough author feedback about using it on real-world scenarios, it will be too late to change it in any way.

It might have been a reasonable weighing of tradeoffs if the use counter showed a much higher percentage, but 0.3% is actually encouraging as an upper bound for breakage. I suspect for many, if not most, cases within that 0.3%, the change will actually fix content that is currently broken. E.g. in the example given of an unclosed <select> that is eating up the rest of the page, I’d argue the change actually makes the content more accessible. And as it was pointed out, in many other cases, the content is still functional, even if it looks broken.

---

Looking at the compat analysis document (thanks @josepharhar for such a detailed report!), there are 19 websites in there of which 13 are actually unaffected by the change (green). Of the 6 that are marked yellow or red:

• 3 are hidden with display: none, so not sure why they are marked yellow. If they are hidden, then the change does not negatively affect them and thus they should be green. What am I missing?
• And in the remaining 3, it appears the select is still functional, just would not look pretty, which is probably appropriate since it actually does have this HTML content inside it.

Looking at chromestatus it appears there are 117 websites total that do this. How were the 19 in the doc selected? Is it the 19 most popular ones? The 19 first ones?

[josepharhar]

Is it a requirement to be able to use table in select in table (maybe a flight seat picker or so)?

The only reason I can think of to use a table in a select would be for laying out the options in a table.

Requiring an opt-in so that HTML content can work in the predictable, obvious way is author-hostile

I agree. I am becoming increasingly convinced that we should do option 1 and deal with the compat fallout.

3 are hidden with display: none, so not sure why they are marked yellow. If they are hidden, then the change does not negatively affect them and thus they should be green. What am I missing?

I agree, you're not missing anything. I was just taking some notes without thinking that I'd publish them and decided to use yellow for these websites since I couldn't figure out if my changes would make a difference or not.

And in the remaining 3, it appears the select is still functional, just would not look pretty, which is probably appropriate since it actually does have this HTML content inside it.

Yeah the only one which seemed more impacted was http://tx.7ma.cn/ since there is a checkbox and submit button which no longer get rendered.

Looking at chromestatus it appears there are 117 websites total that do this. How were the 19 in the doc selected? Is it the 19 most popular ones? The 19 first ones?

If I remember correctly it was just the first 19 ones. I also skimmed the list for websites I could recognize, which there were not many of.

[lukewarlow]

is it a requirement to be able to use input in select?

I can see this being useful for building searchable selects. Like a bounded version of input + datalist?

[LeaVerou]

is it a requirement to be able to use input in select?

I can see this being useful for building searchable selects. Like a bounded version of input + datalist?

That would require far more extensive changes than the parser changes described here to become possible. Also, it's such a common need that it should be possible without having to roll your own UX (in fact, I'd argue browsers should implement it by default for selects with more than say 20 options).

[lukewarlow]

That would require far more extensive changes than the parser changes described here to become possible.

Option 1 would allow it afaik? (Depending on the specifics of what special cases we keep). It would obviously require JavaScript to implement but it's an example of where that parser change could be useful.

[LeaVerou]

That would require far more extensive changes than the parser changes described here to become possible.

Option 1 would allow it afaik? (Depending on the specifics of what special cases we keep). It would obviously require JavaScript to implement but it's an example of where that parser change could be useful.

I could be wrong, but I believe the way the accessibility tree works rn, it doesn't work well with labels and controls nested within other interactive controls.

[lukewarlow]

So I think it would be okay because the input would actually be inside the popover rather than the select button trigger. Having said that the changes to the parser would need changes to the content model and accessibility mappings to make full use of, regardless of the input.

Just wanted to say if we could allow that to work at the parser level it would be good.

[mfreed7]

Looking at the compat analysis document (thanks @josepharhar for such a detailed report!), there are 19 websites in there of which 13 are actually unaffected by the change (green). Of the 6 that are marked yellow or red:

So we're supportive of trying to ship option #1, assuming it's actually web compatible. (For reference, Chromium's "safe" threshold for deprecations and changes is typically 0.001%, so the current 0.3% use counter is considered extremely high.) I spoke to @josepharhar about this, and we'll do an experiment to see what breakage we might see if we go the "option 1" route. If that experiment produces good results, we'll be supportive of trying to spec option #1.

That would require far more extensive changes than the parser changes described here to become possible.

Option 1 would allow it afaik? (Depending on the specifics of what special cases we keep). It would obviously require JavaScript to implement but it's an example of where that parser change could be useful.

I could be wrong, but I believe the way the accessibility tree works rn, it doesn't work well with labels and controls nested within other interactive controls.

I think it's really important to emphasize that this issue ("HTML Parser changes for...") is about just the parser changes. Allowing all content (a.k.a. "option 1") does not necessarily mean all content is fully supported. It means all content is parsed. We still need to work through what content is "allowed", and exactly what it means to be "not allowed". That's #10317.

An important point to make, though, is that similar to the web platform, accessibility is not static. There are great folks working on making AT able to understand more types of things, and represent more complex content in a way that works for AT users. Right now, my understanding is that having something like a button inside an <option> is an a11y no-no. However, there are proposals to expand what ARIA/etc can represent. For example, see this proposal: w3c/aria#1440. If adopted, that would make it possible to have accessible buttons within options. Given that we're potentially stuck on web compat changing the parser now, let's not immediately lock ourselves into another such situation later by trying to proactively restrict content types within the parser. The right course of action here would seem to be allowing as much content as possible in the parser, and then figuring out how to make it accessible now, or keeping folks from using it until it can be made accessible later.

[aardrian]

There are great folks working on making AT able to understand more types of things, and represent more complex content in a way that works for AT users.

FWIW, this goes beyond AT (though keyboards are AT, most non-practitioners think "AT" means screen readers). But what I mean is that this comes down to platform support. As, in the browser and the operating system need to support these things and expose them properly for screen readers (and sure, other AT) to be able to use them. For example, Windows UIA does not support heading levels past 9, so Chrome caps it there despite ARIA imposing no such limit.

If you get parsers to allow literally anything in selects, you can't expect the Orca (random example) developers to just make it work for users (at least not with platform support from all the relevant *nix flavors). Nor the ARIA authors to make it just work (the linked ARIA example has been open for 3 years). While this issue is expressly not about accessibility, if you aren't accounting for it in this discussion then please don't suggest can make sure it "works for AT users".

Probably contribute to #10317.

And this response is not a shot at Mason. His was just the easiest to quote from the speculative discussion of accessibility implications.

[mfreed7]

FWIW, this goes beyond AT (though keyboards are AT, most non-practitioners think "AT" means screen readers).

Fair. I'm not a practitioner, and I likely am using "AT" incorrectly. Is there a better term for all of the affected tech here?

If you get parsers to allow literally anything in selects,

My only point here is that the parser is not a gatekeeper. Any site written in React roughly-speaking doesn't use the HTML parser at all. You can very happily do this today:

render() {
  return (
    <button>
      <button>Nested button!</button>
    </button>
  );
}

So I'd like to take care of accessibility at a different layer than the parser, so React sites and others get the benefit also.

Probably contribute to #10317.

Yes, please. Let's have this conversation there.

And this response is not a shot at Mason. His was just the easiest to quote from the speculative discussion of accessibility implications.

No problem!

[aardrian]

Fair. I'm not a practitioner, and I likely am using "AT" incorrectly. Is there a better term for all of the affected tech here?

No, that is the correct term. My point is that "AT" encompasses more than screen readers. The vast majority of AT does not use the accessibility tree.

[zcorpan]

Looking at chromestatus it appears there are 117 websites total that do this.

No, I believe it just shows a sample of roughly 100 sites. The data is from httparchive, which contains 12,900,240 sites for desktop and 16,193,055 sites for mobile. 0.17% of the mobile dataset is ~27,500 sites.

If the "3 out of 19 sites are broken" is representative, it would mean breaking ~4,300 sites in the httparchive dataset.

Again, I think the "break out of select" tags are most likely to break sites if they were to no longer break out of select.

[mfreed7]

Looking at chromestatus it appears there are 117 websites total that do this.

No, I believe it just shows a sample of roughly 100 sites.

That's correct - that list is just a sample set of the "top" URLs that hit the use counter. Where "top" is defined by this methodology. There are many more than 117 sites hitting this use counter.

The data is from httparchive, which contains 12,900,240 sites for desktop and 16,193,055 sites for mobile. 0.17% of the mobile dataset is ~27,500 sites.

I'm not sure where the "0.17%" in this sentence comes from, so I'm not sure about its correctness.

Again, I think the "break out of select" tags are most likely to break sites if they were to no longer break out of select.

I think I agree with this, with the caveat that the location matters. I.e. I believe most of the broken sites were <select><option>something<input> where the <input> terminated the <select>. But I think our numbers show that the use counters would be much much lower for <select><datalist><option>something<input>. I.e. we might be able to make an <input> that isn't nested within another element continue to terminate the <select> and have that be web compatible.

[LeaVerou]

I think I agree with this, with the caveat that the location matters. I.e. I believe most of the broken sites were <select><option>something<input> where the <input> terminated the <select>. But I think our numbers show that the use counters would be much much lower for <select><datalist><option>something<input>. I.e. we might be able to make an <input> that isn't nested within another element continue to terminate the <select> and have that be web compatible.

Do you mean that <datalist> would only be required if form elements are included inside <option> and not otherwise?

[mfreed7]

I think I agree with this, with the caveat that the location matters. I.e. I believe most of the broken sites were <select><option>something<input> where the <input> terminated the <select>. But I think our numbers show that the use counters would be much much lower for <select><datalist><option>something<input>. I.e. we might be able to make an <input> that isn't nested within another element continue to terminate the <select> and have that be web compatible.

Do you mean that <datalist> would only be required if form elements are included inside <option> and not otherwise?

Right. That would be one way to get around specifically the compat issue, if we wanted to permit form elements to be parsed within <select>. Again, we're just talking about parser rules here, not whether that should be "allowed". That's #10317.

[zcorpan]

The data is from httparchive, which contains 12,900,240 sites for desktop and 16,193,055 sites for mobile. 0.17% of the mobile dataset is ~27,500 sites.

I'm not sure where the "0.17%" in this sentence comes from, so I'm not sure about its correctness.

The graph under "Adoption of the feature on top sites" says 0.17% for Apr 1, 2024, both mobile and desktop.

Again, I think the "break out of select" tags are most likely to break sites if they were to no longer break out of select.

I think I agree with this, with the caveat that the location matters. I.e. I believe most of the broken sites were <select><option>something<input> where the <input> terminated the <select>. But I think our numbers show that the use counters would be much much lower for <select><datalist><option>something<input>. I.e. we might be able to make an <input> that isn't nested within another element continue to terminate the <select> and have that be web compatible.

OK, yes.

[mfreed7]

The graph under "Adoption of the feature on top sites" says 0.17% for Apr 1, 2024, both mobile and desktop.

Ahh! Ok, thanks. I actually never look at that particular plot, since the definition of "top sites" is a bit rough. But anyway, thanks, makes sense.

[hsivonen]

Considering how long the trickle of bugs from SVG integration and template integration (and their combination!) has been, I feel uneasy about poking select parsing at all. However, I realize that there's enough demand for better styling of select that not changing the parsing at all is probably not a practical approach.

Therefore, my preference is explicit opt-in with button and datalist and particular spec writing care taken to consider how svg, math, and template as descendants of select affect coming back through the "in select" and the "in select in table" insertion modes when coming out of SVG and template for any reason (including breakout from SVG on particular special names).

[josepharhar]

Therefore, my preference is explicit opt-in with button and datalist and particular spec writing care taken to consider how svg, math, and template as descendants of select affect coming back through the "in select" and the "in select in table" insertion modes when coming out of SVG and template for any reason (including breakout from SVG on particular special names).

Thanks for the feedback! This was my preference as well, but most of the feedback I've gotten so far is that we shouldn't require datalist to opt into parser changes, so I'm going to try to see how far I can get doing that in chromium.

What if we got rid of the "in select" and "in select in table" modes entirely? I think that would make things a lot simpler to spec and implement. We could also make sure that <select><select> turns into <select></select><select></select> with whatever the same mechanism for <button> is, which apparently doesn't require a parser mode.

Maybe we could keep those special-cases and accept other tags in option? Maybe even in select? Keeping the "break out of select" tags as-is seems especially relevant for web compat.

Of the two websites we've seen that are affected so far, one of them wouldn't be helped by keeping the "break out of select" tags, and hopefully the other can add its missing </select> tag perhaps with some outreach.

I also wonder if we still need to have the "in select" mode in order to break out of select since nested <button>s do this somehow without the need for a special mode?