Relative Locators, iFrames, ShadowDom, RTL .. oh my!

[emanlove]

I am opening up here a discussion on some fundamental design ideas around a set of locater related topics. There is a fair amount here so please read through the entire discussion. Also I understand this has both practical consequences and some deep technical discussions. I strongly feel it is important to take both into consideration and to have input from those approaching at both levels. If you come from the technical aspect please consider how the user will see and think about the solution. And if you come from the usage side don't be overwhelmed by the complexity presented here. Hopefully our solution will be clear and usable by all. Let me start with relative locators ..

Relative Locators

Selenium 4 introduced relative locators. That is locators which you can say something like "find the element to the left of the image". In thinking about an implementation for SeleniumLibrary I thought about taking advantage of the existing cascading or chained locators. One design would be to put the relative locator direction within the chained strategy syntax of >>. For example

Click Element	css:.bar >leftof> xpath://a	# To find a link which is to the left of an element with class "bar"

I have made some really initial test code to try out the parsing of such locators. There is a usability/readability issue with the implementation shown in the example but I want to hold that discussion till I talk about RTL or Right-to-Left languages. Now I want to talk about frames ..

Someone recently shared an issue not solvable with the current library implementation. They wanted to drag and drop an element into an element within an iFrame. Because one needs to select a frame before interacting with an element within a frame and because the drag and drop keyword takes a "drag from" locator and then a target there isn't a way to do both. (Hopefully you are seeing the solution I am going to propose..) But if we could select a frame within a chained locator then we could overcome this short coming. My idea would be to introduce a new locator strategy frame which would be a "shortcut" for select frame. So in the problem outlined above one could do something like

Drag And Drop  css:#moveme  frame:frameOnPage >> css:#dragToTarget    # Drag the element with id moveme to the element dragToTarget within the frame named "frameOnPage"

One extra design consideration with frames is if one has a frame within another frame should we require them to step through the frame as in

Click Element  frame:frameOnPage >> frame:subFrame >> css:#elemWithinSubFrame

or would it be possible to just give the subframe? Here implementation might take precedence over convenience as I think with selenium frame operations it may be "expensive" to have this convenience. Talking about convenience let's now talk about ShadowDom.

ShadowDom support

Locating elements within the ShadowDom has been a weak point on Selenium's side that I would like to resolve. I think some of the above designs for the chaining locators or locator strategies could be used. My initial preference would be to have a locator strategy for the root, something like shadowroot:. But I could also see using the chaining syntax; something like css:.someelem >sroot> css:shadowdomelem. I need to read deeper on ShadowDom elements as it is not something I have worked with and don't have a great amount of knowledge on. One thing though I have heard is this ability, similar to frames, to select a ShadowDom root and then some elements and then another ShadowDom root; that is traversing through root elements. My understanding is that playwright does this automatically. As a convenience I am interested in the question of how does one traverse down a ShadowDom tree easily. Here convenience may out way the implementation costs.

Readabilty of Relative Locators and RTL consideration

Back to the usability and readability problem of the following relative locator

Click Element	css:.bar >leftof> xpath://a	# To find a link which is to the left of an element with class "bar"

Note how it says the link to the left of element with class "bar". But as read read that chained locator - css:.bar >leftof> xpath://a - as one does in English, that is left to right, we read it as "click the element with class "bar" that is to the left of the link". This is also how the current code implements finding the elements a locator like xpath://ice >> css:.cream. First it finds the ice element and then finds the element with class "cream" amongst the children/decedents of the ice element. The question is should it read in the order as one reads a sentence? That is css:.bar >leftof> xpath://a will be read as "to find an element with class "bar" which is to the left of a link"? Should we instead allow for changing the direction of the chaining syntax such that it points from the element you wish to find and towards the one it which it is relation too? What happens if there is several chained locators for one side? For example,

Click Element	id:parentElem >> css:.bar >leftof> xpath://a	# To find a link which is to the left of an element with class "bar" that is the descendent of an element with id parentElem 

At this point I expect I have confused you with the question of how should a relative locator be read. But that fact that it is confusing means to me that it is important to get the design and the explanation of that design right so that we eliminate as best as possible any confusion.

One last consideration to take into account with relative locators is leaving open the possibility that at some point we might have support for true RTL or Right-to-Left support within Robot Framework. RTL means that the line in the script is read starting from the right side and then moving left. Reusing the original LTR example above

Click Element	css:.bar >leftof> xpath://a	# To find a link which is to the left of an element with class "bar"

under a RTL script it would be

xpath://a >leftof> css:.bar    Click Element

with the same meaning. I think this will be fairly easy to implement as it would just be the "reverse" of the LTR syntax. But I want to make sure we include this in our thinking about this design and that I am not missing anything.

Summary

Hopefully this was not TLDR; although I am pretty sure I failed at that. Let me note that implementation will not be all at once. But as I see these designs intertwined I do want to be able to look at it as one whole design. If you have any comments, questions, suggestions, ideas please add them here or over in Slack.