Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Application.GetAllTopLevelWindows() is slow #616

Open
SunnyDesignor opened this issue Mar 3, 2024 · 9 comments
Open

Application.GetAllTopLevelWindows() is slow #616

SunnyDesignor opened this issue Mar 3, 2024 · 9 comments

Comments

@SunnyDesignor
Copy link

Application.GetAllTopLevelWindows is slow, it takes 5 to 10 seconds, but GetMainWindow is fast, <0.1sec. I roughly checked the source code and found that GetAllTopLevelWindows searched the entire desktop. I just want to get a window with a matching name within a specific process. I can't accept the long wait. Is this a flaw in the project, or should I use other methods?

@louislefevre
Copy link

Could you provide some code with what you've done? Would be easier to get a better idea about what you're trying to do

@SunnyDesignor
Copy link
Author

SunnyDesignor commented Mar 8, 2024

var app = FlaUI.Core.Application.Attach("xxx.exe");
using (var automation = new UIA3Automation())
{
  var window = app.GetAllTopLevelWindows(automation).FirstOrDefault(t => t.Name == "Client.Main.Views.AppendViewModel");
  window.FindFirstDescendant(t => t.ByName("Accept")).AsButton()?.Invoke();
}

@Roemer
Copy link
Member

Roemer commented Mar 8, 2024

Can you please measure each call individually (app.GetAllTopLevelWindows, window.FindFirstDescendant) to see which of those 2 is the slower one?
If it indeed is the first one, you can always directly search the window directly:

var window  = automation.GetDesktop().FindFirstChild(cf =>
    cf.ByControlType(ControlType.Window)
    .And(cf.ByProcessId(app.ProcessId))
    .And(cf.ByName("Client.Main.Views.AppendViewModel"))
);

@SunnyDesignor
Copy link
Author

GetAllTopLevelWindows is slower, taking 3 seconds, but the new method you provided is effective, reducing the time to 0.2 seconds. Additionally, GetAllTopLevelWindows is not as slow as before, perhaps because my system has been restarted.

@Roemer
Copy link
Member

Roemer commented Mar 8, 2024

When it gets all windows from the top level (desktop), UIA queries the Windows-Objects underneath and it might encounter a window (application) that is hanging for some time until it responds, making the whole request slow. If you want, you could try to get each window from the GetDesktop individually and see, if there is one or a few that takes longer to get a response.

@SunnyDesignor
Copy link
Author

How do I enumerate it? The code in the loop passes quickly, and the time is concentrated on FindAllChildren.
foreach (var element in automation.GetDesktop().FindAllChildren(a => a.ByControlType(FlaUI.Core.Definitions.ControlType.Window)))
{
element.Name.Dump();
element.FindAllChildren();
}

@maxinfet
Copy link
Contributor

maxinfet commented Apr 14, 2024

How do I enumerate it? The code in the loop passes quickly, and the time is concentrated on FindAllChildren. foreach (var element in automation.GetDesktop().FindAllChildren(a => a.ByControlType(FlaUI.Core.Definitions.ControlType.Window))) { element.Name.Dump(); element.FindAllChildren(); }

In raw UIA you would var firstWindow = AutomationElement.RootElement.GetFirstChild(...); with the condition that its control type is a window. Then you would get the next sibling of the first window by using the treewalker like we do here for getting the parent var nextSibling = Automation.TreeWalkerFactory.GetRawViewWalker().GetNextSibling(firstWindow);. Eventually, this will return null when there are no more siblings. I believe the example in FlaUI would look something like this.

var element = automation.GetDesktop().FindFirstChild(a => a.ByControlType(FlaUI.Core.Definitions.ControlType.Window);

while (element != null) 
{
   //element.FrameworkAutomationElement  gives access to the automation element that FlaUI wraps so that methods that are not implemented can be used  
   //once there are no more siblings this will be set to null and we will exit the loop
   element = Automation.TreeWalkerFactory.GetRawViewWalker().GetNextSibling(element.FrameworkAutomationElement);

   //dump information about the elements
}

You can find more examples of usage of the TreeWalker here. This unfortunately will be slower than just asking for FindAll since this will result in multiple cross-process calls as opposed to a single batched call but if there is one window slowing things down it should still be proportionally slow compared to the other windows just with the added delay of making a call per window.

@SunnyDesignor
Copy link
Author

How do I enumerate it? The code in the loop passes quickly, and the time is concentrated on FindAllChildren. foreach (var element in automation.GetDesktop().FindAllChildren(a => a.ByControlType(FlaUI.Core.Definitions.ControlType.Window))) { element.Name.Dump(); element.FindAllChildren(); }

In raw UIA you would var firstWindow = AutomationElement.RootElement.GetFirstChild(...); with the condition that its control type is a window. Then you would get the next sibling of the first window by using the treewalker like we do here for getting the parent var nextSibling = Automation.TreeWalkerFactory.GetRawViewWalker().GetNextSibling(firstWindow);. Eventually, this will return null when there are no more siblings. I believe the example in FlaUI would look something like this.在原始的UIA中,你需要满足它的控件类型是一个窗口的条件.然后,你可以像我们在这里获取父窗口一样,使用树遍历器来获取第一个窗口的下一个兄弟窗口。最终,当没有兄弟节点时,这将返回null。我相信FlaUI中的例子看起来应该是这样的。

var element = automation.GetDesktop().FindFirstChild(a => a.ByControlType(FlaUI.Core.Definitions.ControlType.Window);

while (element != null) 
{
   //element.FrameworkAutomationElement  gives access to the automation element that FlaUI wraps so that methods that are not implemented can be used  
   //once there are no more siblings this will be set to null and we will exit the loop
   element = Automation.TreeWalkerFactory.GetRawViewWalker().GetNextSibling(element.FrameworkAutomationElement);

   //dump information about the elements
}

You can find more examples of usage of the TreeWalker here. This unfortunately will be slower than just asking for FindAll since this will result in multiple cross-process calls as opposed to a single batched call but if there is one window slowing things down it should still be proportionally slow compared to the other windows just with the added delay of making a call per window.

I tested the following code based on your suggestion, and it speed is basically the same as FindAllChildren().

	FlaUI.UIA3.UIA3Automation automation = new FlaUI.UIA3.UIA3Automation();
	var element = automation.GetDesktop().FindFirstChild(a => a.ByControlType(FlaUI.Core.Definitions.ControlType.Window));
	while (element != null)
	{
		var start = DateTime.Now;
		element = automation.TreeWalkerFactory.GetRawViewWalker().GetNextSibling(element);
		var end = DateTime.Now;
		(end - start).Dump();
		if (element == null) break;
		element.ToString().Dump();
	}

According to the test, I found that the time is concentrated in
"AutomationId:, Name:Program Manager, ControlType:窗格, FrameworkId:Win32",
which consumes more than 2500ms, while other windows usually only take a few tens of milliseconds.

@maxinfet
Copy link
Contributor

maxinfet commented Apr 27, 2024

@SunnyDesignor I have only ever experienced a slow down in getting a window when that window is not responding or the OnCreateAutomationPeer method is taking a long time to finish producing the automation peers for the controls. If it's not a .NET application it could be taking a longer time to respond to the WM_GETOBJECT message but I am not very familiar with how to dig into that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants