Request/Guidance - HowTo make using pywinauto easy

[jjbright]

This is not an issue - but a request

I'm a novice in python, perhaps I was still able to use pywinauto and do a good set of automation in the Unified Service Desk of Dynamics 365.

Now that I struggle a lot to progress quickly to cover more scenarios, I was looking for the below:

• A good tutorial with examples on how to use pywinauto libraries including how to use the uia_control wrappers
• A utility similar to inpect.exe but generates the pywinauto code to the action that we do on the uia control
• The same utility to give the hierarchical path of the uia control. because using the print_control_identifiers output to navigate the controls is way too difficult in complex application like USD
• How to switch between wpf controls and html components

@vasily-v-ryabov @airelil @enjoysmath

[vasily-v-ryabov]

1. There is no any specific guide for "uia" backend. Do you mean something like this list of methods for every "win32" wrapper? Methods available to each different control type. We have no such guide for UIA yet. It's worth filing new docs issue on that.

2. As you've already seen, there is a prototype of script recorder as a command line utility: WIP: Automatic script generator #701

3. One more tool is py_inspect: https://github.com/pywinauto/py_inspect which is just simple analogue of Inspect.exe without any complicated logic. There was an attempt to implement something more intelligent in issue Announcement: I'm making a PyWinAutoUI (graphical) GUI with advanced features #906 but the discussion seems stuck for now.

4. It's not clear what do you mean under "switching"? Switch from WPF app to browser window? Or do you have WPF app with embedded browser engine as a child control?

[jjbright]

@vasily-v-ryabov Thanks for the response.

1. One simple example will be a great help for me to work on the UIA controls, I understand that we do not have a guide for that.

2. The script recorder command line prototype per WIP: Automatic script generator #701 I wasn't able to use. :(

3. I use Inspect.exe now, but have not tried the py_inspect. Announcement: I'm making a PyWinAutoUI (graphical) GUI with advanced features #906 item was impressive, good if we get.

4. The windows application that I'm automating contains a chromium web browser using cefsharp process. So the outer window of the application is window, but the content rendered in the application is html/div/css etc using chromium web browser.
   After I open and enter the application, I wanted to get the browser control and use selenium code and xpath to identify the elements.

Hope I clarified.

[airelil]

Regarding the chromium pls see : #863 (comment)

[vasily-v-ryabov]

1. There is simple example for explorer.exe in the main Readme: https://github.com/pywinauto/pywinauto/blob/atspi/README.md#ms-ui-automation-example Also there are few examples in ./examples/ folder: see UIA related wireshark.py and list_windows_updates.py.
2. It might be not easy since this is just a prototype. But if you tell us which step is failed with some details, we can advise or take a look.

[jjbright]

The simple example that I was seeking is like this, I have a combo box and I wanted to select an item from that by the item name listed there like Dealer, Business Account, Fleet. So I need to select the 'Fleet' - How to do that. This combo box will list more items...
DELETED THE IMAGE DUE TO SECURITY

I get the combo object like this
aType = mainPanel.child_window(auto_id="acc_type", control_type="ComboBox")

Now how to use the aType object as pywinauto.controls.uia_controls.ComboBoxWrapper

[jjbright]

Regarding the chromium pls see : #863 (comment)

@airelil Hope I mislead you...
DELETED IMAGE DUE TO SECURITY & PRIVACY POLICIES

My application is 'Windows Desktop Application' like above. See the New Session tab, inside that tab is a browser control the chrome...

In the top menu I've few buttons that are of windows controls...

I'm able to click and do actions on windows controls as well as the browser controls.

My question was,
if I can use selenium to do actions on the browser controls, then I can use xpath easily, is that possible?
If possible how do I switch the control between with windows controls and the browser controls?

Note : I had to hide the control texts due to security reasons.

[airelil]

For the combobox example, you might find useful unittests for uia backend. Like this:

pywinauto/pywinauto/unittests/test_uiawrapper.py

Line 717 in 51e6793

def test_combobox_select(self):

[airelil]

But I'm not sure if I can suggest much about the chromium control. And I'm not familiar with selenium too...

[vasily-v-ryabov]

@jjbright no, I think @airelil posted correct link to my comment regarding Chromium control. Accessibility support for this control should be enabled explicitly in the code of application. If you don't have access to the app source code or the developers can't help you, there are no more ideas.

ComboBoxWrapper has method select. It could be used so:

aType.select('Fleet')

[jjbright]

@airelil @vasily-v-ryabov Appreciate all your support & help.

I have accessibility to the code of the application. Today I follow the child_window() method to access the UIA controls, I think if its possible to interact with the chromium using the selenium webdriver and XPATH for element identification, it will be pretty easy.

However, without that, I'm able to do my basic requirements.

Now I'm closing this comment.
My next iteration is to implement additional requirements and I hope I'll use the pywinauto library efficiently to do that.
In need of any help I'll reach you.

Closed #926

[vasily-v-ryabov]

FYI #896 is a feature request for XPath implementation in pywinauto.