Return true if button was clicked

[johnyCZS]

Hello, Iam trying to automate a application with pywinauto, it works perfectly when ran on my computer but. we have some issues making it work on PC of our employes. The main issues Iam dealing with right now is clicking an item in popmenu. For the purpose of logging I would like to know if the item in popmenu was clicked. Iam only able to provide a pseudo code but it works like this:

Right click to open popmenu
Select menu item with name
Check if the menu item is enabled
if enabled click_input()

Iam looking for a way to get confirmation that the button was actualy clicked. Something like "if button was clicked return true" because the only realiable feedback I can get is a log I get send after the application was ran and I need to know if the button was actually clicked. The main issue is that I cant get visual confirmation that the button was clicked since if its clicked the pop menu goes aways but the same happens if it clicks somewhere else.

It's a pretty complicated issue so I apologize it this doesnt make sense.

[junkmd]

Hi.

I can only guess about your pseudo code.

I would like to clarify your intention.

Perhaps what you actually want to confirm is not "whether an element was clicked" but rather "whether a screen transition occurred" or "whether a dialog was displayed"?

[johnyCZS]

Hello,

I would like to confirm if a specified element was clicked. I know the popmenu appears and I know the button is "clickable", I need a confirmation that the one specified item in the menu was clicked. So I can confirm that the click did not happen for example next to the menu since I use click_input(). I had this problem in the past but I could visualy confirm that the script didnt perform "click" where I wanted.

In this case I can't do that since when the menu appears if you click somewhere else then on the specified element the menu disapears the same happens when you click on the specified element so there is no visual diference. The program doesnt continue after this step and I need to confirm whether that is because there is a problem with the automated program or with my script not "clicking" properly.

Hope this makes more sense.

[junkmd]

Hello.

The clicked element will usually get focused.
How about using the is_focused or get_focused_element methods to check if the element gets focused?

pywinauto/pywinauto/windows/uia_defines.py

Line 67 in bf7f789

self.get_focused_element = self.iuia.GetFocusedElement

pywinauto/pywinauto/controls/common_controls.py

Lines 568 to 571 in bf7f789

	def is_focused(self):
	"""Return True if the item has the focus"""
	return win32defines.LVIS_FOCUSED == self.listview_ctrl.send_message(
	win32defines.LVM_GETITEMSTATE, self.item_index, win32defines.LVIS_FOCUSED)

After reading your report, I thought that the program terminating abnormally might be because the GUI is frozen for a moment and cannot react.
Have you considered using time.sleep in this case? It is a common practice, although some uncertainty remains.

[johnyCZS]

Yes that could help. Maybe I could use draw_outline too? The bigets issues here is that its a popmenu that appears after a right click and in my experience these behave weird. I have made a function for waiting, I find it quite usefull. Its a while cycle that takes a function as a argument and checks every cycle if the function return is != None and if so the whole function returns True.

[junkmd]

While draw_outline could be helpful for debugging purposes, I'm not certain if it provides a fundamental solution to the problem at hand.

As you mentioned, pop-up menus can be troublesome for GUI Automation, as they may appear and disappear unpredictably when the mouse pointer moves. Does the pop-up menu you are trying to handle have identifiable name?

I am also interested in your workaround. Is it more useful than methods such as wait_enabled?

[johnyCZS]

The draw_outline method is a quick-fix solution for me... which I need right now, but I have to test it first and then I can indetify the exact problem and exact solution.

The pop-menu unfortunately doesnt have a name but the individual items inside the menu do. Thats how I specify the item I want to be clicked. In production I first printed all the menu items and then by the item´s name I specify which item to click.

I´am not sure as I dont have that many experiences using the wait_enabled method. I will try to explain better how I wait for windows and other elements.

`def functionOne()
#some logic to find the element I need

elementNeeded = #specific window or other element specified by logic above

return elementNeeded`

`def wait_for_window(timer, function)
counter  = 0

     while counter < timer:
     if function() != None:
         return True
     timer =+ 1
     time.sleep(1)
  
return False
`

I would then use the wait_for_window in an if statement such as
#some logic
if wait_for_window(timer = 100, function = functionOne):
    keep going
else:
   show error message

Its probably worth noting that we do not use pywinauto like this

app.SaveAs.Save.click()

as we found for our purpose its slower.

But we use the children() method like this:

dsk = desktop("title")
win = dsk.children(title of the window)
win1 = win[0].children().....

Hope this makes some sense. Let me know what you think.

[johnyCZS]

Could you provide your idea of workaround?

It does realy seems that we could exchange useful ideas for both sides

[junkmd]

I use the following function for workarounds. I actually do type checking on the return value and also limit the error type.

def retry_loop(timeout, interval, /, func, *args, **kwargs):
    start = time.time()
    while time.time() - start > timeout:
        try:
            return func(*args, **kwargs)
        except:
            logger.info("...", exc_info=True)
        time.sleep(interval)
    raise TimeoutError

[johnyCZS]

After some testing I wasn't able to make methods you suggested work, mainly due to error: "menuItem doesn't have method".

Then I tried different approach with pywinauto actionlogger with no luck either, it logs all actions and clicks but not the clicks happening inside popmenu, I'm not sure if there are other options to actionlogger for example specify which action to log, but I wasn't able to find any information about this.

The final solution which I implemented and it seems to work is printing .rectangle() of the menu item, this works. Then before and right after .click_input() I print coords of the mousecursor with win32api.GetCursorPos() and I compare them with coords provided by .rectangle().

With this method I can tell that the mouse was inside the rectangle of the menuItem and the click happened there. Its not 100% optimal but its better then nothing and it works for debuging.

[junkmd]

After some testing I wasn't able to make methods you suggested work, mainly due to error: "menuItem doesn't have method".

Indeed, since MenuWrapper is not a subclass of UIAWrapper, I couldn't use methods like invoke or wait_enabled. I'm sorry.

The final solution which I implemented and it seems to work is printing .rectangle() of the menu item, this works. Then before and right after .click_input() I print coords of the mousecursor with win32api.GetCursorPos() and I compare them with coords provided by .rectangle().

If MenuItem can be retrieved, I think it indicates that the GUI is not frozen.
In that situation, you're right that click_input() moves the mouse pointer, so by capturing the coordinates, you can determine if the mouse pointer is within the element's rectangle().

I think that in order to determine whether "the click was truly executed" or not, it is necessary to check the results caused by the click.

[johnyCZS]

Yes I think so too, right now it seems like there isn't a way to say if "the click" was truly executed by return values or other confirmation by log output. It seems that the only way is as you say check the results that is caused by the click.

Thank you for your help and time.

[junkmd]

You're welcome!

It's been a valuable experience to discover that there are others meeting similar problems and my GUI Automation project is not a solitary battle.

[johnyCZS]

I think we could further benefit by exchanging ideas and solutions to problems we encountered in development.