Add pywinauto script generator

[cetygamer]

The idea is to run an application, do some stuff with it, so generator could detect it and print out python code that would do exactly the same operations.

Currently we are able to catch automation events thrown by automation elements.

https://github.com/cetygamer/pywinauto/tree/dev-script-generator/pywinauto/script_generator

The problem is to handle events which were thrown by closing or already closed automation elements. Even with caching enabled I can't get their properties (to be able to generate pywinauto code). See:
https://github.com/cetygamer/pywinauto/blob/dev-script-generator/pywinauto/script_generator/recorder.py#L130

Example usage:

import time

from pywinauto import Application
from pywinauto.script_generator import Recorder

app_path = 'notepad.exe'
app = Application(backend="uia").start(app_path)
time.sleep(0.3)

rec = Recorder(app=app, record_props=True, record_focus=False, record_struct=False)
rec.start()

Example output:

Event: MenuModeStart - Notepad, Untitled - Notepad
Event: MenuOpened - #32768, Help
Event: MenuClosed - , 
Event: Window_WindowClosed - , 
Event: MenuModeEnd - , 
Property Changed: WindowWindowVisualState 3740
Event: Window_WindowOpened - #32770, About Notepad
Event: Window_WindowClosed - , 
Event: Window_WindowClosed - , 

Those empty strings separated by comas are properties of already non-existent automation elements (Help menu, About notepad menu item, About notepad window, Untitled - Notepad window).

@vasily-v-ryabov, @airelil, I could really use some help here.
I could just cache all children of new menus or windows openning, but that wouldn't give me exact information about sender of an event since after window closes all its children get deleted.

[airelil]

It's a quite complicated task that requires a good discussion at least. The design choice to relay on UIA events is arguable as well. But let's say you started it as a prototype to get deeper insights in the problem )).

Regarding the question. I'd expect that StructureChangedEvent will pop-up on its element add/delete but in your example it seems to be disabled.

[vasily-v-ryabov]

On handling WindowClosed events I have only one idea for now. We can remember all the controls properties and especially rectangles (it might be slow though) and then use low level mouse or keyboard events (by win32_hooks) to detect coordinates of the click or keyboard-focused control which caused the window to close.

This is the main problem in the whole task I think.

[hvbtup]

I think it may be necessary to collect all windows and their controls whenever anything changes (probably too complicated) or at least whenever there is no user input for a second (blocking user input while collecting the data) or whenever the user presses a hot-key.
Using a hot-key is probably the simplest solution, but it requires the user to assist the recording.

[cetygamer]

Hi, @hvbtup most of the core functionality has been already implemented in https://github.com/cetygamer/pywinauto/tree/dev-script-generator/pywinauto/script_generator

Main idea: cache tree of application elements with their main properties (position, name, type). Use that tree to determine which element user interacted with and rebuilt the tree whenever we detect 'chage structure' event.

[hvbtup]

AFAIK the current implementation has problems with closing elements. I wonder if it could help to detect any screen changes while recording using the "Windows DXGI Desktop Duplication API". BTW this would allow to record the visual appearance as well (capturing images of the screen or of changed parts). When the DXGI API informs us that something changed on the screen, we could inform the recording thread about that fact by setting a "screen_is_dirty" event. The recording thread could wait for that event, and whenever it is set, clear it and take a new snapshot of the structure of the Windows and their controls.

[Hemant42435678]

Can I get this actual code
https://github.com/cetygamer/pywinauto/tree/dev-script-generator/pywinauto/script_generator

The below link has no code
Please send the link to killme.hemant@gmail.com