Unversioned python used in Popen() calls leads to failing commands

[sbonds]

I've run across some odd errors such as this one from /var/opt/microsoft/omsconfig/onconfig.log where it looks like python2 code is getting run under python3:

2021/05/17 17:02:07: ERROR: Scripts/nxOMSAutomationWorker.pyc(812):
Set_Marshall returned [-1] with following error: Traceback (most recent call last):
  File "Scripts/nxOMSAutomationWorker.py", line 94, in Set_Marshall
    raise Exception("call to omsutil.py --initialize failed. %s, %s" % (stdout, stderr))
Exception: call to omsutil.py --initialize failed. ,   File "/opt/microsoft/omsconfig/modules/nxOMSAutomationWorker/DSCResources/MSFT_nxOMSAutomationWorkerResource/automationworker/worker/omsutil.py", line 60
    print "Successfully added omsagent secondary group to nxautomation user."
                                                                            ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Successfully added omsagent secondary group to nxautomation user.")?

2021/05/17 17:02:07: ERROR: null(0): EventId=1 Priority=ERROR Job OMITTED :
DSC Engine Error :
         Error Message Native provider failed to run SET operation.
        Error Code : 1
2021/05/17 17:02:07: ERROR: CAEngine.c(1744): EventId=1 Priority=ERROR NativeResourceProvider_SetTargetResource failed.

2021/05/17 17:02:07: ERROR: null(0): EventId=1 Priority=ERROR Job OMITTED :

It looks like the affected script (omsutil.py) is correctly using the right python via the #!/usr/bin/env python2 line at the start.

A bit more digging led me here:

PowerShell-DSC-for-Linux/Providers/Scripts/2.4x-2.5x/Scripts/nxOMSAutomationWorker.py

Line 91 in f0e0a53

proc = subprocess.Popen(["sudo", "-u", AUTOMATION_USER, "python", OMS_UTIL_FILE_PATH, "--initialize"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)

And this additional case:

PowerShell-DSC-for-Linux/Providers/Scripts/2.4x-2.5x/Scripts/nxOMSAutomationWorker.py

Line 121 in f0e0a53

args = ["python", REGISTRATION_FILE_PATH, "--register", "-w", settings.workspace_id, "-a", agent_id,

Where you can see that command lines containing an unversioned "python" are created instead of calling the script and relying on it to self-choose the correct python version.

I'm sure there are many other cases like this and any use of Popen() should be checked.

The best solution is (probably) to call the script as an executable and let it decide which python version to use by way of /usr/bin/env python2 or /usr/bin/env python3. This seems to be compatible with https://www.python.org/dev/peps/pep-0394/

The specific error is due to the workaround for Azure/WALinuxAgent#1719 which created the unversioned python pointing to python3. This is also not a good choice, but there doesn't seem to be a perfect choice yet among all the many Azure agents provided by Microsoft.

I plan to revert this back to python2 as the least-bad-choice available (this also seems to match general Microsoft guidance: https://docs.microsoft.com/en-us/azure/virtual-machines/extensions/issues-using-vm-extensions-python-3#resolution) which will mask this problem.

Anyone else tired of python version issues yet? :-)

[Klaas-]

I think this was also fixed via d1c9ac9 by @samarth793