Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for PEP384 + separate user script dir #1954

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

stt
Copy link
Contributor

@stt stt commented May 20, 2018

Todo: assumes scripts directory is under QCoreApplication::applicationDirPath(), if this is ok packaging scripts would need to be updated.

Actually for me doing make install INSTALL_ROOT=/tmp/tiled-pkg doesn't copy any of the examples etc directories to INSTALL_ROOT that dist/distribute.qbs seems to indicate should be copied in the package?

@@ -18,13 +18,15 @@
* this program. If not, see <http://www.gnu.org/licenses/>.
*/

#define Py_LIMITED_API 0x03040000
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this explains why you didn't run into the errors I was running into. The whole plugin needs to compile with this define, not only this file! In particular, we need pythonbind.cpp to also take this define.

Rather than putting this here, I have added the following line to python.qbs (see fa939ef):

cpp.defines: base.concat(["Py_LIMITED_API=0x03040000"])

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I added the define in the Makefile but guess I never did make clean to force it to recompile the bind sources as well.
Now it does give those same errors, will check if there's something that can be done about them.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep it does indeed look like pybindgen would require many changes to generate stable ABI compatible code .. should’ve perhaps read past the PEP384 Abstract and/or realised that just removing one PyRun was way too easy. :)
The biggest hurdle seems to be the new dynamic type objects. There is however a tool that can automate the conversion to some extent http://svn.python.org/projects/python/branches/pep-0384/Tools/scripts/abitype.py ..I’ll give that a go some day next week when there’s a free moment.

@bjorn
Copy link
Member

bjorn commented May 20, 2018

Actually for me doing make install INSTALL_ROOT=/tmp/tiled-pkg doesn't copy any of the examples etc directories to INSTALL_ROOT that dist/distribute.qbs seems to indicate should be copied in the package?

Indeed, the example files were never installed, though they are listed in distribute.qbs to make them part of the released archive (though this may only be used by the hidden 7-zip packages for Windows at the moment). In any case, I'd rather just get rid of the qmake files.

I think I preferred the contents of _preload.py to be in the C++ code though, and I guess we can still keep things that was if we switch to using Py_CompileString and then somehow execute / evaluate the compiled code.

@stt
Copy link
Contributor Author

stt commented May 23, 2018

Ok, having modified pythonbind.cpp to use dynamic type declarations (PyType_FromSpec) + few other changes, the plugin now compiles with -DPy_LIMITED_API, and fotf.py and mappy.py successfully load maps on mac + linux as they used to.
Obviously the file can't be regenerated anymore without losing the changes. I opened an issue with pybindgen project for adding stable API support, if no one picks it up in a day or two I'll look into developing that feature.

Re: _preload, I was thinking maybe there could be a feature for batch converting bunch of levels from custom binary formats to tmx and with that users might want to get any errors to stderr, having the logging redirection in a script gives them an option to customize it as they want to.

Copy link
Member

@bjorn bjorn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stt That's a promising change, thanks!

Obviously the file can't be regenerated anymore without losing the changes. I opened an issue with pybindgen project for adding stable API support, if no one picks it up in a day or two I'll look into developing that feature.

Thanks for opening gjcarneiro/pybindgen#13, let's see what the response is.

How hard is it to do the changes? I guess I should hold off on merging this until it is either mostly automated or working in PyBindGen.

I've provided some more comments inline.

if (QFile::exists(mScriptDir))
mFileSystemWatcher.addPath(mScriptDir);
connect(&mFileSystemWatcher, SIGNAL(directoryChanged(QString)),
&mReloadTimer, SLOT(start()));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This connection is now already made in the constructor, so it can be removed here.

/**
* (Re)load modules in the script directory
*/
void PythonPlugin::reloadModules()
void PythonPlugin::reloadModules(QString path)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use const QString &path to avoid unnecessary reference counting.

@@ -40,7 +41,8 @@ static void handleError()
}

PythonPlugin::PythonPlugin()
: mScriptDir(QDir::homePath() + "/.tiled")
: mSysScriptDir(QCoreApplication::applicationDirPath() + "/scripts")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably would prefer to call this mAppScriptDir.

Unfortunately, initializing it like this will only work on Windows. Check out the LanguageManager constructor for how we could make it work on macOS and Linux (and of course, install rules will need to be written for this - see translations.qbs).

@@ -10,7 +10,7 @@


#if PY_VERSION_HEX >= 0x03000000
#if PY_VERSION_HEX >= 0x03050000
#if PY_VERSION_HEX >= 0x03050000 && 0 // <- TODO: this is not Py_LIMITED_API compatible?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyAsyncMethods is not available in the limited API, so that check should probably be:

#if PY_VERSION_HEX >= 0x03050000 && !defined(Py_LIMITED_API)

@bjorn bjorn force-pushed the master branch 3 times, most recently from 384d5ac to 2e9a0fb Compare June 25, 2022 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants