Embedded libgraphqlparser build fails with latest ctypesgen

[bobh66]

Report a bug

• Explain with a simple sentence the expected behavior
  The build of the embedded libgraphqlparser fails with the latest ctypesgen module.

• Tartiflette version: - latest

• Python version: 3.6, 3.7

Since ctypesgen 0.1.1 (now 1.0.0) it has installed as "ctypesgen" instead of "ctypesgen.py".

The CMakeLists.txt for libgraphqlparser only looks for "ctypesgen.py" and doesn't find "ctypesgen".

I pushed a PR to the tartiflette/libgraphqlparser repo/branch that holds the Py3 version of libgraphqlparser to fix the CMakeLists.txt file to look for ctypesgen if it can't find ctypesgen.py, and I will submit the same change upstream to libgraphqlparser .

• Executed in docker: No
• Dockerfile sample: N/A
• GraphQL Schema & Query: N/A
• Is it a regression from a previous versions? No
• Stack trace N/A

[abusi]

I'll wait a littlebit for a reaction on the official repo, if they merge it, i'll rebase. If nothing happens there, i'll merge you PR on tartiflette/libgraphqlparser and release a tartiflette 1.x.y with it.
And, tartiflette don't need all this python2 things and I don't want to have to maintain all this.
I don't want responsability over libgraphqlparser.
We'll see how it goes. Maybe we'll really fork completly and even remove all this python2 thing. If someone is interested in maintaining it.
I hope the graphql/libgraphqlparser will be updated on the latest GraphQL Spec.
We'll see how it's going in on when it happens.

[florimondmanca]

I also encountered this while trying to upgrade to 1.0 to try out tartiflette-starlette on that version.

Link to the issue on the upstream repo: graphql/libgraphqlparser#89

Here's the full output from that command:

$ pip install -U tartiflette
Collecting tartiflette
  Using cached https://files.pythonhosted.org/packages/c9/a1/fcd8c32f7cf892d1164177dd4f72de6a8170ef932cd351d4d4fd27d181ad/tartiflette-1.0.0.tar.gz
Requirement already satisfied, skipping upgrade: cffi<2.0.0,>=1.0.0 in ./venv/lib/python3.7/site-packages (from tartiflette) (1.12.3)
Collecting lark-parser==0.7.5 (from tartiflette)
  Using cached https://files.pythonhosted.org/packages/00/f9/1a3a7d79bb204096ca0e20b06a076ad8775939cb0a92eace83ddb1e8732a/lark-parser-0.7.5.tar.gz
Requirement already satisfied, skipping upgrade: pytz in ./venv/lib/python3.7/site-packages (from tartiflette) (2019.2)
Requirement already satisfied, skipping upgrade: pycparser in ./venv/lib/python3.7/site-packages (from cffi<2.0.0,>=1.0.0->tartiflette) (2.19)
Installing collected packages: lark-parser, tartiflette
  Found existing installation: lark-parser 0.6.4
    Uninstalling lark-parser-0.6.4:
      Successfully uninstalled lark-parser-0.6.4
  Running setup.py install for lark-parser ... done
  Found existing installation: tartiflette 0.12.5
    Uninstalling tartiflette-0.12.5:
      Successfully uninstalled tartiflette-0.12.5
  Running setup.py install for tartiflette ... error
    ERROR: Command errored out with exit status 255:
     command: /Users/florimond/Developer/tartiflette-projects/tartiflette-starlette/venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/ld/jnnnnsfd7r7cyvbwygss3blw0000gn/T/pip-install-e3l6uy_6/tartiflette/setup.py'"'"'; __file__='"'"'/private/var/folders/ld/jnnnnsfd7r7cyvbwygss3blw0000gn/T/pip-install-e3l6uy_6/tartiflette/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/ld/jnnnnsfd7r7cyvbwygss3blw0000gn/T/pip-record-febtchui/install-record.txt --single-version-externally-managed --compile --install-headers /Users/florimond/Developer/tartiflette-projects/tartiflette-starlette/venv/include/site/python3.7/tartiflette
         cwd: /private/var/folders/ld/jnnnnsfd7r7cyvbwygss3blw0000gn/T/pip-install-e3l6uy_6/tartiflette/
    Complete output (14 lines):
    running install
    running build
    running build_py
    CMake Error at CMakeLists.txt:29 (FILE):
      FILE COPY given no DESTINATION
    
    
    CMake Warning at python/CMakeLists.txt:13 (MESSAGE):
      ctypesgen.py not found; install with pip or easy_install if you want to run
      pythontest.py.
    
    
    make: *** No targets specified and no makefile found.  Stop.
    Libgraphqlparser compilation has failed
    ----------------------------------------
  Rolling back uninstall of tartiflette
  Moving to /Users/florimond/Developer/tartiflette-projects/tartiflette-starlette/venv/lib/python3.7/site-packages/tartiflette-0.12.5-py3.7.egg-info
   from /Users/florimond/Developer/tartiflette-projects/tartiflette-starlette/venv/lib/python3.7/site-packages/~artiflette-0.12.5-py3.7.egg-info
  Moving to /Users/florimond/Developer/tartiflette-projects/tartiflette-starlette/venv/lib/python3.7/site-packages/tartiflette/
   from /Users/florimond/Developer/tartiflette-projects/tartiflette-starlette/venv/lib/python3.7/site-packages/~artiflette
ERROR: Command errored out with exit status 255: /Users/florimond/Developer/tartiflette-projects/tartiflette-starlette/venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/ld/jnnnnsfd7r7cyvbwygss3blw0000gn/T/pip-install-e3l6uy_6/tartiflette/setup.py'"'"'; __file__='"'"'/private/var/folders/ld/jnnnnsfd7r7cyvbwygss3blw0000gn/T/pip-install-e3l6uy_6/tartiflette/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/ld/jnnnnsfd7r7cyvbwygss3blw0000gn/T/pip-record-febtchui/install-record.txt --single-version-externally-managed --compile --install-headers /Users/florimond/Developer/tartiflette-projects/tartiflette-starlette/venv/include/site/python3.7/tartiflette Check the logs for full command output.

[florimondmanca]

And, tartiflette don't need all this python2 things

I can confirm that installing Tartiflette requires the python2 to be available — not sure if this is new (I reinstalled 0.12.5). I had to tweak my pyenv setup to account for this:

$ pyenv global 3.7.3 3.6.8 2.7.15 3.8-dev

[abusi]

And, tartiflette don't need all this python2 things

I can confirm that installing Tartiflette requires the python2 to be available — not sure if this is new (I reinstalled 0.12.5). I had to tweak my pyenv setup to account for this:

$ pyenv global 3.7.3 3.6.8 2.7.15 3.8-dev

Yes, what I mean is that it shouldn't need it. but it's mandatory in the cmakelist of libgraphqlparser.
I don't really know what to do here, cause if I update my branch on my libgraphqlparser, it will go farther from libgraphqlparser main repo, and I really don't want to have to maintain a fork.
I'll see if I found the time to think of something, or at least to finish working for my branch to be merged.
It also looks like the project libgraphqlparser is kind of abandonned ... maybe forking away is not that bad of an idea ... IDK.

Do you have any idea on how we could get rid of this problem? (fastness of tartiflette partly come from using this C++ lib to parse the requests.)

Anyway, thanks for this tips @florimondmanca

[florimondmanca]

Is there any way we can restrict the version of ctypesgen to 0.x on our side? Or maybe push the fix from graphql/libgraphqlparser#90 to our fork and release it while things get sorted out on the official repo?

I'm still encountering this issue on macOS Mojave / Python 3.7.4 / Tartiflette 1.1.0, and this is definitely blocking other users from upgrading to Tartiflette 1.0.

I'm curious why Travis CI builds over at tartiflette/tartiflette-asgi#58 don't catch this compilation error (is it not happening on Linux?), but adding 1.0 support to tartiflette-starlette is also blocked because of this at the moment unfortunately.

[florimondmanca]

As for the Python 2 issue,

but it's mandatory in the cmakelist of libgraphqlparser

Would pushing (as in: contributing) for upgrading the Python 2 scripts in graphql/libgraphqlparser to Python 3 be acceptable?

You wrote:

I don't want responsability over libgraphqlparser.

In my point of view though, libgraphqlparser is our responsability — we're using it to power our parser layer; it's an important dependency, and we should care for it.

If people from the main repo don't have the resources/interest to make what we need happen, perhaps scratching our own itch and contributing back is reasonable.

If they stay unresponsive though, either we find new maintainership for the main repo, we hard fork, or we use a different parser. But I think the state of libgraphqlparser is very much our problem too.

[abusi]

@florimondmanca
I think that it is what i'm going to do. I'm going to clean the tartiflette/libgraphqlparser repository from everything that is python2 dependant. and use this version in tartiflette and produce a 1.1.1 with it.

In the end, we need it to work. What I mean in my previous comment is that I may not have the time to take care for it. But in the end it is very important, so I'll have to do it.

Thanks for your advices.

[abusi]

@bobh66 Can you try using pip install -i https://test.pypi.org/simple tartiflette==1.1.1.dev1570620494 --extra-index-url https://pypi.org/simple and tell me if it's working for you ? Thanks.