Strange issue with transwhat running under Spectrum2

[Hursofid]

Hello,

I've installed, configured prosody and just working fine for months.
I got a problem with using transwhat with Spectrum2

Steps to reproduce as follows:
Note: all sensitive information below were replaced, e.g. password, domain.tld, etc.

• prosody configuration edited - /etc/prosody/prosody.cfg.lua, added the new component:

Component "whatsapp.domain.tld"
        component_secret = "_password_"

• Spectrum2 were installed and transport were configured here: /etc/spectrum2/transports/whatsapp.cfg:

[service]
#user = spectrum
#group = spectrum

jid = whatsapp.domain.tld

server = localhost
password = _password_
port = 5347

backend_host = localhost
backend = /usr/local/bin/transwhat

users_per_backend = 10
more_resources = 1

admin_jid = gojaradmin@domain.tld

[identity]
name = transWhat
type = xmpp
category = gateway

[logging]
config = /etc/spectrum2/logging.cfg
backend_config = /etc/spectrum2/backend-logging.cfg

[database]
type = sqlite3

• cloned transwhat:

# git clone https://github.com/stv0g/transwhat.git
Cloning into 'transwhat'...
remote: Counting objects: 1123, done.
remote: Total 1123 (delta 0), reused 0 (delta 0), pack-reused 1123
Receiving objects: 100% (1123/1123), 416.27 KiB | 65.00 KiB/s, done.
Resolving deltas: 100% (749/749), done.
Checking connectivity... done.

• Installed dependencies that are required - pip install --pre e4u protobuf python-dateutil yowsup2

• Preformed running python setup script: ~/transwhat# python setup.py build and ~/transwhat# python setup.py install

• Checked if backend were installed to expected place:

# ls /usr/local/bin/transwhat
/usr/local/bin/transwhat

And finally, restarting Spectrum2 # spectrum2_manager start and getting the following:

Traceback (most recent call last):
  File "/usr/local/bin/transwhat", line 9, in <module>
    load_entry_point('transwhat==0.2.2', 'console_scripts', 'transwhat')()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 356, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2476, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2190, in load
    ['__name__'])
  File "/usr/local/lib/python2.7/dist-packages/transwhat-0.2.2-py2.7.egg/transWhat/transwhat.py", line 42, in <module>
    from whatsappbackend import WhatsAppBackend
  File "/usr/local/lib/python2.7/dist-packages/transwhat-0.2.2-py2.7.egg/transWhat/whatsappbackend.py", line 30, in <module>
    from session import Session
  File "/usr/local/lib/python2.7/dist-packages/transwhat-0.2.2-py2.7.egg/transWhat/session.py", line 39, in <module>
    from yowsup.layers.protocol_media.mediauploader import MediaUploader
  File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/protocol_media/__init__.py", line 1, in <module>
    from .layer import YowMediaProtocolLayer 
  File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/protocol_media/layer.py", line 2, in <module>
    from .protocolentities import ImageDownloadableMediaMessageProtocolEntity
  File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/protocol_media/protocolentities/__init__.py", line 8, in <module>
    from .iq_requestupload import RequestUploadIqProtocolEntity
  File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/protocol_media/protocolentities/iq_requestupload.py", line 2, in <module>
    from yowsup.layers.protocol_iq.protocolentities import IqProtocolEntity
  File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/protocol_iq/__init__.py", line 1, in <module>
    from .layer import YowIqProtocolLayer
  File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/protocol_iq/layer.py", line 6, in <module>
    from yowsup.layers.network import YowNetworkLayer
  File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/network/__init__.py", line 1, in <module>
    from .layer import YowNetworkLayer
  File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/network/layer.py", line 2, in <module>
    from yowsup.common.http.httpproxy import HttpProxy
  File "/usr/local/lib/python2.7/dist-packages/yowsup/common/http/__init__.py", line 2, in <module>
    from .warequest import WARequest
  File "/usr/local/lib/python2.7/dist-packages/yowsup/common/http/warequest.py", line 4, in <module>
    from yowsup.env import YowsupEnv
  File "/usr/local/lib/python2.7/dist-packages/yowsup/env/__init__.py", line 1, in <module>
    from .env import YowsupEnv
  File "/usr/local/lib/python2.7/dist-packages/yowsup/env/env.py", line 15, in <module>
    class YowsupEnv(with_metaclass(YowsupEnvType, object)):
  File "/usr/lib/python2.7/abc.py", line 87, in __new__
    cls = super(ABCMeta, mcls).__new__(mcls, name, bases, namespace)
TypeError: Error when calling the metaclass bases
    metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

Would you have any idea what's going on?
I'll appreciate any help.

[SmartHoneybee]

Try again with Python 3.

[truenicoco]

I'm experiencing the same issue and I agree it is a python 2/3 related error. However, e4u relies on the deprecated beautifulsoup which seems to only work with python 2, so I could not find any workaround to install transwhat. I'm using debian 9, if anyone has a fix this would make me really happy!

$ pip3 install e4u --user
Collecting e4u
  Using cached e4u-0.2rc1.tar.gz
Collecting setuptools (from e4u)
  Using cached setuptools-36.6.0-py2.py3-none-any.whl
Collecting BeautifulSoup (from e4u)
  Using cached BeautifulSoup-3.2.1.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-uu5hdj16/BeautifulSoup/setup.py", line 22
        print "Unit tests have failed!"
                                      ^
    SyntaxError: Missing parentheses in call to 'print'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-uu5hdj16/BeautifulSoup/

EDIT: downgrading six seems to have resolved this specific issue. Still working on getting everything to work though...

[SmartHoneybee]

If I'm not mistaken it is only yowsup2 which needs python 3.

[opiums9]

after installing through python 3 and running, we get:

Traceback (most recent call last):
File "/usr/local/bin/transwhat", line 5, in <module>
from pkg_resources import load_entry_point
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2876, in <module>
working_set = WorkingSet._build_master()
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 449, in _build_master
ws.require(__requires__)
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 745, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 639, in resolve
raise DistributionNotFound(req)

[stv0g]

I think you have to use Python 2.7 because of e4u for now...

I am looking into a way to replace e4u completely.

[thundergreen]

@stv0g is there a solution already for e4u and Python 3?

[stv0g]

No, I haven't found a replacement yet.
However, I am wondering whether WhatsApp is still using the SoftBank encoding.
I would expect that they switch to much more common Unicode code table...

[vitalyster]

@stv0g what exactly e4u do? convert from SoftBank encoding? How another yowsup-based app do that? We just found an active yowsup fork which requires python3, wonder how it works without e4u :)

[stv0g]

Yes, that's the only purpose of e4u:

transwhat/transWhat/utils.py

Line 49 in 35afbe5

def softToUni(message):

It converts Emojis from Softbank to Unicode encoding.

Could you try to just remove this part? Maybe WhatsApp already switched to Unicode?