Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AMQP 1.0 Parser and Cython code acceleration #29

Closed
Gsantomaggio opened this issue Jan 31, 2023 · 4 comments
Closed

AMQP 1.0 Parser and Cython code acceleration #29

Gsantomaggio opened this issue Jan 31, 2023 · 4 comments
Assignees

Comments

@Gsantomaggio
Copy link
Collaborator

At the moment, we include all the AMQP 1.0 QPID with Azure AMQP 1.0.

We should extract only the AMQP parser avoiding all the client

@Gsantomaggio
Copy link
Collaborator Author

cc @DanielePalaia

@DanielePalaia DanielePalaia changed the title AMQP 1.0 Parser AMQP 1.0 Parser and Cython code acceleration Nov 3, 2023
@DanielePalaia
Copy link
Collaborator

This issue also to see if we can do some tests to improve the performance of the parse using Cython

@DanielePalaia
Copy link
Collaborator

DanielePalaia commented Nov 23, 2023

I made a few tests and exploration on the https://github.com/Azure/azure-uamqp-python library.

This library already uses cython acceleration.

There is a section written in python (inside uamqp folder) which is a wrapper of the cython library inside src (there are .pyx files there). The library is also wrapping a pure C library https://github.com/Azure/azure-uamqp-c.

Anyway despite this the library seems very slow during instantion of Message objects and encoding/decoding.

There are few steps that are very inefficient but I'm not sure how these can be improved.

The struct Message defined in Message.py seems to be a wrapper of c_uamqp.Message.
During every encoding operation a cloning of this wrapped object c_uamqp.Message happens.
This operation really takes a lot of time during enconding.

Also there is a section in util.py:

data_factory(value, encoding='UTF-8'):
"""Wrap a Python type in the equivalent C AMQP type.

This wrapping conversion seems to take a lot of time especially when complex structure like dictionary are passed.

Also the get_encoded_message_size which written in cython seems having few redundant operations.

Given the three layers this library is composed we need to undestand at this point if it is better to create a new one or just refactor this one.

A simple python code (without cython accelleration) as in this branch: https://github.com/qweeze/rstream/tree/uamqp_test which is just encoding a body is giving already better performance of the uamqp library with a body encoded by a lot.

@DanielePalaia
Copy link
Collaborator

I'll close this issue after #194

This can be continued if necessary in the Performance issue #30

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants