Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increasing the size of the WebSocket #70

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

ViniciusLucchesi
Copy link

@ViniciusLucchesi ViniciusLucchesi commented Aug 27, 2023

What is the motivation?

WebSocket only handles 1MB of data and I'm getting this error message sent 1009 (message too large); no closed frames received when I try to do a SELECT * FROM my_table when my_table had more than 1 MB of data.

Type of Change

  • 📚 Examples / docs / tutorials / dependencies update
  • 🔧 Bug fix (non-breaking change which fixes an issue)
  • 🥂 Improvement (non-breaking change which improves an existing feature)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to change)
  • 🔐 Security fix

What does this change do?

These changes allow the user to specify what is the max size of the data that the WebSocket connection to the SurrealDB will be capable to handle.

What is your testing strategy?

  • Create a table on SurrealDB with more than 1 MB of data.
  • Run the script above, that start the connection to SurrealDB with a limit of 3 MB instead of the 1 MB default.
import asyncio
from surrealdb import Surreal

DB = Surreal()

async def main():
    await DB.connect("ws://localhost:8000/rpc", max_size=3000000)
    await DB.signin({"user": "root", "pass": "root"})
    await DB.use(database="test", namespace="test")

    try:
        data = await DB.select("tipi")
    except Exception as e:
        data = str(e) 
    print(data)
        
if __name__ == '__main__':
    asyncio.run(main())

This script will print on the terminal all data from your table, but if you remove the max_size parameter from connect method the max size handle will be only 1 MB as default.

Is this related to any issues?

No

Have you read the Contributing Guidelines?

…pport

WebSocket only handles 1MB of data and I'm getting this error message `sent 1009 (message too large); no closed frames received` when I try to do a `SELECT * FROM my_table` when `my_table` had more than 1 MB of data.

Adding the `max_size` variable to the `Surreal` constructor class and its `connection` method solves the problem.
Copy link

@meppu meppu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just wanted to share my personal opinions, thanks for your work :)

Comment on lines +242 to +243
else:
self.url
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Else block seems doing nothing, is there any reason why it is here?

Comment on lines +249 to +255
# helping people when they type the url in wrong
if "http" in self.url:
self.url = self.url.replace("http://", "ws://")
elif "https" in self.url:
self.url = self.url.replace("https://", "wss://")
if "/rpc" not in self.url:
self.url = "".join([self.url, "/rpc"])
Copy link

@meppu meppu Aug 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may lead some unexpected behaviour:

url = "https://meppu.boo"
if "http" in url:
  print("something is wrong")
elif "https" in url:
  print("still not a good way to do it")

# prints "something is wrong"

The easiest solution is of course just switching condition's order and appending ://, But if something like this needs to be implemented, using something like urllib would be a better choice: https://docs.python.org/3/library/urllib.parse.html

The default value of 1MB was being set in the connect method, which made the value passed to the surreal class constructor invalid.

That is, every time the connect method was called but did not receive any value from its `max_value` variable, it would set the default value of 1 MB, overwriting the value set at object initialization.
@ViniciusLucchesi ViniciusLucchesi changed the title Making it possible to increase the maximum size that WebSocket can su… Increasing the size of the WebSocket Aug 29, 2023
@meppu
Copy link

meppu commented Aug 30, 2023

Hi,

I noticed that the latest commit appears to be unrelated to the feedback I provided in my previous review. Please let me know if you have any questions about the feedback I provided or if you'd like any clarification on specific points.

I want to clarify that I don't have merge access and my initial review was offered from a personal standpoint to provide insights and suggestions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants