Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for server side markdown processing #3018

Closed
wants to merge 5 commits into from

Conversation

Dubzer
Copy link
Contributor

@Dubzer Dubzer commented Feb 29, 2024

I have added an option to handle markdown on the backend instead of the frontend.
This is for users who prioritize load time and are more limited in computing resources on their devices rather than on the server. For example, if a memos instance is self-hosted and used primarily from a phone.
Also, this can be useful for first time accessing public instances using slow internet connection.

This PR is WIP

Also, to avoid dublicating node definition, https://github.com/yourselfhosted/gomark-wasm should be published to the buf repository

@Dubzer Dubzer requested a review from boojack as a code owner February 29, 2024 17:03
@Dubzer Dubzer marked this pull request as draft February 29, 2024 17:03
@Dubzer
Copy link
Contributor Author

Dubzer commented Feb 29, 2024

@boojack could you please explain the difference between workspace settings and system settings? I'm thinking on where to put server side markdown setting.
the same goes for Workspace Profile in workspace_service.proto and Workspace Setting Service in workspace_setting_service.proto.

@boojack
Copy link
Collaborator

boojack commented Mar 3, 2024

Thank you for your contribution. But we probably don't need to do this. Here are some reasons or background about our parser.

At the beginning of the design, it was designed as if it was transferring nodes through the backend, but we realized that if there were some large content, the whole API would be very slow to respond and server-side performance would become an issue. So this is definitely not a long term solution, what can be done on the client side should be done on the client side.

We plan to use wasm to put the go parser on the frontend. The results show that it works very well, except that the file size is a bit large. But this problem has been solved (yourselfhosted/gomark-wasm#1)

@Dubzer
Copy link
Contributor Author

Dubzer commented Mar 3, 2024

@boojack I don't quite understand the reasoning on avoiding slow API responses.

my current setup is a private home server with the AMD Ryzen 7 CPU. and I mostly use memos from my phone. obviosly, markdown will be proccessed much faster on the backend, with potential on the future improvments on taking advantage of multithreading and SIMD instructions, which both are not really possible with the browser's WASM runtime.
as I see it, using server-side markdown parsing will lead to overall faster rendering. while the API response may take longer, for me as a user, I'm at a net plus in terms of content load time.
the negative effect may occur if I host memos on some slow single board PC, such as a RPI, which is slower than most of my client devices. but that's why I would like to introduce this feature as an option

despite all this, after reducing the size to just 1.1mb, this PR might be not as relevant, since most of the problems with the WASM blob were because of it's size. so I understand if you prefer to not spend additional resources supporting this feature in the future

@boojack
Copy link
Collaborator

boojack commented Apr 29, 2024

Implemented with 155c5ba

@boojack boojack closed this Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants