Model-like StreamField blocks

[tbrlpld]

@mnieber raised an interesting question in the issue on StreamField migration size. It was deferred as too big to cover there, which makes total sense, but I did not find that the discussion was ever taken up. I wanted to do so, because it is something I have been wondering myself.

The question raised was

What if - instead of inserting a StructBlock - I could insert a new django.Model instance into a StreamField?

I have been wondering this question recently too. I was not drawing the similarity to only Djangos Model but even to Wagtail Page model. Pages and blocks hold data and know how to render themselves (or at least which template should be used to render them).

I was wondering why are defining blocks in the less structures JSON way instead of the Django'ish way of storing all data in models. Would the table joins be too big of a slow down or what other considerations are there that I am probably missing?

[nrsimha]

If each Streamfield's block will be saved in database as own record, it will be easy to make translations of each block, or we could enable users to comment on paragraph level.

[ababic]

So if I want to translate individual block and see in which languages it is already translated? Or if I want to categorize/tag specific paragraph, is it really doable without some crazy hacking?

@nrsimha I see your point. Block values do have a unique ID that is preserved over time, which can be used for this sort of thing - but I know that isn't the same thing.

I would question the value of translating individual blocks though. Blocks can be pretty granular, and translation work very often isn't a straight-forward like-for-like swap in terms of content structure.

[nrsimha]

@ababic Yes, I see each block has unique key, but yes, it is not same.

I for example work on website with books in multiple languages. (https://vedabase.io/en/)

Idea is that each paragraph, quote, verse text is record. Benefit is that users can easy bookmark specific paragraph (although they can do on Streamfield too, it is somehow hacky solution to search where the content is). Or they can show text side-by-side with other language, and each paragraph starts on same line (e.g.: https://vedabase.io/en/library/bg/1/1/side-by-side/es/)

Each text is being translated in available languages (some books are translated in many languages, some in few). Currently I am using for paragraph, verse text, etc... Page model (but not using it as Page - practically if you would visit url of that page it redirects you to parent page and use # to jump to specific text on that page), but it is BAD SOLUTION. I could off course use own model, but to keep it nicely integrated I went with Page.

In ideal case I would be able to use StreamField, or something very similar, but each block to be saved as own record in database.

I keep it separate like this instead of using StreamField, so user can bookmark specific paragraph, or once categorization is enabled, we can assign topic/category to any paragraph and same time we know which paragraphs are translation of that, so we can categorize all all languages at once. Maybe is hard to explain, but it will be really useful to have ability to have StreamField blocks saved separately. But as StreamField UI is very nice and practical to use, it will be sad to not be able to use it somehow (but for database model instead).

[nrsimha]

Currently each text (something like StreamField block is saved page model as children):

[ababic]

Thanks for explaining further @nrsimha - your comments makes a lot more sense now.

Would be interested to hear how this all works for you more generally. For example, is it performant? I remember migrating to Wagtail from something called 'page-cms' which took exactly this approach, but this was back in the days of Django 1.3 (before prefetch_related was really a thing), and it was all super slow.

[nrsimha]

@ababic Before 2017 I used for the project Drupal, but then I moved to Wagtail. Performance would not be so bad, but on big pages like advanced chapter view where I show more than 300 verses on one page it would be slow (e.g.: https://vedabase.io/en/library/cc/adi/17/advanced-view/) if I will not use cache.

I just use Redis cache, so for first user it is little slower, but for all other visits it don't need to generate HTML for all those verses separately, but will get full content part through Redis which is quite fast (of course there is still size of whole page - which is huge).

Upcoming new version Wagtail will be headless as the front part of site will be served by Nextjs (most texts stay same all the time only sometimes we fix some error, or import new content), so performance will be even better. Nextjs will get necessary data through Wagtail API and will SSG the page, so users will get whole page pre-rendered and served only by Nginx, and only custom data or functions (bookmarks, notes, etc...) for signed in users will be loaded additionally.

As my current approach when paragraphs are saved as Page model is "nasty" I was considered either to use completely custom solution for data, or custom Model (non-Wagtail) for those records which are not Pages, or was thinking to make something like StreamField, but database based. But Wagtail 4 Streamfields are so nice, specially splitting.

I love Wagtail, and StreamFields concept is amazing, but for my case it is somehow limited. But I understand that most website StreamFields are perfect.

But if there will be plan to make something like StreamFields but with proper models I will be happy to help as much as I able to.