Draft - Http cache / rule rework for next major

[OliverSkroblin]

This merge requests reflects the current work in progress state of the cache/rule system rework which also has a lot of dependencies and many different tasks.

The topics are already discussed in many discussions, adrs or other merge requests:

• RFC: Event focused extension system #3489
• Scalling shopware storefront #3165
• RFC: Rule system v2 #3406
• RFC: Catalog Import API #3391

I will try to split all commits into separated topic and keep the task list up to date.

I publish it here on github to have it public as fast as possible and clear communicated for all of you. And start discussing different solutions.

Goal of the changes is:

• Reduce global state complexity
• Increadible performance boost and cache hit improvement (now: ~5%, goal: 100%)
• Less cache purges
• Reduce indexing ressources (message queue overload right now)
• Better extendability
• Clear and documented APIs for important business classes

Here the task list:

General

• Rector rules for easy update
• Upgrade guide
• Add profiler element which tags would be set for the current page

Context rules

• Rules are no longer available in context object
• Rewrite ActiveRulesDataCollectorSubscriber and show rules of the cart (if cart was loaded) > maybe via event + listener?
• Remove usage of area rule ids

Cart rules

• Rules in cart will be validated after each processor
• Attach matching rules to cart
• Adapt classes in cart domain which rely on context.rules and change it to cart.rules
• Remove cart rule loader

product (pricing) api

• New php class to load the product prices
• Different scopes listing/pdp/cart
• New pricing concept (customer group, sales channel, delivery_country?)
• Rework of cheapest price updater
• rework the cheapest price accessor builder in sql
• Rework the cheapest price in elasticsearch
• Price templates
  • storefront
  • feeds
  • exports
• Implement new event to invalidate product cache
• provide new http api endpoint to import product prices
• instant purge on http cache for out-of-stock/price-changed products
• New pricing concept
  • customer group
  • customer group + channel
  • customer group + channel + country
  • % discount

Cart widget

• Store cart widget in local storage for X minutes and refresh on cart actions (add, delete, change)

Wishlist widget

• Just fetch the count
• Cache result in local storage

http cache

• Remove rules from cache key
• Dispatch event to calculate http cache key
• Add cookie list to allow key extension
• Catch event in CacheStore and Reverse proxy
• Remove cache decorator
• Rework cache invalidator, to optimize tags and hit invalidation count
• New route to invalidate "delay" cache
• Refactor admin shortcuts for cache delete, maybe also remove "delete all caches"
• Remove cart access in cache response subscriber, otherwise we always access the cart again to set the states (cart-filled, etc)
• Hide AddCacheTagEvent dispatching behind feature flag
• Global cms-page tagging handling for cache invalidation

Store api

• Use new AddCacheTagEvent event to add tags to http cache
• Store api full page cache
• Remove cached routes
• I would like to remove abstract routes and integrate new event-decoration pattern
• Remove route decorations (Resolved...), there is no more cache layer between so we don't need to resolve the request before calling the "real" route

delay cache

• Activate delay cache by default
• Delay everything
• Shortcut in admin for clear + dropdown(?) button in cms/category/product module to save+invalidate-delayed

Esi integration

• Header and footer via esi
• header pagelet no more in page
• Replace activeLanguage
• Replace activeCurrency
• Replace minSearchLength
• Cms category sidebar element own resolver
• Replace active navigation id
• Add http cache support esi requests
• Breadcrumb via id
• Remove header from page object
• Remove footer from page object
• Reduce data shipping and payment method for footer
• Remove caching of shipping method / payment method route tag
• Remove header.serviceMenu (and just use footer.serviceMenu)

Navigation route

• Better cache invalidation
• Break navigation route to just provide main, service, footer function
• Reduce data which is returned

Flow improvement

• Flow can no more rely on context.ruleIds
• Should also not rely on order object loaded
• Rules in flows should be allowed to query data by their own
• Remove big data objects from flow data and migrate rules to new rule system
• Dont restore context / order in flows to evaluate rules

Theme caching

• Rework theme caching to have one tag for all theme related stuff because of shared on each page
• Includes theme config, theme assignment, snippets, etc

---

Nice to have

Twig rendering performance

• Improve filter panel rendering
• Improve product list rendering
• Improve navigation rendering

Document storage

• Require new document storage as composer dependency
• Use document storage for product listing > At least minimal data stack for filter
• Use document storage for navigation > At least minimal data stack for navigation

Decorator extension

• Introduce new decoration pattern via event

Documentation system

• Introduce new documentation system
• Added documentation for new store-api route decorations

Pricing concept

Flow Concept