Redesign the caching to be 'static assets' somehow?

[beddari]

I'm thinking the cache approach is somewhat non-optimal for my workflow. What about a design where the caching is somehow implemented "in-place"?

As an example, I have these files

technical/network (master)$ ls -la site_design.*
-rw-r--r--. 1 beddari beddari 13085 Mar  6 11:07 site_design.drawio
-rw-rw-r--. 1 beddari beddari  1492 Feb 26 12:53 site_design.md

but instead of writing the build outputs to the cache dir, write it directly to the same dir as above, so that I get perhaps (multipage example):

technical/network (master %)$ ls -la site_design*
-rw-r--r--. 1 beddari beddari 35832 Mar  6 11:22 site_design-0.svg
-rw-r--r--. 1 beddari beddari 33467 Mar  6 11:22 site_design-1.svg
-rw-r--r--. 1 beddari beddari  6542 Mar  6 11:22 site_design-2.svg
-rw-r--r--. 1 beddari beddari 13085 Mar  6 11:07 site_design.drawio
-rw-rw-r--. 1 beddari beddari  1492 Feb 26 12:53 site_design.md

Is this something you'd like to consider - or just not interesting? I'm aware I could just wrap this completely on the outside and forgo the markup althogether (just ref the svgs) but I think an integrated approach still has value.

[LukeCarrier]

Hey @beddari, thanks for reaching out.

I'll admit upfront that I'm uncomfortable with writing directly into the source directory (at least by default -- I'm open to an option for it) for a couple of reasons:

1. It'd be a lot easier for us to accidentally overwrite some manually crafted diagram with a similar name.
2. It makes source control difficult without just ignoring *.svg or similar.

Are you able to share any more information about your workflow to help contextualise?

[beddari]

I've been giving this some more tought. What I'm really getting at is that build times on large documentation sets are ever increasing if we don't keep the cache around. Thus, I feel like the fix is to do that - keep the cache around, in other words, keep the compiled svgs as static assets somehow, in version control. Need to try that.

Also perhaps I'll need to look into how you determine if a drawio file is already built or not. So far our builds have been done in containers, throwing away the cache every build - obviously takes a long time with a lot of drawio files :)

[LukeCarrier]

@beddari yep, I totally understand the concern here. It's something we're starting to notice in our builds at work, too.

I'm still wary of checking generated content into repositories, and I suspect other users would be too. For me this rules out putting the files in-place, but I guess we could directly link to the files in cache_dir rather than doing that final copy? This would support both use cases and leave the final decision down to the user.

Note that the work in #16 will require us to start delving into the source Draw.io file contents a little more anyway. This would allow us to identify which pages within the document have changed rather than blindly rebuilding all of them because the file's utime changed.