Support older TeX Live versions

[benwebber]

The LaTeX3 code we've been developing assumes recent versions of TeX Live, or at least the ability to upgrade l3kernel and l3packages.

Some users may not be able to upgrade their version of TeX Live, or may not have access to upgrade individual packages. As discovered in #196, Overleaf offers TeX Live 2016.

As I mentioned in that issue, I don't think we should alienate Overleaf users. Many current Overleaf users are future TeX hackers. But we should also be sensible about the versions we support.

There are a bunch of different strategies we can employ to maintain compatibility with old systems:

• Lift the macro (or variant generation) code directly from l3kernel and stick it in compat.
• Document workarounds for older versions of TeX Live.
• Fall back on LaTeX2e solutions to accomplish the same tasks.
• Fall back on Lua code, which should (hypothetically) be more portable between systems (as long as they offer LuaLaTeX).

Each has its own merits. I doubt there's a single universal solution.

I figure we should pick the oldest TeX Live version we're willing to support and aim for compatibility with it. We can configure CircleCI to build under multiple TeXLive versions.

[BrianCriswell]

Using CircleCI to test TeX Live versions from the one on Overleaf (2016) through the current version to find the pain points sounds like a good way to cover most bases. It probably makes sense to lift the code as needed to fix the problems we find.

I guess Overleaf's strategy has changed since this (blog post)[https://www.overleaf.com/blog/435-tex-live-upgrade-november-2016]. Ubuntu seems to be at version 18 now with at least TeX Live 2017 as a supported package. When Overleaf finally upgrades their default TeX Live configuration, we can drop support for versions older than what they have installed.

[BrianCriswell]

This also brings up what is the best way to utilize your skills and time, @benwebber, at least in the short term. We currently have 7 PRs waiting for review (and more on the way). We have also talked about ensuring XeLaTeX and LuaTeX compatibility through CircleCI, and now this recent issue has shown the value of continuous integration against previous distributions of TeX Live.

I think having you complete the proposed CI enhancements would be of the biggest benefit to the project. Once all of the PRs are merged, they are still going to need the CI, and it would be beneficial to have that up and running sooner rather than later, especially with the internals being primarily in LaTeX3. At the same time, I would prefer to not delay the new interface further, and there are enhancements I have not started developing yet because not having the various PRs integrated (other than my test integration branch) is creating too much of a moving target.

We could see if @anoderay is available or bring on some additional collaborators to perform reviews. Alternatively, I could work on integrating the PRs into the new-interface branch, and you could review the PR from new-interface to master as you have time.

[BrianCriswell]

Back more on topic to the OP, we may be able to create a compatibility package for Overleaf that has the user drop the newer versions of packages into their Overleaf project folder. Maybe a Lua script that the user could run in Overleaf that would drop the CTAN packages into their Overleaf project directory? This post mentions that Overleaf V2 (which went live January 8) supposedly has TeX Live 2017, but I was unable to verify that anywhere else.

If we solved the Overleaf problem through a script, we could then solely support the current and the previous TeX Live distributions through compat.sty, or a script to pull down up to date package files into the project folder may be another option for those users unable to run the most recent TeX Live distribution.

[benwebber]

We have also talked about ensuring XeLaTeX and LuaTeX compatibility through CircleCI

That's already set up 😄

CircleCI apparently doesn't build PRs from forks automatically. That is probably why we haven't seen any build updates on your PRs (coming from your fork). I flipped that switch:

https://circleci.com/docs/2.0/oss/#build-pull-requests-from-forked-repositories

I triggered builds using the CircleCI API for all the current PRs. Now CI is actually useful! We won't have to do that again in the future. GitHub sends webhook requests to CircleCI, but previously CircleCI ignored them.

It looks like some of the builds failed for reasons other than the changesets themselves. I'll take a look at them.

I think having you complete the proposed CI enhancements would be of the biggest benefit to the project. Once all of the PRs are merged, they are still going to need the CI, and it would be beneficial to have that up and running sooner rather than later, especially with the internals being primarily in LaTeX3.

Yeah, I agree.

One of the reasons the build is fast now is because we only install the packages we need. To do the sort of testing we're talking about, we'd need to install the whole TeX Live distribution.

CircleCI lets us run our own Docker containers, so we could create a Docker image for every TeX Live version.

On its own, that's not complicated, but I don't want it to block reviewing the work in progress. How about making that a goal for the final merge of new-interface into master?

Back more on topic to the OP, we may be able to create a compatibility package for Overleaf that has the user drop the newer versions of packages into their Overleaf project folder. Maybe a Lua script that the user could run in Overleaf that would drop the CTAN packages into their Overleaf project directory?

That's a little clunky, because you actually have to "compile" your LuaTeX document to run the command. But it's definitely flexible.

You can also upload a zip file to start a project. We could offer an Overleaf-ready zip file to bootstrap new projects.

That being said, if we can nail compatibility with TeX Live 2017 (see below), a script or bundle like that might not be necessary.

[T]here are enhancements I have not started developing yet because not having the various PRs integrated (other than my test integration branch) is creating too much of a moving target.

Yeah, the current state of affairs is a little chaotic. But it's good, and temporary.

Mind logging issues for the planned enhancements?

We could see if anoderay is available or bring on some additional collaborators to perform reviews. Alternatively, I could work on integrating the PRs into the new-interface branch, and you could review the PR from new-interface to master as you have time.

I personally don't see the need to rush 0.8, but I understand where you're coming from. It's frustrating to wait on review, especially when it blocks future work.

I definitely don't want to be a bottleneck here. Maybe it'd help if we established some time constraints. What sort of time frame are you thinking of for 0.8?

This post mentions that Overleaf V2 (which went live January 8) supposedly has TeX Live 2017, but I was unable to verify that anywhere else.

For #131 I imported the project into Overleaf. According to the logs they're running TeX Live 2017. That still makes sense, the change that spurred this issue would have been part of TeX Live 2018.

[BrianCriswell]

Here is the link to the article about running lua. I don't think they allow any other scripting language.

If we help Overleaf users by getting up to date modules into their project directory, I think I would prefer a scripting solution (maybe reading packages.txt) to pull the modules from CTAN over us providing a startup zip (because then we need to keep it updated as modules update).

However, since Overleaf is on 2017, that may mitigate the issue somewhat. Can you tell if the modules available on Overleaf are up to date (for 2017) or part of the original distribution?

[benwebber]

However, since Overleaf is on 2017, that may mitigate the issue somewhat. Can you tell if the modules available on Overleaf are up to date (for 2017) or part of the original distribution?

Working on that now. I couldn't reproduce the issue from #196 in a local TeX Live 2017 (Ubuntu 18.04 in Docker). So it's possible Overleaf runs an earlier TeX Live 2017 version.

EDIT: Some confusion on my part. It looks like Ubuntu issues periodic updates to their texlive package, so their TeX Live 2017 isn't the same as the official release. Overleaf seems to use the official release (it's installed to /usr/local). Not sure if they update specific packages with tlmgr yet. I'll keep digging.

[rjbprime]

With regards to the upcoming changes, and the option of using Overleaf, I have two questions.

1. How would one link images into the document? Having never used Overleaf, but have previously used both Homebrewery and GMBinder, is it as simple as those two?

For example, in my HB/GMB docs, I usually just upload my images to my Google Drive, share them, grab the url, change a part of said url to allow it to render outside GDrive, then paste the resulting url in the img tag.

2. How simple would it be to add my style file to the mix of Overleaf and the zip package, after I've updated said style file for the current release?

The reason I'm asking, is I've been without internet since the end of January until middle of this March, as I am on holiday out past the black stump, and with no mobile reception, I've been having to go 20 km each way by car to get to the local internet cafe to check things, hence I go in once a fortnight, for family to do shopping, and for me to tool around on the net. I understand that's not your problem, but as I am unable to use my laptop at the internet cafe, it's one of those ones that have prebuilt machines that are locked down. Country Australia is backwards like that.

Anyway, my question is, if I can use Overleaf like I used to use HB/GMB, and add my styling to the resultant document, that means I can update said document easier, and have it backed up offsite.

Would that be possible?

[BrianCriswell]

With regards to the upcoming changes, and the option of using Overleaf, I have two questions.

1. How would one link images into the document? Having never used Overleaf, but have previously used both Homebrewery and GMBinder, is it as simple as those two?

For example, in my HB/GMB docs, I usually just upload my images to my Google Drive, share them, grab the url, change a part of said url to allow it to render outside GDrive, then paste the resulting url in the img tag.

The images must be stored within the Overleaf project, just as LaTeX would require them to be on your computer if you were compiling your document on your computer.

1. How simple would it be to add my style file to the mix of Overleaf and the zip package, after I've updated said style file for the current release?

The reason I'm asking, is I've been without internet since the end of January until middle of this March, as I am on holiday out past the black stump, and with no mobile reception, I've been having to go 20 km each way by car to get to the local internet cafe to check things, hence I go in once a fortnight, for family to do shopping, and for me to tool around on the net. I understand that's not your problem, but as I am unable to use my laptop at the internet cafe, it's one of those ones that have prebuilt machines that are locked down. Country Australia is backwards like that.

Anyway, my question is, if I can use Overleaf like I used to use HB/GMB, and add my styling to the resultant document, that means I can update said document easier, and have it backed up offsite.

Would that be possible?

Yes, it is possible. You can look at the preview of one of my documents to see how different the style is from the default. https://www.drivethrurpg.com/product/246746/War-of-the-Burning-Sky-5E-2-The-Indomitable-Fire-Forest-of-Innenotdar

I have an example file here to get you started: https://github.com/rpgtex/DND-5e-LaTeX-Template/wiki/CustomStyleExample

I am working on a LaTeX 3 version of the example style file for the new interface and am working on enhancements to the module specifically to make it easier to add your own style.

[rjbprime]

Got a bonus trip into town. Anyway, seems easier to use a local installed version. Thanks for the advice, I will most likely use my already installed copy of MiKTeX and update as needed.

On a related note, I noticed that a website called RPGChef.com (that I signed up for gods know when) uses a variant of this package. The website has their source on github, (link) but said source hasn't been updated since June 2016, and isn't their latest release (github release v0.9.57 versus v0.9.58 on the RPGChef site). The reason I mention this, is that the PDFs rendered from there have a centred footer. I'm going to download their code and see if they note how they do that, then incorporate it into my style file. If I can get that working, can I create an issue with minimal code on how to do so? As I see that as being useful to beginner LaTeX-ers, such as myself, it could be added to the CustomStyleExample, as an option.

[BrianCriswell]

Got a bonus trip into town. Anyway, seems easier to use a local installed version. Thanks for the advice, I will most likely use my already installed copy of MiKTeX and update as needed.

On a related note, I noticed that a website called RPGChef.com (that I signed up for gods know when) uses a variant of this package. The website has their source on github, (link) but said source hasn't been updated since June 2016, and isn't their latest release (github release v0.9.57 versus v0.9.58 on the RPGChef site). The reason I mention this, is that the PDFs rendered from there have a centred footer. I'm going to download their code and see if they note how they do that, then incorporate it into my style file. If I can get that working, can I create an issue with minimal code on how to do so? As I see that as being useful to beginner LaTeX-ers, such as myself, it could be added to the CustomStyleExample, as an option.

Just take a look at the fancyhdr documentation. It has everything you need. The C option does the centered portion of a footer, and you can clear the existing footer to start from scratch with \fancyfoot{}.