A Makefile for the anthology

[akoehn]

This is an issue to track the creation of a Makefile fo building the anthology. A first wip is #347. See also some preliminary discussion in #316.

Todo:

• clean up the commit
• add documentation
• make the http serve target work as expected (serve as /anthology/...)
• investigate how to have real targets instead of just phony ones
• make all targets safe for parallelization (i.e. make the targets depend on each other)
• collect feedback
• make PDF host configurable

As this is not a critical issue, I will work on it in my spare time; no progress does not imply abandonment.

[akoehn]

I updated #347 with the first three items of the todo list.

I had a short look at what is actually needed to change the PDF storage path so that a completely separate anthology website could be hosted. The URL sanitation is in a single place and with some work, it should be feasible to sanitize to a different host/url and expose this feature to the command line (see #156). Once that is done, the Makefile could use that similar to the ANTHOLOGYHOST variable.

[mjpost]

The PDF storage path will be much easier to do when #324 is done. I am behind on this and not sure I'll be able to get it done before the weekend.

#324 is close to done and will close a large number of issues relating to paths, frontmatter display, whether to show PDFs, including extra conference material, and so on.

[mjpost]

Feedback: the makefile is awesome.

It seems the remaining issues here are more properly thought of as secondary to other tasks—maybe we could close this?

[akoehn]

If this is also of interest to someone else, I would like to see whether we can make some targets non-phony. The bibtex conversion e.g. could be done that way, which would make the Makefile actually shorter and parallelization would by handled by make.

Regarding the PDF host: i just want to have that noted as a TODO somewhere.

[mjpost]

+1 to non-phony targets and parallelization.

[mjpost]

Maybe the PDF host is relevant to #295, so we could note this there and close this out?

[mbollmann]

I'm just trying to develop some UI changes for the first time after the introduction of the Makefile, and I have a hard time figuring out the best workflow.

I used to rely on the hugo server command to serve the local version of the website, as it comes with the benefit of a) automatically rebuilding the site when a source file changes, and b) doing so in an efficient manner that is way faster than a full rebuild.

Now, I thought I could simply run hugo server in the build directory, but of course that loses the automatic rebuild since that's not where the source files are. For some reason, copying the whole directory over with cp hugo/* build does not trigger Hugo's change detection; copying individual files works. I've settled on doing rsync -ruv hugo/ build now when I've changed something, but maybe I'm missing some easier way?

[akoehn]

Maybe do the changes in the build directory and once you are happy with it copy the changes to hugo? That way, you can still use hugo server.

Of course, do a clean rebuild before a commit to check whether everything works.

[mbollmann]

I thought about that, but that also requires me to remember which files I've changed (and copy those individually), since I can't sync back the whole build dir. And there's the chance of screwing things up with an accidental make command...

[akoehn]

something like for i in $(cd hugo; find); do cp build/$i hugo/$i; done to move back the files that were already in the hugo dir? Completely untested of course.

[mbollmann]

I think doing it the other way around makes more sense; I've settled for

python3-pyinotify -r hugo/ -e IN_MODIFY,IN_CREATE -v -c "rsync -ru hugo/ build"

now to trigger the copying to the build dir automatically when something changes. I'll think of adding something to the detailed README or the wiki for future developers.

(It's not possible to add a target to the Makefile that does this, is it? As it involves keeping two commands running in the background?)

[akoehn]

The code in the Makefile is evaluated using bash, so you can do anything you like.
However, each line is evaluated in its own sub-shell to prevent contamination from the environment. Therefore, you will have to write everything in a single line or escape the newline with a \ (in that case, the newline will be stripped and you need a ; to denote the end of a command).

Example:

foo: bar
    hugo serve & \
    python-notify-stuff

will be just as writing hugo serve & python-notify-stuff on the command line.

[mjpost]

I think we moved into the build directory per my request; I don't understand all the consequences, but I think I'd be open to moving back to @mbollmann's original formulation, so long as we have good Makefile targets for clean, upload, and serve.

[akoehn]

I think that moving everything back is not a good idea. I strongly dislike projects which do not build out of tree; it is much easier to distinguish between source files and generated ones with the current set-up.

Instead, let's have a new target for developing the files as outlined above.

[mjpost]

Anyone here have tips for hugo server? When I run this on OSX, I get an error about too many open files, and I haven't been able to fix this with ulimit -n (which only goes up to 49152 on my machine).

[mjpost]

I've similarly tried sudo launchctl limit maxfiles 500000 1000000, per this document, with no success.