create_hugo_yaml.py should clean up output directory

[davidweichiang]

like its sister scripts do. Or was this left out on purpose? This caused a build problem in #285.

[mbollmann]

I think I just saw that as unnecessary since files would only ever be added or overwritten, never removed. Of course, the file structure looked a bit simpler at that time. Also, they are written to Hugo's main data directory where, in theory, we might add other files manually or via other scripts. If we establish that we won't ever do that, we can have the script clean it out, of course.

That said, I'm not sure how this behavior could have caused a build problem?

[davidweichiang]

That PR was an unusual case where one file (J79.yaml) was removed.

[mjpost]

I would also suggest either:

• Having a "clean-only" mode, a script that will clean up everything
• Writing all generated files (particularly from the Python scripts) to a separate build dir (which would make cleanup easy)

[mbollmann]

• Writing all generated files (particularly from the Python scripts) to a separate build dir (which would make cleanup easy)

Not fully possible, as generated pages must go to hugo/content, where other (non-generated) page content also lives.

I'm not sure if Hugo's configuration allows nested directories; if so, hugo/data and hugo/data-export could be subsumed under hugo/gen/{data,data-export}, but I'm not sure that changes much tbh.

[akoehn]

You could copy the non generated files to the output directory when building the site. They could reside in hugo-src or similar.

[akoehn]

As I really enjoy writing makefiles, I could write one to automate the build process into an out of tree target. It also seems cleaner to me to have a main Makefile and people can clone the repository, run make, and have an up-to-date anthology site.

Are you interested? I won't promise a date as I would view it as a relaxing activity in my spare time.

[mbollmann]

What would be the advantage of that over, say, extending the current bin/build_hugo? At least for full, first-time generation, the script should do exactly that. It could be nice to be able to clean or specify only some of the targets (e.g., to skip bib-file generation), but is there more that would be easier/better with a Makefile?

(I'm all for letting you do it, I mainly wonder whether it'll be harder to maintain compared to a plain bash script.)

[akoehn]

A simple Makefile is essentially a plain bash script. The advantage is: 1) make clean / make is standard, so it should be obvious to more people what to do, 2) make can do dependency tracking. Depending on the sophistication of the Makefile, re-generating the website when only some of the data is modified could be sped up.

[mjpost]

I'd thought about adding a Makefile myself, so I think this would be great. At the very least, typing make is slightly easier than ./bin/build_hugo. And having a clean target would be nice. Finally, here is the command I usually run for publishing changes (which would fit naturally in a sync target).

./bin/build_hugo && rsync -azve ssh --delete hugo/public/ aclweb:anthology-static

[mjpost]

A Makefile is also a sort of universal cue to Unix folks about how things are laid out.

[mjpost]

We also have a few directories with a .keep file in them, presumably to help with builds. With a Makefile, maybe we could remove these from the repo.

[mjpost]

I’m hoping @akoehn will find time to add that Makefile. We could reopen this or create a new issue.

[akoehn]

Didn't forget it, just did not have time yet to build more than the minimal makefile which essentially moves the previous build script into a Makefile target. I will create a pull request when there is something more interesting.