Clean deletes folder: dangerous for submodules

[igorsantos07]

I'm implementing a Sculpin site using GitHub pages. It needs the static files in the root of a repository (in case of @username pages), and thus I've set the output_prod folder to be a submodule referencing the actual GHPages repository.

However, running sculpin generate --clean --env=prod deletes the output_prod folder itself, thus deleting the entire submodule tree. That's quite dangerous, while it seems to be the only way to make sure every build will be a clean one (without files that got deleted from previous builds, for instance).

Do you think it would be possible if the --clean command would actually delete folder contents, except for hidden files? Or else, this could be set in a sculpin_kernel.yml config (a "clean_whitelist" option)?

[davedevelopment]

I'm generally agreeable, but I must admit I have deliberately had sculpin create hidden files (.htaccess springs to mind), which I would want to be deleted.

[Incognito]

Is sculplin git-agnostic?

[simensen]

@Incognito Sculpin is indeed git-agnostic.

@davedevelopment Same here.

@igorsantos07 I think that having Sculpin delete the directory itself is probably not necessarily required. However, simply not deleting "output_prod" doesn't seem like that would be safe enough for you as deleting the contents of output_prod would also delete your submodules git metadata, correct?

It seems like the best solution here would be to setup something along the lines of exclude/ignore for source files but apply them to the output directory. output_clean_ignore or clean_ignore seem like good candidates for this.

I don't think that the clean operation currently does anything smart or fancy at all. That code would need to change to actively delete individual files and directories which would likely make the whole operation slower. It could be optimized to just do the quick delete if no ignore rules are specified, though, which would keep it quick for people who currently have no problem running --clean as-is.

[igorsantos07]

100% agreed with you, @simensen :)

From my tests, the only thing that must be kept for the submodule to stay alive is the root's .git* and the folder's .git. The root one specifies git information and .gitmodules defines the state of the submodules; the folder's .git specifies that module's configs. As long as it's there, git knows how to update the folder and so on.

@Incognito it is, that's why I mentioned hidden files in general, or a whitelist :)

[Incognito]

I wonder if the files could be merged with a sort of skeleton directory. Does that suit the need?

[igorsantos07]

I'm sorry @Incognito, I think I didn't follow your idea.

[simensen]

@igorsantos07 Are you going to try and implement this on your own?

[Xerkus]

I believe proper approach is not to have anything volatile in output_*

Here is my build and deploy script. It is a sloppy example, but still might be interesting for you: https://github.com/Xerkus/Xerkus.github.io/blob/18612560ada6549ae2f8be6a44a917872211c542/build.sh

[carbontwelve]

Off topic, but @Xerkus you need to start blogging again ;)