Configurable navmesh cache and auto generation

[Gireen]

Add the possibility to disable or only partial enable navmesh auto generation and caching.

the load priority of navmeshes should be:
navmesh in config folder > navmesh in map or mod dpk > autogenerated navmesh
with the options for excluding or forcing a specific source

Usecase would be for mappers, modders and serveradmins to create or test specific navmeshes like for example hand made ones.

[slipher]

In the navmesh config, which can be customized on a global or per-map basis, we could add a key for disabling generation and/or ignoring config mismatches (e.g. generated with different step size). However on its own, this wouldn't reduce the pain of the navmesh customizer so much as the custom navmeshes would still be rendered unusable by any file format changes. Configuration settings are serialized by just copying C structs into the file so it is hard to make any updates without backwards-incompatibly changing the file format.

We could try to redesign the file header format to use a text-based format or protobuffers or something, which would be more amenable to backwards compatibility. But of course navmeshes created before the file format redesign would still be unusable.

[sweet235]

There are more problems. Once upon a time, we had the navmesh and navcon files in each map package. Then we had a paradigm shift. Now we only have the navcon files in each map package. But this does only work in specific cases.

Both ends of a navcon have to be placed on the navmesh. But when you place these, you cannot tell anymore where the navmesh will be. You might think that you can play this defensively, and place so many navcons that enough of them will hit the navmesh by luck. But this is not possible, the maximum number of navcons seems to be around 128.

You might think: this is a minor problem. It is not. The navcon files are what makes our alien bots attack the top of crates etc. Without them, they are quite blind.

[sweet235]

this wouldn't reduce the pain of the navmesh customizer so much as the custom navmeshes would still be rendered unusable by any file format changes

lol. The only person I know who changed this format is you. Maybe we should not change it until we have a better idea of what we want to do?

[slipher]

There are more problems. Once upon a time, we had the navmesh and navcon files in each map package. Then we had a paradigm shift. Now we only have the navcon files in each map package. But this does only work in specific cases.

Sure, that's one of the things that could be mitigated if we were to realize backwards-compatible navmeshes and disableable autogeneration as proposed in this issue.

The only person I know who changed this format is you.

Last 3 file format version bumps were not made by me as shown by git log -G'NAVMESHSET_VERSION ='. (Although there are other effective version bumps besides those because if the size of the header changes, the code detects that it is incompatible and invalidates the cache even if we forgot to bump the version number. There was at least one change like that.)

[Gireen]

Would it be possible to completely ignore NavMeshSetHeader in the navmesh by relying on default values and navmesh config until there is a proper backwards compatible format?

[illwieckz]

Something I would like to see is a navmesh cache that is not class-based but property-based.

If a given class requires a navmesh with a given step size, then generate and store a navmesh for this step size.
But if a navmesh with this step size already exists, reuse it, even if it was generated for another class or even another mod.

One problem with this way of doing things is that we would lose the ability to purge what is not needed anymore.

[slipher]

Would it be possible to completely ignore NavMeshSetHeader in the navmesh by relying on default values and navmesh config until there is a proper backwards compatible format?

We can't ignore it completely because at least int numTiles;, dtNavMeshParams params;, and dtTileCacheParams cacheParams; are needed to initialize the navmesh. But maybe it's possible to make a hacky multi-version loader. The "params" structs are at the end of the header, so we could exploit headerSize to find the location of those. The numTiles offset would just have to be hard-coded based on version.

Another complication with loading old navmeshes would be that they never had Advanced Granger meshes. You'd need to configure them to substitute Granger somehow.

[Gireen]

I think its not needed to work out of the box with all old navmeshes. Its enough if all new generated navmeshes have a header that can be ignored except for the necessary parts.

Would it be enough to move all needed variables to the start?

Manual generated navmeshes can then adapt this "minimal header" and if needed maybe historic navmeshes could be patched to that state to work with any future version.

[DolceTriade]

That sounds feasible. We can have a "core" minimal header and then future additions can be considered extensions. It's much more complicated, but I agree that requiring recompilation of all navmeshes for every version is undesirable.