Moving game data definitons to data files

[BottledByte]

Continuation of discussion from #665 about de-hardcoding game data into data files.

---

TLDR:

1. I want to move as much as possible game data definitions out from Java code.
2. It must be some common format, not a custom one.
3. I don't want to reinvent the wheel, writing custom parsers/serializers.
4. I don't want to pull in large and/or unnecessary dependencies.
5. As a programmer, I want simple API to work with stored data.

---

[Zireael07]

As the most recent comment says, JSON is widely used for that purpose. (Cata:DDA is indeed a great example)

[BottledByte]

For idea how game data in JSON would look like, here is a sample of de-hardcoding SpeechFactory.
This is a snippet from file for SPACE_PIRATES speech:

[
  {
    "kind": "TRADE_EMBARGO_REALM_CHOICE",
    "texts": []
  },
  {
    "kind": "BORDER_WARS",
    "texts": [
      "Yarr, scurvy fleet %1$s spotted. Time to die!"
    ]
  },
  {
    "kind": "ASK_PROTECTION",
    "texts": [
      "Are you interested in insurance? That's against pirates!",
      "Ohoy, looks like you have nice fleet here. Not to lose it will cost for you.",
      "Yarr! It would be a shame to scratch your fleet, but that can be avoid by paying this."
    ]
  }
]

I believe the sample is quite obvious, but here is explanation anyway:
The file is an unnamed JSON array, which contains variable number of objects. The objects have two attributes:

1. kind - SpeechType enum value, just encoded as string. It says what speech kind the object describes.
2. texts - Array of 0 or more Strings, each being one speech line that is available for that kind of speech. Game can select one of the lines at random, as needed. If the texts contains empty field or null, it could fallback to default

I made a functional prototype of this. It works nicely and although it might look better, I think it serves it's purpose well. The TRADE_EMBARGO_REALM_CHOICE value technically should not be there 🙂. Using specially written code, I attempted to extract data from the game at runtime and serialize it, and value for this key is generated at game time, therefore it is empty. But even with this oversight, the prototype is working without problems.

I think that de-harcoding data to any common, flexible data format is a plus, because the data then can be easily transformed, edited and/or visualized using conventional tools. It is much easier to migrate data to other scheme or format, if it is actual data and not Java source file. If the need for different form of data arises, both in terms of object attributes/composition and stored representation, such conversion can be done way easier than now. (And it would prevent such "bugs" of including generated fields or the like, like mentioned above 🙂 )

[tuomount]

I think there should be an issue for each data definition and first needs to define how to store data in JSON. For example in speech there could root level Speech, under it each space race and under each spacerace the actual lines. And yes I like that variation.

So obvious ones are factories: ShipHullFactory, ShipComponentFactory, ArtifactFactory, TechFactory, BuildingFactory
Then not factories, these might be a bit more trickier to implement: SpaceRace, Government, PlanetTypes, WorldType, PlanetTypes, LeaderPerks

I would not try convert all of these at same time, since it would cause massive amount of testing. Plus I am also working with new ascension victory where I need to touch a bit there and here in code. I have been rebasing quite often so I am still good.

[BottledByte]

For example in speech there could root level Speech, under it each space race and under each spacerace the actual lines.

That is not a good idea.
With it, you bound SpeechSet (that's how I call the class and it's data) to a SpaceRace. I do this inside the factory (which has now only like 150 lines 😁 ). At data files level, I have just bunch of JSONs in the format described above (see picture below). Therefore, I could use "human.json" easily as speech for Homarians, if I want to.

And not just that! There could be even more SpeechSets for a single SpaceRace and the code could choose which one to use based on other conditions, like GovernmentType of the realm. So Human Empire could have different narrative than Human Utopia, for example. That is, assuming the overall design would be tweaked.

I would submit a PR with these changes. As you did not gave me a green light to use external JSON library, I am not submitting it AS-IS. (I did not had the time to rewrite it to use the in-house JSON parser).

these might be a bit more trickier to implement: SpaceRace

The SpaceRace enum is one of the most sprawling design issue I have encountered so far. I already tried to redesign it's internal workings, but failed miserably due to the spaghetti of code that is attached to that all-ruling enum 😢 . The SpaceRace will indeed be one of the last things to dehardcode, unfortunately.

However, of the factories you mentioned, TechFactory and ArtifactFactory are likely my next targets, along with redesign of Tech, to increase modularity while at it. That is, after getting that SpeechFactory done (there is some polish I have to do and that JSON library thing...).

---

I want to kill two birds with one stone. Not just to de-hardcode data. I want the game to be (optionally) modular, which calls for redesign of internal code. Such changes would massively increase potential content of gameplays. And as a player, I want that a lot. ❤️ And as programmer, it makes things way more easier to work with. Component, data-driven designs are way more superior for game development, exactly due to the combination possibilities it can provide.

Also it is the reason why I submitted all the refactors so far. I need code that can be safely refactored and easily understood. Currently, that's not the case. And to clarify my intents, any redesign I speak about is on internal, code level. I am trying to keep the gameplay exactly the same, without breaking or removing anything. Which is hard 🙁

---

I am also working with new ascension victory where I need to touch a bit there and here in code.

Yes, I am aware of this. I hope you will finalize it soon 🤞 , since there are many changes needed to be able to close this issue 🙃 . Also, I highly recommend postpone all new features for the game "indefinitely" (bad name for a milestone, I know 🙂 ). Each game "subsystem" that will get cleaned-up and dehardcoded before adding new feature will mean less work overall. And as this project does not have deadlines or dependents, the need for new features is mostly driven by you yourself 😉

[tuomount]

I am actually thinking that if 0.25.0 release should be done without Ascension victory. Current version has highly improved map drawing which increases performance on lower pcs plus other improvements. Then there would be just testing that all the features work.

After this internal redesign could be started more safely. Also that new JSON-in-Java could be bring in. These redesign could be started on separate branch and later merge that to master.

Yes, I agree your plan sounds really good.

[BottledByte]

I am actually thinking that if 0.25.0 release should be done without Ascension victory. Current version has highly improved map drawing which increases performance on lower pcs plus other improvements. Then there would be just testing that all the features work.

Then all the better. 👍 Any feature that does not exist in the code is one less thing to worry about and/or break during redesign/refactor. 🚀
Plus as the release process was also changed, I think it is a good idea to try it soon. (and adjust accordingly in case of issues).

These redesign could be started on separate branch and later merge that to master.

I highly recommend keeping the redesign/refactor work on master .

1. It simplifies workflow.
2. No other development should occur during redesign, maybe except bugfixing. Having massive "redesign" branch does not make sense overall, nor it's continual merging into master.
3. In case of bugfixing 0.25.0, branch should be created from the tag and 0.25.1 released from such branch. Patches could be backported to master, if applicable.

---

Anyway, should I wait for you releasing 0.25.0 before submitting further cleanup PRs? I have two more that I can submit, if you want to add them to 0.25.0, although I think it does not worth it now.

I would suggest that you focus on getting the release out of the door, while I will continue experimenting with new designs locally. And after the release some serious cleanup/refactor/redesign can begin! 😁

[tuomount]

Redesign/refactor is easier on master but if you have lot's merge requests waiting, these could be place on separate branch. But I agree it is easier keep master where features/redesign/refactors are merged.

I check all the current issues which are open there are still two perks missing, but these could be also moved to 0.26.0 and then there is the feature you suggested that governor should not choose for next building project. After this there is testing left. I would like to play at least one full game, and hopefully it can played without no issues. If that's the case I think next release can be done.

I think maybe it is better to wait for cleanup PRs, less things to test and go wrong...

[tuomount]

Lot's of data moved into JSON. Closing.