AGS 4 Draft: major engine refactor

Summary

The idea of a big refactor is as following: we theorize a number of components which the engine may be divided into, and then restructure the code into these components. For each component we try to define following:

• its clear purpose and area of responsibility;
• how would its interface and internal implementation would be separated; in other words, what would the component expose to the external users, and what keep private to itself;
• clear and minimal dependencies on other components.

The division may be continued further by picking out parts of one bigger component into smaller ones.

I propose following basic structure:

1. Engine, aka "mainframe".
   • Purpose: creates other parts and connects them together. Startups and shuts down the whole thing.
   • API: getting other component interfaces; general init and shutdown.
   • What's inside: initializes and shutdowns everything; reads backend/system events and passes them as "commands" to the Game in the way it can handle them.
   • Dependencies: System, Utils, Game engine, other components that it starts directly.
2. System.
   • Purpose: provides interaction with the operating system.
   • API: A generic interface for anything that engine or utils might want from OS, which is not provided by a backend lib we're using (such as SDL2).
   • What's inside: OS-dependent implementation.
   • Dependencies: is ONLY allowed to depend on Utils.
3. Utils.
   • Purpose: a collection of utility and helper functionality for anything common that does not have a direct connection to the engine and game logic. In other words, anything that may be used separately.
   • What's inside: data containers, streams, math, data processing, raw graphic manipulation, etc etc.
   • Dependencies: is ONLY allowed to depend on other parts from Utils, and System, if OS interaction is necessary.
4. Game engine.
   • Purpose: complete game behavior.
   • API: start and shutdown the game; advance a logical frame. Wholesome API for issuing commands and changing properties of the game and its objects, that may be called by the engine itself or connected to a script interpretor.
   • What's inside: purely game logic, manipulates other things like graphics or audio using their respective APIs, but should not know or depend on anything about its exact surroundings (what OS, how it's presented on screen, and so on). Receives events from the engine and react to these.
   • Dependencies: System, Utils, Script VM, Graphics renderer, Media api, Filesystem api, else...
5. Game data utils.
   • Purpose: a collection of utilities related to game structures, but not relying on game logic.
   • What's inside: game data serialization, conversion, and similar.
   • Dependencies: only Utils.
6. Script VM.
   • Purpose: runs scripts in a completely isolated enviroment.
   • API: start, interrupt, shutdown script execution. Allows to register "imports" (external functions from the engine or plugins).
   • What's inside: script bytecode execution.
   • Dependencies: only Utils.
7. Graphics renderer.
   • Purpose: rendering a list of commands (texture objects & other effects) onto screen or another "surface".
   • API:
   • What's inside: performs render of textures and effects, taken from a list of commands, which is placed by the external code.
   • Dependencies: only Utils.
8. Media subsystem.
   • Purpose: generic audio & video playback.
   • API:
   • What's inside: plays audio & video data placed inside by the external code.
   • Dependencies: only Utils.

Proposed directory structure:

- system OR platform - handling system-specific operations
- main OR engine - engine mainframe
- script - script vm
- data - game data classes and serialization / conversion utils.
- game - game logic ; aka game engine; may have subdirs for various game parts: room, gui, etc
- gfx OR graphics - renderers, maybe gfx utils too; only related to generic gfx operations, no game logic whatsoever
- audio - audio subsystem; only related to generic sound playback, no game logic whatsoever
- video - video subsystem; only related to generic video playback, no game logic whatsoever
- util - various utilities

NOTE: other utilities may have separate dirs here, e.g. "fonts" (but not game logic - that goes to game subdirs)

Question of Common dir

Historically the Common dir contained the C++ code that may be shared between the Engine and the Editor. As we plan to rewrite Editor and avoid having any C++ code there, we must decide whether to keep Common dir and what to have there.

Supposedly, standalone tools written in C++ might still benefit from sharing a code with the Engine. On another hand, if the Engine's code is already separated into a structure as noted in previous chapter, that may already be enough, and further picking out of this code into some Common dir may become redundant.

Hence the Common dir should go.

---

Game data classes

My experience working with AGS code showed that it may be inconvenient to have everything object-related in one class. For example: we might want to use a list of object's fields and their serialization in common format when writing stand-alone tools. If we use a "full" object class, we'll bring lots of dependencies along.

For this reason, I propose to pick out structs with only object's properties and make them "public", so that anyone could use these. These structs will also be used in the serialization or any processing that requires only knowing object's fields. They will have only minimal methods, like ones that validate the set property values, and have no complex dependencies.

Then come classes that are responsible for the runtime behavior (update, render). These may either inherit the "raw fields struct", or nest one, whatever is more convenient.

For example, say we have a Character class. We pick out a CharacterInfo or CharacterData (etc) struct which contains the serializable fields. Character class would have this struct as a member. This is applicable to any object type.

The data separation may look like the following code:

// Data exclusive to a (base) Entity component
struct EntityData {
   int ent_params;
};

// Data exclusive to a parent GUIControl component
struct GUIControlData {
   int control_params;
};

// Data exclusive to Button type
struct ButtonData {
   int button_params;
};

Having that, the game logic classes may be designed like:

// GUIControl behavior class
class GUIControl {
   EntityData ent; // add basic entity params
   GUIControlData guictrl; // add control params
};

// Button behavior class
class Button : public GUIControl { // inherits all params
    ButtonData but; // add button params
};

Finally, if you need to have a "All button data" struct, e.g. in game compiler tool, or to pass around, you may have:

struct ButtonDataAll {
   EntityData ent;
   GUIControlData guictrl;
   ButtonData but;
};

Initializing a runtime Button class with pure data read from somewhere else:

void Button::SetData(const ButtonDataAll &butdata) {
    // copy over "components"
    ent = butdata.ent;
    guictrl = butdata.guictrl;
    but = butdata.but;
}

---

Ideas on various managers

This is a bit more advanced than refactoring, but I'd put this here for the time being. I've been thinking on how to structure the data loading by the engine. Here's a draft proposal:

• FileManager - maps virtual file location to physical one. Provides read/write streams (maybe uses physfs library).
• LoadManager - threaded job manager that loads data by request; Requests are processed by queue and/or in parallel: synchronously or asynchronously.
• ObjectLoader - helper class, controls loading data for game objects. Issues complex orders for the LoadManager: e.g. "Load View" would mean - cache all sprites and optionally cache frame sounds, "Load Character" would mean load all character's views, and so on.
• ObjectManager - contains ObjectCache (see below) and either contains or links to ObjectLoader. Keeps loaded resources in memory and provides by request; orders ObjectLoader to get one if it's not available. Stores resources in ObjectCache, which is configured in some way.
• ObjectCache - a MRU cache, which stores resources and keeps them ordered according to their recent uses. Has a memory limit; when a new resources is put into the cache, the oldest ones (the ones that were not use longest) are freed until the total size of data in cache is not below limit again.

Then the object data retrieval chain would look like:

Engine -> ObjectManager -> ObjectLoader -> LoadManager -> FileManager
                |
           ObjectCache