Yomi is a C++ 3D game engine project built entirely from scratch, with the intent to learn what goes into the heart of a game engine.
A space game is included in this project which is built using the engine, currently a work in progress.
See the end of this document for a bunch of videos which show off the development progress.
Yomi is portable and currently runs on Windows, Linux, MacOS, and Android. More platforms may be added in the future and the porting process is fairly straightforward (see the cmake build files).
Clone with:
git clone --recurse-submodules https://github.com/Aleksbgbg/YomiYou must have the Vulkan SDK, cmake, and your favourite C and C++ compilers installed on your system for the project to build.
You can build and run the project on desktop systems using these commands from the project's root directory:
cmake -B cmake-build-debug
cmake --build cmake-build-debug -j 8
cd cmake-build-debug/src && ./yomiOn Windows, you can also open the project in Visual Studio and run it.
Alternatively, for any desktop platform, use JetBrains CLion which will automatically figure out building and running for you, as long as you have the dependencies listed above installed.
When building for Android, use Android Studio with the appropriate environment variables configured and cmake version >= 3.20.
Open src/platform/android/java as the root project directory.
If Android Studio defaults to a lower version of cmake, you might need to create a local.properties file in the src/platform/android/java directory that contains this line:
cmake.dir={path to your cmake root directory, either in the Android SDK directory, or a different location}
Ensure you have symbolic links enabled in git using git config --global core.symlinks true before cloning, otherwise the project won't build, since the Android project symlinks back to the top-level cmake project.
Gradle doesn't seem to re-run cmake when resources change, so consider deleting build files and rebuilding if you run into resource issues (or commit a fix for the issue if you know of one). Particularly on the first build, you might need to perform a clean rebuild because gradle doesn't seem to copy the generated shaders into the assets folder on the first try.
Yomi has very few dependencies:
- C++ standard runtime library
- Vulkan (for its rendering backend)
- GoogleTest (for unit tests where appropriate)
- SDL (for cross-platform abstraction and windowing)
- FreeType (for text rasterization)
- glm (for 3D math and feeding data to the GPU in the correct layout)
- stb_vorbis (for decoding vorbis audio files)
glm will definitely be removed in the future, and stb_vorbis could potentially be removed in the future (in favour of a custom OPUS implementation) if time allows. However, the other dependencies would take too much time to implement from scratch.
Many interesting programming problems have been solved during the development of Yomi.
Some of my favourites are listed below, in no particular order.
Yomi has a built-in UI framework which takes large inspiration from Microsoft's WPF framework, using a similar XML syntax, as well as binding syntax, and encouraging use of the MVVM (Model-View-ViewModel) pattern. Everything in the framework is built from scratch, including the XML parser. Calls from the ViewModel can update the view and changes in the view are fed back to the ViewModel.
See the example pause menu in XML syntax, as well as the ViewModel.
When deciding the rendering style of the menu, I went with a near-identical copy of GTA V's pause menu. GTA V is one of my favourite video games, and the GTA franchise has been a big inspiration whilst developing my programming and mathematics skills. I've been impressed for years at the amount of detail and technological marvel in GTA V at launch, and it has been a dream of mine to one day lead the development of a programming project with reach as large as GTA, and use the reach to have a positive effect on its audience. I've never tried implementing anything from GTA, though I've always wanted to see if I'm up to the challenge - so I thought the pause menu would be a good start.
A video of the pause menu working is included at the top of the videos section.
An interesting problem I faced during development was describing the game in an abstract way which isn't coupled to the renderer implementation or the structure of the code. Pondering on how to solve the problem, I looked to Unity's actor, property, and script model. You create actors in the scene, which have different properties such as colliders (which themselves have a bunch of settings), and then you can program how these objects interact with scripts.
I created a similar system, albeit much simpler and written in C++ instead of edited in a UI. You can add actors to the scene, which have properties such as visibility and a transform (position, scale, rotation), and these properties can be modified through 'behaviours' (scripts).
Here is an example of adding an NPC spaceship which moves constantly forward:
scene_.Actor()
.Attach(BEHAVIOUR(ConstantMovement,
ConstantMovement::ParameterPack()
.SetTransform(actor.RetrieveProperty<Transform>())
.SetForwardVelocity(1.0f)))
.Mesh(npcMesh)
.Child(spaceshipExhaust)
.Spawn();BEHAVIOUR is a macro which creates a lambda function that creates instances of behaviours, so that you can spawn an actor composition multiple times. Each time you call .Spawn(), the actor receives new instances of all the behaviours and children set on it with previous calls. The lambda accepts two parameters that you have access to when creating behaviours, actor and parent, which are the current actor and it's parent, respectively. You can use these actor instances to retrieve properties from them that you would like to modify in a script, thus allowing arbitrary programs to be applied to objects in the game.
The ConstantMovement behaviour can have parameters set on it, such as the forward velocity of the object, which makes it reusable. It could also move any actor with a transform, whether it be the current actor, or it's parent. You can see that in the example above, the constant movement is applied to the current actor's transform.
You can see the Scene constructor where the entire game is described using this syntax.
Near the start of the project, I was using bitmap images for textures but noticed that those take up a lot of space. Deciding that at some point I would need to implement PNG images to save space during deployment, I deep dived into implementing a barebones PNG decoder for the textures I was using, based directly off of the PNG specification, the ZLIB specification, and the DEFLATE specification, without any other supplementary material.
The minimal implementation took a week to write as some parts of the specifications were particularly tricky to understand (in particular the encoding format in DEFLATE). The decoder is not particularly efficient, as the decoding algorithm is used directly as described in the DEFLATE specification. Some other implementations like the LiteSpeed Web Server use fast huffman decoding (via more efficient data structures) which drastically speeds up the decoding process. Decoding a PNG texture in the game takes about 1 second in debug, which is unsatisfactory. In the future, I will adjust the implementation to use better data structures to achieve fast huffman decoding.
Keys were required to keep track of resources and actors in a scene, so that resources can be released by key when they run out of scope, and actors can be deleted from the scene graph when they are despawned.
I wrote a key generator which encodes the current time, thread ID, and sequence number (a number which increases on every key generation) into a 64-bit number to use as a key, with the bit widths of these values set so that the generator could run in a loop on multiple threads and still not produce duplicate keys (on modern hardware). The code is well abstracted, so the bit width of the number and encoded values could be varied and optimised for a particular use if necessary. 64-bit was just an easy choice without much optimisation.
The idea is based on Twitter's snowflake format. Although it may be slightly overkill since there is no multi-threaded key generation in the game and separate instances of the generator are used where needed (so clashes are unlikely in the first place), it was fun to implement and play around with this idea. As the game stands, the key could have been implemented just as a monotonically increasing sequence and sufficed.
Since Vulkan is such a low-level API, it requires manual management of GPU memory. Although most projects might use Vulkan Memory Allocator (VMA) due to it's high quality of implementation, I decided I wanted to write my own memory manager for learning purposes.
For simplicity, my memory manager is a growing allocator (no deallocations, only further allocations) which starts off with a memory block of a base size, then allocates additional blocks of 1.5x the size of the previous block if required. It tries to allocate free memory first if possible, before allocating bigger blocks. The memory manager implementation lives here, and is called by Vulkan code from a wrapper here.
The archived networking branch demonstrates the first working trial of a multiplayer implementation of the game, where players can join the same server and fly together. Although the implementation was very naive and unoptimised (and was only implemented on Windows), it worked really well and was very fun to make and play test with friends. In particular, I was surprised to find that playing with my friends in South Korea (very far from the UK!) worked without any noticeable lag! In the future, networking will be implemented again, this time using server time steps and client-side prediction.
I wrote a light object-oriented (RAII) wrapper on top of Vulkan to somewhat isolate my C++ code from the C-style API. One of the main issues I wanted to address was filling out structs in C-style - I wanted to be able to fill in struct parameters inline, in the same statement as my Vulkan calls, and also sometimes modify these parameters as they pass through the call stack of my Vulkan wrapper. To do this, I used the builder pattern, creating a builder class for every Vulkan struct I use, which has methods setting each of the struct's values. I took extra care to ensure the structs were composable, so that nested structs could be set in one full, readable statement. This was an excellent solution, resulting in some very readable code when creating Vulkan objects.
However, creating over 70 individual structs manually seemed too time consuming. Instead, I wrote a small Python script, which given a Vulkan C-style struct definition straight from the specification, generates the relevant builder class. It worked great and as a result I have very readable Vulkan structs.
All of the structs are in this directory. The structs are generated with macros (defined in this file) to allow me to modify the bodies of all the builders at the same time. A bonus advantage of these structs is that template objects can be built ahead of time, then used with a builder and have only some properties customised. For an example, see this file which contains all of my frequently used template objects, and an example of them being consumed by a builder and then modified further.
Although this project aims to be a complete implementation of a game engine from scratch (within reason), two main problems remain unaddressed at the time - multithreading and CPU memory management.
Although the game does run multiple threads (one for event processing and one for rendering, as well as additional threads for networking when that is implemented), there is no overarching multithreaded design where calculations for actors in the game can be run in parallel or GPU command buffers can be generated in parallel.
Although this is an interesting problem to solve, it would require deep design thought and a very time-consuming implementation with many cross-cutting concerns, whilst at the same time not being necessary due to the simplicity of the implemented game.
Still, there is space to solve this problem in the future should the game become inefficient when running single-threaded.
On a related note, heavy computations such as the particle exhaust from a spaceship are run multithreaded on the GPU via compute shaders, so no heavy processing is currently done by the CPU.
Although the project contains a GPU memory manager, there is no CPU memory manager which optimises memory allocations and deallocations for the engine. Everything is allocated through the C++ standard runtime library. Surprisingly, even without optimising this process, memory allocations are pretty fast and my attempt at writing a CPU memory manager was slower than the standard library, so there is no reason to spend time solving this problem at the moment.
As the project was developed, I learnt more about C++, which evolved my programming conventions. As a result, the style in the project has changed several times, so some files may be inconsistent with others. I use the most appropriate conventions based on my latest knowledge when I add new code, but porting all the old code each time I learn something new would be too time consuming. I mostly refactor code only when I am editing it for other reasons in the first place, so generally files edited a long time ago will be out of sync with the latest conventions.
Branches in this project follow a defined naming scheme.
The main branch contains the latest code that builds and works as intended.
wip/ branches are used for working on multiple features simultaneously if required, and will eventually be merged into main.
archived/ branches contain functionality of historical interest. They are used to keep features that will be reworked and included in the game in the future or features that won't make it into the final version of the game.
obsolete/ branches contain abandoned code for reference. The code may be reworked and included in the game in the future, but for the time being isn't worth working on.
Here are some videos of the space game in action. Videos at the top are the most recent. Some videos have sound, but lower your headphone volume as they can be quite loud.