Three + ECS #16
Replies: 2 comments 8 replies
-
|
Great write-up! I have many thoughts here, but let me begin by supplying a sixth option: Option 6: ProxiesI still haven't fully tested this idea yet, but I don't see why this wouldn't be possible. Essentially, custom proxies could be created (not ES6 E.g. abstract class BaseProxy {
protected store: IComponent
protected eid: number
constructor(store, eid) {
this.store = store
this.eid = eid
}
load (eid) { this.eid = eid }
}
class Vector3Proxy extends BaseProxy {
constructor(store, eid) { super(store, eid) }
get x () { return this.store[this.eid][0] } // or this.store.x[this.eid]
get y () { return this.store[this.eid][1] }
get z () { return this.store[this.eid][2] }
set x (n) { this.store[this.eid][0] = n }
set y (n) { this.store[this.eid][1] = n }
set z (n) { this.store[this.eid][2] = n }
}
class QuaternionProxy extends BaseProxy {
constructor(store, eid) { super(store, eid) }
get x () { return this.store[this.eid][0] }
get y () { return this.store[this.eid][1] }
get z () { return this.store[this.eid][2] }
get w () { return this.store[this.eid][3] }
set x (n) { this.store[this.eid][0] = n }
set y (n) { this.store[this.eid][1] = n }
set z (n) { this.store[this.eid][2] = n }
set w (n) { this.store[this.eid][3] = n }
}
class Object3DProxy extends THREE.Object3D { // extending probably won't work but this can be worked around
position: Vector3Proxy
quaternion: QuaternionProxy
scale: Vector3Proxy
constructor(store, eid) {
this.position = new Vector3Proxy(store.position, eid)
this.quaternion = new QuaternionProxy(store.rotation, eid)
this.scale = new Vector3Proxy(store.scale, eid)
}
load (eid) {
this.position.load(eid)
this.rotation.load(eid)
this.scale.load(eid)
}
}Usage: const TransformComponent = defineComponent({
position: Vector3Schema,
rotation: QuaternionSchema,
scale: Vector3Schema,
})
const e = addEntity(world)
const obj3d = new Object3DProxy(TransformComponent, e)
obj3d.position.x = 2.3
console.log(TransformComponent.position.x[e]) // => 2.3
const v = new THREE.Vector3(1, 2, 3)
obj3d.position.copy(v)
console.log(TransformComponent.position.x[e]) // => 1
console.log(TransformComponent.position.y[e]) // => 2
console.log(TransformComponent.position.z[e]) // => 3Due to Data is now available in both formats: objects, and SoA. Best of both worlds. Use the Three.js API by default, and then switch to SoA syntax when (re)writing the hot paths of code where needed. Incremental and selective optimizations. |
Beta Was this translation helpful? Give feedback.
-
|
Just thought of another advantage, sort of off-topic but related: proxies can be used to manage memory in different ways without interfering with the object interface itself. E.g. class TightlyPackedVector3Proxy extends BaseProxy {
constructor(store, eid) { super(store, eid) }
get x () { return this.store[this.eid][0] }
get y () { return this.store[this.eid][1] }
get z () { return this.store[this.eid][2] }
set x (n) { this.store[this.eid][0] = n }
set y (n) { this.store[this.eid][1] = n }
set z (n) { this.store[this.eid][2] = n }
}
class UnpackedVector3Proxy extends BaseProxy {
constructor(store, eid) { super(store, eid) }
get x () { return this.store.x[this.eid] }
get y () { return this.store.y[this.eid] }
get z () { return this.store.z[this.eid] }
set x (n) { this.store.x[this.eid] = n }
set y (n) { this.store.y[this.eid] = n }
set z (n) { this.store.z[this.eid] = n }
} |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for sharing all of this. I've thought about this a little bit more. Maybe there's a middle ground here that'll work well for the three.js API (which I expect most user-defined systems to use). Maybe instead of proxies, there's a dedicated Object3DEntity object for each Object3D type. We preallocate a bunch of them so that there's less memory allocation. These classes will act like the proxies in that they'll reference data in the component stores by overriding various properties on the object 3D classes. They'll work with the Three.js renderer, raycasting, animations, etc. because they'll conform to the same API. The biggest difference between this and using a couple proxy objects is that they will represent a scene graph that the three.js renderer can use directly. It's a lot of work to do this, but by starting with a few properties like BitECS backed transforms and working our way outwards, we can do this iteratively. @antpb BMO and I were brainstorming some things around this last night. We can convert between the output of the Three.js GLTFLoader to these new Object3DEntity classes such that the whole scene graph uses these. I like the ability to then query for just specific components like |
Beta Was this translation helpful? Give feedback.
-
|
@robertlong here's a quick and dirty POC i created to demonstrate the concept of proxying the https://github.com/NateTheGreatt/bitECS-threejs-poc/blob/master/src/index.ts The gist: mesh = new THREE.Mesh( geometry, material )
const world = createWorld()
const e = addEntity(world)
Object.defineProperty(mesh.rotation, 'eid', { get: () => e })
Object.defineProperty(mesh.rotation, 'store', { get: () => TransformComponent.rotation })
Object.defineProperty(mesh.rotation, '_x', {
get () { return this.store.x[this.eid] },
set (n) { this.store.x[this.eid] = n }
})
Object.defineProperty(mesh.rotation, '_y', {
get () { return this.store.y[this.eid] },
set (n) { this.store.y[this.eid] = n }
})
Object.defineProperty(mesh.rotation, '_z', {
get () { return this.store.z[this.eid] },
set (n) { this.store.z[this.eid] = n }
})function animate() {
requestAnimationFrame( animate )
mesh.rotation.x += 0.005
TransformComponent.rotation.y[0] += 0.01
mesh.rotation._onChangeCallback()
renderer.render( scene, camera )
}as you can see, you can modify either the rotation object or the bitECS component, the data is rendered correctly either way. the only caveat being that if you mutate the bitECS component data you need to manually call
|
Beta Was this translation helpful? Give feedback.
-
|
after pondering this a bit more, i think the above method might be the right way forward. it's unobtrusive, doesn't require any changes to Three, and yet has Three treating the ECS state as the single source of truth for anything via the proxy getters/setters. the dedicated |
Beta Was this translation helpful? Give feedback.
-
|
i organized and updated the repository to reflect my thoughts here's a sandbox of it in action (thx @SupremeTechnopriest 🍻) the movement system ended up pretty clean minus the export const movementSystem = (world: World) => {
const ents = movementQuery(world)
for (let i = 0; i < ents.length; i++) {
const e = ents[i]
const obj3d = world.objects.get(e)
TransformComponent.rotation.x[e] += 0.001
TransformComponent.rotation.y[e] += 0.03
TransformComponent.rotation.z[e] += 0.005
obj3d!.rotation._onChangeCallback()
TransformComponent.position.x[e] += Math.sin(world.time.elapsed/1000) / 10 * world.time.delta
TransformComponent.position.y[e] += Math.cos(world.time.elapsed/1000) / 10 * world.time.delta
TransformComponent.position.z[e] += Math.cos(world.time.elapsed/1000) / 10 * world.time.delta
}
return world
}the render system is laughably simple now 😆 export const renderSystem = (world: World) => {
world.renderer.render(world.scene,world.camera)
return world
} |
Beta Was this translation helpful? Give feedback.
-
|
This work is now happening in the Web-ECS organization. If you'd like to get involved, go there to check out the various repositories! |
Beta Was this translation helpful? Give feedback.
-
Three + ECS
I've worked on a few different attempts to create an ECS library for Three.js. I've also worked extensively with AFrame. There are a lot of lessons to be learned from each attempt and I'd like to do my best to summarize them here as well as brainstorm how to build a better solution.
AFrame
AFrame's ECS system is not "pure" in the sense that components are not just data, they can have behaviors, and entities are not just identifiers, they have data and are backed by a DOM node. The systems are also a bit different in that they are just essentially singleton components that you can use to process other components.
Things to Avoid
I think AFrame is a great library and has really laid the foundation for WebXR development. It's still a great framework today, but it does have quirks and performance issues that are hard to avoid.
1. Unpredictable Component Execution Ordering
AFrame does not execute components in a consistent and easily predictable order. They are executed as the scene graph is traversed and in the order that they are added. Adding, removing, and adding a component can have a different behavior since it executes in a different order. This introduces a fair number of bugs that can be hard to track down. There are many different actions whose side effects can result in component execution ordering changing.
2. Two Scene Graphs
AFrame relies on the DOM to construct its scene graph. In more recent versions it avoids mutating the DOM in most situations except adding/removing entities and components so it's mostly performant (more on that later). The real complexity that this adds is in managing two interconnected scene graphs. You have the AFrame scene graph and then you have the ThreeJS scene graph. The AFrame scene graph is represented by entities which are both DOM nodes and a ThreeJS Object3D. However, this Object3D can also have children. In fact, adding components like light, or geometry also add child Object3Ds which are not represented with entities.
This becomes more complicated when working with imported models. When you import a glTF, you may want to add behavior to some of those objects. However, you can't do that via standard AFrame components because the model's hierarchy is not made up of Aframe's entities, but Object3Ds which do not accept components.
3. DOM Performance
AFrame's scene graph uses DOM nodes. Adding/removing DOM nodes is relatively fast, but big changes to the DOM hierarchy can cause performance problems. This is particularly evident when loading scenes.
4.
setAttributeAPIAFrame also uses the
el.setAttributeAPI to set component properties. This method does a lot of work. It can parse component properties from a string to an object. It deals with multiple components of the same type. It has to deal with mixins and components. You can add components or update components via this method. It also has to clone component properties as a result of calling this method. It's a lot of work and it's a core API. The workaround is to not use this method. Instead set properties likeel.positiondirectly. It's weird to sidestep the core API, but you kind of need to for things that are called every frame.5. Duplicate Object 3D Performance
Because AFrame represents it's scene graph as entities, a scene graph with 3 three.js
Meshobjects will have at minimum 6Object3Ds. This is due to entities needing to have their ownObject3Dcontainer. As a result, an entity can have both aLightand aMesh, but the tradeoff is that you have moreObject3Ds and this makes updating matrices and traversing the scene graph slower.Key Takeaways
I think there are also a lot of good things to take away from AFrame.
1. ECS + Three
AFrame proves that ECS + ThreeJS can be a very productive workflow. When building with AFrame you can easily build scenes via composing entities out of components.
2. HTML Syntax
The HTML syntax makes it fairly familiar for web developers to get started, but it kinda gets in the way after a while. I think a higher level scene composition language is needed and AFrame showed us one possibility for what it could look like.
3. Input System
AFrame has built-in controls for a number of different devices and this was all done pre-WebXR. Making WebXR experiences with AFrame control components was easy.
4. Link Element
AFrame came with a link component before most other 3D web frameworks did. This lets you navigate to another experience and is core to building out metaverse experiences.
5. Library Ecosystem
AFrame has a couple key libraries that make it a powerful engine that would take a lot longer to set up any other way. networked-aframe, aframe-physics-system, and superhands being some of the most notable ones. ThreeJS definitely has some libraries, but these libraries unlock a whole lot of power with very little effort and I think ThreeJS still lacks some of that.
After AFrame
I think AFrame is still the most mature and successful ECS framework for ThreeJS. Even with it's flaws it does a pretty good job. During my time at Mozilla we created a couple prototypes for a simpler ECS library for Three.js
ECSY was the main one, but I'd also like to spend some time talking about two other projects that helped shape my view of Three.js and ECS and those are Mozilla Spoke and bitECS.
ECSY
Fernando Serrano, myself, and others collaborated on this ECS library. It had a lightweight core that didn't include a library like Three.js or Babylon. You'd then write systems and components for those specific libraries. I may be biased, but I think it was a pretty good library!
I think there are a few key takeaways from this project.
1. Javascript performance is first and foremost about memory management
Javascript has a garbage collector. When you create objects the language tracks your usage of them and when you aren't using them anymore it throws them away. In most webapps you are calling methods that create or dispose of some memory in response to a user action. However, in browser-based games you are running code every animation frame. Any function that allocates memory during every frame will add up pretty quickly. This means that every so many frames you might experience what is called a major GC event. This is where the garbage collector needs to clean up unused memory over a certain threshold. And that takes time. Not a ton of time, it would likely be imperceptible in most web apps, but when you are running animations, you'll likely see a hitch.
ECSY was designed to help make memory management easier. It pre-allocated components and kept them in object pools. Then when you went to create a new component, it would grab one that was already allocated in the pool. Only when you resized the pool did memory allocation or collection happen. This meant that adding/removing components every frame was basically free! You didn't need to worry about GC pauses and this enabled new ECS patterns that would otherwise be expensive.
2. Removing components is hard
Removing components from an entity is pretty easy, you remove a reference from an array or a bitmask. It's not too difficult. What is actually hard about removing components is managing what queries do in response to removing components. Do you remove components immediately or at the end of the system or at the end of the frame? If you do remove them immediately, how do you tell what components were removed earlier in the frame? If you defer removal, how do systems that run after the component was removed respond to the removed components? There's no one right answer and each has it's own quirks. I think the important point is to pick one and make sure it is 100% predictable and easy to understand. I think our inclusion of the
entity.hasRemovedComponent(),entity.getRemovedComponent(), and immediate remove flag inentity.removeComponent()were code smells. We probably could have come up with a simpler design that didn't need these extra methods to deal with removed components.3. Bolting a "pure" ECS library onto an object oriented rendering library is hard
Building ECSY was a lot more straightforward than coming up with patterns to work with it and Three.js. At least it was at first. Getting used to a new programming pattern is hard! Mixing in something that doesn't comply to your new programing pattern is even harder!
This was most evident when trying to come up with how to handle Object3D transforms withing ecsy-three. Do we make Object3D's entities? That wouldn't make it a "pure" ECS, because entities then aren't just identifiers. Or maybe we come up with "pure" components for transforms and then apply those transforms back onto the Object3D? This led us down a rabbit hole of figuring out how to translate three.js' scene hierarchy into ECSY's ECS model.
What we didn't understand at the time was that we were headed back towards the "two scene graph" pitfall from AFrame. Luckily, we eventually backtracked and realized that we should keep the Object3D scene graph in place and treat it as our single source of truth. Rather than try to shift more of that data into ECS, we'd make it easier to query for in systems. Entities with Object3Ds would get tag components which would tell you what type of Object3D you were querying for.
This also has its challenges though. What happens when you remove an Object3D from the scene graph, does it clean up its associated entity? How should we create different types of Object3D entities in the first place? We ended up creating some helper functions for these operations and that worked relatively well.
4. It helps to be building a project using your ECS alongside your ECS
For a while we were building ECSY without actually using ECSY on a real project. It was dream code. We all wanted to use it, but couldn't because all our projects were using legacy frameworks. Fernando eventually ported Hello WebXR and a few other projects which gave him some real world experience with it. I worked on demos for ecsy-three in my spare time. I think if we had gone further with ECSY, we would need a larger project to test it on.
Mozilla Spoke
Spoke is the scene composition tool for Mozilla Hubs that I worked on for a large chunk of my time at Mozilla. I had been bitten enough by AFrame when I started on the project that I decided to try my best to stick to a traditional object oriented Three.js project. Before this, I hadn't truly built a Three.js project where I made full use of extending the built in objects.
Here's what I learned from working on Spoke:
1. The "happy path" for Three.js is extending built-in objects
In Spoke, all of the "nodes" surfaced to the creator are Object3D classes like
MeshorAmbientLightthat have a mixin applied to them. This mixin adds some common functionality to each "node" like lifecycle methods including one that is called every frame, when loading an object from a .spoke file, when serializing to a .spoke file, or when exporting a .gltf file. Then I would apply additional custom behaviors on top ofMesh+NodeMixinclass like adding a animation mixer. Few are probably aware of this, but Spoke has an experimental "preview" mode which lets you preview your scene's animations and sounds before you publish them.On top of the ability to define lifecycle methods, I found one of the best capabilities was being able to override methods like
object3D.copyorobject3D.clone()which allowed me to implement features like copy/paste and the undo/redo system. I piggybacked off of these methods to make all of my custom "nodes" able to be copied/pasted and their actions able to be undone.2. OO is cool, but you still want systems
In most game projects you're going to end up with a feature that requires some code that manages a set of game objects. You can call it a "manager" or in ECS we call this a "system". Whatever you call it, every game is going to have a bunch of these. In Spoke, my systems were for rendering, managing input, moving the camera around the scene, and selecting/manipulating objects. These systems were not written in classes extending from Object3D, they managed a set of objects or traversed an object, but were otherwise just plain ol' Javascript classes.
3. OO is cool, but you still want components
After a while, Spoke nodes started to have some features in common like playing audio, or playing an animation, or loading a glTF model. I really wanted to create a reusable class for these behaviors but it was pretty difficult in some cases because they differed in ever so slight ways. I think if I had an ECS at my disposal, I could have written these as components that I could add to my Object3Ds.
Also, as we came up with more and more features for Hubs, it became a chore to keep these nodes up to date with our Hubs components. It would be cool for Spoke to have a components system to manage these bits of data that are just need to be attached to an object and exported into Hubs where they have systems that give these components behavior.
I think Spoke really could have benefited from it's own behavior system. Maybe instead of implementing all of these individual "nodes" I could have wrote the node mixin and included a behavior system that would have let me compose components to create new node types. It wouldn't be a "pure" ECS, but it'd resemble something more like AFrame. Only it would avoid some of AFrame's pitfalls discussed above.
bitECS
bitECS is the newest project I've gotten invested in. It's created by @NateTheGreatt and it's a great little Javascript ECS library. Currently Thirdroom is written against bitECS and aside from some utility functions I created to work with Three.js and external component data, I basically use it as-is.
Here are my takeaways from bitECS:
1. Javascript performance is first and foremost about memory management
bitECS tries to preallocate all of its memory. It does this by defining it's components up front via schemas which are stored in typed arrays. If you can't fit your component in a typed array, you need to find your own way of storing that data. bitECS focuses on typed array component storage because it can use them to gain performance from data locality. This refers to component properties that are stored sequentially in memory which are loaded in chunks. Loading a chunk or page of memory from your RAM into one of your CPU's caches takes some time, but if you can use multiple pieces of information from the same page of memory then you spend less time waiting for new pages to stream into your CPU's cache. So, not only do we improve performance by reducing garbage collection and preallocating all of our memory up front, we reduce the time it takes to retrieve and use that memory. If you do this well, good Javascript can run just about as fast as good WebAssembly or Native code.
2. Bolting a "pure" ECS library onto an object oriented rendering library is hard
bitECS is even more strict about its ECS pattern than ECSY was. You can really only store numbers or arrays of numbers in components. Technically everything happening in your computer is just dealing with numbers, but when we're working with Three.js in Javascript, sometimes we want to refer to components with objects. I've tried a couple methods of working with Three.js and bitECS. The first way was storing Three.js objects in a javascript
Mapwhere the key is the entity id and the value is whatever the component data was. This worked pretty well, but I still felt like I was going against the grain of the library. Adding a component using the bitECS API didn't add any new object to the map. It also didn't add any new values to it's component data arrays when using a standard bitECS component, but at least it would zero out that data when you removed the component. Also, I was allocating objects again, and not pooling objects when I could be. In my next attempt I tried to address these issues in a more robust set of utility functions. However, I realized just how many of these I'd need to cover the gamut of the Three.js API when I saw just how many PhotonStorm, the creator of Phaser, was using in their implementation of Phaser 4.3. Performance isn't everything and "pure" ECS is the enemy of a working game
When working on ECSY we sacrificed some performance here and there to get the API we wanted. We figured, we wanted something we could actually work well with instead of something that was fast but hard to use. bitECS takes the opposite approach. It has a pretty simple API, don't get me wrong. But it is designed to work with only its own component data. You can take bitECS and wrap it in whatever you want though to make it less pure. You won't benefit from its data locality performance benefits but you can still benefit from its great query engine.
It's also still pretty hard to work with when using it with Three.js and it feels like reinventing the wheel when I'm writing all of these utility functions to do basic Three.js things in a new way. These things aren't getting any faster either because they are wrapped up in legacy code. What I do get though is the composability of components and systems and that eventually has boosted my productivity quite a bit. I feel like if I wasn't spending quite as much time figuring out how to work with and keep track of external components, I could have an excellent time with bitECS + Three.js
What's Next?
I think Thirdroom needs to be approachable for Three.js developers and burying the Three.js API beneath a layer of abstraction around bitECS is a bad idea.
I think bitECS has an excellent query system though that'd be hard to beat.
I think supporting more complex component data types like we did with ECSY and AFrame was a good idea.
I think that I was most productive of all when working with OO code in Spoke.
I think the 3D web ecosystem needs the modularity that ECS can provide.
So how do we consolidate all of these learnings into a workable foundation for ECS + Three.js in Thirdroom?
I think we need to build yet another ECS library.
bitECS + Three.js Take 3
I really like bitECS, but I'm starting to grow tired of "pure ECS" again. If I had all the time in the world, I'd write a ECS-based WebGPU rendering engine for bitECS, but I don't. I'm working on Thirdroom in my free time these days and I don't think it's reasonable to rebuild over a decade of work on a massively successful 3D rendering library just so I can use a relatively new and possibly overhyped programming pattern.
So first and foremost I need to make Three.js work with this new library because I'm going to spend time building an app/game instead of a rendering engine. Likewise, I don't actually want to write another ECS engine, so I'm going to use bitECS to make this one.
Option 1: Make Object3D's Entities
Object3Ds are now the entity class and support adding/removing components to them.
Option 1a: Monkeypatch Object3D entities
This is brittle and likely won't work with external libraries especially with isolated modules being a thing.
Option 1b: Fork three.js to add this change to Object3D
This requires us to maintain this fork of Three.js and makes libraries using Three.js largely incompatible with our fork.
Option 1c: Get ECS into Three.js core
This would be really hard unless the whole Three.js ecosystem supports the change.
Option 2: Make an Object3D Entity Mixin
Turn any Object3D class into an Entity class with this mixin. This is cool, but it makes it hard to integrate into things like the GLTFLoader or anything else that instantiates Object3D classes.
Option 3: Make Object3D's Components
This solves all the issues with figuring out how to instantiate Object3D entities, but now we're back to the component/entity lifecycle problem.
Option 4: Wrap bitECS to use Object3Ds as entities
Object3Ds become the entityId which also doesn't solve the entity/component lifecycle problem so much.
Option 5: Make an Object3D entity container class
This class would hold a reference to an optional Object3D, the entity, and the world. It could have all the methods for dealing with Object3D / entity / component lifecycles. It also can be applied to existing Object3D objects, not requiring them to be instantiated as the class with the mixin applied. This starts to smell like the double scene graph problem though. Perhaps it could proxy methods onto the wrapped Object3D? Maybe it event could expose "fake" components for things like light properties or transform properties. How much slower would this be and would it matter?
Beta Was this translation helpful? Give feedback.
All reactions