Another project that I am very proud of is my computer animation software development project, the Multi-Axis Pose Space Deformer Tool (MATRIX). The MATRIX adds functionality to Autodesk Maya, the leading 3D computer graphics application for creating assets for movies. The MATRIX is an innovative rigging and deformation tool for highly realistic human characters that provide anatomically-correct movement and skin deformation. The tool gives full control of the movement and deformation for any pose of the character. The results of the project can be seen in my demo reel (http://youtu.be/0BYrtUnnLdU).
I decided to teach myself computer animation with the help of online courses and videos. For my portfolio, I wanted to create a highly realistic human character. (As a classical figurative painter, I wanted to continue in this style.) Initially, I thought the challenge would be in only modeling the character realistically. But, I discovered that the real challenge was making the character move and deform in a realistic manner and even the major studios were having difficulty with this. For example, in Spider-Man (2002), the CG version of Spider-Man had shoulders that did not raise in a natural way when he was suspended from his web. And, in Hulk (2003), the Hulk character was reviewed by The New York Times as looking like a “giant green Gumby.” After searching for a solution, and not finding one, I decided to develop a solution of my own. I thought it would be an excellent “thesis project” for my computer animation independent study.
So, why is it so hard to make realistic human 3D characters? Computer animation is very computationally expensive. So, there is a concerted effort to minimize how much computation needs to be done for each frame of a film. This restriction has led to a “close is good enough” approach. So, human characters have been constructed in a simplified manner. Most 3D characters are built with a very simple skeleton with far fewer bones and simple single-point joints. Fortunately, the human body is much more complicated. We have a wide array of joint types with complex movements. For example, the shoulder is composed of the clavicle, the scapula, and the humerus with all three bones rotating like a ball as well as rotating along their axis. And, even the knee which appears to be a simple hinge joint is more complex. When the knee bends, the main lower leg bone, the tibia, glides over the oblong surface of the bottom of the femur, the upper leg bone. So, we have a translation of the bone to the rear in addition to the rotation. Visually, this creates the large area of the bent knee and visually lengthens the lower leg. And, it is a difference the human eye can easily detect. People are very good at noticing less than perfect replications of the human form and movement which has led to the popular term “uncanny valley.”
The solution was comprised of the model and the Multi-Axis Pose Space Deformer Tool (MATRIX). The model was highly realistic, anatomically correct, and detailed. Realism was also included in the three deformation systems: joints, muscles, and skin. The joint system included an anatomically-correct skeleton that contained over 90% of the human skeleton. A complete muscular system was developed. The skin mesh of the model was designed to support poses covering the full range of human motion without losing the look of realism.
And, the MATRIX is a control and deformation layer that sits on top of Maya’s built-in mechanisms: joints, muscles, and blend shapes. With the MATRIX, moving the various animation controls applies the appropriate anatomically-correct pose out of a library of hundreds. Each pose consists of all the elements of the model that was influenced: joint rotation and translation, the shape of muscles, and the stretching, compression, and folding of the skin. For example, as the arm is raised out to the side and above the head from the hanging position, the clavicle and scapula move very little until the arm reaches horizontal then the scapula rotates dramatically rotates out to the side and the clavicle rotates on its axis and slides the scapula forward around the side of the ribcage.
One of the key design decisions in designing the MATRIX was to move away from Maya’s built-in XYZ coordinate system to a custom coordinate system that combined polar coordinates with twist (rotation along the axis) and additional translations as needed. This allowed me to give each joint as many degrees of freedom as necessary to very accurately copy human motion. The code used Maya’s API to automate the building of the MATRIX in the application. All of these subsystems combined to create a convincing animatable CG human character. The results of the project can be seen in my demo reel (http://youtu.be/0BYrtUnnLdU).
The MATRIX is a collection of Python scripts that are run within Maya that build components of the model, set up control equations, and connect components together. This GitHub has an example that covers the shoulder.
Version 2.0: If I were to attempt this again, the system would use deep learning to create the poses from video similar to "3D Human Pose Machines with Self-supervised Learning" (https://arxiv.org/abs/1901.03798) and "Subspace Neural Physics: Fast Data-Driven Interactive Simulation" (https://static-wordpress.akamaized.net/montreal.ubisoft.com/wp-content/uploads/2019/08/27140237/deep-cloth-paper.pdf).