Developer Diary - Deo Perez, Lead Animator

Game of "Firsts" - Jade Empire, the MoCap Sessions

August, 2004

Jade Empire is a game of "firsts" at BioWare. By now you know this is BioWare's first intellectual property, our first action-roleplaying game, and our first game inspired by mythical China. What's not widely known is that this is also the first time we've built an animation skeleton and system from scratch, and the first time we've used motion capture to animate our characters.

Wanted: 500 Action Animations!

When we started brainstorming for the game, we wanted the character to learn multiple martial arts styles as he explores the world of Jade Empire. Around how many styles we were talking about? At least 30! 30... hmmm... that's at least 24 more "styles" than our previous games, if you consider collections of similar moves a "style". In games like Baldur's Gate II, Neverwinter Nights, and Star Wars: Knights of the Old Republic, we only had a single unarmed style, a single "magic" style, and around four different weapon styles. Jade Empire needed 30--at least 30--unique styles.

Jade Empire's combat system is a major departure from our previous games since it gives complete real time control of the action to the player, something like "Devil May Cry", "Ninja Gaiden", or "Soul Calibur." Custom damage reactions for different attacks would also need to be animated. So, more than 30 styles, plus combat reactions like damages, knock backs, and combat movements, came out to roughly five hundred human actions. This did not include the actions used in non-combat situations, such as conversations, walking, and crowd actions.

The First Mocap - A Trip To The Hospital

On top of the huge number of animations, we also had very high requirements for their quality. We wanted each martial arts style to look unique, polished, and true to their form, and for every action to look as realistic as possible. To accomplish this, we decided to use motion capture ("mocap") for the first time in BioWare's history. Before committing to this process, we did some testing to see if it was worth the time and money, and if we could work with the data. During the pre-production stage, we took sample motion capture data from the internet and played around with it in our 3D animation package. Once we came up with a draft of our motion capture pipeline, our next step was to capture actions ourselves.

The nearest facility with a motion capture set up was the Glenrose Rehabilitation Hospital located in downtown Edmonton. They use a sophisticated motion capture system to study the posture and walk of patients with joint and balance disorders. We had a different purpose in mind for their system. A volunteer was needed to play the role of mocap actor and I was volunteered to do the job. With this as our first session, we came with just some theories about how we were supposed to capture actions. At first we didn't even have a proper mocap suit, so we had to improvise. I wore the tightest clothes I could find and taped the markers all over my body, including my shoes and a headband. An advantage of capturing in a hospital was that the staff knew were to place the markers to best represent each joint, and they were an enormous help as we took these first baby steps into the process. My martial arts skills are limited, but we were able to capture enough actions at that first session, with many false starts and redos, to assemble a single, prototypical fighting style, which we dubbed "Deo style."

Prototyping Begins

With "Deo style" imported to the engine, we were able to start playing with our combat system, and begin to prototype controls, camera, and a rough list of actions and styles. Now the fun part could begin: we had to gather video and photo reference for all the varied styles in the game. We watched many, many hours of martial arts movies, training videos, and videos of martial arts expos to get the reference we'd need, and assembled a huge library of these clips as reference for our mocap performers.

With more than 30 different styles, we had to somehow make each one as unique as possible in terms of look and feel. We subdivided the styles to fast, strong, magic/defensive, and weapons. Fast styles were more upright, had quick moves, but delivered the least damage. Strong styles were lower and slower than the fast styles, but each hit would be devastating. Magic or defensive styles were more graceful and evasive. Some moves would rely on spells instead of physical contact. We examined different martial arts not only from China but from different parts of the world to generate ideas for our own unique styles. Once we gathered references for all the styles, step 2 of our mocap pipeline commenced.

A Visit To Giant Studios

We decided to go with a commercial mocap studio for the roughly 500 combat moves, trusting that their experience in working with stunt professionals and props would help reduce the risk of our maiden voyage into mocap territory. After a long search, we chose Giant Studios in Los Angeles, the same company that did motion capture for the Lord of the Rings movies. Jim Bishop, the producer on Jade Empire, Harvey Fong, our lead technical artist, and myself flew down there to tour their facility and to see what they can do. The studio at that time was based in an aircraft hanger in the Santa Monica municipal airport. It was huge and they had over 30 cameras placed on layers and layers of steel beams. When we arrived at the reception area of the studio, we were shown a test on a video camera where more than a dozen mocap actors were being captured at the same time. It was very impressive!

Then they brought us into the huge studio space with an actor all "markered" up for test captures. We tried to break their system by having the actor do the most complicated moves in our combat animation list--our nightmare scenarios. He rolled on the ground, jumped and ducked, and performed many different acrobatic moves, but the captures remained pristinely clean. Even more impressive to us, the entire time of the capture we were able to watch a 3D actor performing the same actions in realtime on a TV monitor. Finally, unable to break the system through conventional means, we draped a blanket over the mocap actor. Even here the system pulled through, using its advanced understanding of human physiology to track down the lost markers immediately. We went back to our hotel with our jaws dropped, convinced that Giant Studios would do a spectacular job on Jade Empire.

Later that year we went back to their newer, bigger, and better studio. We spent roughly a week there and captured our entire list of animations for the game, working at a pace that exceeded even our most optimistic estimates for scheduling. Overall, Giant Studios has been terrific to work with, and we were lucky enough to have some of the best stunt professionals in the industry work on our shoots, including guys who had worked on the Matrix films, Spider-Man and Spider-Man II, multiple national martial arts champions, and even the stunt double for Lucy Liu. At the end of the session we had hundreds of top-quality combat animations, and seeing them today in the game it's clear that all the hard work was worthwhile--it's some of the best animation I've ever seen in an RPG.

Overall, Giant Studios has been terrific to work with and very professional. They have a lot of experience in working with some of the major movie studios, and we were lucky enough to have the stunt double for Lucy Liu do the mo-cap for the female characters in the game including Wu the Lotus Blossom. The animations turned out great, and the martial arts movements look fantastic as you're playing the game!

Biography

Enrique Deo Perez started out studying in Providence, Rhode Island and earned his Bachelor of Fine Arts in Industrial Design. From there he headed to Canada and studied at the DigiPen Institute of Technology in Vancouver, B.C., earning a Certificate in 3D Computer Animation. Once he finished his schooling he went directly to BioWare where he has worked there for the last 4 1/2 years. He started out as an animator on Baldur's Gate II: Shadows of Amn and MDK2, then moved on to be a Lead Animator on Neverwinter Nights. Once Jade Empire started production, Deo moved on to there as the Lead Animator. And, in case you didn't know, Deo's face was used as one of the PC model heads in Star Wars: Knights of the Old Republic. Next time you see the game, have a look!