As I descend further into the depths of animation Hell, I thought it would be good to expose my animation process to the world. At this stage of the project, I’ve completed half of the cutscenes in the game. I’m able to tackle one cutscene every week. I have a good sense of the workflow and I’m getting “in-the-zone”, so I feel ready to talk about the steps I take to bring these scenes to life.

Long time readers of this blog (all three of you) may remember the first time I did this. Last year, when I was still animating the Demo, I wrote two articles about my process. Here are the links to that old article, parts one and two. Enough has changed since then that it warrants a new piece.

( NOTE: I won’t go into detail about how I animate the character’s faces, since last week’s blog was a deep dive into that system. Feel free to check that out. )

Now, how do we make the cutscenes in Where Shadows Slumber? To make this process more concrete, I’m going to focus on a relatively innocuous cutscene that I just got finished animating this week called Beach.

 

Step 1: Write A Story

You may not have expected this to be the first step on our journey, but it is! Writing is the most important step, by far. Before any work begins on a cutscene, Jack and I have to agree on the game’s story and how it will be told to the player during the game. This process took place a long time ago, at the beginning of 2017, when we locked ourselves in a room and did not emerge until the game’s narrative was pinned down.

The original script was revised two times (officially) with some extra cuts happening unofficially in conversations between the two of us. Our original idea was to have a lot of cutscenes – I think 15 or 16 cutscenes in total – in the entire game. Since there are 8 Worlds, we wanted an intro cutscene to each World and a “finale” cutscene after the player completed all the cutscenes in that World. I agreed to this not only because I loved animation, but also because I vastly underestimated the scope of the work.

However, we eventually decided to eliminate a lot of the intro scenes. They weren’t really necessary, and it was jarring for players to watch two cutscenes in a row (a finale for one World, and an intro to the next) when they really wanted to get back to the gameplay. We only kept intro scenes for moments where the Player would be genuinely confused without them. The best example is the cutscene called Escape which takes place very early in the game. Prior to this cutscene, Obe is captured by human-like animals in a finale cutscene, and his Lantern is taken. In the very next Level, he’s freely walking around a volcanic prison with his Lantern in hand. Without a cutscene like Escape, players might wonder what happened to the animals, how the Lantern returned, and why Obe is not still in some kind of cell.

Therefore, the Beach scene is a bit of a relic as far as cutscenes go. It’s one of just two intro cutscenes left in the game, taking place at the beginning of World 3, the Aqueduct. I felt it was important to show the transition between the River World and the Aqueduct World because they are quite different, and the River finale doesn’t hint at the Aqueduct in the slightest bit.

Here’s the short version: The scripting process is important, because if we can’t agree on whether or not a cutscene should be in the game, I can’t go forward and spend 40 hours creating it!

 

Step 2: Sketch the Scene

Execution begins with sketching the scene on pen and paper. There is a long gap between the writing process and the actual execution of the cutscene. For reference, I began this cutscene 1 week ago on May 22nd 2018, but the story was written in January of 2017. That’s over a year! As I mentioned above, one reason for this is because puzzles are more important to the game than cutscenes are. Puzzles get top priority! Also, since edits to the script happen sporadically as the game evolves and our scope shrinks, it’s good to sit on the script for a while. That’s why I’m doing cutscenes last.

There’s one more good reason, though! Since cutscenes happen after the game’s art has been completed, the sketching process is a lot easier. Most of the game’s artwork is done using a modular set of puzzle-piece 3D models that can be arranged along a grid to form pathways, bridges and obstacles. I’ve also created a bunch of materials for each World. That means when it’s time to lay out how a cutscene is going to look, I have a wealth of building blocks to work with. Really, all I need to do is draw a few pictures to determine the camera’s position, and I’m good to go.

When I sketch a scene, I’m trying to make it look just like the puzzles. My goal for cutscenes is that you never even feel like you’ve left the game. The camera is in the same position and rarely moves, just like the game. I use the same models, colors, camera effects, and even some ambient audio, to keep that feeling of similarity. So when I draw a picture of the scene, I’m trying to get everything in one shot. I need it to work in portrait mode on an iPhone, with room for superfluous art on the sides that only iPad users can see.

That’s why for Beach I composed the scene with the outlet pipe near the top of the scene. I know Obe is going to wake up, walk to it, and climb in. Arranging the scene this way avoids a messy camera transition, and lets us focus on the stillness of the moment.

With a good picture to work from, we’re ready to set things up in Unity 3D.

 

 

 

Step 3: The Unity Smoke Test

You were probably expecting Step 3 to be “model everything in the scene” or “begin animation” – but I don’t dive into that right away. I’ve gotten into the habit of doing a smoke test whenever possible, before beginning a large amount of work. This is an old phrase from computer programming that refers to plugging in a machine and seeing if it starts to smoke, or light on fire. It’s also known as a sanity test.

As the picture above indicates, the modern version of a smoke test is when I douse my computer in gasoline, light it on fire, change my name and move to Mexico. (Wait, that’s not a smoke test. That’s Operation: Secret Grongus. Whoops! Jack, please remind me to delete this paragraph before I hit Publish)

The modern version of a smoke test is when you intentionally do placeholder work just so you can test it and see if something is going to function correctly. After all, if it doesn’t work now, it won’t magically work later. It’s especially important when making a transition from one tool (3DSMax) to another (Unity). The game’s cutscenes will be animated in 3DSMax, but they’ll be viewed by the player in a build of the game generated by Unity. We need to make sure that pipeline works before we dedicate 40 hours of work to something.

First, I create a scene in 3DSMax to work with. I import (3DSMax calls it “merging”) Obe’s character model, and the models of any other characters that are in the cutscene, into the file. I also merge in a few models that I know I will need. For example, in Beach, I know I need to use my ladder pieces so Obe can climb into the pipe.

I give Obe some basic placeholder animations. Really, it’s just a few frames that will all be deleted later. I make Obe wave his hands, move in a T-pose, do jumping jacks, or something silly. My goal is to make sure the animations are properly translating over to Unity. I do a similar process for objects in the scene and other characters. Obe is animated separately from them because I’m using the same Unity prefab that is used in the real game. This adds another step, but it’s worth it in case there are crucial last-minute changes to his prefab. Along with that, there’s a lot of little things to do – Animation Controllers for each FBX file, setup in the scene, camera positioning, light adjustments, and much more. Anything could go wrong, so I’d rather find out before I’ve done a few grueling hours of animation.

I have a small checklist of things I go down:

• Can I animate Obe?
• Can I animate his Lantern separately from him?
• Can I animate his Lantern if he’s holding it and it follows his hand?
• Are Obe’s hands, feet, and pelvis “Linked to World”?
• Can I animate other characters?
• Can I animate other characters holding objects?
• Can I animate objects on their own?
• Do other characters require their body parts to be “Linked to World”?
• Is there a light? Is that light attached to the Lantern?
• Is the Lantern flickering properly?
• Does the camera need to be re-positioned, or zoomed in?

When I’m confident that Obe’s animations and the animations of everything else in the cutscene are working well, I can begin modeling the scene in earnest. Now I’ve made sure there won’t be any surprises during the next step.

 

Step 4: Model Static Objects

We’re ready to bring my ink sketch to life by creating the scene in 3DSMax. This is done by using modular building blocks wherever possible, and also creating new 3D models. Beach is a bit of a hybrid in this regard. The ladder, for example, is the same model and material used throughout the game whenever Obe climbs a ladder during a puzzle. The water is the same rig we use during Levels, albeit with a special material. But other specific objects, like the sandy beach, the wall, and the outlet pipe are unique to this scene. I gave up the strictly modular approach a little while ago, and I think the game is better for it. (Above, the scene in 3DSMax. Below, the same scene in Unity.)

Now that the models are in place, and nothing is going to change, I can go forward with confidence. I place Obe in an initial pose that matches the terrain, and begin animating the scene by hand.

 

Step 5: Keyframe Animation

Recently, when I was at PAX East 2018, someone asked me if the cutscenes in our game were animated using motion capture technology. I took this as a compliment, because I think most people assume motion capture animations are an indicator of high quality. Thanks, random person!

For those unfamiliar with motion capture, think of the character Gollum in Peter Jackson’s Lord of the Rings trilogy. Gollum was animated in 3D, but not by hand – rather, the actor Andy Serkis dressed up in a silly motion capture suit and performed the role himself. Later, computer imagery was placed on top of the scene using data captured from his performance. This technology has also been used to great effect in the Uncharted series of games. As it grows in popularity, there are boundless examples to use. I can’t name them all!

However, that is not how animations are created for Where Shadows Slumber.

Motion capture is the proper tool to use when your resulting animation is intended to be life-like, gritty, and serious. Characters like Joel and Ellie from The Last of Us work well because they are intended to be portraits of real people, so it makes sense to have actors play them. Motion capture also requires a financial investment that only AAA studios can afford. If you’re using motion capture, that means you’re paying actors some money, purchasing a large studio room to perform in, purchasing high-speed cameras, and purchasing (or creating) software to bring it from the stage into the virtual world. We don’t have the resources to afford that, and I don’t want to work that way anyway!

Animation for Where Shadows Slumber is done the old fashioned way – by mouse-and-click keyframe setting. I’m fairly certain Pixar does this as well, albeit with more complex tools than the 3DSMax Animation Timeline. If you’ve ever seen a documentary on how Walt Disney created those first frames of Mickey Mouse by hand on cell sheets, you get the idea – the lead animator sets a pose for one period of time, and then sets a different pose for a different period of time. His subordinates fill in the gaps, and the result is the illusion of animation.

I don’t have any subordinates, so 3DSMax fills in the gaps for me. Sometimes I work with it, and sometimes I have to fight it because it filled in the gaps wrong. You need a lot of key-frames, but animation frames are just a fraction of a second ( 1/30th a second, in our game ). That means an hour of work may get you just 3 quick seconds of animation. The process is painstaking, and easily takes the longest amount of time in the cutscene creation process. Beach, a relatively simple 50 second cutscene, required 7.5 hours of animation to complete. The previous cutscene, Wolf, which is a very involved fight scene that lasts 100 seconds, required 48.5 hours of animation!

 

Step 6: Special Effects

We’re not even close to done yet. Animating the characters in a scene is not enough to bring it to life! Every cutscene needs some kind of special effects, whether it’s footprints in the sand or the drip-drip-drip of a leaky pipe. This never takes as long as actual animation, but it can still be a painstaking process. For example, in the Wolf scene I mentioned above, every time an object fell into the water I had to trigger a particle burst to make it seem like the objects were splashing. That was as fun as it sounds!

To achieve my special effects, I wrote a script called Cutscene Manager. This thing will fire off effects based on the time of the animation, and I save it only for things I can’t animate by hand. Here’s two examples to show you the difference:

Example 1: Footprints in the sand

These footprints can be animated by hand, so I don’t need to use my script. Notice how they appear after Obe touches his feet to the ground – what’s happening here? Well, they are actually just hiding under the sand! I triggered their animations using keyframes, just like anything else in the scene. Above, you can see one that I have selected that is still burrowed under the ground, waiting to rise up.

Example 2: Obe’s Lantern light grows, and then shrinks

We use the solid color black a lot in this game. It represents total darkness, which makes it handy for scene transitions. Every Level and cutscene begins with the world in total darkness, and then a light grows somewhere and the animation begins. I think this helps focus the attention of the player, and it makes transitions less jarring. However, since Lights are a Unity component, their Range values can’t be animated in 3DS Max. 3DS Max has no idea they even exist! Instead, my Cutscene Manager script knows to change the Range of a specific Light at a specific speed at a specific point in the animation. It may seem like a crude solution, but it’s the best we came up with. At the end of the scene, the Light gets another trigger to shrink down to zero – pitch black.

You can see why special effects necessarily need to come after principal animation. So many of these things require specific timing! If the underlying animation changes, they’d have to change, too. It’s better just to wait.

 

Step 7: Recording for Alba and Noah

Recording the cutscene is my final step, although the cutscene is not done yet. Using OBS, I record my screen with the animation playing. I mute the sound in the game, and I talk during the cutscene to tell our audio engineers what is happening. Some things are obvious, and I don’t need to say them (e.g. he’s walking in sand, which sounds like the sound of someone walking in sand). Other times, a noise comes from off-screen and has no visual representation. Without my direction, Alba and Noah couldn’t possibly guess at what is happening in the scene. My recording is set to be the exact same time-frame that it will be in the game, which means they can “score” this video as if it was a short film. From the work they’ve done so far on earlier cutscenes, I can tell the cutscene audio is going to be incredible.

I briefly flirted with the idea of using a high-quality recording of the cutscene in the game, instead of having people view the cutscenes in real-time. However, I don’t trust Unity’s ability to play videos across multiple iOS devices and countless Android platforms. I also wanted to avoid including 10 large MP4 files into the game’s databanks, for fear it would clog up the game. The last reason is that our final cutscene transitions seamlessly into the credits, which need to be translated into multiple languages. This would result in 15 different movie files! I prefer to have that done on the fly using Jack’s JSON file setup.

Once Alba and Noah score the cutscene, I’ll put that file into the game and the audio will play in-sync with the animation, all in real time! Players can pause the cutscene from a top menu, go to the level select screen, skip the cutscene, or resume the animation seamlessly.

 

 

---

 

I don’t exactly know how Alba and Noah score these cutscenes, so I’ll leave that for another blog post. I invite them to share their knowledge with you, dear readers, whenever they feel the desire to do so. (Maybe I’ll interview them about it?)

That’s all for now. I need to go back to the animation mines and make more cutscenes… I’ll see you back here next Tuesday for the June State of the Art. Don’t miss it!

 

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

We hope you enjoyed this look at the cutscene animation process. You can find out more about our game at WhereShadowsSlumber.com, ask us on Twitter (@GameRevenant), Facebook, itch.io, or Twitch, and feel free to email us directly at contact@GameRevenant.com.

Frank DiCola is the founder of Game Revenant and the artist for Where Shadows Slumber.