I have chosen these questions based on the film-side qualities we identified (emotional engagement, believable world) since our next task, presumably, will be to produce those qualities.
- How can we produce compelling facial animation an order of magnitude more quickly than the way we've been taught? Why will your method be so much faster? For background, the "way we've been taught" is the keyframed viseme/blendshape/morph target method. I think the Wikipedia entry on computer facial animation is not half bad for those wanting a little more background.
- How can we produce compelling pantomime/body performance an OOM more quickly than the way we've been taught? Why will your method be so much faster?
- This one is tricky to phrase. How can we translate the visual designs for an original character into a form that we can use an OOM more quickly than the way we've been taught? Note I didn't say "how can we model and rig a character..." because that assumes we will be modeling and rigging like we're used to. We may decide that's the only way to go, but I want to encourage people to think of other ways they might make our character exist in the final film without, perhaps, existing as a fully-rigged 3D object.
- How can we create the illusion of a believable world an OOM more quickly? Why will your method be so much faster?
- Also tricky to phrase. How will we generate the appropriate color field for a frame of our film an OOM faster than we currently do? Once again, I avoid "light" because that assumes some kind of true 3D world model. We aren't yet committed to having that.
- What questions would you ask the producers of today's feature animated films to help guide us in our mission? Why?
- What question(s) should we be asking instead of the previous 6? If you choose this route, please also answer at least one of the questions you propose.
I think a good place to start when asking a producer questions is to verify that our assumptions on what takes the longest in a production are correct. Before we jump in full force we want to make sure that the areas we are going to focus on(if we do focus on just a few) are the most time consuming in productions.
ReplyDeleteAnother idea for questions would be after we have figured out what consumes the most time do these producers feel that there are ways to improve that area. The producers themselves may have thoughts on the subject but not able to pursue them because they are already plugged into a system, while time consuming, produces a quality animation.
Speeding up animation is really a tough question and its been approached in many different ways. I recently read an article in a computer graphics journal that described a rigging system that would allow for the traditional squash and stretch animation style to "arise" from normal motion. It was basically a procedural way to approach a certain aesthetic quickly. This of course assumes we use the set way of rigging a character.
ReplyDeleteProcedural animation is something that could take a long time to develop, and honestly is something I am hesitant to endorse. I have seen very well executed procedural animation made using Massive (the crowd-sim software developed by Weta and used in Lord of the Rings), but is this something we have access to? Massive is very expensive software.
I am saying I really don't have an answer, but out of all of these processes, the animation worries me the most.
Careful with Massive: there is proceduralism in how the agents interact, for sure, but to my knowledge the well-known uses of Massive (like LOTR) depend on libraries of mocap and/or keyframed data which the procedural mechanism switches between. For instance, Massive says, "use the orc walk cycle, now the run cycle, now the falling off the bridge cycle..." etc. Those cycles themselves come from other places.
ReplyDeleteI'm taking a stab at #1 and #3.
ReplyDelete#1:
I think that my face is the best facial animation tool I have, so what I want to be able to use my own face. Without a thought, I can simultaneously move dozens of muscles and strike emotive expressions. Given a shot, a line of dialogue, some direction, and some practice, I think we all could get compelling emotional performances this way.
Sounds like facial mocap. An area worth investigating, to be sure. I think a few tracking markers would be acceptable as a part of the process. I imagine practicing, videotaping a few takes, picking one take we like, then bringing the clip into shake or some other 2d tool and tracking the features as 2d features in screen space. Then, here' s perhaps the innovative part, these tracked features are exported into maya and they act almost as IK goal objects for various facial shaping controls. Or perhaps they act as constraints for the same controls. Either way, there is a manual but not costly step of someone connecting the tracks to the proper controls on the face.
Bake out the results of that process and you have keyframe data, which the original actor can come back to and see how it worked.
Other ideas to inject into the mix: There really are a finite number of extreme face poses we strike. The eyebrow, for example, only does so much. If we ran someone through as many little sub-expressions as possible, and catalogued them, we'd have a fairly dense catalogue of emotions which correspond to a number of different facial controls set at particular values. The tracked feature data could then be fairly automatically transferred into a higher-level representation, which could then be used in maya and turned back into keyframes (part of character setup would be to make those same poses and name them, so that the mapping is one-to-one).
#3:
I think of this question as "model building without model building" and "rigging without rigging."
My idea here is to use a pre-existing rig. Take one of the many good free rigs that are online. At least for bipeds, they're fairly fully-functional. Then, the problem becomes one of re-shaping the existing mesh to the new character's mesh, or perhaps you actually animate the entire show with a proportionally-adjusted version of the one rig and save the mesh step until later.
For example, you have a close-up shot of your character. You stage it with the default rig, act it out (see #1 and #2 for that), then "flip the switch" to make it the new character. This magic could be done with part-for-part retargeting (there is a lot of software that exists for this, where one mesh is used point by point, like a lattice deformer, to deform a new mesh). You only have to fix errors in the retargeting on the shot itself (if you haven't solved them for all poses earlier). And then you never have to rig the new model, other then identify the correspondences between it and the original rig. You do still have to make the geometry for the new model, however.
Here's a patent that DreamWorks got that talks about the relevant "wrap" methods like I'm discussing: http://www.patentstorm.us/patents/7236170/description.html.
There is also the idea of making a standard bi-pedal/humanish mesh that can just be tweaked into new characters.
ReplyDeleteFor question #2: A lo-fi version of motion capture that might help speed up the animation of a (human-like) body could be simply video taping someone acting out the scene. The camera could be set up at the same angle with similar movement, and the animator could then just reference the video for the precise timing, poses, etc. A lot like using reference footage, but reference footage tailored to exactly what you're going to animate.
ReplyDeleteThe idea of true motion capture intrigues me a lot though - I have the feeling that it would be a lot faster, but I'm not sure why. Even if motion capture was used just for laying out the animation, I think it would save a considerable amount of time.
My thoughts on #3 were very similar to Chris and Tati. I was thinking of Spore again - how each character seems to a have the same skeleton rig, but the look of the limbs varies. One generic rig could be applied to each character and tweaked if necessary. I'm not sure which would come first though - modeling the character and applying the rig or modeling the character around the existing rig (like what Chris was saying about reshaping the mesh). From my memory of working in Maya, I seem to remember painting weights taking the most amount of time. So I suppose that aspect of articulation would need to be faster rather than the rig itself, but I'm not really sure - I haven't done a lot of rigging.
Also, what exactly does OOM stand for? I assumed it was Object-Oriented... Something? Or Order-Of-Magnitude?
4. At least as far as the "physical laws" part of creating a believable world, if we could figure out a system of assigning objects attributes like mass, flexibility, etc., it could take care of the more technical footwork in animation. This seems like something video games have been doing for a while; is there something about game engines that isn't applicable to film animation? I feel like the bulk of time spent animating a given shot is on trying to emulate the physics, rather than honing a performance.
ReplyDeleteOf course, this is something a producer (or professional animator) could verify: does a simulation-heavy (not necessarily special effects, but... Newton-intensive?) shot take more or less time to animate than a performance-driven shot of the same length? In other words, does it take longer to animate a character balancing a stack of books on his head, or giving a monologue from Hamlet?
I'm just growing curious if there are more specific problems the animation step poses than just the fact that it uses keyframes. If a chunk (but not necessarily all) of the animation can be sidestepped, that can still cut time spent down by a lot.
In response to question 2: A few years ago I came across this research project on what is called spatial keyframing. The idea is that instead of key frames there are key poses, and a program interpolates between these poses based on the animator's 'performance'. This method can make short work of what would otherwise be fairly complex movements to animate. I can't explain it very well, so you should look at the website and try out the program. The examples given are crude and not exactly what I would call compelling, but I think there is potential for this concept to be applied to higher quality animation. I'm not in the class, but I thought you all might be interested in seeing this.
ReplyDelete(http://www-ui.is.s.u-tokyo.ac.jp/~takeo/research/squirrel/index.html)
That is a very interesting system. Id like to see it applied to more complex models than the ones they used. The snake model is pretty cool. It would be really cool i think if you use that technology combined with the rigs we use now to get really precise poses to interpolate through.
ReplyDeleteI really feel like its something to investigate.
Great reference, Fletcher. The qualities I like of this system are first that it is trying to bake down an arbitrary number of free variables (all the rotation values of all the nodes) into a few (the 3D position of the cursor). The second is the capture-based aspect, where performance can be captured live.
ReplyDeleteThe latter was clearly made possible by the former: since we're largely limited to a 2D input device (the mouse), then the only way to get live performance to pose more than a single node (or part of a node) was to translate 2D movement into a higher dimension.
Can we use more input devices? Two mice? What about multiple people, each with a mouse, using multiple computers (like Japanese Bunraku puppetry in a weird way) each controlling a different node or set of nodes in 3D? I forget if this film has been posted here, but check out the animation called "Tyger" (http://www.guilherme.tv/tyger/) and enjoy the real-time performances of a fairly complex character.
I like the idea (thinking of question 2) of shooting video performance as a foundation, then perhaps one or more live performers controlling body parts in sync with the reference performance. Maybe.