Video Capture to Animation Pipeline

The main objective of this is to take what these guys are wearing:  

And turn it into motion for this guy(and any other character):

So here is a list of things that will be needed to complete this task:

• A Video Camera (or two)
• A Suit similar to the ImoCap one
• Some sort of software for capturing the points on the suit that can translate it into 3d motion (Bujou or whatever)
• A rig and presumably Maya
• A room relatively clear of clutter ( a solid colored wall would be ideal)

Pipeline Proposal

1.        Get actor into suit and the place where the animations motion will be filmed.  Set up the camera to reflect the shot's camera as best as possible.  

2.        First take a reference shot of actor standing in a neutral pose, arms and legs clearly visible.  This shot is important because it is setting up a reference for the distance of all the rest of the shots based off how far our dots on the suit are away from the starting position.

3.        Begin filming and acting out the animation.  The actor needs to act for animation and perhaps odd proportions if the character is not realistic.  This may take some practice, it's one thing to act for live action but acting for exaggerated motion is harder.  

4.        Get a good take that everyone is happy with.  Next bring the video into our motion capturing software.  Define the points on the first shot and have it track those for the shot.  Hopefully this will give us the basic motion of our actor.

5.        Next is to constrain this motion in Maya onto the corresponding parts of our rig.  I assume we would use a parent constraint without maintain offset on.  Now the part I am not sure about is does each dot correspond to a dot on a control of our rig that is almost identical to our suit or before we bring the data in do we somehow average all the dots on our bands to one point in the center.  Then once we have that one solid point that moves around then bring it in and constrain to something similar to a rig we are use to in maya.  

6.        The data we brought in should be keyframe date for our controls, so next we could go in a delete or change some of those keyframes that we feel do not contribute to believable motion.  

7.        The last step would be to refine the motion with more traditional techniques.  We need to animate the face, hands and feet by hand.  As they say in animation, an animation is never finished until the deadline, so keep refining.

I am very aware of how complicated this sounds.  I don't know if it's beyond our scope or not.  There are few things I don't know,  an example is what the best constraints in maya would be to use.  Some other questions are is this actually faster?  I feel that if  we do get a system down where you can record motion and track it then put it into maya, it will be faster.  I think a huge problem is actually making this system.  We are only 6 people with only a couple programmers.   The math involved appears to  be very high.

 

I'd love to see it work though and would really appreciate feedback, ideas and anything else to contribute to this idea.