|
|
Plugin idea for facial animation
About Truespace Archives
These pages are a copy of the official truespace forums prior to their removal somewhere around 2011.
They are retained here for archive purposes only.
Plugin idea for facial animation // Roundtable
Post by frank // Dec 12, 2006, 12:01pm
|
frank
Total Posts: 709
|
The folks at Aardman (makers of Wallace and Gromit, Creature Comforts, etc.) know how to do fantastic facial animation.
Why does it look so good? Because it is snappy and not floaty like a lot of the 3D lipsynch that we see.
Stop-motion / claymation animators often use replacement mouths to speed up animation. This keeps them from having to re-shape the mouth each time to form a specific mouth shape / viseme. The really cool by-product of this is the quick change between shapes. They may "lean" one shape into another a bit to smooth it out but in general, a character may go from an "OOH" to an "AHH" in a few frames or so. What tends to happen with 3D and standard vertex blending is there is too much linear inbetweening going on for it to look very snappy.
Alright, here is what I have in mind. It's actually something I thought about a couple years back but just kinda wrote it off as "silly". I'm revisiting it now because the more I think about it, the more I like it:
REPLACEMENT MOUTH PLUGIN FOR TRUESPACE
"Weeehaaaaaaaaaaaaaaa!"
Basically you have a standard or default mouth shape on a character that is the target. You scrub along on the keyframe and want to change the mouth shape to a closed "B/M/P" so you just hit the appropriate button on the plugin and it replaces the target mouth with the pre-made "B/M/P" object, ensuring that the translation and rotation of the second object matches the target. It basically works off of setting Invisible keys. Once you hit a button to change the mouth shape, the currently displayed shape is invisible and the new one is visible.
A reason I revisited this idea recently is because it hit me just how flexible this system is and how much freedom it would give for more "spontaneous" animation. The main reason for that is because you aren't limited to modeling the mouth shapes by copying a standard model (you have to do this currently in vertex blending plugins so that the vertex numbering/order matches).
With this type of system you can, for instance, have a closed mouth shape that may have a small poly-count and then a high-poly shape with teeth visible. That's another added bonus - the system would allow not just single-mesh objects, but objects with sub-objects attached.
The interface would be pretty simple. (I can do a mock interface if anyone is interested in going forward with this.)
Who can write this for tS 6.6?
:) |
Post by frank // Dec 12, 2006, 12:24pm
|
frank
Total Posts: 709
|
I've attached a quick and simple interface idea.
In this case, the brighter green block shows which object is visible (all the lighter green ones would of course be hidden).
Clicking ADD NEW would give you another object on the list and allow you to select it from the scene. |
Post by Nez // Dec 12, 2006, 11:21pm
|
Nez
Total Posts: 1102
|
Not sure about Truespace, but am I right in thinking this is something that is available in Poser? I've got an older version (3 I think) and I'm pretty sure that the libraries including a ramge of facial positions for different letter sounds.
Don't know how you'd go about it in TS though... |
Post by Délé // Dec 13, 2006, 2:35am
|
Délé
Total Posts: 1374
|
Hey Frank,
Look in the "My Activities" library. There is a script called the "Interactive replace tool". If I understand you correctly, this script does kind of what you're talking about. When you click on an object, it replaces it with a pre-selected object. I bet with a little modification you could get it to work exactly how you imagine.
Perhaps some of the scripters around here could modify it to work ideally for your situation. I'll give it a look myself but can't make any promises. |
Post by RichLevy // Dec 13, 2006, 3:35am
|
RichLevy
Total Posts: 1140
|
sorry if I am misunderstanding you frank, but isn't multimorph from ckgamefactory basically the same thing? http://ckgamefactory.hp.infoseek.co.jp/tsxe/multimor.html
rich |
Post by frank // Dec 14, 2006, 4:34am
|
frank
Total Posts: 709
|
Look in the "My Activities" library. There is a script called the "Interactive replace tool".
Hrm. Thanks, Délé... but any ideas on how to go about it in tS6.6?
sorry if I am misunderstanding you frank, but isn't multimorph from ckgamefactory basically the same thing? http://ckgamefactory.hp.infoseek.co..../multimor.html
rich
I've used both MultiMorph and (especially) SpaceTimeMorph - both of which are excellent vertex blending tools - but I'm looking for more of a "shape-swapping" tool because:
- Blended animation in tS tends to be drifty unless you add many keys to break up the curved interpolation between poses (another reason "held keys" or "stepped keys" would be usefull)
- You are limited to using the same mesh, having to perform tedious vertex manipulation on a base model for each mouth shape
The type of system I'm describing would force the animator to work a bit differently, causing them to rely less on automatic tweening, but also affording the opportunity to use whatever type of mouth shape models desired. I think the end result would look better in terms of achieving snappy cartoon style animation.
Oh, and it wouldn't necessarily be strictly for mouth shapes. Eyes and any other objects could be swapped around as well, each having their own target object on the face. |
Post by frank // Jan 18, 2007, 5:31am
|
frank
Total Posts: 709
|
Hrm...
I've been doing some experimentation with mouth replacement animation and I really like the results.
For a while I was so hung up on figuring out how to mix bones and vertex blending in the same mesh, or how to build a facial rig using just bones, or modeling 40 blend shapes for every possible expression (ala Jason Osipa's book "Stop Staring"), so animating with replacement mouths is kinda...refreshing. A lot less burdensome, anyway.
It is so cool to make mouths without worrying about vertex ordering. The "E" shape can have teeth objects, the "F/V/PH" shape can have only top teeth, the "TH" sound can have a tongue object, etc... So the mouth shape can be a bunch of smaller bits glued together. A bonus of this method is the simplicity of adding extra shapes as needed.
The first test looked pretty neat - really poppy like I expected, but still readable.
In the second test, I did some psuedo-interpolation. The way I did it was to "lean" each shape into the next shape, by setting scale keys. If an "AAH" shape was about to be swapped by an "OOH" shape in 3 frames, I would set a scale key for the AAH shape where it started, move ahead a few frames, scale the AHH shape down a bit, closer to the shape/size of the OOH, and set a key. This causes the shapes to blend together and it looks pretty cool in a way. However, I find that I would use it sparringly and only in short frame changes. Where a shape stays on screen for over 3 frames, I think it's best to just hold it static with no scaling. I likes ta keep it snappy.
I'd like to record some dialog and do an actual sync test before posting an animation.
It's a bit tedious doing this manually - selecting the various objects from the drop-down list, toggling visibility, setting keys, etc. I still say a plugin for this would be NICE! ;)
Here's a render of the character and some mouth shapes: |
Post by Bobbins // Jan 18, 2007, 6:32am
|
Bobbins
Total Posts: 506
|
Frank, I've not tried it myself to see if it will animate but would the object replace tool make it simpler to exchange the mouth shapes? |
Post by frank // Jan 18, 2007, 7:58am
|
frank
Total Posts: 709
|
Thanks for the suggestion, Bobbins! I just gave it a try but doesn't seem like it would work in an animation scenario. May require further investigation, though.
What I did in my last test was to glue ALL of the mouth shapes to the head object, in the correct position on the face, before animating. That way they all go along with the rotation and translation of the head when it is moved. So when I set the various mouth shapes to visible, they are where they should be. |
Post by jamesmc // Jan 18, 2007, 9:49am
|
jamesmc
Total Posts: 2566
|
Interesting project Frank.
Are you going to tie the mouth shapes to other facial expressions? (eyebrows, eyes, forehead, cheeks, nose, etc.) |
Post by frank // Jan 18, 2007, 10:02am
|
frank
Total Posts: 709
|
Interesting project Frank.
Are you going to tie the mouth shapes to other facial expressions? (eyebrows, eyes, forehead, cheeks, nose, etc.)
They could have their own interchangeable shapes, and work the same way the mouth would.
That's another reason a plugin would be handy - you could just select a target object and give it a list of "replaceables" that can be swapped at the click of a button. The plugin could have the mouth parts, eyes, etc. all in the same interface, and it would know which target object to swap out with. |
Post by Burnart // Jan 18, 2007, 11:51am
|
Burnart
Total Posts: 839
|
Frank, this is a great idea and I've had a yen for something similar myself. I know you are specifically interested in ts6.6 but I'm wondering if this is precisely the kind of thing that a script could be written for in ts7 - yes/no?
Is it possible to have a grouped object where all the mouths are overlaid on top of each other but only the required one is marked visible? Bit clunky and not exactly a "one click" solution but it could work. The advantage would be that regardless of how the body and head have been moved the mouths would be correctly oriented. Perhaps that would make for an easier plugin to write - i. all your doing is assigning visibilty instead of position in 3d space. |
Post by frank // Jan 18, 2007, 12:27pm
|
frank
Total Posts: 709
|
That's what I'm after, precisely. My last test was done with all the mouth objects aligned and grouped, and attached to the same MoStu bone as the head.
Correct - we are basically after a visibility toggling tool. ...and then maybe we can add some other features. ;) |
Post by trueBlue // Jan 18, 2007, 5:26pm
|
trueBlue
Total Posts: 1761
|
Hello Frank,
Have you seen Johny's Fighting game tutorial? I think it would fit your idea perfectly.
http://forums1.caligari.com/truespace/showthread.php?t=1871 (http://forums1.caligari.com/truespace/showthread.php?t=1871) |
Post by frank // Jan 19, 2007, 3:44am
|
frank
Total Posts: 709
|
Thanks, trueBlue! I've seen some of Johny's work - interesting stuff!
Seems like what I want to do would be possible in tS7 but as a lot of my rigs and such are in MotionStudio and I am still relying heavily on it, I'd like to write a plugin for tS6.6. |
|
