Creating and Blending Corrective Shapes for the Visemes

Creating or Editing Corrective Shapes

The shape manager is the tool that provides you with an environment for creating and editing corrective shapes. To help you work efficiently, the shape manager has a viewer that immediately displays the results of the changes as you make them to the object.

For general information on the shape manager, see Overview of the Shape Manager.

1. Select LipSync Setup Visemes from the Library panel.

   

2. In the Visemes dialog box, select a Phoneme for which you want to create a corrective shape and click the Apply button. The mouth animation controls take on the pose for that phoneme.

3. Do either of the following:

   • If you want to create a corrective shape, click the Create Corrective Shape button.

     This automatically stores a shape key using the selected phoneme name for the cluster on the face's geometry, and opens the shape manager with that corrective shape key selected.

     or

   • If you already have a corrective shape created for that viseme, click the Edit Corrective Shape button. The shape manager opens with the corrective shape selected. By default, only the points of the cluster that are affected are highlighted in green (select View Affected Points in the shape manager).

     

4. Select the points in the cluster on the face and adjust them as you want for that phoneme.

5. Click the Animate tab in the shape manager and set the weight to use for the shape as you like.

6. You can then blend these shapes in globally via the Speech Blend property editor, and subsequently per phoneme in the Lip Sync view. See the following sections for information.

Blending in Corrective Shapes Globally

The corrective shapes that you created for the visemes can be blended in with the lip-sync animation in a general way using the Corrective Shapes > Shape Weight and Falloff parameters in the Speech Blend property editor. These parameters determine how much all corrective shapes contribute to the lip-sync animation.

You can also animate this weight value so that the corrective shape usage changes over the duration of the lip-sync animation. On frames where you don't want to use the corrective shapes, set the Shape Weight to 0, then set it to a value over 0 for frames where you do want to use the corrective shapes.

The corrective shapes are available only for the Lips animation channel because only these mouth controls are affected by the shapes, not the jaw or tongue.

Blending in Corrective Shapes Per Phoneme

You can change the amount of contribution that the corrective shape makes to the lip-sync animation per phoneme.

The contribution of the corrective shapes is first determined by its Shape Weight value in the Speech Blend property editor. Once this is set, you can set the Shape Weight and Falloff In/Out values for each phoneme in the Lip Sync view.

If the Shape Weight value for each phoneme is set to 1.0, it uses 100% of the Shape Weight value set in the Speech Blend property editor. You can set it to be within a range of 0.0 to 2.0 (which doubles the weight).

You can also tweak how smoothly the corrective shapes go into and come out of each phoneme by using the Shape Falloff In and Shape Falloff Out commands. For example, you could lower the Shape Falloff In value so that the shape eases smoothly into the phoneme, then raise the Shape Falloff Out value so that the shape goes out of the phoneme more abruptly.

Decoupling the Corrective Shapes from their Viseme Poses for Mocap Animation

With motion capture, you can capture the position of the lips, but not the shape deformation of the lips (think about how the lips thicken with the "oo" phoneme). This is where corrective shapes come in: they allow you to get the shape deformation of the lips, and you can make it look just right for each viseme.

One of the cool things about the corrective shapes you make for the visemes is that they don't actually need to be linked to the viseme pose. This lets you automate the corrective shape work for motion-captured facial animation: you can load in mocap animation for the face after you have finished the lip sync, then use the lip-sync tools to blend in the corrective shapes without needing to manually keyframe them.

You can have the mocap animation control the lips/jaw and have the lip-sync animation drive the corrective shapes and the tongue.

1. Use the lip-sync tools to generate a speech clips using the audio/text files as usual (see Creating a Speech Clip from an Audio File).

2. Import the visemes as usual and set up corrective shapes (see Creating or Editing Corrective Shapes).

3. Turn the Weight Scale Factor value of the Jaw and Lips channels to 0 (zero) in the Speech Blend property editor (see Weighting the Lip-Sync Animation).

4. Deselect the Corrective Shapes Apply Channel Weight option in the Speech Blend property editor. By default, this option is active, which means that the corrective shapes consider the Weight Scale Factor for the Lips channel.

Now you can apply the mocap and use the lip-sync animation to drive only the corrective shapes (and tongue as well, if you like) and use the mocap animation to drive the lips and jaw movement.