Creating and Blending Corrective Shapes for the Visemes

 
 
 

In addition to editing viseme poses and creating viseme variations, you can create corrective shapes on the face per viseme. A corrective shape is any modification that you make to the points in a cluster on the face's geometry and then save that change as a shape key; for example, you may want to adjust the mouth shape so that the lips stick out more for the "ou" viseme. This corrective shape is simply a shape that is driven with the viseme, but it is completely separate and not stored with the viseme.

After you have created corrective shapes, you can set their global (overall) blending with the lip-sync animation using the Corrective Shapes parameters in the Speech Blend property editor — see Blending in Corrective Shapes Globally.

Then you can have more precise control over their blending per phoneme by using the Shape Weight and Shape Falloff In/Out commands in the Lip Sync view — see Blending in Corrective Shapes Per Phoneme.

You can also use the corrective shapes independently from their viseme poses to automate these corrections when using motion-captured facial animation — see Decoupling the Corrective Shapes from their Viseme Poses for Mocap Animation.

Creating or Editing Corrective Shapes

The shape manager is the tool that provides you with an environment for creating and editing corrective shapes. To help you work efficiently, the shape manager has a viewer that immediately displays the results of the changes as you make them to the object.

For general information on the shape manager, see Overview of the Shape Manager.

  1. Select LipSync Setup Visemes from the Library panel.

  2. In the Visemes dialog box, select a Phoneme for which you want to create a corrective shape and click the Apply button. The mouth animation controls take on the pose for that phoneme.

  3. Do either of the following:

    • If you want to create a corrective shape, click the Create Corrective Shape button.

      This automatically stores a shape key using the selected phoneme name for the cluster on the face's geometry, and opens the shape manager with that corrective shape key selected.

      or

    • If you already have a corrective shape created for that viseme, click the Edit Corrective Shape button. The shape manager opens with the corrective shape selected. By default, only the points of the cluster that are affected are highlighted in green (select View Affected Points in the shape manager).

  4. Select the points in the cluster on the face and adjust them as you want for that phoneme.

  5. Click the Animate tab in the shape manager and set the weight to use for the shape as you like.

  6. You can then blend these shapes in globally via the Speech Blend property editor, and subsequently per phoneme in the Lip Sync view. See the following sections for information.

Blending in Corrective Shapes Globally

The corrective shapes that you created for the visemes can be blended in with the lip-sync animation in a general way using the Corrective Shapes > Shape Weight and Falloff parameters in the Speech Blend property editor. These parameters determine how much all corrective shapes contribute to the lip-sync animation.

You can also animate this weight value so that the corrective shape usage changes over the duration of the lip-sync animation. On frames where you don't want to use the corrective shapes, set the Shape Weight to 0, then set it to a value over 0 for frames where you do want to use the corrective shapes.

The corrective shapes are available only for the Lips animation channel because only these mouth controls are affected by the shapes, not the jaw or tongue.

To set the weight and falloff of all corrective shapes

  1. Open the Speech Blend Property Editor — see Blending the Lip-Sync Animation.

  2. On the Lips tab in this property editor, set the weight value for Corrective Shapes Shape Weight.

    A value of 1 is 100% of the corrective shapes' weight, but you can set it to be any value between 0 and 1.

  3. Set the value for Corrective Shapes Falloff.

    If the Falloff value is set to 0, a uniform falloff is used, but you can set this value to be within a range of -1 to 1 (you can enter these values in the text box even if the slider doesn't show it).

    • A negative falloff value makes the corrective shape ease in/out over a larger area so the shape blends in sooner and more gradually.

    • A positive falloff value makes the corrective shape ease in/out over a smaller area so the shape blends in later and more abruptly.

    Once you have the corrective shapes' weight set in a general way, you can then set it per phoneme in the Lip Sync view, as described in the next section.

Blending in Corrective Shapes Per Phoneme

You can change the amount of contribution that the corrective shape makes to the lip-sync animation per phoneme.

The contribution of the corrective shapes is first determined by its Shape Weight value in the Speech Blend property editor. Once this is set, you can set the Shape Weight and Falloff In/Out values for each phoneme in the Lip Sync view.

If the Shape Weight value for each phoneme is set to 1.0, it uses 100% of the Shape Weight value set in the Speech Blend property editor. You can set it to be within a range of 0.0 to 2.0 (which doubles the weight).

You can also tweak how smoothly the corrective shapes go into and come out of each phoneme by using the Shape Falloff In and Shape Falloff Out commands. For example, you could lower the Shape Falloff In value so that the shape eases smoothly into the phoneme, then raise the Shape Falloff Out value so that the shape goes out of the phoneme more abruptly.

To set the weight and falloff corrective shapes per phoneme

  1. Set the overall (global) weight value for the corrective shapes using Corrective ShapesShape Weight in the Speech Blend Property Editor (see the previous section).

  2. Open the Lip Sync view—see The Lip Sync View.

  3. Select one or more phonemes.

  4. Do either of the following:

    • Right-click on one of the selected phonemes and choose Shape Weight or Shape Falloff In/Out from the menu, then select a value from its submenu.

      If the Shape Weight is set to 1.0, the phoneme uses 100% of the Shape Weight value set for the corrective shapes in the Speech Blend property editor. You can set it to be within a range of 0.0 to 2.0 (which doubles the weight).

      If the Shape Falloff In/Out value is set to 0.0, a uniform falloff is used, but you can select a value within the range of -0.5 to 0.5.

      • A negative falloff value makes the corrective shape ease in/out over a larger area so the shape blends in sooner and more gradually.

      • A positive falloff value makes the corrective shape ease in/out over a smaller area so the shape blends in later and more abruptly.

      or

    • Right-click on one of the selected phonemes and choose Edit from the menu. In the Phoneme dialog box, set the Shape Blending Weight and In Falloff and Out Falloff values.

      You can select a preset value from the parameter's list, or enter a value in its text box. The In Falloff and Out Falloff values can be in a range of -1 to 1.

      Click the Apply button to apply the values for these parameters to the selected phonemes.

    TipYou can move easily among phonemes on the lip-sync track using the Previous and Next buttons in this dialog box.

Decoupling the Corrective Shapes from their Viseme Poses for Mocap Animation

With motion capture, you can capture the position of the lips, but not the shape deformation of the lips (think about how the lips thicken with the "oo" phoneme). This is where corrective shapes come in: they allow you to get the shape deformation of the lips, and you can make it look just right for each viseme.

One of the cool things about the corrective shapes you make for the visemes is that they don't actually need to be linked to the viseme pose. This lets you automate the corrective shape work for motion-captured facial animation: you can load in mocap animation for the face after you have finished the lip sync, then use the lip-sync tools to blend in the corrective shapes without needing to manually keyframe them.

You can have the mocap animation control the lips/jaw and have the lip-sync animation drive the corrective shapes and the tongue.

  1. Use the lip-sync tools to generate a speech clips using the audio/text files as usual (see Creating a Speech Clip from an Audio File).

  2. Import the visemes as usual and set up corrective shapes (see Creating or Editing Corrective Shapes).

  3. Turn the Weight Scale Factor value of the Jaw and Lips channels to 0 (zero) in the Speech Blend property editor (see Weighting the Lip-Sync Animation).

  4. Deselect the Corrective Shapes Apply Channel Weight option in the Speech Blend property editor. By default, this option is active, which means that the corrective shapes consider the Weight Scale Factor for the Lips channel.

Now you can apply the mocap and use the lip-sync animation to drive only the corrective shapes (and tongue as well, if you like) and use the mocap animation to drive the lips and jaw movement.