doModifyPoly()

doModifyPoly() is the most complex piece and the core of the polyModifierCmd. In here all the data is parsed and the appropriate action given. Rather than implementing the entire process in a single method, doModifyPoly() only focuses on quickly scanning the node states and passes control over the appropriate method based on the state:

MStatus polyModifierCmd::doModifyPoly()

{

 MStatus status = MS::kFailure;

 if( isCommandDataValid() )

 {

 // Get the state of the polyMesh

 //

 collectNodeState();

 if( !fHasHistory && !fHasRecordHistory )

 {

 MObject meshNode = fDagPath.node();

 // Pre-process the mesh – Cache the old mesh

 //

 cacheMeshData();

 cacheMeshTweaks();

 // Call the directModifier

 //

 status = directModifier( meshNode );

 }

 else

 {

 MObject modifierNode;

 createModifierNode( modifierNode );

 initModifierNode( modifierNode );

 connectNodes( modifierNode );

 }

 }

}

The skeleton of doModifyPoly() shows the concepts of how to interface with a polyShape node given its state. If you have no history and recording history is turned off, you cache the mesh data for undo purposes (explained with undoModifyPoly()) and proceed to call the directModifier callback. Otherwise the modifierNode approach is taken. Calls are made to create the modifier node, initialize it (through the callback) and subsequently passed on to attempt to connect the nodes.

Note that doModifyPoly() processes only the construction history states. Though tweaks do play a role in the process, they are considered further on in the process separately since they are independent from construction history. The following table shows the code path that is followed based on the node’s construction history state:

The directModifier line of code is implemented by the derived command and as such has nothing complex to worry about. It passes the derived command the mesh node to operate on directly—all the control rests with the derived command.

In contrast, the three other cases regarding history are more involved. Beginning with the creation of the modifierNode:

MStatus polyModifierCmd::createModifierNode( MObject& modifier )

{

 MStatus status = MS::kFailure;

 if( fModifierNodeTypeInitialized ||

 fModifierNodeNameInitialized )

 {

 if( fModifierNodeTypeInitialized )

 {

 modifier = fDGModifier.createNode(fModifierNodeType,

 &status);

 }

 else if( fModifierNodeNameInitialized )

 {

 modifier = fDGModifier.createNode(fModifierNodeName,

 &status);

 }

 // Check to make sure that we have a modifier node of the

 // appropriate type. Requires an inMesh and outMesh

 // attribute.

 //

 MFnDependencyNode depNodeFn( modifier );

 MObject inMeshAttr;

 MObject outMeshAttr;

 inMeshAttr = depNodeFn.attribute( “inMesh” );

 outMeshAttr = depNodeFn.attribute( “outMesh” );

 if( inMeshAttr.isNull() || outMeshAttr.isNull() )

 {

 displayError(“Invalid Modifier: inMesh/outMesh needed”);

 status = MS::kFailure;

 }

 }

 return status;

}

createModifierNode() uses the initialized data for the modifier node type or modifier node name to create the modifier node via the local DG modifier. Use of the DG modifier is essential to keep the undo/redo relatively simple. Though the DG modifier has not created the node yet, it allows you to queue up actions for that node such as connections, so that once the DG modifier’s doIt() is called everything is executed in order. This helps alleviate rollback issues in case of errors. createModifierNode() also does a few checks to ensure that you have an inMesh and outMesh attribute on the node. Though the names may seem restrictive they keep a standard in the chains of modifier nodes. A helper class named polyModifierNode (discussed in a later section) automatically generates these two key attributes.

Following the createModifierNode() the callback to the initModifierNode() is made through which a derived class can initialize node data. This also lies under the control of the derived command. From there you enter the final stage of the process—connectNodes().

connectNodes()

connectNodes() is a larger method which processes all the variables of the polyShape and modifier nodes and connects them. Look at the connectNodes() implementation for a higher level view of what it controls:

MStatus polyModifierCmd::connectNodes( MObject& modifierNode )

{

 MStatus status;

 // Declare our internal processing data structure

 //

 modifyPolyData data;

 // Get the mesh node attrs and plugs

 //

 status = processMeshNode( data );

 MCheckStatus( status, “processMeshNode” );

 // Get the upstream node attrs and plugs

 //

 status = processUpstreamNode( data );

 MCheckStatus( status, “processUpstreamNode” );

 // Get the modifierNode attributes

 //

 status = processModifierNode( modifierNode, data );

 MCheckStatus( status, “processModifierNode” );

 // Process the tweaks on the meshNode

 //

 status = processTweaks( data );

 MCheckStatus( status, “processTweaks” );

 // Connect the nodes

 //

 if( fHasTweaks )

 {

 status = fDGModifier.connect( data.upstreamNodeShape,

 data.upstreamNodeSrcAttr,

 data.tweakNode,

 data.tweakNodeDestAttr );

 MCheckStatus( status, “upstream-tweak connect” );

 status = fDGModifier.connect( data.tweakNode,

 data.tweakNodeSrcAttr,

 modifierNode,

 data.modifierNodeDestAttr );

 MCheckStatus( status, “tweak-modifier connect” );

 }

 else

 {

 status = fDGModifier.connect( data.upstreamNodeShape,

 data.upstreamNodeSrcAttr,

 modifierNode,

 data.modifierNodeDestAttr );

 MCheckStatus( status, “upstream-modifier connect” );

 }

 status = fDGModifier.connect( modifierNode,

 data.modifierNodeSrcAttr,

 data.meshNodeShape,

 data.meshNodeDestAttr );

 MCheckStatus( status, “modifier-mesh connect” );

 status = fDGModifier.doIt();

 return status;

}

connectNodes() is broken down into several subsections. First a general processing data structure is constructed. This contains all of the processing variables required between each of the processing methods. The structure was created to reduce the amount of argument passing between processing methods. From there a set of processing methods are called to collect the necessary data to connect the nodes and also to process node state specific intricacies. Following that it uses the collected data to connect the nodes.

For details on each of the processing methods that follow, refer to the source code in polyModifierCmd.cpp, which is documented thoroughly.

processMeshNode()

The first process method that the connectNodes() runs through is processMeshNode(). It processes all that’s necessary of the polyShape node, which is comprised of the node shape, node transform and connection data (the inMesh plug and attribute). This data is stored inside the passed in modifyPolyData data structure to be used further on in the other process methods. The order that these process methods are run is important.

processUpstreamNode()

For the second process method, processUpstreamNode(), there is a difference between the history exists case and the history does not exist case. ConnectNodes() is only called in the case where history exists or the case where history does not exist but history recording is turned on. The reason for this is that in each of those cases, the addition of history is permissible and more stable and flexible so we process it that way.

If history already exists, processUpstreamNode() uses the inMesh plug obtained in processMeshNode() to retrieve the node directly upstream from the polyShape node. Once you’ve obtained the node, you disconnect the node from the polyShape node so that you’re prepared to insert the modifier node further on. Since all DG connections take place between the mesh node and the upstream node you only retrieve the outMesh plug and attribute for connections further on.

If history does not exist, you need to create the history chain. At the start of each history chain there is a hidden intermediate node that represents the initial state of the polyShape node at the time history was created. processUpstreamNode() does this by calling an MFnDagNode::duplicate() on the polyShape node. Since the duplicate() method also creates a new transform for the duplicate shape, you re-parent the shape node under the same transform as the original shape and delete the transform through the DAG modifier. To the DG, this duplicate shape node behaves the same as the upstream node processed in the case where history exists. That is, all connections take place between this duplicate shape node and the original shape node. Thus you set the upstream node information to the duplicate shape and retrieve the outMesh plug and attribute for connections further on.

processModifierNode()

Like processMeshNode() the third process method, processModifierNode(), retrieves the inMesh and outMesh attributes of the modifier node for the connection of the nodes at the base of connectNodes().

processTweaks()

Because of history, tweaks need to be extracted prior to the addition of the modifier node to maintain the order of operations. So if tweaks exist, you go through two stages:

• Tweak extraction
• Tweak application

Begin by creating a tweak node (polyTweak) to store the tweaks and begin to parse the pnts attribute for tweaks. The tweaks are individually cached into a class array member for undo purposes as well as in a local MObject array for the purposes of transferring over to the tweak node. Although tweaks are composed of the vector arrays that you just extracted, that is not all that you must account for during the extraction. Tweaks are stored on a pnts attribute. As an attribute in the DG, it can also contain connections. These connections must be preserved and transferred over to the tweak node to maintain the DG structure.

In addition to each vector array, you extract a plug array of each node that is connected to the pnts attribute as well as each plug in the pnts attribute that is connected. Note that each attribute in the pnts array attribute is a compound attribute. Each attribute in the compound attribute is associated with a single axial translation for the given vertex tweak vector (that is, x, y, z).

Once you’ve extracted the data, you apply the tweak vectors on the polyTweak node and reconnect any connections on each tweak vector. Now you have a tweak node that is ready to be connected to the history stream. But if you connect it as is, you would expect the resulting mesh to have double the tweaks. This is because the tweaks have not been removed from the mesh node. So the tweaks would be applied twice, once from the tweak node and once from the mesh node. However, recall that you used an MObject array to retrieve the plugs. These retrieved an MObject reference to the compound plug that was contained in the array attribute. Setting these back onto the tweak node moved them over to the tweak node. There is a large segment of code that is commented out in processTweaks() that removes the tweaks from the mesh node. Executing this code gives the same result, however it slows down the performance with no apparent benefit.

Like the other process methods, processTweaks() also retrieves connection data for the tweak node to allow connectNodes() to connect all the nodes.

connectNodes() revisited

All the necessary nodes have been set up and the connection data extracted. The node connections are set up via MDGModifier::connect calls(). And all the operations are executed via a final MDGModifier::doIt() call. From there you can implement undo support by calling the undoIt() in the opposite order that the DG modifiers were executed in, and implement redo support as the doIt() calls in the same order as before. This is accomplished since the DG modifier caches all the executions that it made. Likewise, polyModifierCmd caches all the necessary data through the doModifyPoly() call. This leaves the undoModifyPoly() and redoModifyPoly() relatively simple.

undoModifyPoly()

undoModifyPoly is very similar to the doModifyPoly() method in structure and level of abstraction. It looks at the scenario from which it is being called (that is, node states) and caters the control over to the appropriate methods:

MStatus polyModifierCmd::undoModifyPoly()

{

 MStatus status;

 if( !fHasHistory && !fHasRecordHistory )

 {

 status = undoDirectModifier();

 }

 else

 {

 fDGModifier.undoIt();

 // undoCachedMesh() must be called prior to

 // undoTweakProcessing() because undoCachedMesh()

 // copies the original mesh without tweaks back

 // onto the existing mesh. Any changes made before

 // the copy will be lost.

 //

 if( !fHasHistory )

 {

 status = undoCachedMesh();

 fDagModifier.undoIt();

 }

 status = undoTweakProcessing();

 }

 return status;

}

undoDirectModifier()

The undoDirectModifier is not as simple as directModifier. Since polyModifierCmd is not aware of what the directModifier does, it must revert the entire mesh back into its original state. It does this through a unique use of MObject handles, making use of the knowledge of how MObjects work.

MObjects are said to be handles to internal objects owned by Maya. That is partly true. MObjects are handles, but Maya does not always own them. There are certain objects in Maya that are reference counted. That is, for each reference to the object a count is incremented. Once each reference is deleted the count is decremented until it reaches zero where the object is deleted. It’s like an object that lives only if it is being used. MObjects may refer to such reference counted objects that increment the reference count. Thus even though Maya owns these objects, you can have some control over the lifetime of that object by holding onto the MObject handle to such an object. For other types of objects the general tip for not hanging onto an MObject is valid and still highly recommended as the data can change between calls to a plug-in (for example, a deleted node).

The types of data that are reference counted are objects classified under the MFnData and MFnComponent class hierarchy. Fortunately the MFnData object type includes the entire contents of a mesh. Using this concept as a foundation for backing up a mesh we can properly undo the direct modifier.

cacheMeshData()

cacheMeshData() caches the data on the use of the mesh prior to the directModifier(). Caching the mesh data makes use of the MObject concept (see undoDirectModifier()). To make a backup of the data you must be careful not to retrieve the reference to the current object, as you will then be holding onto a reference of possibly dirtied data. To get around this duplicate the current mesh and retrieve an MFnData object by extracting an MObject off the outMesh attribute of the duplicate mesh. The MObject that you retrieved is a reference to the original mesh data and thus it increases the reference count. To make this transparent to the user finish up by deleting the nodes created by the duplicate. Note that this does not delete the mesh data since you now have a reference to it.

cacheMeshTweaks()

This method is similar to the processTweaks() method except it does not deal with tweak nodes. Instead it parses the pnts attribute and extracts the tweak vectors into the tweak cache data members.

undoDirectModifier() revisited

You can put the caching of data required to backup our mesh to use in undoDirectModifier(). There is a distinct difference in the data flow of a polyShape node with tweaks and without. This directly affects the way the backup mesh is reapplied.

Tweaks affect the point of reapplication of the backup mesh for reasons of the data flow inside the node. For a node without tweaks you can set the value of the outMesh to the backup mesh.

For a node with tweaks you have to use a trick to force the copy of the backup mesh to the cachedInMesh to keep the node in sync. In this case you duplicate the polyShape node, set the outMesh of the duplicate shape to the backup mesh, and then connect the duplicate shape node to the original node. After forcing an evaluation, disconnect the duplicate shape node and delete it. This implicitly forces the outMesh to hold the backup mesh. Then reapply the initial tweaks to the node via undoTweaksProcessing, forcing the node to copy the backup mesh to the cachedInMesh and perform the internal node synchronization.

undoModifyPoly() revisited

For the case where connectNodes() needs to be undone, the MDGModifier::undoIt() method recalls all the connections and nodes created by connectNodes(). Recall that processUpstreamNode() also made use of a DAG modifier in the case where there was no history initially. In this case you need to perform some extra operations prior to calling the MDagModifier::undoIt().

The reason for this is that since you created history and you’re undoing the operation you need to remove history. Removing is straightforward, however the node’s outMesh attribute still holds the last known evaluation of the node—it still contains the modifier. As a result you need to undo the “cached” mesh on the node. To do this, call the method undoCachedMesh(). Following the restoration of the mesh data, restore the tweaks leaving ~ with the original mesh.

undoCachedMesh()

Similar to the undoDirectModifier() case, the operations contained in undoCachedMesh() are dependent on the presence of tweaks due to the change in data flow through the node. This is because you are reverting to a node without history and you need to restore the mesh by directly interfacing with the node.

For the case where tweaks do not exist, you only need to restore the outMesh since it becomes the geometry that represents the node. To do this, use the duplicate shape node that was created to start a history chain (recall that this is only done if history was not initially present). Retrieve the outMesh of the duplicate shape node and copy that mesh data over to the outMesh of the shape node, restoring the node to its initial state.

For the case where tweaks do exist, you need to access the node in a similar manner to the way undoDirectModifier handles the tweaks case, however this time you don’t need to duplicate the shape node since you already have one. Similarly reconnect the outMesh of the duplicate shape to the inMesh of the original shape through a local DG modifier and force a DG evaluation. Then undo the connection via the same DG modifier. The original mesh data is subsequently forced into the outMesh prior to the reapplication of tweaks whereupon the outMesh will be copied to the cachedInMesh.

redoModifyPoly()

redoModifyPoly() is straightforward because doModifyPoly() initialized and set up all that was necessary for the operation. redoModifyPoly() holds a similar structure to its counterparts. For the directModifier() case, the doModifyPoly() recalls that method without caching any of the mesh data again, since the class already has it. Otherwise, in the connectNodes() case, it recalls the MDGModifier::doIt() to redo the operations previously set up by doModifyPoly():

MStatus polyModifierCmd::redoModifyPoly()

{

 MStatus status = MS::kSuccess;

 if( !fHasHistory && !fHasRecordHistory )

 {

 MObject meshNode = fDagPath.node();

 // Call the directModifier – No need to pre-process the

 // mesh data again.

 //

 status = directModifier( meshNode );

 }

 else

 {

 // Call the redo on the DG and DAG modifiers

 //

 if( !fHasHistory )

 {

 status = fDagModifier.doIt();

 }

 status = fDGModifier.doIt();

 }

 return status;

}

polyModifierFty

From the inner workings of polyModifierCmd, you see that there are two spots where we could potentially have redundant code: directModifier() and inside the modifier node. Effectively they do the same thing, except have a different means of retrieving their inputs. To reduce code duplication, the concept of a factory is introduced. The factory exists solely as a class structure which implements a modification to a mesh. It possesses a basic interface through which the modifier can be called.

To help guide you, a base factory class named polyModiferFty is provided from which a factory can be derived. Though it serves no functional purpose, it provides an outline through which you can implement your modification:

class polyModifierFty

{

public:

 polyModifierFty();

 virtual ~polyModifierFty();

 // Pure virtual doIt()

 //

 virtual MStatus doIt() = 0;

};

polyModifierNode

Similar to polyModifierFty, another guidance class is provided to give a framework for all modifier nodes to be used in association with polyModifierCmd. This class suggests things such as, the modifier node requires an inMesh and outMesh attribute to work:

class polyModifierNode

{

public:

 polyModifierNode();

 virtual ~polyModifierNode();

 // There needs to be an MObject handle declared for

 // each attribute that the node will have. These handles

 // are needed for getting and setting the attribute

 // values later.

 //

 static MObject inMesh;

 static MObject outMesh;

};

For further details on implementing a polyModifierCmd command, refer to both the splitUVCmd example (next) and the source code. Inside the source code a general set of guidelines for each class is provided.