Write MEL Scripts for the Paint Scripts Tool
 
 
 
Note

This section assumes knowledge of MEL scripting and programming. For more information about MEL, see the MEL and Expressions guide.

The Paint Scripts Tool is based on the concept of a two-dimensional array of numeric values super-imposed on a NURBS or polygonal surface. This array is either defined by the vertex positions of the surface or an arbitrary 2D grid that is evenly spaced in the surface parameter space. Think of this array of numbers as a 2D grayscale image, where each pixel corresponds to an array position and the grayscale value corresponds to the number associated with that array position. The scripts associated with the Paint Scripts Tool determine how Artisan interprets this array of numbers. Artisan calls the script when it needs to know the number assigned to one of the array positions as described above. Artisan also calls the script when it changes a number assigned to one of the array positions.

Script Paint MEL procedures

The Paint Scripts Tool is defined by a set of MEL procedures. The names of the MEL procedures display in the Setup section of the Tool Settings editor. You can also set up the MEL procedures using the artUserPaintCtx MEL command. This chapter describes the following Artisan MEL procedures:

In the following descriptions, the artUserPaintCtx flag used to set the command, is in brackets.

Tool Setup Command (-tsc "ToolSetupCommand")

If defined, ToolSetupCommand is called just after the Paint Scripts Tool is selected (either by using the Modify > Paint Scripts Tool menu item or selecting it from the Tool Box or the shelf). This procedure is often used to set up all the other Artisan MEL procedures. Define this procedure as follows:

global proc ToolSetupCommand (
	string $toolContextName
)
{
	// This is an example artUserPaintCtx command that would setup
	// the following Artisan procedures. This is not a realistic
	// example because you would never have all the procedures
	// defined.
	//
	artUserPaintCtx -e
		-tcc "ToolCleanupCommand"
		-gac "GetArrayAttributeCommand"
		-ic "InitializeCommand" -fc "FinalizeCommand"
		-svc "SetValueCommand" -gvc "GetValueCommand"
		-gsc "GetSurfaceCommand"
		$toolContextName;
}

... where $toolContextName is the name of the tool context that was selected. It is meant to be used as the final argument to the artUserPaintCtx command.

Tool Cleanup Command (-tcc "ToolCleanupCommand")

If defined, ToolCleanupCommand is called just before a Paint Scripts Tool is exited. Define this procedure as follows:

global proc ToolCleanupCommand (
	string $toolContextName
)
{
	...
}

... where $toolContextName is the name of the tool context that is exited.

Get Array Attribute Command (-gac "GetArrayAttributeCommand")

If you define this command, any commands defined for the Initialize Command, Finalize Command, Set Value Command and Get Value Command are ignored.

If defined, GetArrayAttributeCommand is called once for every surface that is selected for painting. This procedure returns a string, which is interpreted as a list of names referring to double array attributes on some dependency node. Define this procedure as follows:

global proc string GetArrayAttributeCommand (
	string $surfaceName
)
{
	string $arrayAttributes
	// determine array attributes that correspond to
	// $surfaceName
	...
	return $arrayAttributes;
}

... where $surfaceName is the name of the surface which will be associated with the returned array attributes. If more than one array attribute is returned, the first one is read when Artisan requires a value but all of them are written by Artisan.

Initialize Command (-ic "InitializeCommand")

If defined, InitializeCommand is called once per paintable surface before the beginning of every stroke. Define this procedure as follows:

global proc string InitializeCommand(
	int $surfaceName
)
{
	string $flags;
	// build up $flags
	//
	...
	return $flags;
}

...where $surfaceName is the name of the surface. For examples, see spherePaint.mel and geometryPaint.mel in the following directories:

  • (Windows) Drive:\Program Files\Autodesk\Maya2012\scripts\others
  • (Linux) mayapath/scripts/others
  • (Mac OS X) /Applications/Autodesk/maya2012/Maya.app/Contents/scripts/others

This procedure should:

  • determine a unique integer surface ID for this surface. This ID is passed to future invocations of SetValueCommand, GetValueCommand and FinalizeCommand associated with this surface. If your script maintains per surface information, it is convenient to make this ID an array index, which can be used to easily access the per surface information.
  • determine what sort of information is required by future invocations of SetValueCommand associated with this surface.
  • determine whether to paint on the surface vertices or on an evenly spaced grid in the surface’s parameter space.

All of this information is passed back to Artisan via the $flags variable passed back from the procedure. This is a string that contains a sequence of flags and arguments very similar to the flags and arguments that can be passed to standard MEL procedures. These are the flags that are understood by the Paint Scripts Tool. Each flag has a short and long version (that is, -short/long) plus some arguments:

  • -id/identifier int

    This flag indicates the integer surface ID that is to be used for this surface. If this flag is not specified, the surface ID for this surface is -1.

  • -uv/uvlong string

    This flag indicates whether the (U,V) location should be sent to SetValueCommand. The possible values for this are none, surface or normalized. The default is none.

  • -p/position string

    This flag indicates whether the (x,y,z) position should be sent to SetValueCommand. The possible values for this are none, local or world. The default is none.

  • -n/normal string

    This flag indicates whether the (nx,ny,nz) normal should be sent to SetValueCommand. The possible values for this are none, local or world. The default is none.

  • -g/grid int1 int2

    This flag indicates that painting should occur on an evenly spaced grid in parameter space of size int1 in the surface’s U direction and int2 in the surface’s V direction. If this flag isn’t specified, painting occurs on an array defined by the surface’s vertices.

  • -j/jitter boolean

    This flag is only valid in conjunction with the -grid flag. If it is true, the grid positions are jittered before they are passed to the SetValueCommand. The default value is false.

  • -d/dither boolean

    This flag is only valid in conjunction with the -grid flag. If it is true, Artisan uses a 16x16 dithering matrix to determine whether SetValueCommand should be called for it.

  • -dt/directionType string

    This flag indicates whether the (U,V,W) stroke direction should be sent to SetValueCommand. The possible values for this are none, screenV (a vector in screen space) or worldV (a vector in world space). The default is none. For projection painting, both screenV and worldV return a vector in screen space. The direction option is not supported for reflection painting. The third value (W) will be invalid if the user has requested direction as screen vector.

  • -sp/stampPosition string

    This flag indicates whether the (spX, spY, spZ) stamp position should be sent to SetValueCommand. The possible values for this are none, local or world. The default is none.

Finalize Command (-fc "FinalizeCommand")

If defined, FinalizeCommand is called once per paintable surface after the end of every stroke. Define this procedure as follows:

global proc FinalizeCommand(
	int $surfaceID
)
{
	...
}

... where $surfaceID is the surface identifier assigned to the surface by this surface’s corresponding InitializeCommand.

Set Value Command (-svc "SetValueCommand")

If defined, SetValueCommand is called when Artisan has set the value associated with a particular "location" on the surface. This procedure is called once per location. Therefore, if some Artisan painting operation changes the values at 10 locations on a surface, SetArrayAttributeCommand is called 10 times. Define this procedure as follows:

global proc SetValueCommand(
	int $surfaceID,
	int $index,
	float $value,
	//
	// The following arguments are only passed if requested by
	// InitializeCommand. However this is the order that they
	// will be passed in.
	//
	float $u,			// ($u,$v) is UV location on surface
	float $v,				
	float $px,			// ($px,$py,$pz) is position on surface
	float $py,
	float $pz,
	float $nx,			// ($nx,$ny,$nz) is surface normal
	float $ny,
	float $nz
)
{
	...
}

... where:

  • $surfaceID is the surface identifier assigned to the surface by this surface’s corresponding InitializeCommand
  • $index is the vertex or grid index
  • $value is the value to be assigned to the passed index

The extra arguments passed to this procedure depend on what was returned from the surface’s InitializeCommand.

Get Value Command (-gvc "GetValueCommand")

If defined, GetValueCommand is called when Artisan wants the value associated with a particular "location" on the surface. Define this procedure as follows:

global proc float GetValueCommand(
	int $surfaceID,
	int $valueIndex
)
{
	float $value
	// determine the value associated with $valueIndex location
	// on surface $surfaceID
	//
	...
	return $value;
}

... where:

  • $surfaceID is the surface identifier assigned to the surface by this surface’s corresponding InitializeCommand
  • $valueIndex is the index for the "location" for which Artisan requires the value

Get Surface Command (-gsc "GetSurfaceCommand")

This command is rarely defined. If it is defined, GetSurfaceCommand is called once for every dependency node on the selection list, whenever Artisan processes the selection list. This usually occurs whenever the Paint Scripts Tool is selected. Define this procedure as follows:

global proc string GetSurfaceCommand (
	string $selectedDependencyNode
)
{
	string $surface;
	// Set $surface based on $selectedDependencyNode.
	// Set $surface to NULL string (""), if no surface 
	// is to be painted because of $selectedDependencyNode.
	//
	...
	
	return $surface;
}

This procedure overrides Artisan default handling of the selection list.

Example Script Paint script

This section explains the overall script layout and gives an annotated script example.

Overall script layout

The scripts used for the Paint Scripts Tool can be arranged in many ways. This example script layout is used in the sample Artisan scripts available in:

  • (Windows) Drive:\Program Files\Autodesk\Maya2012\scripts\others
  • (Linux) mayapath/scripts/others
  • (Mac OS X) /Applications/Autodesk/maya2012/Maya.app/Contents/scripts/others

The layout keeps all the MEL procedures in one file and allows all the scripts to be defined by only filling in the Tool Setup Cmd box in the Setup section of the Tool Settings window. This procedure sets up all the other MEL procedures using the artUserPaintCtx command. Here is a very high-level view of this:

// Define global variables used throughout the script
//
global ...;
// Define the Tool Setup Cmd procedure. This procedure will
// setup all the other MEL procedures used by the Paint //Scripts tool.
//
// "spherePaint" would be entered into the Tool Setup Cmd field in
// the Setup section of the tool settings window.
//
global proc spherePaint( string $context )
{
	artUserPaintCtx -e
		-ic "initSpherePaint"
		-fc "finishSpherePaint"
		-svc "setSpherePaintValue"
		-gvc "getSpherePaintValue"
		-gsc ""
		-gac ""
		-tcc ""
		$context;
}
// Define various procedures that were mentioned in the Tool
//Setup Cmd procedure above. All the procedures defined by
//the artUserPaintCtx command have to be global.
//
global proc string initSpherePaint( string $surfaceName )
{
	...
}
...

spherePaint.mel

This is an annotated version of the spherePaint.mel file supplied on:

  • (Windows) Drive:\Program Files\Autodesk\Maya2012\scripts\others
  • (Linux) mayapath/scripts/others
  • (Mac OS X) /Applications/Autodesk/maya2012/Maya.app/Contents/scripts/others

This script works on NURBS surfaces only.

//
// This is a simple example script for the Artisan Paint 
//Scripts tool. It will paint spheres onto the selected
//surfaces. The size of the spheres are controlled by the
//painted values.
//
// Usage:
// 1) Place this script into your scripts directory (usually
//the maya/scripts directory in your home directory
// 2) Select the Paint Scripts Tool (Modify > Paint Scripts
// Tool) and bring up the Tool Settings window
// 3) Go to the Setup section and enter "spherePaint" into 
// the "Tool Setup Cmd" field and hit enter
// 4) Paint Geometry
//
// Tips:
// Once you have the Geometry Paint Tool setup you may want
// to it from the toolbar to the shelf so that it is always
// accessible
//

These global variables are used to determine how many surfaces are being painted and to determine surface IDs. $sphereNamePrefix is an array of strings, one string per active surface. If an entry is the empty string, it means that it’s array index is available to be used as a surface ID. If the entry is non-empty, this string is used as a prefix when creating the spheres. $spherePaintFreeSlot is the first free entry in $sphereNamePrefix and $spherePaintSlots is the current size of $sphereNamePrefix. This approach of using arrays indexed by the surface ID to store per surface information is used heavily in some of the other sample Artisan scripts.

// These are global variables used to keep track of multiple
// surfaces and the name prefixes used for the spheres on each
// surface
//
global string $sphereNamePrefix[];
global int $spherePaintFreeSlot = 0;
global int $spherePaintSlots = 0;

spherePaint is the procedure that initializes the Paint Scripts Tool by telling it what procedures should be called under various circumstances.

// This procedure should be set as the "Tool Setup Cmd" in the 
// Setup section of the Maya Artisan Paint Scripts Tool’s tool settings
// window. The tool context is supplied as an argument.
//
global proc spherePaint( string $context )
{
	// initialize all the other commands in this scriptable 
	// paint tool context.
	// 
	artUserPaintCtx -e
		-ic "initSpherePaint"
		-fc "finishSpherePaint"
		-svc "setSpherePaintValue"
		-gvc "getSpherePaintValue"
		-gsc ""
		-cc ""
		-tcc ""
		-gac ""
		$context;
}
// This is the "Initialize Cmd". This procedure is called once
// for every selected surface when an initial click is received
// on any surface. The argument is the name of the surface. This
// procedure returns a string which indicates to the scriptable
// tool how to behave for the duration of the stroke. 
//
global proc string initSpherePaint( string $name )
{
	global string $sphereNamePrefix[];
	global int $spherePaintFreeSlot;
	global int $spherePaintSlots;
	int $slot;

The first thing to do is determine a surface ID to associate with this surface. This is done by going through the $sphereNamePrefix array, looking for the first free entry. Once it finds an entry, the entry index is used as the surface ID ($slot).

	// find a free slot for this surface in the global arrays
	//
	for ( $slot = $spherePaintFreeSlot; $slot < $spherePaintSlots; $slot++ )
	{
		if ( $sphereNamePrefix[$slot] == "" ) {
			break;
		}
	} 
	if ( $slot == $spherePaintSlots ) {
		$spherePaintSlots++;
		$spherePaintFreeSlot = $spherePaintSlots;
	}

The next thing to do is determine whether the passed in $name corresponds to a NURBS surface. If it does, a prefix is generated for the painted spheres and stored into the appropriate entry in the $sphereNamePrefix array.

	if ( ‘nodeType $name‘ == "nurbsSurface" ) {
		// save the name of the parent of this shape as well
		// as a prefix to use when creating the spheres
		//
		string $parent[] = `listRelatives -p $name`;
		$sphereNamePrefix[$slot] = $parent[0] + "Sphere";
	}

The final function for the initialize procedure is to return a string which indicates to Artisan how to handle this surface. The string is a sequence of special flags and their arguments, as described earlier. In this case, the string includes flags telling Artisan:

  • what surface ID should be used for this surface
  • that painting should occur on a 20x20 grid, evenly spaced in the surface’s parameter space
  • that the world space position should be passed to setSpherePaintValue whenever it is called.
	// Return an argument string which:
	// - tells the tool what surface ID to use for this surface
	// - indicates that values should be distributed on a 20x20
	// grid on the surface
	// - indicate that the associated world space position
	// should also be passed to the "Set Value Cmd".
	//
	return ( "-id " + $slot
		 + " -grid 20 20"
		 + " -position world");
}
// This is the "Finalize Cmd". This procedure is called at the
// end of the stroke. It is passed the surface ID, that was
// generated by the "Initialize Cmd".
//
global proc finishSpherePaint( int $slot )
{

This is called at the end of the stroke. Use $slot to clear out this surface’s corresponding entry in $sphereNamePrefix. Also update $spherePaintFreeSlot with $slot, if $slot is earlier in the array.

	
	global string $sphereNamePrefix[];
	global int $spherePaintFreeSlot;
	// clear out the slot that was used for this surface
	//
	$sphereNamePrefix[$slot] = "";
	if ( $slot < $spherePaintFreeSlot ) {
		$spherePaintFreeSlot = $slot;
	}
}
// This is the "Set Value Cmd". It is called everytime a value
// on the surface is changed. A surface ID, a grid index
// on the surface and the value associated with that grid index
// is passed. There can be additional arguments depending on the
// options generated by the return value of the "Initialize Cmd".
// In this case the (x,y,z) surface position for this grid point
// is also passed.
// 
global proc setSpherePaintValue(
	int $slot,
	int $index,
	float $val,
	float $x,
	float $y,
	float $z
)
{
	global string $sphereNamePrefix[];

Determine if $slot is a valid surface ID by checking if the corresponding entry in $sphereNamePrefix is not the empty string.

	if ( $sphereNamePrefix[$slot] != "" ) {

All the painted spheres have a unique name which combines the $sphereNamePrefix for this surface plus the grid index. This is how this script determines whether something has already been painted at this location. One disadvantage to this method is that it is sensitive to the grid size specified in initSpherePaint. For example an index of 44 would correspond to grid position (4,4) on a 10x10 grid. However if the grid size was changed to 20x20, index 44 would now correspond to grid position (2,4).

		// determine the name of the sphere associated with this
		// grid location 
		//
		string $objname = $sphereNamePrefix[$slot] + $index;
		if ( ‘objExists $objname‘ ) {

In this case the sphere already exists. Therefore, the passed value, $val, is used as the overall scale factor for the spheres. As a special case, if the value is less than or equal to 0, the corresponding sphere is removed.

			// if the sphere already exists, use the value to
			// adjust the size of the sphere. If the value is
			// 0, the sphere is deleted
			//
			if ( $val > 0 ) {
				scale $val $val $val $objname;
			} else {
				delete $objname;
			}
		} else if ( $val > 0 ) {

In this case, there is no painted sphere corresponding to this grid location. Create a sphere using the sphere MEL command using the -name option to give the newly created sphere the desired name. Once the sphere is created, uniformly change the sphere’s scale factor to the passed in value ($val). At this point the sphere is also positioned so that it appears at the right place by moving it from the origin (this is where spheres are created by default) to the world space position that was passed into this procedure ($x,$y,$z).

			
			// the sphere doesn’t exist
			//
			string $sname[];
			// create a sphere with the proper name, scale it by
			// the passed value and parent the sphere to the same
			// parent as the surface we are painting on
			//
			$sname=‘sphere -ch off -name $objname‘;
			if ( $sname[0] != $objname ) {
				print ("SPHERE NAME FAILED: wanted "
					+ $objname + " got " + $sname[0] + "\n");
			}
			scale $val $val $val;
			move $x $y $z;
		}
	}
}
// This is the "Get Value Cmd". It is called everytime a value
// on the surface is needed by the scriptable paint tool. A
// surface ID and a grid index is passed in. This procedure should
// return the value for this grid location on the specified surface.
// 
global proc float getSpherePaintValue( int $slot, int $index )
{
	global string $sphereNamePrefix[];
	if ( $sphereNamePrefix[$slot] != "" ) {
		// if this slot is valid, generate the name for the
		// sphere at this grid index
		//
		string $objname = $sphereNamePrefix[$slot] + $index;

This procedure is called whenever Artisan needs a value corresponding to a specific grid location. In this script, the values are the sphere’s scale factor.

		
		if ( ‘objExists $objname‘ ) {

If a sphere has been painted at the location referenced by $index, then the sphere’s X scale factor is returned. There is nothing magical about the X scale factor. It could just as easily have been the Y or Z scale factor, or the average of all scale factors for that matter. It is entirely up to the script.

			
			// if the sphere exists, return the X scale factor
			// as the value for this grid location
			//
			return ‘getAttr ($objname + ".sx")‘;
		} else {

If there is no sphere at this location, just return 0.0 as the value for this location.

			// the sphere doesn’t exist, therefore return 0 as
			// the value for this grid location
			//
			return 0.0;
		}
	} else {
		return 0.0;
	}
}

Here are some experiments you can try on this script to get an idea of how it works.

Change Grid Size

Change line 89 from:

		 + " -grid 20 20"

to:

		 + " -grid 40 30"
		

Paint Cones instead of Spheres

Change line 154 from:

			$sname=‘sphere -ch off -name $objname‘;

to:

			$sname=‘cone -ch off -name $objname‘;

Jitter the Grid

Add the line:

		 + " -jitter true"

before line 90.