Geometry shaking from frame to frame when exported to .mi
In some cases, files exported to .mi for rendering with mental ray standalone appear to have shaking geometry from frame to frame. In this case, there are three possible solutions.
When exporting to a .mi file, export using the binary file format instead of ascii to prevent data loss due to conversion.
The loss in precision is due to mental ray for Maya exporting only six digits of precision for float point positions, and 15 digits of precision for double point positions. For example, the float point value 1000.2249 is rounded to 1000.22 when exported.
To increase the precision, you can use the Export Float Precision, and Export Double Precision dynamic attributes.
Create these attributes by entering the following in the status line:
addAttr -type short -ln “exportFloatPrecision” mentalrayGlobals
addAttr -type short -ln “exportDoublePrecision” mentalrayGlobals
Set these attributes as follows:
setAttr mentalrayGlobals.exportFloatPrecision <value>
setAttr mentalrayGlobals.exportFloatPrecision <value>
where <value> is the number of digits of precision you want to use.
Avoid using large object offsets with very tiny shape deformations, which results in vertex positions such as 1000.000001. This exceeds the precision of floating point representation in general, and the .mi format does not have a double representation of vertex positions.In situations where the default precision is not sufficient to represent the scene in a .mi file, the Maya model should be reviewed to avoid extreme values:
Some Maya modeling features such as skinning are known to bake global transformation into the final shape properties, which may lead to the described problems in extreme cases.
mental ray for Maya rendering errors
If you experience mental ray for Maya rendering errors, and you save your Maya scene, you should not try to re-render your scene immediately. Instead, exit and restart Maya.
Error message try -memory error appears
When there is not enough
memory to render your scene, you may see an error message that is
similar to "MEM 0.3: TRY -MEMORY 314".To troubleshoot, select
Render > Render Current Frame > 
and
increase the Memory Limit to be higher than or equal to that stated
in the error message.In general, however, when an out of memory
condition occurs, you should also review your scene for possible
optimization.
Some older derive from Maya settings no longer apply
Because of the (as of Maya 5.0) unified render settings (see Open the Render Settings window), some of the mental ray for Maya derive from Maya settings no longer exist. You may need to modify some older files for which derive from Maya was set by setting the mental ray for Maya render settings.
For known differences between mental ray for Maya and Maya, see mental ray for Maya renders look different than Maya renders.
Object still renders when primary visibility is turned off
When an object is occluded by a transparent object, for example, an image plane, the object still renders even if its Primary Visibility flag (under the Render Stats section of the object’s Attribute Editor) is turned off.
Turn off the Visible in Transparency flag instead (under the mental ray section of the object’s Attribute Editor).
This behavior only occurs in mental ray for Maya. This is because, unlike the Maya renderer, mental ray for Maya distinguishes between Primary Visibility and Transparency. When Primary Visibility is turned off, mental ray still recognizes the geometry that is behind the transparent occluding object and includes it in the render. To exclude this geometry, disable the Visible in Transparency flag.
mental ray for Maya renders look different than Maya renders
The following sections describe known differences between Maya and mental ray for Maya and provide hints on how to handle those cases.
Scanline only rendering is the default render mode in Maya. Since scanline rendering is limited in various aspects, it may show artifacts when used together with the following: motion blur, volume rendering, shadow tracing, and others. Raytracing should be turned on if render problems appear.
mental ray uses an adaptive tesselation approach to better fit the details of the displacement map. The initial tesselation (for NURBS: after surface approximation) is further subdivided to fit secondary criteria, controlled by the displace approximation settings. By default, mental ray for Maya creates appropriate settings that lead to curvature dependent subdivision of the triangles, one form of feature-based displacement mapping. You can change these settings by creating or selecting a custom displace approximation node in the Approximation Editor. For more information on Approximation Nodes, see Approximation nodes).
If the mental ray shadow map attributes on Maya lights are disabled, mental ray for Maya derives most parameters for shadow maps from Maya's depthmap settings, although most control values are not comparable. This causes different results, especially when modifying the filter values to achieve soft shadows. For sharp shadows the Filter Size should be set to 0. Other controls, like Use Mid Dist and Use Auto Focus are not used to derive mental ray shadowmap values.
For information about the way mental ray for Maya handles shadow mapping, see Shadow in Maya in the Lighting guide.
The bump mapping implementation in the mental ray shaders may handle the Filter settings in the bump nodes slightly differently compared to Maya. This filter usually produces view-dependent bump mapping details, but might not be appropriate in animations. As soon as the filter has been disabled by setting the Filter value to zero, the Filter Offset determines the bump map lookup, both in Maya and mental ray. This leads to comparable bump mapping render results that are not view-dependent.
Transparent objects produce an appropriate alpha channel in the final image according to Maya’s Matte Opacity settings. However, some restrictions apply:
For more information on transparency, see Surface texture in the Shading guide.
This effect is supported with a mental ray lens shader, not as an output filter as in Maya. Therefore, it is dependent on the global sampling settings in mental ray. Raising the minimum or maximum sampling level improves quality and but slows down the rendering accordingly. On the other hand, as a true 3D effect, it won't show any artifacts with problematic scenes, where output filters are not able to produce correct depth-of-field blurring.
When assigning a group to a render layer, mental ray for Maya assumes all the members of the group also belong to that layer and inherit its attributes. However, Maya allows members of the group to be assigned to different layers than the group parent.
Particle system translation and rendering
Particle types for software rendering are supported on all platforms for rendering with mental ray. To render particles, a new mayabase shader library is required, which is also provided for all platforms. Particle data are also exported to .mi files. These files cannot be rendered on machines with different byte ordering (big end or little end) than the machine where Maya has been used (for example, mixing Linux and Windows). Particle translation and rendering is limited and may not be able to handle large particle counts. Light linking is not supported with particles. However, software particles and particle instancing should work together seamlessly.
Tearing off mental ray for Maya Render menu may crash Maya
If you tear off the Maya Render menu, then load the mental ray for Maya plug-in and tear off the mental ray for Maya Render menu, Maya may crash.
mental ray for Maya crashes or won’t load when a firewall is active
Firewall software may block command ports. When you load mental ray for Maya, your firewall prompts you if you want "to allow the Maya application to access the internet: IP 127.0.0.1 port 1333." If you select no, Maya instantly exits without a warning or error message. If you select yes, mental ray for Maya starts up as expected.
Rendering speed is slower than expected when submitting multiple renders on a muli-core machine
When several renders are submitted on a multi-core machine, the amount of memory being used is much higher. On the other hand, a single render across multiple cores yields better rendering speed because memory can be shared.