Using Memory-Mapped Textures

 
 
 

When a texture is memory-mapped, it means that the texture is never loaded into memory when rendering. Instead, it is accessed directly from disk whenever a shader uses it. When used properly, memory mapped textures can speed up rendering considerably.

For example, if scene and its textures are large enough that they don't fit into physical memory (RAM), loading a non-memory-mapped texture means loading the file into memory, decompressing it, and copying it out to swap (the disk partition that acts as a low-speed extension of the system's RAM). From then on, the texture is accessed from the swap.

If you use memory-mapped textures in the same situation, the "read-decompress-write" step is eliminated and the textures are accessed from the file system instead of from the swap; therefore, less space is needed. In other words, instead of accessing the entire texture, mental ray only accesses the parts that it needs to access to render a given tile.

If a texture is memory mappable, the mental ray renderer recognizes it and memory maps it automatically. Any file format can be mapped, as long as mental ray recognizes the file format from the file's header.

To create memory mapped textures, you can either have Softimage create them automatically, as described Creating Memory-Mapped Textures Automatically, or you can create them manually, as described Creating Memory-Mapped Textures Manually.

When to Use Memory Mapped Textures

  • Memory mapping works best if there are extremely large textures containing many tens of megabytes that are sampled infrequently, because then most of the large texture files are never loaded into memory.

    Note

    Memory-mapped textures are not decoded or converted by mental ray when they are read in: mental ray expects memory-mapped textures to be in the exact format that it uses during rendering.

    Normally, mental ray does attempt to automatically convert image-data formats. For example, if a color image file is to be used as a scalar texture, mental ray converts the color pixels to scalars as the texture is read in. This does not apply to memory-mapped textures.

  • Memory mapping also works well with elliptical filtering, see Filtering Image Clips for Rendering. This is especially true when you auto-generate memory mapped textures (see Creating Memory-Mapped Textures Automatically), since the auto-conversion process automatically creates pyramid-mapped textures that are well suited for elliptical filtering.

When Not to Use Memory Mapped Textures

If used incorrectly, memory-mapped textures can have a negligible, or even negative effect on rendering speed. For example:

  • If the texture and scene are not large and fit into memory, memory-mapped textures are slower than regular textures because the swap would not have been needed to load the textures in the first place. This is especially true if the textures are accessed frequently.

  • If the texture is applied in such a way as to be completely visible in the rendered image, memory-mapped textures have the same memory requirements as their non-memory-mapped counterparts.

  • In a distributed rendering setup where textures are stored on a remote server, using memory mapped textures can slow down rendering considerably. This is because the textures are accessed frequently, and the resulting network traffic can cancel out the speed increase afforded by memory mapping. You can avoid this by copying your memory-mapped textures onto each render slave. For more information, see the following section, Memory-Mapped Textures and Distributed Rendering.

Memory-Mapped Textures and Distributed Rendering

If you are using memory-mapped textures in a distributed rendering environment, it is crucial that before you begin rendering, each such texture is copied to the same location on every render slave machine.

This is because memory-mapped textures are automatically set to Broadcast mode, which means that during rendering, mental ray broadcasts the texture image's path and filename to all of the render slaves, instead of sending the image's pixel data (see Image Clip Property Editor [Properties Reference] for more information).

If the memory-mapped textures are stored on an NFS-mounted file system (one that is imported across the network from another host), then all of the render slaves have to access the texture from the same single location. The resulting network traffic reduces the effectiveness of memory mapping and can easily make rendering slower than it would have been using regular textures.

There are numerous ways to prepare your memory-mapped textures for distributed rendering. One of these is described in the following procedure:

Example: Preparing memory-mapped textures for distributed rendering

  1. As you are creating your scene, make sure to activate the Automatically Convert Images to .map Files rendering preference. For more information, see Creating Memory-Mapped Textures Automatically.

  2. Copy the resulting .map files to the same location on each render slave machine. For this example, assume the location is C:\mr_map.

  3. Before you render your scene, open a Softimage command prompt and define a new environment variable. For this example, you would type: set MR_MAP=C:\mr_map

  4. Start Softimage and load the scene you want to render.

  5. From the main menu, choose File External Files to open the External Files Manager.

  6. Set the File Type to Pictures and, for each memory-mapped texture, change the path and file extension to look like this. Change:

    C:\projects\pictures\wood.pic

    to:

    $MR_MAP\wood.map

  7. Edit the render region options, and the render options for each pass. On the Optimization page of each property editor, set the Automatic Map Conversion Type option to Off for this pass.

  8. On each render slave machine, edit the setenv.bat file to add the environment variable that you created earlier:

    set MR_MAP=C:\mr_map

And you're done. The scene is now ready to be rendered using distributed rendering.

Creating Memory-Mapped Textures Automatically

The easiest way to create memory-mapped textures is to have Softimage generate them automatically by converting some, or all of the scene's image clips to .map format at render time.

The auto-generated .map files are stored in a subdirectory (named mr_map) of the directory where the original image sources are stored. The .map files are used for rendering only, while the original image clips are used to display the textures within Softimage.

Some of the advantages of auto-generating .map files are:

  • It's automatic, so you don't have to spend time manually converting your textures to .map format.

  • By default, the auto-generated .map files are used automatically so you don't have to update paths with new file extensions before you render.

  • Because the .map files are created from the scene's image clips, changes that you make to the clips (using the compositing and/or 2D paint tools, for example) are automatically incorporated into their .map counterparts.

Although auto-generating .map files is controlled by user preferences that affect the scene globally, you can override these preferences both in the render region and in any individual render pass. This is described in Controlling Auto-Generated .map File Use.

To have Softimage auto-generate .map files

  1. Open the Preferences view (choose File Preferences from the main menu) and click Rendering in the explorer pane.

    The Rendering Preferences property editor opens in the right pane.

  2. Set the following two Map Conversion options:

    • Automatically Convert Images to .map Files: when activated, all of the image clips that are used in your scene, and meet the Minimum Size requirement, are converted to .map files.

    • Minimum Size: specifies the minimum resolution necessary to convert an image to .map format.

      Images whose resolution in either axis is greater than or equal to this value are automatically converted. Images whose resolution in both axes is less than this value are not converted.

      This helps you avoid converting images that are small enough that memory mapping them does not affect rendering speed.

Controlling Auto-Generated .map File Use

Whether or not you've activated automatic .map file conversion in the Rendering preferences (as described in the previous section), you can specify whether it should be used for a given render pass, and/or in the render region, by doing the following:

  1. Do one of the following:

    • Choose Render Render Options from the Render toolbar to open the Render Options property editor for the current pass.

    • Choose Render Regions All Options to open the View Render Options property editor for the render regions.

    • Choose Render Regions Active Viewport Options to open the View Render Options property editor for the last viewport you clicked in (highlighted with a white border).

  2. On the Optimization tab, set the Automatic Map Conversion Type parameter to one of the following:

    • Use Preferences: .map files are generated and used at render time only if automatic .map file conversion is activated in the rendering preferences.

      or

    • On for this pass/for region render: .map files are generated and used when you render the pass/region, regardless of whether or not automatic .map file conversion is activated in the rendering preferences.

      or

    • Off for this pass/for region render: .map files are not generated when you render the pass/region, regardless of whether or not automatic .map file conversion is activated in the rendering preferences. Previously auto-generated .map files are not used when rendering the pass as long as you maintain this setting.

Creating Memory-Mapped Textures Manually

If you didn't (or don't want to) have Softimage automatically convert your image clips to .map format, you can manually create memory-mapped textures in any of the following three ways:

  • Using the Image Clip Viewer.

  • Using the flipbook.

  • Using the imf_copy standalone.

Use whichever method is most convenient for you. Once you have created the textures, you'll have to update your scene to use them, as described in Updating the Scene File.

Tip
  • The mental ray renderer recognizes .map textures even if they have an extension other than .map. This means that you can continue to use the "non-.map" texture references in your scene file.

  • If you want to paint or retouch a manually created memory-mapped texture, you should give it a .tga or .pic file extension until the image is finalized, then convert it to .map format.

ImportantA file that has been made mappable on a Windows platform cannot be copied to the Linux platform and vice-versa.

To create a memory-mapped texture from the image clip viewer

  1. Open the image clip viewer (choose Views Rendering/Texturing Image Clip Viewer from the main menu).

  2. From the image clip viewer's menu, choose File Import. A browser is displayed. Select the image to import and click OK.

  3. From the image clip viewer's command bar, choose File Export. In this browser, select a destination for the image, make sure that the File Type is set to .map, and click OK.

To create a memory-mapped texture from the flipbook

  1. Open the flipbook by choosing Flipbook from the Playback menu in the Playback panel.

  2. In the Open Images dialog box, select an image to open and click OK.

  3. From the flipbook's command bar, choose File Export.

  4. In the Export dialog box, save the image to the appropriate directory, making sure that the File Type is set to .map.

To create a memory-mapped texture using the imf_copy standalone

  1. Open a Autodesk Softimage command prompt.

  2. Type the path of the folder where the file(s) that you want to convert are stored (for example: C:\projects\myproj\pictures).

  3. Do one of the following:

    • To convert a single image, type the following command (all on one line):

      imf_copy [-v] [-p] inimage outimage [outformat [outtype]]

      For more information about using imf_copy, type imf_copy -h in the command prompt for command line usage or see Standalones.

    • To batch convert multiple images in the same directory, type the following command (Windows):

      for %i in ([informat]) do imf_copy [-v] [-p] %i %~ni.map

      For example, if you have a directory full of .pic, .tga, and .jpg files that you want to convert, you would type (all on one line):

      for %i in (*.pic *.tga *.jpg) do imf_copy [-v] [-p] %i %~ni.map

      Tip

      In both of the previous examples, the -p option outputs the .map file in pyramid-mapped format. Pyramid mapping pre-filters the texture into multiple resolutions. At render time, the resolution appropriate to the texture's distance from the camera is used.

      This can speed up rendering, especially when the higher-resolutions of the texture are not required. However, it does require more space to store the textures. If disk space is a concern, you can use imf_copy without the -p option to generate your .map files.

Updating the Scene File

After manually converting your textures to .map files, you must update the scene file to reference these files instead of the original files.

NoteOnce you've updated your scene, working with the memory-mapped textures is the same as working with any other textures. You can still preview textures in Textured display mode and you can still see thumbnails of the .map files in the Image Clip property editor.

To update the scene file

  1. Load the scene and choose File External Files from the main menu.

  2. In the External Files Manager, click the Search and Replace button to open the Search and Replace dialog box.

  3. Set the File Type to Pictures.

  4. Using the String to Search and String to Replace text boxes, search for each file format's extension (.pic, .tga, .jpg, and so on) and replace it with .map.