Using and Writing Shaders

All color, displacement, contour, and other computation in mental ray is based on shaders. There are various types of shaders for different situations, such as material shaders to evaluate the material properties of a surface, light shaders to control the light-emitting properties of a light source, lens shaders to specify camera properties other than the default pinhole camera, and so on. Much of the power of mental ray relies on the possibility to write custom shaders and call them during rendering in mental ray. Traditional custom shaders are written in C/C++, using all language features and the complete shader interface in mental ray for advanced effects. On the other hand, MetaSL shaders may be used to write functions for common shading tasks like defining material behavior in a standardized form independent of the computing platform or the final renderer.

Vendors and integrators of mental ray typically provide custom shader libraries which implement compatibility with their products, like Autodesk 3ds Max and Maya, Softimage, Dassault Systèmes CATIA and SolidWorks, and others.

MetaSL Shaders

MetaSL shaders are provided in files with .msl or .xmsl file extension. If such a file name is announced to mental ray, for example with the $include statement in a .mi file), it is treated as MetaSL shader source code. mental ray will read, parse, and convert it to an internal representation. This internal representation is then used to create a shader declaration, as well as intermediate shader code in a native language of the current platform for final compilation/linking and execution in mental ray. The shader code generation is performed by so-called MetaSL back-ends. mental ray currently provides two back-end alternatives :

C++ back-end using an external compiler installed in the target machine,
.NET back-end on Windows platforms using built-in just-in-time compilation.

The C++ back-end is currently the default. An external C++ compiler and linker is automatically invoked on intermediate source files to create a dynamically linked shared library in the directory containing temporary files. That resulting shared object .dll or .so is then loaded into mental ray. In order for C++ compilation to succeed, the C++ compiler needs to be pointed to include directories containing mental ray headers files (like shader.h and others), as well as MetaSL C++ header files (in the directory rooted at mi/metasl_runtime/*). For convenience, mental ray now applies the common mental ray include path also to the compiler invocations automatically. In other words, the include paths may be specified on the mental ray standalone command line (option -I) or with a registry variable ({_MI_REG_INCLUDE}).

On Windows platforms, shaders need to be linked against shader.lib or ray.lib. If shader.lib is not present in the compiler library search path, it needs to be specified explicitly (like with command-line options -ld_libs and/or -ld_path).

Note, that the C++ compilation and linkage is done on demand. For shader declaration, parsing and analysis of the MetaSL source code is done when the MetaSL file is loaded.

The .NET back-end is optional, because it has several known limitations in the current implementation. It is using the .NET Framework from Microsoft available for all modern Windows platforms. To enable it the {_MI_REG_METASL_CIL} mental ray registry variable need to be set to any non-empty value. This will automatically disable the C++ MetaSL back-end. Once enabled it works as follows:

The built-in MetaSL back-end generates .NET assembly code in CIL language (CIL - common intermediate language). An external Microsoft .NET (version 2.0 or higher) CIL assembler executable is invoked on that .cil file and generates a managed code DLL which is linked into mental ray. The assembler is part of the Microsoft .NET Framework redistributable package, so no installation of a separate C/C++ compiler is required.

Native Shaders

Here are the steps necessary to create a native shader:

Write a .mi declaration for the shader, providing the shader name, return type, shader parameter names and types, and the version to mental ray.
Write a C shader parameter data structure that agrees exactly with the .mi parameter declaration. (The mkmishader utility can do this translation.)
Write the shader function in C or C++, using the correct signature. The shader computes a result from its shader parameters and the state, and by calling shader interface functions provided by mental ray.
Write a version shader (same name with _version appended to the name) that returns a version number that matches the version number in the .mi declaration.
If required, write initialization and exit shaders (same name as the shader with _init and _exit appended, respectively).
Compile and link the shader, and create a shared shader library (DSO or DLL). This step can be omitted but DSO/DLLs are easiest to use and much faster to link than objects or sources. Libraries must be installed on all machines on the net that are used as masters or network rendering slaves.
Use a $include statement in the scene .mi file to load the .mi declaration and a link (or code or $code) statement to load the shader DSO/DLL (or source code).

Note that if the shader is expected to work on Windows NT, all four function definitions (the shader, the version function, and the init and exit shaders) must be preceded with DLLEXPORT. This is a pseudo type specifier that makes the functions visible to users of the generated shader library. On Unix systems DLLEXPORT evaluates to an empty word.

The shader is then ready to be used in the scene. These steps are described in detail below.