Dynamic Linking of Shaders

Shaders are written as C or C++ subroutines, stored in files with extensions like .c or .cpp. To execute these shaders during rendering of a scene they must be dynamically linked into mental ray at run time. mental ray accepts shaders in three forms:

As dynamically loadable shared library, further referred to as DSO, or Dynamic Shared Object. DSOs are supported on all major platforms. They have the special file extension .so on Unix/Linux/MacOSX systems and .dll on Windows systems. mental ray automatically substitutes .dll for .so on Windows platforms and vice versa, so the same library filename is accepted on any platform regardless of the actual file extension.
Directly as source code. The .mi file offers the code statement, and the standalone application has a −code command line option. Both accept a source file with a .c extension.
Deprecated In object format. Source files can be compiled to object code. This is the most common form on systems that do not support DSOs; it is faster than source because mental ray does not have to compile the shader. A compiler will create an object file like myshader.o with the extension .o (or .obj on Windows) in the current directory.

Using DSOs is the fastest way to load shaders, there is very little overhead. DSOs and object files are loaded using link statements in the scene file or -link command line option.

To create a DSO two steps are typically required:

compile the shader source to native object format,
link the object file as a dynamically loadable library.

The commands to create a DSO depend on the operating system type. To build a DSO named example.so or example.dllfrom a source file example.c, the commands in the following section can be used in a shell (outside an IDE or Integrated Development Environment, like Microsoft Visual Studio or Apple XCode). The commands create a performance optimized version of the shader for final rendering. Further compiler options may be used to tune for specific requirements, like dedicated target CPUs.

During development phase of shaders it is useful to generate debug information with the -g (/Zi on Windows) compiler option, and disable compiler optimizations at the same time by omitting any -O or /O options. On some platforms like Linux, -g and -O can be combined, to find problems which may be introduced only during the optimization process of the compiler. Refer to the compiler documentation of the target platform for details.

Windows x86, cl (Microsoft Visual Studio Compiler)

cl /c /O2 /MD /W3 -DWIN_NT example.c
link /nodefaultlib:LIBC.LIB /OPT:NOREF /DLL /OUT:example.dll example.obj shader.lib

This applies to all Visual C++ 6, 7, 8, or 9, and Microsoft Platform SDK compilers. The shader.lib [13] is a stub library that is delivered together with mental ray. It is required to resolve mental ray shader interface references at link time.

Additional options like /G6 or /G7 apply processor-specific optimizations and may run certain code faster on that target CPU (only).

Windows x86, icl (Intel Compiler Suite)

icl /c /Ox /G7 /MD /W3 -DWIN_NT example.c
xilink /nodefaultlib:LIBC.LIB /OPT:NOREF /DLL /OUT:example.dll example.obj shader.lib

The /G7 compiler option applies to special target CPU only, and may need to be adjusted to benefit from capabilities of newer CPUs.

Windows x64, cl (Microsoft Visual Studio Compiler)

cl /c /O2 /MD /W3 -DWIN_NT -DBIT64 example.c
link /nodefaultlib:LIBC.LIB /OPT:NOREF /DLL /OUT:example.dll example.obj shader.lib

Almost identical to 32bit compilation, with the BIT64 conditional defined for mental ray public headers.

Linux x86, gcc (GNU Compiler)

gcc -c -O3 -fPIC -Bsymbolic [14] example.c
ld -export-dynamic -shared -o example.so example.o

Linux x86, icc (Intel Compiler Suite)

icc -c -O2 -KPIC -Bsymbolic -tpp7 example.c
ld -export-dynamic -shared -o example.so example.o

Linux x64, gcc (GNU Compiler)

gcc -c -O3 -fPIC -Bsymbolic -DBIT64 example.c
ld -export-dynamic -shared -o example.so example.o

Mac OS X

cc -c -O3 -fPIC -dynamic -fno-common example.c
libtool -flat_namespace -undefined suppress -dynamic -o example.so example.o

The following section proposes build rules for other, less frequently used platforms.

Windows IA 64, ecl (Intel Compiler)

ecl -c /O2 /MD /W3 /GR /Zp8 /W3 /GX /GR /GF -DWIN_NT -DBIT64 example.c
xilink /nodefaultlib:LIBC.LIB /OPT:NOREF /DLL /OUT:example.dll example.obj shader.lib

Linux IA 64, gcc (GNU Compiler)

gcc -c -O3 -KPIC -Bsymbolic -DBIT64 example.c
ld -export-dynamic -shared -o example.so example.o

IRIX, 32 bits

cc -n32 -O3 -shared -o example.so example.c

IRIX, 64 bits

cc -64 -O3 -DBIT64 -shared -o example.so example.c

HP/UX HPPA

cc -c -Aa +z example.c
ld -b -o example.so example.o

HP/UX IA 64

cc -c +O3 +Onolimit +DD64 -Ae -Bprotected_def -Bdefault:keepsym.exp example.c
cc +DD64 -b -o example.so example.o

IBM AIX

cc -c -O3 example.c
ld -o example.so example.o -bE:example.exp extra.so -bI:keep.exp -bM:SRE -T512 -H512 -lc

The file example.exp is a user supplied export file for the shader and must contain the names of the exported symbols to be seen by mental ray, one entry per line. The file keep.exp is the list of symbols exported by mental ray, either for use by shaders or for internal linking purposes. It is supplied with mental ray and can be referenced using an absolute path name. The optional file option extra.so needs to be supplied only when functions from other shader libraries that are pre-loaded during run time (such as physics.so) are used directly in the shader.

Sun Solaris (Sun CC)

cc -c -xO3 -KPIC -D_REENTRANT example.c
ld -lm -G -o example.so example.o -ldl

The GNU compiler gcc can also be used, see Linux above.

Tru64 Unix

cc -c -newc -O2 -ieee_with_no_inexact -DBIT64 -D_REENTRANT example.c
ld -expect_unresolved '*' -shared -o example.so example.o

To be able to compile and create DSOs requires that a C or C++ development environment is installed on the system. If the cc, ld, cl, link, etc. commands are not found, then the development environment may not exist or is not set up for the user. On most platforms, such development tools are separate products that must be purchased separately. Additionally, certain setup commands may need to be executed prior to compilation, to establish environment variables and pathes to the correct compiler, tools and system libraries. On the other hand, the capability of dynamic loading a DSO does not require any compilers or development options.

If a host machine supports multiple programming models, like running both 32bit and 64bit versions of software, the build options must agree with the version of mental ray. The use of system commands like the Unix file command applied to the mental ray executable and the shader library will inform if they can work together. If there is a mismatch, the system may emit error messages like:

/my/example.so: rld: Fatal Error: cannot successfully map soname '/my/example.so'

The runtime linker always tries to match executables and libraries, which allows multiple target binaries to reside on the same system without conflicts. If the filename of the DSO given as an absolute path name mental ray only tries to load that particular file. If it is given as a relative path, mental ray searches for the DSO in a list of paths, by default in /usr/local/mi/lib;.. This list can be adjusted on the command line of the standalone renderer (-ld_path) or an environment variable (MI_LIBRARY_PATH). A search list is a semicolon-separated, or colon-separated (Unix-like platforms) list of paths (Windows uses colons for drive letters).

Note, that shader source code is normally portable, unless non-portable system features (such as fsqrt function on SGIs) are used. This means that the shader will run on all other vendors' systems unchanged. If the compilation fails and therefore the shaders are undefined, mental ray will return miFALSE for any calls of the undefined shaders, which will generally leave the pixel sample black.

The binary DSOs (.so extension) or object files (.o extension) are not portable. They must be compiled separately for each platform and, usually, for each major operating system release. For example, a Linux object file will not run on a Windows system even if they run on same type of processor. Also note, that pointers are 32bit values on some systems and 64bit values on others, and that certain processors require that 64bit values such as doubles are stored at memory addresses evenly divisible by 8 (which is typically enforced by compiler defaults).

On SGI systems, a shader can be debugged after it has been called for the first time, which attaches it to the program and makes its symbols available to the debugger. For this to work, the -g option must be given to the cc and ld commands in all stages - compilation, linking, and shared-library building. Also, only DSO shaders are debuggable, not shaders loaded in object ( .o) or source ( .c) form.

On older systems where debugging shaders in dynamically loaded objects is difficult, or even to watch progress in regular shaders, it is helpful to use mi_info or mi_debug calls in the shader code. Direct calls to system functions like printf should be avoided because the output is lost when running on a slave host, and because it can cause problems when running on multi-threaded systems.

Windows shader libraries must use at least one shader interface call (any C shader interface function beginning with mi_). Otherwise an error message like "DLL_SetModuleHandle not found" may occur.

When a shared library is loaded that provides an externally visible function module_init(), this function will be called right after the library has been loaded, and before the first shader in it is called. Conversely, if it contains a function module_exit(), that function will be called just before the library gets unloaded, and after the last call to a shader in this library. Note, that in the case of non-recoverable errors at any time after the library was loaded the module_exit is not guaranteed to be called. Neither module_init nor module_exit should rely on any shader interface services that assume that a rendering operation is in progress. Message functions such as mi_info are available. Note, that this feature is not supported on IBM systems.

Note, that if a C++ compiler is used to compile a shader, all function names intended to be visible to mental ray (shader, init, exit, version) need to be declared with extern "C". This is necessary because C++ compilers "mangle" symbols by encoding type information into the exported name, so mental ray cannot find the shader in the library. This will result in a "declaring nonexisting function" warning, and the shader will not be called.

[13] Windows is not able to resolve external symbol references dynamically at load time only as supported by Unix-like platforms. Therefore, the shader.lib library provides all the symbol names of the mental ray shader interface functions to please the linker. At load time, mental ray (Standalone, or Library in a DCC application) will resolve the stub function references with the actual functions provided by the mental ray process, thus performing a similar dynamic symbol searching as on Unix-based platforms.

[14] The -Bsymbolic option is supported by RedHat and SuSE Linux 8.0 and later. It makes it possible to use a different standard C++ library in the shaders than in mental ray, even though both are part of the same executable. This is important because mental ray uses a lowest-common-denominator library that is likely to differ from the default version used when compiling shaders.