Dynamic Linking of Shaders

Shaders are written as C or C++ subroutines, stored in files with the extension " .c". To use these shaders in a scene, they must be dynamically linked into mental ray at runtime. mental ray accepts shaders in three forms:

Directly as source code. The .mi file offers the code statement and the standalone renderer has a -code option. Both accept a source file with a .c extension.
In object format. Source files can be compiled to object code using Unix commands such as "cc -O -c source.c" (assuming that the shader source is in a file named source.c). The command may vary on some platforms, consult the manual (see " man cc)". On Windows NT systems, shared libraries are called DLLs. This is the most common form on systems that do not support DSOs; it is faster than source because mental ray does not have to compile the shader. The compilation leaves a source.o file with the extension .o (.obj on Windows) in the current directory. This file can be passed to mental ray using the link statement in the .mi file, or with the -link option of the standalone renderer.
In DSO format. DSO stands for Dynamic Shared Object. DSOs are supported on all platforms except Windows NT, which use a simpler but similar system called DLL (Dynamic Link Library). DSOs have the extension .so on Unix/Linux systems and .dll on Windows NT systems. mental ray automatically substitutes .dll for .so on Windows NT platforms and vice versa, so the same library filename can be used everywhere.

To create a DSO, first compile the shader source to object format using cc as described above, then run the Unix command " ld -shared source.o", again assuming that the object file is called source.o. This ld command applies to Unix/Linux. Using DSOs is the fastest way to load shaders, there is very little overhead. DSOs, like object files, are loaded using link statements or -link options.

The commands to create a DSO depend on the operating system type. To create a DSO named example.so from a source file example.c, use the following commands. Insert the -g command line option after -c to insert debugging information, or insert -O to compile an optimized version. On Linux, -g and -O can be combined. Refer to the compiler documentation for details.

Linux x86, gcc

gcc -c -O3 -fPIC -Bsymbolic [14] shader.c ld -export-dynamic -shared -o shader.so shader.o

Linux x86, icc

icc -c -O2 -KPIC -Bsymbolic -tpp7 shader.c ld -export-dynamic -shared -o shader.so shader.o

(Using the Intel compiler suite.)

Linux x86, 64 bits

gcc -c -O3 -fPIC -Bsymbolic -DBIT64 shader.c ld -export-dynamic -shared -o shader.so shader.o

Linux IA 64

gcc -c -O3 -KPIC -Bsymbolic -DBIT64 shader.c ld -export-dynamic -shared -o shader.so shader.o

Linux Alpha

gcc -c -O3 -ansi -fPIC -mieee -DBIT64 shader.c ld -export-dynamic -shared -o shader.so shader.o

IRIX, 32 bits

cc -n32 -O3 -shared -o example.so example.c

IRIX, 64 bits

cc -64 -O3 -DBIT64 -shared -o example.so example.c

MacOS X

cc -c -O3 -fPIC -dynamic -fno-common example.c libtool -flat_namespace -undefined suppress -dynamic -o example.so example.o

HP/UX HPPA

cc -c -Aa +z example.c ld -b -o example.so example.o

HP/UX IA 64

cc -c +O3 +Onolimit +DD64 -Ae -Bprotected_def -Bdefault:keepsym.exp example.c cc +DD64 -b -o example.so example.o

IBM AIX

cc -c -O3 example.c ld -o example.so example.o -bE:example.exp extra.so -bI:keep.exp -bM:SRE -T512 -H512 -lc

example.exp is a user supplied export file for the shader and must contain the names of the exported symbols, one per line. keep.exp is the list of symbols exported by mental ray, either for use by shaders or for internal linking purposes. It is supplied with mental ray and can be referenced using an absolute path name. extra.so must be supplied to allow linking with shader libraries that are preloaded during runtime (such as physics.so) if functions from that library are used directly.

Sun Solaris

cc -c -xO3 -KPIC -D_REENTRANT example.c ld -lm -G -o example.so example.o -ldl

(Using Sun CC. gcc can also be used, see Linux above.)

Tru64 Unix

cc -c -newc -O2 -ieee_with_no_inexact -DBIT64 -D_REENTRANT example.c ld -expect_unresolved '*' -shared -o example.so example.o

NT x86, VC

cl /c /O2 /G6 /MD /nologo /W3 -DWIN_NT example.c link /nologo /nodefaultlib:LIBC.LIB /OPT:NOREF /INCREMENTAL:NO /DLL /OUT:example.dll example.obj shader.lib

VC means Visual C++ 6 or 7; earlier versions are too buggy. shader.lib is a stub library that is provided with mental ray. It is necessary because Windows NT cannot automatically share symbols between modules sharing a process.

NT x86, icl

icl /c /Ox /G7 /MD /nologo /W3 -DWIN_NT example.c xilink /nologo /nodefaultlib:LIBC.LIB /OPT:NOREF /INCREMENTAL:NO /DLL /OUT:example.dll example.obj shader.lib

(Using the Intel compiler suite.)

NT x86, 64 bits

cl /c /GR /Zp8 /W3 /GX /GR /GF /O2 /MD /nologo /W3 -DWIN_NT -DBIT64 example.c link /nologo /nodefaultlib:LIBC.LIB /OPT:NOREF /INCREMENTAL:NO /DLL /OUT:example.dll example.obj shader.lib

NT IA 64

ecl -c /GR /Zp8 /W3 /GX /GR /GF /O2 /MD /nologo /W3 -DWIN_NT -DBIT64 example.c xilink /nologo /nodefaultlib:LIBC.LIB /OPT:NOREF /INCREMENTAL:NO /DLL /OUT:example.dll example.obj shader.lib

Compiling and creating DSOs requires that a C development environment is installed on the system. If the cc, ld, etc. commands are not found, make sure that a development environment exists. On most platforms, it is a separate product that must be purchased separately. Dynamically loading a DSO does not require compilers or development options.

If a host supports multiple programming models, such as IRIX with 32 and 64 bits, the options must agree with the version of mental ray. Use the Unix file command on mental ray and the shader library and make sure that they agree. If there is a mismatch, the run-time linker will print error messages like:

    /my/shader.so: 1234:ray: rld: Fatal Error:
       cannot successfully map soname '/my/shader.so'
       under any of the filenames /my/shader.so

The run-time linker always tries to match executables and libraries, which allows multiple ABIs to reside on the same system without conflicts. If the filename of the DSO given as an absolute pathname mental ray only tries to load that particular file. If it is given as a relative path, mental ray searches the DSO in a list of paths that can be supplied via the command line parameter ( -ld_path) or the environment variable ( MI_LIBRARY_PATH). There is also a default search path ( /usr/local/mi/lib;.). A search list is a colon-separated (Unix) or semicolon-separated (Unix or Windows NT) list of paths to be searched. (Windows NT uses colons for drive letters.)

Note that source code ( .c extension) is normally portable, unless nonportable system features (such as fsqrt on SGIs) are used. This means that the shader will run on all other vendors' systems unchanged. If the compilation fails and the shaders therefore are undefined, mental ray will - when calls of the undefined shaders are attempted - return miFALSE, which will generally leave the pixel sample black.

Neither object files ( .o extension) nor DSOs ( .so extension) are portable. They must be compiled separately for each platform and, usually, for each major operating system release. For example, a Hewlett-Packard object file will not run on an SGI system, and an SGI IRIX 4.x object file cannot be used on an IRIX 6.x system, and vice versa. Also note that pointers are 32-bit values on some systems and 64-bit values on others, and that most but not all (IBM) processors require that 64-bit values such as doubles are stored at memory addresses evenly divisible by 8.

On SGI systems, a shader can be debugged after it has been called for the first time, which attaches it to the program and makes its symbols available to the debugger. For this to work, the -g option must be given to the cc and ld commands in all stages - compilation, linking, and shared-library building. Also, only DSO shaders are debuggable, not shaders loaded in object ( .o) or source ( .c) form.

On non-SGI systems, debugging shaders is, unfortunately, difficult. The reason is that most debuggers cannot deal with parts of a program that have been dynamically linked. In general, the debugger will refuse to set breakpoints in dynamically linked shaders, and will step over calls to these shaders as if they were a single operating system call. Some vendors are working on fixing these problems, but at this time the only option on non-SGI systems is using mi_info or mi_debug statements in the shader sources. Avoid printf because it is lost on slave hosts and because it can cause problems at runtime.

Windows NT shader libraries must use at least one shader interface call (any true C function beginning with mi_), or an error message "DLL_SetModuleHandle not found" will occur.

When a shared library is loaded that contains a function module_init, that function is called just after the library was loaded, before the first shader in it is called. Conversely, if it contains a function module_exit, that function is called just before the library is unloaded, after the last call to a shader in this library. Note that in the case of irrecoverable errors at any time after the library was loaded module_exit is not guaranteed to be called. Neither module_init nor module_exit may rely on any shader interface services that assume that a rendering operation is in progress. Message functions such as mi_info are available. Note that this feature is not supported on IBM systems and should not be used for portable shaders.

Note that if a C++ compiler is used to compile a shader, all names (shader, init, exit, version) need to be declared with extern "C". This is necessary because C++ compilers "mangle" symbols by tacking on type information, so mental ray cannot find the shader in the library. This will result in a "declaring nonexisting function" warning, and the shader will not be called.

[14] The -Bsymbolic option is supported by RedHat and SuSE Linux 8.0 and later. It makes it possible to use a different standard C++ library in the shaders than in mental ray, even though both are part of the same executable. This is important because mental ray uses a lowest-common-denominator library that is likely to differ from the default version used when compiling shaders.