Output Shaders

Output shaders are functions that are run after rendering has finished. They modify the resulting image or images. Typical uses are output filters and compositing operations. Output shaders can directly access all rendered frame buffers and make modifications to these frame buffers, but they do not return a result like other shaders do. They still get a result pointer as their first argument, for symmetry with the other types of shaders, but this pointer should not be used.

Until version 2.0.20 of mental ray, output shaders had a different prototype (there was no result pointer argument) and a different state called miOutstate, which contained frame buffers but very few other state variables, so output shaders could not use any shader interface function that required an argument of type miState. For backwards compatibility, these shaders are still supported, but not recommended and not described in this book. mental ray distinguishes old-style output shaders by a null or missing version number in the declaration. Old-style output shaders did not support shader initialization and cleanup and the mi_eval family of functions; new ones do.

It is important to specify a non-null shader version number in the declaration of output shaders! If none is specified, mental ray\ issues a warning and calls the output shader with an miOutstate, which will probably crash it.

All output shaders must be declared like any other type of shader, and the same types of arguments can be declared. This includes textures and lights. Nonprocedural textures can be looked up using functions like mi_lookup_color_texture and mi_query. Lights can also be looked up with mi_query. Since rendering has completed, it is not possible to look up procedural textures or to use tracing functions such as mi_sample_light.

Here is a simple output shader that depth-fades the rendered image towards total transparency:

    struct out_depthfade {
        miScalar   tnear;   /* no fade closer than this */
        miScalar   tfar;    /* farther objects disappear */
    };

    miBoolean out_depthfade(
        void                  *result,
        register miState      *state,
        struct out_depthfade  *paras)
    {
        register int          x, y;
        miColor               color;
        miScalar              depth, fade;
        miScalar              tnear, tfar;
        miImg_image           *fb_color, *fb_depth;

        tnear    = *mi_eval_scalar(&paras->tnear);
        tfar     = *mi_eval_scalar(&paras->tfar);
        fb_color = state->options->image[miRC_IMAGE_RGBA].p;
        fb_depth = state->options->image[miRC_IMAGE_Z].p;

        for (y=0; y < state->camera->y_resolution; y++) {
                if (mi_par_aborted())
                        break;
                for (x=0; x<state->camera->x_resolution; x++) {
                        mi_img_get_color(fb_color, &color, x, y);
                        mi_img_get_depth(fb_depth, &depth, x, y);

                        if (depth >= tfar || depth == 0.0)
                                color.r=color.g=color.b=color.a = 0;
                        else if (depth > tnear) {
                                fade = (tfar - depth) / (tfar - tnear);
                                color.r *= fade;
                                color.g *= fade;
                                color.b *= fade;
                                color.a *= fade;
                        }
                        mi_img_put_color(fb_color, &color, x, y);
                }
        }
        return(miTRUE);
    }

Note the call to mi_par_aborted, which stops the shader if the user has instructed mental ray to stop whatever it is doing. This call was inserted in the line loop because it should be called periodically, but not too often to avoid slowing it down. Also, note that the parameters are named tnear and tfar instead of near and far because some older early-1980's compilers reserve these as keywords.

This shader is stored in a file out_depthfade.c and installed in the .mi file with a code statement and a declaration:

     code "out_depthfade.c"
     declare shader
         "out_depthfade" (scalar "near", scalar "far")
         version 1
     end declare

Frame buffer access has changed substantially in mental ray 3.4. For example, to access the RGBA color and Z depth frame buffers, it is necessary to open and close the frame buffers instead of reading their tags from the state:

     miImg_image *fb_color = mi_output_image_open(state, miRC_IMAGE_RGBA);
     miImg_image *fb_depth = mi_output_image_open(state, miRC_IMAGE_Z);
     ...
     mi_output_image_close(state, miRC_IMAGE_Z);
     mi_output_image_close(state, miRC_IMAGE_RGBA);

See the definition of mi_output_image_open^3.4 for details. This method allows an arbitrary number of frame buffers, not just 13 as in mental ray 3.3 and earlier.

It is actually more efficient to precompile the shader, store it in a DSO file, and use the link statement instead of code. See the beginning of this chapter (page shaders) for details. The declaration should appear before the first reference to the shader in a camera statement. Note the non-null version statement. The shader is referenced in an output statement in the camera:

     camera "cam"
         output "rgba,z" "out_depthfade"
                              ("near" 10.0, "far" 100.0)
         output "rgb" "filename.rgb"
         samples 0 1
         ...
     end camera

Note that the output shader statement appears before the output file statement. The output shader must get a chance to change the output image before it is written to the file filename.rgb. It is possible to insert another file output statement before the output shader statement; in this case two files would be written, one with and one without depth fading.

Note also that the output shader has a type string "rgba,z". This string tells mental ray to render both an RGBA and a Z (depth) frame buffer. The RGBA buffer would have been rendered anyway because the file output statement requires it, but the depth buffer would not have been rendered without the z in the type string. In this case, all depth values returned by mi_img_get_depth would be 0.0.

By default, depth buffers are not interpolated; instead the min depth within the pixel is used. If the depths should be interpolated, use the output type "rgba,+z". Refer to the Functionality chapter for more information on frame buffer interpolation.

This shader does not anti-alias very well because there is only one depth value per pixel. The shader makes pixels for which a depth of 0.0 is returned totally transparent to fade edges of objects correctly that have no other object behind them. By definition, mi_img_get_depth returns 0.0 for a position x, y if no object was hit at that pixel.