About Image Processing Tools			Formatting Tools
Chapter 12, Image Processing Tools

Topics on this Page:

Remove Dust Tool

About Blurs

Lens Blur Tool

Median Tool

Sharpen Tool

Unsharp Mask

You can use Filtering tools to apply a range of looks, feels, and transitions to your composites.

Remove Dust Tool

Use the Remove Dust tool to remove dust, dirt, hair, and scratch artifacts from sequences of images. Dust removal, encompassing what is sometimes called dust-busting, dust repair, scratch removal and scratch repair, is traditionally a time-consuming task, requiring visual inspection and manual correction steps for each frame of film or video. This tool provides you with a means to automate this task as much as possible, and to easily tweak the results manually. The tool automatically detects dust and scratches, and automatically repairs them. You can view the results of the detection phase and easily correct anything using a simple mouse stroke.

The Remove Dust tool detects dust in images and removes the dust from the images. The tool has a primary color image input and outputs two images, the repaired color image and a mono-channel defect matte. An optional second input can be used to specify the defect matte as a mono-channel image, and any dust detected by the tool is added to the dust supplied by the defect matte. If the defect matte input is not present, the tool initializes the defect matte to empty. Optional third and fourth inputs supply forward and/or backward motion vectors to the tool which it can use for the detection and/or the correction phase.

The dust removal workflow can be divided into three steps:

• Dust Detection -- This automatically creates the defect matte by examining the images, using motion vectors if available. This step is optional, as the defect matte may be supplied to the tool as the second input. Even if no defect matte is supplied, you may choose to bypass detection and perform all labeling by hand. A single tolerance parameter controls the dust detection, where a value of 0 means no dust is detected, and a value of 100 means all pixels are dust. You can tweak the value slightly from its default of 50 to get a reasonable set of dust pixels. Two other parameters control the expansion of the dust to make sure the whole dust object is covered, not just the center.

• Manual correction of the defect matte -- If any pixels have been incorrectly labeled as dust or not dust in step 1, manual correction of the defect matte can be performed by drawing appropriate shapes on the dust matte. These locally change the detection and repair parameters within the geometric region of the shape. Each object drawn has its own dust detection parameters, which overrides the global (automatic) values used in step 1. In addition, each object has other parameters to control how correction is performed. Of course, drawing shapes is optional.

• Defect repair -- Using the final defect matte, an image processing operation is performed to fill in the corresponding pixels in the output image with corrected pixels. This process uses motion vectors if they are available. Your can choose between a spatial repair, which uses only the current frame, or a temporal repair, the default, which uses neighboring frames and motion vectors. Spatial repair should only be used in areas of the image where the motion vectors are inaccurate or there are occlusions making it difficult for the algorithm to find the corresponding correct pixel on neighboring frames.

The Global Detection UI has the following parameter controls:

The Global Correction UI has the following parameter controls:

Occasionally there will be parts of the image where some of the Global (automatic) parameters need to be changed locally to improve the dust detection and repair. For this, you can use a shape drawing facility based on the Garbage Mask tool. The UI is very similar to the Garbage Mask but does not have the mask list user settings or edge gradients, however all Garbage Mask hot keys affecting control points are the same--see Chapter 18, Masking. The objects that are drawn are non-animated and appear only on one frame of the composition. You can select and edit the objects only by direct manipulation on the image wireframe overlays.

When in creation mode (either Rectangle, Ellipse, or Spline is selected) you can set the Detection and Correction values that will apply only within the shape to be drawn. When in selection mode (either Select or Transform), the UI looks the same as in creation mode, except that the values are for the currently selected shape. The two garbage can buttons provide deletion of selected shapes, and all shapes on the current time frame, respectively.

Each drawn shape has several values associated with it, the first 3 of which override the corresponding global (automatic) parameter in the area of the shape:

The defect matte is output on the second output, thus allowing the use of context points--see Setting Context Points to view the input defect matte and the defect matte after dust detection has been applied. You can get the second output node from the Utilities folder in the Tools tab.

About Blurs

Use the Blur tool to finish shots that require directional, radial, modulated, and vectors blurs. This includes shots that require some amount of depth of field or motion blurs. In modulated blurs, you can vary the amount of blur from pixel to pixel. The ability to vary the amount of blur applied at each pixel is sometime useful to model specific physical processes or be used for purely artistic goals.

The Blur tool UI is composed of five tabs:

The Directional Gaussian blur filter lets you scale and rotate its elliptical shape. This allows you to blur an image by a certain amount in a given direction, and by a different amount in a direction perpendicular to it. Each pixel is blurred by the same amount.

The Radial Blur simulates the effect of motion blur that would be generated by the movement of the camera due to zooming and spinning around a given point. The zooming and spinning movements are simply converted into motion vectors representing the local displacement at each pixel. These motion vectors are then used to construct blur ellipses aligned with each vector. The size of the major axis matches the length of the displacement vector. The size of the minor axis is kept fixed at a small value. That value is chosen to keep aliasing artifacts at an acceptable level while not introducing too much blurriness.

A particular color channel (red, green, blue, alpha or luminance) of the modulation input image can be used to modulate a modulated motion blur. The end effect is that the modulation image will apply a varying amount of scaling at each pixel. The elliptical shape of the blur filter is scaled uniformly by this modulation amount.

The Modulation blur tool has the following parameters:

Although the Modulation Blur tool can simulate depth of field, artifacts may appear for a number of reasons and are inherent 2.5D motion blur and depth of field.

Depth of field is a process that occurs in 3D. Out-of-focus objects blur on top of objects that stand further away. They never blur on top of objects in front of them. The modulated blur does not make this distinction about objects that are in front or behind thus causing artifacts every time a silhouette edge appears in an image. Rendering depth of field therefore requires segmentation of the image along the silhouette edges. Please note that it is not always possible to partition a scene into separate objects using silhouette edges.

Anti-aliased images also cause some problems in that the Z-channel cannot be used to determine which portion of the pixel belongs to the background object and which part belongs to the foreground. When an object becomes out of focus, one starts to see parts of the scene that were not visible before through the blurred edge. To replicate that effect as a 2D post-process, the depth of field tool must guess what the background pixels might look behind foreground objects. To do this, segmenting and matting out foreground objects' then reconstructing the background using in-filling techniques become necessary and which can be a difficult task.

To correct some of the Z-channel problems, the 3D elements of a scene can be rendered as separate layers. The scene would be manually segmented into objects all having approximately the same Z. A single layer would not have any significant Z discontinuities. Each layer can be properly anti-aliased. The Z-information of the foreground element and the background element would be available at partially covered edge pixels and the color of the background objects behind foreground elements would also be available. However, the Z-information is not defined at the transparent pixels of each layer. A renderer is likely to assign them some very far-away value. To the modulated blur tool, this represents a huge discontinuity in its blur modulation and the Z-discontinuity ends up as a discontinuity in the blurred image. To help remove the resulting artifacts, the parameters (Min. Alpha and Max Radius) under the Extend label can be set to generate the missing Z information.

Use the Vector Blur tool to specify the length and the width of the blur ellipses. The blur ellipses are then oriented independently for each pixel in the direction of the vectors of the Forward Vector input image and uniformly scaled by the length of the vectors. You can also apply an extra constant rotation to all the blur ellipses.

Using this definition of vector blurs you can blur only in the direction of the vector field. To do this, set the amount of blur in the perpendicular vector direction to zero. No matter how much it is scaled, it will always stay null. Then, use the minimum blur radius control to clamp to a finite value to avoid sampling aliasing.

Note: If the ROD of the Forward Vectors image is smaller than the ROD of the image produced by the Blur tool, the pixels lying at the edge of the Forward Vectors image are repeated to cover the missing region.

The Vectors blur tool has the following parameters:

Using the Modulation blur tool to simulate motion blur can be difficult due to some of the same inherent issues as when trying to simulate depth of field using the Modulation blur tool;

• Need to segment out at silhouette edges.

• Can not deal with anti-aliased images at silhouette edges.

• Needs the background pixels behind foreground elements.

• Discontinuity in the motion vector field ends-up as discontinuity in the blurred image.

• Needs to out-fill motion vectors over transparent region.

These limitations can be worked around, if the 3D department has already partitioned the image into layers of constant movement. You can then use the Min. Alpha and Max Radius parameters of the Vectors blur tool to extend the motion vectors field to get more realistic motion blur effects.

Note: The Blur tool has a masking input--see Pixel Masking. The Blur tool's X and Y attributes are both animatable by setting keyframes or using expressions--see Setting Keys Manually and Validating and Applying the Expression String.

The Output UI allows you to control the following parameters:

Lens Blur Tool

The Lens Blur tool let you simulate rack defocus and add photographic effects to 3D rendered scenes or live footage. You can also create lens flares, streak lines, and halos around bright lights and reflections.

The Lens Blur UI is composed of four tabs: Lens Blur, Modulation, Output, and Masking.

The Lens Blur tab consists of a set of parameters for setting the main blur effect, as well as controls to create bloom, flares, and halos.

To create the main blur effect, use the controls on the left side of the UI.

The following images show the types of effects you can generate using the main blur controls.

The Bloom controls let you remap the higher luminance range to increase the bloom appearance of the blur. This is especially useful for 8-bit images which cannot represent values greater than 1. With bloom, you can remap the values to make it seem like an HDR image, giving you more realistic blooms around bright light sources and reflections.

The Flares parameters let you add lines and streaks emanating from bright lights and reflections. You can set the intensity of the flares, as well as the number of flares. An intensity of 0 (the default) will disable flare generation. The rotation of the lines can be animated to simulate flares that seem to rotate as the camera moves relative to the light sources. The Randomness parameter controls how evenly distributed the lines are. A value of 0 makes the flare lines uniformly distributed around a circle, while a value of 1 makes them completely randomly distributed. You can also set a seed value to choose a particular sequence of lines. The effect of adding flares will be somewhat like the linear patterns in the right image below.

The halo is a ring of brightness of a specified radius from the center of bright lights, and with a specified width. If the radii differ among the color channels, the effect is a rainbow halo like in the following right image. This is controlled by the Aberration parameter. There is also an intensity parameter to adjust the strength of the halo; the default value of 0 disables the halo creation.

The Grating is a set of lines of random intensity, uniformly distributed around the circle. Increasing the Grating Density parameter increases the number of lines, and the smoother it appears.

The Modulation tab looks just like that of the Blur, including the extrapolation (extending) capability--see Modulation Blur. You can choose what channel of the modulation image is used and apply an offset to the modulation image. The modulation can affect the X and Y radius, or the rotation of the polygonal or ellipse shape.

The Output tab is the same as that of the Blur tool, but without the Min Radius parameter--see Output Controls.

The Lens Blur tool has a masking input and a Masking tab--see Pixel Masking.

Median Tool

The Median tool is an edge-preserving smoothing filter that works particularly well for removing impulse noise.

For grayscale images, the median filter works by ranking the pixels under the kernel according to their value and selecting the median to replace the pixel at the center of the kernel. This approach effectively removes spikes in the original image without the blurring typically introduced by common smoothing kernels (e.g. Gaussian).

For color images, there is no single concept of ranking, so several different criteria are supported. Component-wise ranking computes the median of each color component independently, however, this can result in pixel colors that did not belong in the original image. Luminance ranking computes the median of the luminance of each pixel under the kernel. This approach is fast and does not introduce new colors in the original image, however, luminance is not the best criteria for similarity in a color image. RGB Vector does not really rank the pixels under the kernel, but rather it chooses as the median the one pixel with the smallest sum of square distances (in RGB space) to all the other ones under the kernel. It selects the pixel that is closest to the center of the point cloud obtained by looking at the pixels under the kernel as points in 3D space. This approach is computationally intensive, but can yield better results than either of the other ranking criteria.

The median filter uses a square neighborhood and can round the corners of axes-aligned rectangular objects in an image.

As any noise reduction filter, the median filter may also affect the sharpness of small details in the input image. When this problem arises, the result of the median filter can be blended with the original image to decrease its effect.

Finally, the median filter can be applied recursively. Repeated applications using a small kernel size yield better results than a single application using a large kernel.

Note: When using the filter recursively, the blending described above is applied at each iteration.

The Median tool has the following parameters:

Sharpen Tool

The Sharpen tool lets you increase the clarity and focus of an image. The Sharpen tool applies a sharpening filter to a number of adjacent pixels in the input image and increases their contrast.

1. Drag the Sharpen tool from the Tool tab to the dependency graph in the Schematic view.

   

2. Adjust the sharpen amount by dragging the value slider, or by typing in a value.

   

Note: The Blur tool has a masking input--see Pixel Masking. The Sharpen tool's Amount value is animatable by setting keyframes or using expressions--see Setting Keys Manually and Validating and Applying the Expression String.

Unsharp Mask

The Unsharp Mask tool lets you sharpen the edges and other fine details in the input image.

Unsharp masking provides more control on the sharpening process and will generally produce better results than the Laplacian filter, but at a higher computational cost.

Unsharp masking works by removing the low frequency spatial information from the image and emphasizing the high frequency details. The algorithm basically uses a Gaussian filter to produce a smooth version of the original image, called unsharp mask. The unsharp mask is then subtracted from the original image, removing low frequencies.

The Unsharp Mask tool has the following parameters:

• Unsharp Mask Radius X, Radius Y, and Link -- Control the radius of the blur (just like in the Blur tool). By default, the X and Y radii are linked and set to 1.20; otherwise, their range is the same as that of the Blur tool.

• Strength -- Controls the strength of the effect of masking that is added. Basically, the output image is computed as I+s*(I-M) where I is the original image, M is the unsharp mask, and s is the strength parameter (in 0 to 500 percent; default 100%).

• Threshold -- Defines how large the difference between the original image and unsharp mask must be before the original pixels are changed. A pixel is changed if the difference in any of its components is larger than or equal to the threshold (in 0 to 100%; default is 0%).

Note: This tool can either affect the RGB components or the Alpha component, but not both at the same time.