XPD File Specification

Overview

XGen uses XPD files for per-primitive data baking. This format is specifically designed for use in the case where thousands, if not hundreds of thousands, of primitives need to have values stored. The format stresses space saving but also improves read performance. While writing the files is not slow, performance is definitely tweaked to provide the best possible read performance.

Design

The XPD file is broken into four main pieces: header, face, block, and primitive. The header consists of data describing the contents of the file. It is laid out as follows:

magic

4 bytes - The constant XPD3 is written to identify the file type (and major version).

fileVersion

unsigned char - Version of the header data to allow backward compatibility if additional data is added to the header. This can be thought of as a minor version number (3.x).

primType

enum - The type of primitive in the file. Valid values are Point, Spline, Card, Sphere, Archive, or CustomPT. This enum is defined such that there is plenty of space to add new values if necessary.

primVersion

unsigned char - Version for the data contained in each primitive within the file. This version can be used by the reader to allow it to read in older XPD files and properly unpack the data. This allows for future modifications without invalidating all XPD files.

time

float - Frame number or current time when the data was written.

numCVs

unsigned int - Number of CVs for each primitive. This implies, and enforces, that all primitives have the same number of CVs.

coordSpace

enum - The coordinate space for the points in the file. Valid values are World, Object, Local, Micro, and CustomCS . Object space is the default space of XGen. Local space refers to the space of the primitive, meaning surface coordinate frame space. Micro is used if the values are relative to a coordinate frame placed at each CV of the primitive. This enum is defined such that there is ample space to add new values if necessary.

numBlocks

unsigned int - Number of named data blocks per face.

blockSize

unsigned int - Number of characters for all block names combined (including the end of string character for each block).

blockNames

char * - Names of the blocks. The name ordering also defines the ordering of the data in each face.

primSize

unsigned int - An array of values that represent the size of a single primitive for each block. This implies, and enforces, that each primitive must be a constant size.

numKeys

unsigned int - Number of keys stored in the file. Since all stored data is of type float, any string data to be saved, such as CV attribute names, must be saved using this key mechanism. The keys are stored in the header and then an index for the key is saved in the data.

keySize

unsigned int - Number of characters for all key names combined (including the end of string character for each key).

keyNames

char * - Names for the keys. The name ordering also defines the index for each key that should be used in any reference.

numFaces

unsigned int - Number of faces contained in the file. Not all are guaranteed to have data.

faceid

int - An array of faceids for each face in the file. The order of this array is also the order of the faces within the file.

numPrims

unsigned int - An array containing the number of primitives for each face. The values match up by index with the faceid array.

blockPosition

unsigned long int - Jump table for each block within each face. This table holds absolute offsets for the start of a block within a face that can be used to jump directly to the block and its first primitive.

Data Layout

Immediately following the header is the primitive data. The data is laid out in face order (as defined by the header). Within each face, the data is laid out in block order (as defined by the header). Each block must be present in each face and in the proper order. Within the block lies the primitives. The primitive itself is just a vector of floats. The writer provides the vector, and it knows the order of the data, as well as how to unpack it. The reader reads in a single primitive at a time and then, using the primitive version, knows how to unpack the data. There is no header per face or per block. In reality, after the file header, the file is a constant stream of primitive data.

When reading the file, you must find the face you want, find the block you want, and then read the primitives in order. Once you read all the primitives you want, you can then find the next block and read the primitive data that matches up with the previous block. The blocking structure allows an application (such as Paint3d) to export the data XGen needs in one block (for example, Location), but then export data it needs into its own block (for example, PointCloud ). XGen can then read only the data it needs and easily skip over the other data. Furthermore, both applications can own their data format without having to negotiate with the other.

Accessories

There are a couple of programs available to help with using XPD files. One of these is xpd2txt. This program converts the binary data within the XPD file into a text format that is easily readable. This can also be used to convert the data to a form that can then be parsed and used by production scripts (for example, to convert the data to Inventor format, or read it into Maya). Another program is xpdcmp, which can compare two XPD files to verify if they are the same. This program compares the content of the files, which includes the number of faces, blocks, keys, primitives, and so forth, but does not compare actual primitive values. Therefore, it can be used to verify that a file is “compatible” with another, but not to see if the files are identical.

Examples

Two examples are provided in the XGen source code to read and write XPD files that hold primitives of type Point. This is also referred to as a XUV file. Typically, this type of file is created in Paint3d and used to drive either primitive locations (for example, using the PointGenerator ) or clump locations (that is, via the Clumping FX Module). The examples can be found in wxuv.cpp and rxuv.cpp.

Another special type of XPD file is the curve/guide file targeted for Paint3d consumption. This is a lighter file that contains the id, u, v, and CVs for each primitive, but does not contain any other XGen bake information that Paint3d does not need. This lighter format allows Paint3d to use a format that it can load without worrying about changes to XGen's baking format. This format can be detected by checking the blocks and finding one called “Paint3d”.