Guide to getting raw data in and out of the movie library.

This document has the following goals:

• Explain the organization of a QuickTime movie file in the SGI movie library
• Some basic information on movie library calls
• Provide background on how to write raw data into a QuickTime file.
• Provide examples of writing raw data (eg, uncompressed, or raw JPEG data) into a QuickTime movie file.
• Provide background on how to read raw data from a QuickTime file.
• Provide examples of how to read raw data (eg, uncompressed, or raw JPEG data) out of a QuickTime movie file.

Organization of a QuickTime file, and how to access it through the SGI movie library.

Movie Library API Organization

1. Basic calls (used by essentially all movie lib application): eg, mvOpenFile
• special version of Open: mvOpenFD, if you want to open the file descriptor yourself and issue reads/writes yourself. Note that this requires installing a patch on Irix 6.3/Irix 6.4. The patch numbers are 1905 (or successor, eg, 1694, 2095, 2223, 2275, etc) for 6.3 or 1765 (or successor) for 6.4. No patch required for 6.2 or 6.5.
2. High level playback calls: eg, mvBind*, mvPlay*
3. High level data access calls.
• These map through the edit list, and do data IO, decompression, etc, for you.
• Examples: mvRender*
4. High level Edit list calls. These change the edit list for you, although they don't let you find out what the edit list actually looks like. eg, mvDeleteFramesAtTime, mvCopyFramesAtTime
5. Low level Edit list calls.
• These let you get a peek at what the edit list does. The edit list maps a time in the track into a time in the media's timeline. Within the media's timeline, each "chunk" (single image frame, or multiple audio frames) is associated with a "dataIndex." The dataIndex is a unique integer handle to a piece of data within the track. (Labeled #1 in picture above)
• Note that a particular data index may be in the edit list more than once-or may not be in the edit list at all.
• Examples: mvGetTrackDataIndexAtTime, mvGetTrackBoundary
6. Data description calls. These calls let you set or get information about a particular piece of data in a track, via its data index. eg mvGetTrackDataInfo (Line labeled #2 in picture above)
7. Data access calls which do IO for you
• These calls are available in two varieties, the "frames" version and the "fields" version. Which one to use depends on how the data is (or should be) stored in the file.
• These calls work with a data index and a void* (or pair of void*'s)
• Examples: mvInsertTrackData, mvInsertTrackDataFields, mvReadTrackData, mvReadTrackDataFields
• mvClose flushes and writes out all the changes made to the movie.
8. Data Access calls which do not perform IO
• These calls are available in two varieties, the "frames" version and the "fields" version. Which one to use depends on how the data is (or should be) stored in the file.
• These calls (which are actually internally called by the calls described above in #7), work with a data index and one or more file offset and size values. (Line labeled #3 in picture above)
• Examples: mvInsertTrackDataAtOffset, mvGetTrackDataFieldInfo, mvGetTrackDataInfo

Background: Writing raw data into a QuickTime movie file

If you choose to do the latter, keep in mind that the movie library will require space for its first QuickTime atom at the beginning of the file as well as extending the file with the remainder of the QuickTime header. The minimum empty space required at the beginning of the file is 128 bytes (assuming that you will want to take advantage of the movie library's ability to generate QuickTime files with 64 bit file offsets). In practice, however, most often 1K or 4K (or some other convenient size, perhaps derived from the disk block size) is more commonly used (extra space is simply ignored).

Do not call mvInsertTrackDataAtOffset with an offset less than the minimum headersize of 128. If you do, when the movie header is written to disk(during mvClose for instance) the header will overwrite the first few bytes of the first block of data. This will appear to the user as a few bad pixels in the first field/frame. Also remember mvInsertTrackDataAtOffset does not move the file pointer for you.  This means to leave space for the header you must call a function like lseek with your initial offset before your first write.

If you do your own file I/O it is important that you return the file pointer to the position it was in when the movie library lasted used it for file I/O.  If you do all your own file I/O this often means returning it to position 0 (with a call to lseek) just before calling mvClose which flushes the movie header.

To insert raw buffers of data (eg, perhaps compressed JPEG data that you have received from the dmIC library, or uncompressed data you have received from the VL), you would normally do the following.

• Create a file using mvOpenFile and mvAddTrack
• call mvInsertTrackData or mvInsertTrackDataFields (passing in your void* or pair of void*'s, and appropriate size values). This will cause libmovie to write your data to disk.
• flush the QuickTime header to disk using mvWrite (Or: flush the header to disk and close the file by using mvClose.)

To insert data which is already on disk into the movie's edit list (so that its location is stored in the file, and can be played back using standard QuickTime compatible tools), you would normally perform a multiple step process centered around mvInsertTrackDataAtOffset and friends. Note that mvInsertTrackDataAtOffset does not write anything to disk (nor does mvGetTrackDataIndex, nor mvSetTrackDataFieldInfo); it simply updates in memory movie data structures, which will be flushed to disk when mvWrite or mvClose is called.

Also, note that calling mvInsertTrackDataAtOffset need not be called between each write; you can also write all of your data to disk in one pass (and maintain your own data structure keeping track of where each frame/field is on disk), and then have a post-processing step that calls mvInsertTrackDataAtOffset. In some cases, this may be preferable, since mvInsertTrackDataAtOffset may call malloc; however, in other cases, it may be preferable to call it after each write, as this will provide some amount of CPU/IO overlap.

• open a file descriptor read-write
• turn the file descriptor into a movie using mvCreateFD, mvAddTrack. (see example)
• seek on the file descriptor to leave space for the initial QT Atom.
• enter a loop which does: (see example)
• the write system call to store your data into the fd. (of course, you may also use other forms of the write call, such as aio_write(), writev(), pwrite(), etc.)
• call mvInsertTrackDataAtOffset to tell the movie library the area of the disk containing the current frame/field pair, and where in the duration of the movie it should be played
• If you are writing field pairs, call mvGetTrackDataIndexAtTime to get the dataIndex the movie lib assigned to the chunk you just inserted, and then call mvSetTrackDataFieldInfo on that dataIndex to set the size of the first field, gap (measured from the end of the first field to the start of the second field), and the size of the second field. For more information on gaps between fields, see Motion JPEG on SGI Systems.
• Increment your current time counter...
• Return the file pointer to position 0 with a seek.
• flush the QuickTime header to disk by using mvWrite. (Or: flush the header to disk and close the file by using mvClose.)

Creating a Movie from a file descriptor (eg, "creating the QuickTime header")

DMstatus initMovie( int fd, int width, int height, MVid *movie, MVid 
                    *track)
{
     DMstatus s = DM_FAILURE;
     DMparams *movieP = NULL;
     DMparams *trackP = NULL;

     *movie = NULL;
     *track = NULL;

     if (dmParamsCreate(&movieP) != DM_SUCCESS)
          goto ERR;

     if (mvSetMovieDefaults(movieP, MV_FORMAT_QT) != DM_SUCCESS)
          goto ERR;

     if (mvCreateFD(fd, movieP, NULL, movie) != DM_SUCCESS)
          goto ERR;

     if (dmParamsCreate(&trackP) != DM_SUCCESS)
          goto ERR;

     if (mvSetImageDefaults(trackP, width, height, MV_FORMAT_QT) 
         != DM_SUCCESS)
          goto ERR;

     if (dmParamsSetEnum(trackP, DM_IMAGE_ORIENTATION, 
         DM_IMAGE_TOP_TO_BOTTOM) != DM_SUCCESS)
          goto ERR;

     if (dmParamsSetEnum(trackP, DM_IMAGE_INTERLACING, 
         DM_IMAGE_INTERLACED_ODD) != DM_SUCCESS)
          goto ERR;

     if (dmParamsSetEnum(trackP, DM_IMAGE_PACKING, 
         DM_IMAGE_PACKING_CbYCrY) != DM_SUCCESS)
          goto ERR;

     if (mvAddTrack(*movie, DM_IMAGE, trackP, NULL, track) != DM_SUCCESS)
          goto ERR;

     s = DM_SUCCESS;

ERR:
     if (movieP != NULL)
          dmParamsDestroy(movieP);
     if (trackP != NULL)
          dmParamsDestroy(trackP);

     return s;
}

Example: Inserting already-written field-based data into a tracks edit list.

MVtime frameDur = 1001;
MVtime TS = 30000;
MVtime curTime = 0;

// Note that before this loop is entered, you should have already
// seeked far enough to leave space for the first QT Atom.

while( !done )
{
    off64_t off1, off2;
    size64_t sz1, sz2;
    int index, dummy;
    void * data1, * data2;

    // this function writes data to disk, and fills in off1, sz1, 
    // off2, sz2.
    // it is the thing that actually calls the write system call.
    // 
    // In its most basic form it could be something along the lines
    // of the following:
    //
    //    off1 = tell64( fd );
    //    write( fd, data1, sz1 );
    //    off2 = tell64( fd );
    //    write( fd, data2, sz2 );
    //

    myWriterFunction( &off1, &sz1, &off2, &sz2 );

    // Now, tell the movie lib about the data just written to disk.
    // This assumes that off1 < off2, although that is not strictly
    // required.

    s = mvInsertTrackDataAtOffset( track, 1, curTime, frameDur, TS,
                                   off1, 0, 
                                   MV_FRAMETYPE_KEY, 0 );
    // Since we will be immediately calling mvSetTrackDataFieldInfo,
    // which overwrites the size set by mvInsertTrackDataAtOffset,
    // pass in a zero as the size above.

    if ( s != DM_SUCCESS ) 
        ERROR();

    s = mvGetTrackDataIndexAtTime( track, curTime, TS, &index, &dummy );

    if ( s != DM_SUCCESS )
        ERROR();

    s = mvSetTrackDataFieldInfo( track, index, sz1, off2-(off1+sz1),
                                 sz2 );

    curTime += frameDur;
}

Background: reading raw data from a QuickTime file (via the QuickTime edit list)

Note that when reading raw data from disk, you must pay attention to whether the chunk of data was written as a frame or as a field pair. Pay special attention the the mvTrackDataHasFieldInfo call below.

Also, it is important to note that the QuickTime file format allows the compression/size/etc parameters of chunks within a track to change at any point within the track. For this reason, you need to use mvGetTrackDataInfo (and possibly mvGetTrackDataParams) to make sure the data for the current chunk is in a format you are expecting. Although it is uncommon to encounter files with varying parameters for chunks within one track, it does happen, and therefore is something that any code reading raw data from a QuickTime file must be aware of. Depending on your application, you may wish to abort when this happens, change codecs/etc, or, simply fall back to the general purpose movielib function mvRenderMovieToImageBuffer or mvRenderMovieToAudioBuffer. The mvRender* functions in the movielib have full support for changing chunk parameters, so they make a good fallback case. The correct behavior will obviously depend upon your application.

Just as with the "writing raw data into a QuickTime file" (example above), you have a choice as to how to read the data: you may either pass the movielib one or two void* buffers (mvReadTrackData/mvReadTrackDataFields), or you may ask the movielib for the on-disk offset of a chunk of data (using mvGetTrackDataInfo/mvGetTrackDataFieldInfo) and perform the read operation yourself. Obviously, the former is simpler for non performance critical applications, but the latter allows you to do things like combine multiple reads into one system call (eg, readv), or use directio, allowing you to (potentially) avoid extra bcopy operations.

Since all but one step is the same for the void* method as for the read method, I've combined them into one description.

• mvOpen the file
• mvFindTrack(ByIndex or ByMedium) the track you are interested in
• get the track's edit list timescale using mvGetTrackTimeScale
• loop through the track from mvGetTrackOffset to mvGetTrackDuration and:
• use mvGetTrackDataIndexAtTime to map a track time into a media chunk.
• if index is -1, then you have a track gap (skip to the "increment" step below).
• if image track, ignore the frameOffset (it tells how many audio frames to skip at the start of an audio chunk, but image tracks have one frame per chunk, so frameOffset is 0 for image tracks)
• use mvGetTrackDataInfo to determine the paramsId that this chunk is stored with, as well as this chunk size, type (keyframe/nonkeyframe), and number of frames (1 for image tracks)
• use mvGetTrackDataParams on paramsId returned from mvGetTrackDataInfo to get the DMparams describing this chunk (change codec etc as needed)
• if you want libmovie to read the data for you:
• use mvReadTrackData or mvReadTrackDataFields (Depending on mvTrackDataHasFieldInfo for this data index)
• else: if you want to do the read yourself:
• use mvGetTrackDataInfo to find the offset and size (if !mvTrackDataHasFieldInfo), or to determine the offset of the first field (if mvTrackDataHasFieldInfo).
• If mvTrackDataHasFieldInfo, then use mvGetTrackDataFieldInfo to determine the size of the first field, the relative offset of the second field (relative to the end of the first field), and the size of the second field.
• call read (once or twice, depending on whether you have field info) (of course, readv, aio_read, etc also can be used)
• increment current time, either by a constant, or by using mvGetTrackBoundary

Example for reading data: