UST/MSC: Using the Frontier MSC to Detect Errors

By Chris Pirazzi. Some material stolen from Wiltse Carpenter, Doug Cook, Bryan James, and Bruce Karsh.

In most of Introduction to UST and UST/MSC, we carefully assumed that:

These assumptions allow us to use the frontier MSC (returned by alGetFrameNumber() and vlGetFrontierMSC()) to associate an MSC with each piece of data we read or write.

You can also use the frontier MSC to detect overflow and underflow conditions, and determine their precise length. To understand how this works, you have to remember that the MSC "slots" we described earlier are entries in the input or output signal and not necessarily your data. Usually your signal perfectly matches your data:

If overflow occurs on input, the timeline of your signal and data tear apart and you are left with slots (MSCs) of the input signal that never appear in your data:

If underflow occurs on output, the timelines again tear apart and you are left with slots (MSCs) of the output signal that contain pad data:

The pad may consist of black/silence, colorbars, or a duplicate of earlier data, depending on the device.

In the next sections, we'll show you how the tearing effect above affects the frontier MSC, and how you can use the frontier MSC to detect overflow and underflow.

Underflow on Input or Overflow on Output?

If you were wondering,

In other words, you can wait but the device can't!

Detecting Underflow on Audio Output

We will begin with output underflow since it is simpler than input overflow. Say we have an an AL output port:

This diagram is like the AL output diagrams in Introduction to UST and UST/MSC, except:

The program in the diagram has one frame to write, and it calls alGetFrameNumber() to retrieve that frame's MSC, the frontier MSC of 40.

Now the program writes the frame:

After writing one frame, the AL ringbuffer now has 5 filled entries instead of 4. The program sees that the frontier MSC has gone up by 1. This makes sense since alWriteFrames() now addresses the next slot in the output signal. If there is no underflow, then every time we put N items into the buffer, we would expect the frontier MSC to go up by exactly N.

Now say the program goes off and does something else, and during this time the audio system pulls out 5 frames:

Even though the buffer is now empty, the audio system is still happy: every time it has needed a frame from the buffer, one was available, so there is no underflow.

Note that the frontier MSC is unchanged. This makes sense because alWriteFrames() still addresses the same slot in the output signal. If there is no underflow, and we do not write any data, then the frontier MSC should remain unchanged.

Now say the program dallies a little longer and the audio subsystem unsuccessfully tries to pull out one more audio frame:

Now the buffer is underflowing. Interestingly, the frontier MSC has gone up, even though we did not write any data. This follows directly from the underflow timeline picture. When the timeline of your signal and your data are torn apart by an underflow, MSCs track the signal, not your data. If we were to repeatedly get the frontier MSC without writing any more data, we would see it increment once per audio frame period. The frontier MSC will return to a stable state as soon as we write some more data.

You can use this behavior to detect underflow:

{     
  stamp_t newmsc, oldmsc=-1;
  
  /* this is your main data-writing loop, not a special one */
  while (1)
    {
      alWriteFrames(port, buf, nframes);
     
      alGetFrameNumber(port, &newmsc);
      if (oldmsc > 0)
        {
          stamp_t M = (newmsc-oldmsc) - nframes;
          if (M != 0)
            printf("we underflowed for %lld MSCs!\n", M);
        }
      oldmsc = newmsc;
    }
}
First, write nframes frames. If the port was in underflow, it will now be satisfied. Then, get the frontier MSC (alGetFrameNumber()). If no underflow occurred, the frontier MSC should have gone up by exactly nframes. If you see that it has instead gone up by nframes+M, then you know there was an underflow, and that underflow lasted exactly M sample periods.

Some applications will simply abort if they ever see M > 0. Other applications will use the value of M to implement the fastest possible recovery scheme.

Detecting Overflow on Audio Input

Input buffer overflow causes the same symptom as output buffer underflow: if you read nframes audio frames, and then see that the frontier MSC has gone up by nframes+M, then you know that the port was in overflow for M sample periods:

{     
  stamp_t newmsc, oldmsc=-1;
  
  /* this is your main data-reading loop, not a special one */
  while (1)
    {
      alReadFrames(port, buf, nframes);
     
      alGetFrameNumber(port, &newmsc);
      if (oldmsc > 0)
        {
          stamp_t M = (newmsc-oldmsc) - nframes;
          if (M != 0)
            printf("we overflowed for %lld MSCs!\n", M);
        }
      oldmsc = newmsc;
    }
}
If your application wants to abort on overflow, or your application is not concerned with getting the precise UST of each piece of data in the buffer after the overflow, then this code is all you need.

If your application cares about accurately determining the UST of the samples which did make it into the buffer, or if you want a pictorial example of how the frontier MSC behaves on input overflow, read on.

First, your program opens an AL input port:

This diagram is like the AL input diagrams in Introduction to UST and UST/MSC, except:

Since we're doing input, the frontier MSC of 54 refers to an audio frame currently in the audio subsystem. Say your program reads 3 frames:

After reading 3 frames, the AL ringbuffer has 1 filled entry instead of 4. The program sees that the frontier MSC has gone up by 3. This makes sense since alReadFrames() now addresses a slot which is 3 slots later in the input signal. If there is no overflow, then every time we take N items out of the buffer, we would expect the frontier MSC to go up by exactly N.

Now say the program sits around, and during this time the audio system puts in another 5 frames:

Even though the buffer is now full, the audio system is still happy: every time it has needed space to put a new frame in the buffer, it was available, so there is no overflow.

Note that the frontier MSC is unchanged. This makes sense because alReadFrames() still addresses the same slot in the input signal. If there is no overflow, and we do not read any data, then the frontier MSC should remain unchanged.

Now say your program delays even longer, and during this time the audio system unsuccessfully tries to put in one frame:

Now the buffer is overflowing. The frontier MSC has gone up, even though we didn't read any data. Your application can immediately tell that the buffer is overflowing. If we were to repeatedly query the frontier MSC without reading any more data, we would see it increment once per audio frame period. The frontier MSC will return to a stable state as soon as we read some more data.

Notice in the diagram that the labels A-I for the data on the right do not match the labels for the MSCs B-J on the left. While the buffer is overflowing, the 6 frames of data in the buffer (D-I), and the 3 frames we read earlier (A-C) stay the same. Yet the frontier MSC tells us that the 3 frames we read earlier have MSC 55-57 (B-D), and that the next frame we read from the buffer will have MSC 58 (E). An overflow causes the frontier MSC to "mislabel" the MSC of data that arrived before the overflow. If we were to use a UST/MSC pair to compute the UST of our 3 frames, we would get the wrong UST.

This is why it is crucial to use the frontier MSC to compute USTs (with the UST/MSC pair) only when the buffer is not underflowing and not overflowing, and why extra caution is required even after an overflow.

How long will this mislabeling of MSC last? Fortunately, only the audio frames that arrived before the overflow are affected. Say we relieve the overflow by reading 2 frames with alReadFrames(), and then the audio system deposits 2 new frames into the buffer:

Notice that the labels for the data and MSC of slots K and L again line up. When you read frame K, L, and all subsequent frames up to the next overflow, you will be able to compute their MSC correctly using the frontier MSC. So if you do nothing special, then your program will recover to a state of computing correct USTs based on correct MSCs within an amount of time roughly equal to your AL ringbuffer size.

Since you can use the frontier MSC to compute the exact length of each overflow, it turns out that you can compute the correct MSC of every frame in your buffer, even during overflows. In the worst case, this requires that you store as many stamp_t's as there are entries in your AL ringbuffer, since repeated overflows could generate that many discontinuities in the MSC of items in your buffer. So far, no audio developer has needed to do this, so we won't bother going into details.

Because video MSCs are relatively infrequent compared with audio MSCs, video applications can solve this "MSC mismatch during overflow" problem more easily. The VL provides input timestamping mechanisms, described in Introduction to UST and UST/MSC, which allow your program to extract a UST (and in some cases an MSC) for every piece of incoming data. Because the UST and MSC/sequence become associated with each VLInfoPtr or DMbuffer as soon as the data enters the computer, successfully captured data never becomes mislabeled. Video developers might consider using these timestamping mechanisms instead of UST/MSC for programs that only do video input.