Synchronization Across Machines

By Chris Pirazzi. Some material stolen from Wiltse Carpenter, Doug Cook, Bryan James, and Bruce Karsh.

The UST and UST/MSC support described in Introduction to UST and UST/MSC lets you relate the times at which signals enter and exit one machine. Sometimes you want to relate the timing of signals on multiple machines. For example:

UST and UST/MSC solved the problem on one machine by mapping all interesting signal events onto a common clock, the UST clock.

To solve the problem across machines, you need:

Sometimes, it's convenient to map the machine-global clock onto the machine-local UST clock, since then you can use the UST support on the machine to map to other machine-local events. Sometimes you map directly from the machine-global clock to the particular machine-local event you are using.

Global Clocks

For our purposes, a clock is any periodically changing numerical quantity which we can measure from software, such that two pieces of software measuring the clock at the same instant will retrieve the same numerical value (to some stated accuracy). In some cases, that numerical quantity may appear as a timecode (see Timecode).

These are the machine-local clocks we're familiar with:

How many kinds of clocks can be distributed to more than one machine (requirement 1 above)? The number may surprise you:

There are many more such clocks. We have chosen some clocks which also satisfy requirement 2.

SGI's libraries let you accurately map each of these global clocks (except for video+GPI) onto the machine-local UST clock. For example:

You can also bypass UST and map many of these global clocks onto other local events. For example:

With the tools described above, and a little imagination, you can build a cross-machine synchronization mechanism for your application.

The rest of this document contains more detailed explanations of a few particular cross-machine synchronization mechanisms.

Relating the UST of Two Machines Using Network Time

The most obvious way to relate the UST of two machines on a network is to sync up their gettimeofday() clocks using your favorite network time protocol (timed, NTP, etc.) and then use dmGetUSTCurrentTimePair() on each machine to pair UST with gettimeofday(). The network time protocols often give you accuracy in the millisecond range on LANs. When using this technique, be sure to update your UST/gettimeofday() relationship with dmGetUSTCurrentTimePair() periodically, for reasons described in Introduction to UST and UST/MSC.

Relating the UST of Two Machines Using a Digital Audio Signal

For machines which are not connected by a network, or for cases where you require more accuracy, consider connecting the machines with another clock. The following hack shows a way to relate the UST clock of two machines to at least ±80us (±3us on newer audio platforms) using a digital audio signal.

This hack works if you are willing to dedicate one digital audio channel to the task (or at least some of it), and plug the connector containing that channel from the output of one machine to the input of the other machine. Say machine A has the digital audio output, and machine B has the digital audio input.

On machine A, use the UST/MSC support in the AL to determine the UST of the next audio sample you are sending out. Say the next sample you're sending out has UST 0xaabbccddeeffgghh. Transmit the following data over the normal audio bits of your digital audio channel using alWriteFrames() (this example assumes 16 bits per sample):

  0xFEED 0xBABE 0xDEAD 0xBEEF 0xaabb 0xccdd 0xeeff 0xgghh 
Since 0xFEED was the first frame we transmitted, the UST 0xaabbccddeeffgghh is the UST of 0xFEED. Machine A continues to transmit valid pairs of "0xFEED 0xBABE 0xDEAD 0xBEEF" and UST. The program generating the data on machine A need not call alGetFrameTime() and alGetFrameNumber() between transmission of each pair: once per second should be fine. The program can safely interpolate the remaining USTs, or it could even transmit a "0xFEED 0xBABE 0xDEAD 0xBEEF" and UST pair only once every second or so, and transmit zeroes the rest of the time.

Now, on machine, B, you have another program running which reads 16-bit audio in the normal way with alReadFrames(). The program searches the incoming samples for the pattern "0xFEED 0xBABE 0xDEAD 0xBEEF." When it sees this code, it knows that the next 4 samples (next 8 bytes) contain the machine A UST of 0xFEED. Like machine A, machine B uses the UST/MSC support in the AL on its side to determine a machine B UST for 0xFEED.

Now machine B has a machine-B-UST and a machine-A-UST which coincide. The accuracy of this pair depends on the accuracy of the UST/MSC support used to generate it on each machine. The AL's UST/MSC support will give you USTs that are ±1.5us accurate on Octane, Origin, Onyx2, and O2 digital audio. The USTs are 4-audio-sample-accurate (±40us at 48k) on other systems. So if both systems are ±1.5us accurate, this will give you a machine-A-UST/machine-B-UST pair which is accurate to ±3us (since the uncertainty in the transmitter's and receiver's time accuracy are cumulative).

If both machine A and machine B need to know the pairing of their UST's, then you can do one of these:

If you were wondering, we purposely chose the code:
because, in addition to being silly, it is an essentially unachievable UST. This means that the receiver will never receive a UST with the value 0xFEEDBABEDEADBEEF and mistake it for the special code that indicates a UST is next.

Why is it unachievable? UST measures elapsed nanoseconds since system startup. In order for the UST to reach 0xFEEDBABEDEADBEEF, system A would have to be up for:

  0xFEEDBABEDEADBEEF == 18369543784056602351
  18369543784056602351 / (1000000000 * 60 * 60 * 24 * 365) = 582 years
You can even do this hack without dedicating an entire digital audio channel to it. If you transmit and receive with 24 bits of precision, you can transmit the timing signal 0xFEEDBABEDEADBEEFaabbccddeeffgghh one bit at a time using the least significant bit of the audio samples. Thus the timing information overlays the audio information like a water mark overlays an image. This low bit is very likely to be inaudible. Whether or not you actually need it intact depends on your application.

Relating the UST of Two Machines Using Other Signals

The hack above can easily be adapted to other signals: