Displaying Video With the ev1 Video to Screen Path

By Michael Portuesi.

Introduction

Because the VL supports so many different pieces of video hardware, you might be tempted to think that for a given task, such as displaying incoming video on the workstation screen, that you may write one piece of code for all the devices VL supports. Unfortnately you cannot, and nowhere is this more apparent than when you are programming video display using Galileo (ev1) and its relatives.

With most video devices that support a full rate video path to memory, you can create a video to memory path and display the video in your application using OpenGL. Your application wakes up whenever the transfer buffer is full, retrieves a frame from the buffer, and draws the frame to the display.

Galileo does not offer you this option; its video to memory path does not support a full-rate stream into memory. Instead, it offers a VL screen node. You create a video to screen path using the VL, set some controls on the path to tell the screen node the X window it should use and where the video should be located, then call vlBeginTransfer() to display the video to the user.

The code to display video using ev1 differs from other devices not only in the path setup, but in the maintenance work your application must perform. Using ev1, the display work is handled automatically; your application need never service a transfer buffer. But your application must keep the video positioned and sized appropriately, and update its position if the user moves the window. This task is the central topic of this document.

The ev1 Screen Drain Nodes

The ev1 video device (in all its forms) provides the programmer with three screen drain nodes. Here's how they are described by vlinfo:

lipstick.esd.sgi.com % vlinfo
name of server: 
number of devices on server: 2

device: ev1 1
 nodes = 14

[entries omitted]

 Screen Drain A, type = Drain, kind = Screen, number = 0
 Screen Drain B, type = Drain, kind = Screen, number = 1
 Screen Drain C, type = Drain, kind = Screen, number = 2

[entries omitted]

However, in practical use, you will likely use only the first two nodes - numbers 0 and 1. Node 1 can be divided into two half-sized 12-bit screen displays. When you divide screen drain node 1, the second of the two half-displays is denoted by node number 2. This special "half size" mode is not discussed in this document, and judging from the restrictions on its use over and above those discussed here, it seems inappropriate at best for general-purpose application software.

Positioning Video, or Why Does videoin Jump All Around the Screen?

By far, the biggest headache facing the ev1 programmer is simply positioning the video at the right place on the screen. The video drain doesn't render into the framebuffer, as does all other visual display on the system. Instead, the ev1 device provides a hardware-generated "overlay" which is merged with the contents of the frame buffer. As such, all attributes of the video display including screen position and size are controlled through the VL, rather than through X or OpenGL.

To make matters worse, the ev1 hardware imposes a raft of restrictions on where and and under what circumstances the video overlay can be positioned. None of them are particularly friendly to the application developer. Here is a list of the problems:

• The coordinates for the video overlay are expressed using screen (root window) coordinates, not the coordinate system of the window containing the overlay.
  
  
• You cannot position a video overlay such that it shares the same raster lines with another video overlay that may already be on the screen. Because the other video overlay might belong to another application, you must deal with this problem in your code if your application might possibly be run in tandem with any other video applications.
  
  If your application attempts to move the video overlay to a position where it conflicts with another, the overlay will jump to an arbitrary screen position which does not cause a conflict. Additionally, the VL will not return an error to your application. It is your application's responsibility to check for and deal with this problem explicitly
  
  This problem, and one solution to it, can easily be seen by running the bundled videoin utility. If you start up two instances of videoin for ev1, and try to position them on the screen, the windows will often jump to random locations when you drag their window frames. videoin contains code to check the VL_ORIGIN control immediately after it sets it, and then moves its X window to where the video landed! See below for advice on how to handle this situation in a better, if not totally, acceptable way.
  
  
• You cannot position the video overlay even partially offscreen. If you attempt to do so, the edge of the video overlay will "bump up against" the edge of the screen and go no further.
  
  
• You cannot position the video overlay such that any portion of the overlay occupies the last seven pixels on right hand side of display. If you attempt to do so, the right edge of the video overlay will "bump up against" pixel 1271 on the display and go no further.
  
  
• To determine if video was positioned properly in all these instances, you must immediately get the VL_ORIGIN control after setting it and check to see if it is in the same spot you asked for.

How to Position Video Properly in a Motif application

A civilized application program must allow its windows to go where the user places them. We can forgive videoin due to its simplicity. But a better solution for most applications is to put the window where the user wants it, but to post a notifier if the ev1 video display cannot be placed to match the window location. This might disappoint the user, but nevertheless it still leaves them feeling like they have control over the windows on their screen.

• The first step is to determine when the user moves the topmost window of your application to another place on the screen. This is done by installing an Xt event handler for the appropriate events on the topmost window of your application. When the user moves the window, the X server sends an event to the window, and Xt will then invoke a callback function in your code.
  
  There is one major complication. Not only must you install an event handler on the topmost shell widget, you need to install one on every widget in the tree between the shell and the widget displaying video. This reason for this complication is because Xt and Motif geometry operations can move the video window as well.
  
  The most subtle form of this problem happens when the video is displayed in an XmMainWindow widget. If the XmMainWindow needs to make the menu bar span two rows instead of one, it will move the widget displaying video lower on the screen to accomodate the fatter menu bar. As a Motif programmer, you have no control over this behavior; your only option is to recognize it and react accordingly.
  
  Here is a code sample to install event handlers all the way up the widget hierarchy, starting with the widget displaying video and ending with the application's top-level shell widget:

  void installStructureNotifyHandler()
  {
      if (handlerActive) return;
  
      Widget shell = _WidgetDisplayingVideo;
      
      while (!XtIsShell(shell)) {
  	XtAddEventHandler(shell, StructureNotifyMask,
      			 FALSE, &structureNotifyHandler,
      			 (XtPointer) this);
      	shell = XtParent(shell);
      } 
  
      XtAddEventHandler(shell, StructureNotifyMask,
      		     FALSE, &structureNotifyHandler,
      		     (XtPointer) this);
      handlerActive = True;
  }
  

  
  The code to undo this mess looks very similar:

  void removeStructureNotifyHandler()
  {
      if (!handlerActive) return;
  
      Widget shell = _WidgetDisplayingVideo;
      
      while (!XtIsShell(shell)) {
  	XtRemoveEventHandler(shell, StructureNotifyMask,
      			 FALSE, &structureNotifyHandler,
      			 (XtPointer) this);
      	shell = XtParent(shell);
      }
  
      XtRemoveEventHandler(shell, StructureNotifyMask,
      		     FALSE, &structureNotifyHandler,
      		     (XtPointer) this);
  
      handlerActive = False;
  }
  

  
  
  The event handler callback must check for the appropriate event type and react accordingly:
  
  

  void structureNotifyHandler(Widget w, XtPointer client_data,
  			    XEvent* event, Boolean*)
  {
      switch (event->type) {
      case ConfigureNotify:
        // we've been moved around onscreen.  update our video display
        if (XtIsRealized(w)) {
  	//
  	// figure out dimensions and screen coordinates of the 
  	// video widget
  	//
  	Position x, y;
  	Position rootx, rooty;
  	Dimension widgetW, widgetH;
  	
  	XtVaGetValues(w,
  		      XmNx, &x,
  		      XmNy, &y,
  		      XmNwidth, &widgetW,
  			XmNheight, &widgetH,
  		      NULL);
  	XtTranslateCoords(XtParent(w), x, y, &rootx, &rooty);
  
  	//
  	// armed with root x and root y, we now calculate the
  	// placement of the video overlay.
  	//
  	if (!displayPath) return;
  
  	// determine size of video display.
  	VLControlValue val;
  	vlGetControl( svr, displayPath, src, VL_SIZE, &val );
          int sizeW = val.xyVal.x;
          int sizeH = val.xyVal.y;
  	// divide by current ZOOM setting.  assume that zoom
  	// control is 1/n.
  	vlGetControl( svr, displayPath, drn, VL_ZOOM, &val );
  	sizeW /= val.fractVal.denominator;
  	sizeH /= val.fractVal.denominator;
  
  	//
  	// find the origin coordinates of the video overlay,
  	// taking into account that the video overlay cannot
  	// go even partially off the screen.
  	//
  	int originX = rootx + ((Position) (widgetW / 2) -
  			       (Position) (sizeW / 2));
  	int originY = rooty + ((Position) (widgetH / 2) -
  			       (Position) (sizeH / 2));
      
  	// did we go off the screen?  if so, make adjustments.
  	Dimension screenWidth = WidthOfScreen(XtScreen(w));
  	Dimension screenHeight = HeightOfScreen(XtScreen(w));
  
  	if (originX < 0) {		       	// left edge
  	  originX = 0;
  	}
  	if (originY < 0) {		       	// top edge
  	  originY = 0;
  	}
  	if (originX + sizeW > screenWidth) {	// right edge
  	  originX = screenWidth - sizeW - 1;
  	}
  	if (originY + sizeH > screenHeight) {	// bottom edge
  	  originY = screenHeight - sizeH - 1;
  	}
  
  	// place the video overlay.
  	val.xyVal.x = originX;
  	val.xyVal.y = originY;
  	vlSetControl(svr, displayPath, drn, VL_ORIGIN, &val);
  
  	// verify the video was placed where we want it.
  	checkVideoOverlayLocation(originX, originY);
        }
        break;
      default:
        break;
      }
  }
  

  
  
• Finally, your code for checking VL_ORIGIN shouldn't just check to see that the origin is different than you asked - it should try to determine whether the problem is due to overlapping video, or bumping against the edge of the display. You may not want to complain to the user or take drastic action if the video is merely running against the edges of the screen.
  
  

  void checkVideoOverlayLocation(Position originX, Position originY)
  {
      VLControlValue actualVal;
      vlGetControl( svr, displayPath, drn, VL_ORIGIN, &actualVal);
      
      //
      // Collisions between video overlays occur only when they are
      // on the same raster lines (horizontal), so comparing just the
      // y-value is sufficient to detect these cases.
      //
      // We do not worry about offscreen placement,
      // because that is taken into account elsewhere.
      //
      if (actualVal.xyVal.y != originY) {
        //
        // video overlay didn't position properly.  Your app should
        // take some sort of action here.  Keep in mind that if you
        // post an error dialog, you should also set a state flag so
        // that you do not post the dialog more than once.  This
        // code will (unfortunately) get called multiple times,
        // because the event handler for ConfigureNotify events is
        // installed on several widgets in the hierarchy.
        //
      }
  
      //
      // Compare the x values to determine
      // if the right edge of the display
      // cannot go past pixel 1271
      //
      if (actualVal.xyVal.x != originX) {
        // your app might want to do something here
      }
  }
  

Sizing Video

• When you set the VL_ZOOM control, you also need to set the VL_SIZE control at the same time to guarantee the video displays properly.

Get Out of Jail For Free (almost)

Yet another unpleasant side effect of the ev1 video overlay are the "jailbars" which appear when the window is larger than the video, or when the user moves around on the screen a window containing ev1 video.

The "jailbars" happen when you assign an X window to the video node by setting the VL_WINDOW control on the path. The graphics hardware ignores the framebuffer contents for that region of the display, replicating the last value that was in its hardware registers. That last value is usually the pattern for the window manager border, which when replicated looks like a row of brass bars on the display.

If the window is larger than the video overlay, the regions of the window above and below the video overlay which don't contain video information will display jailbars. There is no way around this. There are a few ways to deal with the problem:

• Make sure the widget or window containing the video is never larger than the video display itself. Sometimes, this is easier said than done.
  
  
• A second way to deal with the problem is most applicable in situations where you may not have absolute control over the window size (perhaps your code might be part of a library which is used by several applications). You put the widget containing video together with two other child widgets inside an XmForm. The two child widgets "cover up" the jailbar area above and below. The parent Form widget moves and resize them horizontally as the window moves and resizes. You must move and resize them vertically at the same time you place the video overlay.
  
  

  void createJailBarWidgets()
  {
      //
      // top widget
      //
      topArea = XtVaCreateManagedWidget
        ( "topArea",
  	xmDrawingAreaWidgetClass,
  	form, 
  	XmNtopAttachment, XmATTACH_FORM, 
  	XmNbottomAttachment, XmATTACH_NONE, 
  	XmNleftAttachment, XmATTACH_FORM, 
  	XmNrightAttachment, XmATTACH_FORM, 
  	XmNrightPosition, 0, 
  	XmNtopOffset, 0, 
  	XmNbottomOffset, 0, 
  	XmNleftOffset, 0, 
  	XmNrightOffset, 0, 
  	XmNwidth, topAreaWidth, 
  	XmNheight, topAreaHeight,
  	XmNbackground, BlackPixelOfScreen(XtScreen(form)), 
  	(XtPointer) NULL ); 
  
      //
      // bottom widget
      //
      bottomArea = XtVaCreateManagedWidget
        ( "bottomArea",
  	xmDrawingAreaWidgetClass,
  	form, 
  	XmNtopAttachment, XmATTACH_NONE, 
  	XmNbottomAttachment, XmATTACH_FORM, 
  	XmNleftAttachment, XmATTACH_FORM, 
  	XmNrightAttachment, XmATTACH_FORM, 
  	XmNtopPosition, 0, 
  	XmNbottomPosition, 0, 
  	XmNrightPosition, 0, 
  	XmNtopOffset, 0, 
  	XmNbottomOffset, 0, 
  	XmNleftOffset, 0, 
  	XmNrightOffset, 0, 
  	XmNwidth, bottomAreaWidth,
  	XmNheight, bottomAreaHeight, 
  	XmNbackground, BlackPixelOfScreen(XtScreen(form)), 
  	(XtPointer) NULL ); 
  }
  
  void placeJailBarWidgets(Dimension widgetW, Dimension widgetH,
  			 Position originX, Position originY,
  			 Position rootY, Dimension videoH)
  {
    Dimension bottomAreaWidth, bottomAreaHeight;
    Dimension topAreaWidth, topAreaHeight;
  
    bottomAreaWidth = topAreaWidth = widgetW;
      
    int newTopHeight = ((int) originY) - ((int) rooty);
    if (newTopHeight < 0) {
      topAreaHeight = 0;
    } else {
      topAreaHeight = newTopHeight;
    }
      
    int newBottomHeight = ((int) widgetH) -
      (((int) originY) - ((int) rooty) + ((int) videoH));
    if (newBottomHeight < 0) {
      bottomAreaHeight = 0;
    } else {
      bottomAreaHeight = newBottomHeight;
    }
  
    XtVaSetValues(topArea,
  		XmNheight, topAreaHeight,
  		(XtPointer) NULL);
    XtVaSetValues(bottomArea,
  		XmNheight, bottomAreaHeight,
  		(XtPointer) NULL);
  }
  

• Another way to handle this inside a Motif program is to nest the widget displaying video inside two XmForms. The widget is attached positionally to the edges of the innermost XmForm, and the innermost XmForm is attached relatively to all edges of the outermost XmForm. We do not present code for this solution here, but it is a simple matter for any moderately experienced Motif developer.

IMPACT Graphics do not display jailbars when used with ev1. But unless you know your application will run only on IMPACT, you need to worry about jailbars.

Finally, once a window is used with the ev1 video display node, it cannot be used for anything else. You cannot render into the window using X or GL drawing operations, even if you destroy the ev1 path which was displaying into the window. Once a window is in jail, it's a life sentence. The only surefire way to "get out of jail" is to destroy the window and create one anew.

Why Can't I See the Video?

There is a very common problem encountered by people who try to display ev1 video within an X or Motif program. If you call vlBeginTransfer() before the window intended to display it is fully created and mapped (shown) to the screen, the video will not display.

This problem occurs because the X window system is asynchronous. The X server does not immediately create and display windows when the application invokes the appropriate calls. Instead, the requests are queued up for the server, which performs them at some arbitrary point in the future. If you start the transfer immediately after calling a function like XtManageChild() or XMapWindow(), the video display won't work.

There are two ways to deal with the problem.

• If you are writing an Xt or Motif application, or have access to the X event loop, set an event handler on the widget to display video, using the StructureNotifyMask. When you receive a MappingNotify event, the X server has mapped the window to the display, and it is safe to start the video transfer.
  
  You can code this easily by adding a new clause to the switch statement in the StructureNotifyHandler sample function shown above. This is the preferred, though the more complicated, method of dealing with the problem.
  
  
• If you are writing a non-Motif X application where you do not have access to the X event loop, try calling XSync() after you make the call to display the window. This will force all outstanding requests to be sent to the X server, and XSync() will wait until the server has processed them.
  

Handling Signal Preemption

When your code receives a VLStreamAvailable event for the display path, your code should do the following:

• call vlSetupPaths() to set the path up once again:
  vlSetupPaths(svr, &displayPath, 1, VL_SHARE, VL_SHARE);
  
  
• set the VL_ORIGIN, VL_SIZE and VL_ZOOM controls for the path, since the previous app using this path have left it in an unknown state.
  
  
• If you have windows mapped to cover the jailbars (see "Get Out of Jail For Free," above), update their size and position.
  
  
• call vlBeginTransfer() to restart the video transfer:
  vlBeginTransfer(svr, displayPath, 0, NULL);