Jan 21, 2012

Virtual Object Interaction

I am really interesting on virtual reality concepts and its application. Therefore I thought to note down some knowledge, I gained while reading for my research. Please note that if you are using these details you must refer original papers in your publication. I just note down them in point form.

When you are navigating in a virtual world you may need to interact with virtual objects sometimes. Those objects may be dynamic or static virtual objects.

Let's consider a foot ball that you are given to play in a foot ball game. What are you going to do if the ball coming towards you? Depending on the team which you are playing, you will hit to ball along your desired direction. And what happen if you are the referee in the game? In that kind of situations you may need to grab the attention from team players and sometimes you also need to grab the ball.

Considering about an immersive virtual environment the situation may be different than previous example. Someone can argue that issuing an abstract command is sufficient instated using metaphors. But if the user is navigating on a immersive virtual environment, metaphors are needed to be interact with virtual objects in the VE. The virtual interaction can be described in two cases. Those are grabbing or selection and manipulation. In this article I discuss about virtual object grabbing and manipulation techniques which I encountered during the literature review in my research. 

Grabbing and Manipulation  

  • Arm Extension Technique 
When considering of a natural metaphor such as an arm user must be within the length of arm. In an arm extension techniques user is able to grow the virtual arm to the desired length.In this technique grabbing is easy but manipulation may be difficult due to virtual hand must be positioned within the object, which may be small or distinct.  
    •   Go-Go Technique
In this technique a virtual hand is moving in one-to-one mapping with physical hand of the user. At some point, say a threshold, the virtual hand is growing faster than physical hand. Therefore this technique allows user to reach objects which are situated at greater distance from the user. Another advantage of one-toone mapping area is user able to manipulate virtual objects as he does in the normal manner. Similar to this technique, another technique has been introduced which named as 'fast go-go' techniques. This has quick growing function of virtual hand in the no local area of the user.

  •   Ray Casting Techniques
 A virtual light is used in this technique to grab the attentions of a virtual object. The direction of the light should be along with users' hand.  Therefore grabbing task would be easy since user needs to pointing to an interested object. But using this method manipulation is become difficult.  
The major problem of this technique is called "lever-arm" problem. As the object is attached to the end of the ray, there is no way to rotate around the object except axis of the ray itself. And it gives 1 digree of freedom which rotation around the axis may be controlled. If user needs to move the object vertically he must also translate it and rotate it in another direction. 

  • Stretch Go-Go
In this technique the different constant growing speeds for the users' hand is assigned in three different regions. Using this techniques any length of the arm can be obtained. Only difference is in this technique user allows stretching his arm using arm motion controlling. If the arm is brought back virtual hand retracts in the same speed. This is cognitively challenging for the go-go technique but it does also allow for the arm to be placed at any distance. 
  • Indirect Stretching
In this technique user is able to stretch or retract his arm using two buttons of a 3D mouse.This is far away from natural metaphor of stretching the arm hence we cannot use this kind of technique for our research. But this technique is more precise and easy to use. 

  • Fishing Reel Metaphore 
 As earlier mentioned ray casting has certain limitation with object manipulation. This technique is an enhancement of ray casting technique. Using this metaphor user able to reel the object closer or further away with two mouse buttons. Therefore it would improve one additional degree of freedom of ray casting although it is indirect way of object selection.  

User Study
  • Indirect stretching and go-go technique was popular among users but it was noted some difficulties.
  • Users  almost  unanimously found  go-go to be the  most  natural  technique,  but  many  evaluators  preferred other techniques.
  • Indirect stretching was preferred by several subjects as it offers more precise control of the hand and less physical work on the part of the user. 
  • Due to lack of physical work has to be done ray casting with reeling was preferred by some of the user.
  • Almost every users are commented that ray casting is the best way to grab an object other than any of arm extension techniques. It requires no arm stretching and just need to point to an object and releases the mouse buttons while arm extension techniques one must place the hand within the object. Hence for grabbing ray casting is preferred. But non of them are easy to use for entire iteration: grabbing, manipulation and releasing.
Hybrid Approach

Based on user feedback, we have designed and  implemented  new techniques which  are  hybrids of the best features from  those discussed above,  but  still retain  ease of  use and  simplicity. These techniques use  ray-casting for  object grabbing  and  hand-centered manipulation, since these methods  were thought to be the easiest  to  use and  most efficient by almost all  subjects. We call this the HOMER  ( H and-centered
O bject M anipulation  E xtending  R ay-casting) technique.


  • D. A. Bowman and L. F. Houges, “Formalizing the design,  evaluation,  and  application  of  interaction techniques for immersive virtual environments,”  Journal  of  Visual Languages and Computing, vol. 10, no. 1, pp. 37-53, Feb. 1999.
  • S.  Lee,  J.  Seo,  G.  P.  Kim,  and  Park  Chan-Mo, “Evaluation  of  pointing  techniques  for  ray  casting selection in virtual environments,” in Third International Conf.  Virtual  Reality  and  Its  Application  in  Industry, 2003, pp. 38-44. 

Jan 17, 2012

Head Tracking in Detail

Head tracking is one thing that should consider when implementing the interactive VR applications. It is needed to calibrate the users' head position and orientation.

  • Optical Tracking

In optical tracking it uses camera and video stream as the input. Then the video frames are enhanced by image processing techniques and identify the certain feature of the image using vision technology. So that this feature identification can be determine the head position and orientation.  This can be divided into two parts.

  1.  image feature based tracking

 Shape, color and organization of the organs in the face are considered as feature parameters in feature based tracking. Just to mention few examples are  Elliptical head tracking systems, example based head tracking system, color based head tracking systems using intensity gradients and color histograms.

  1.  Marker based tracking

Two or more cameras are using for 6DOF tracking.  Marker based tracking can be divided into two parts. They are active maker based tracking and passive marker based tracking. Both are using simple vision algorithms to calibrate head position and orientation. Active marker based tracking are accurate and fast and passive marker based tracking is simple and inexpensive. 

  • Wii remote tracking

Comparing with optical marker based tracking, Wii mote tracking is more accurate. And marker based tracking has certain limitation.
  1. Light condition may affected for accurate tracking
  2. Markers may got invisible.
  3. LED markers may be good but it is also confused with light illumination problems.
  4. Environment should not contain colors that related to markers 
Using four IR lights wii mote tracking can be done but dur to inability of identify IR lights separately can not get the orientation correctly.

  •  Head Tracking with Consumer Products 

  1. FAAST
FAAST is middleware to facilitate integration of full-body control with games and VR applications.  The toolkit relies on upon Microsoft Kinect sensors. includes a custom VRPN server to stream the user’s skeleton over a network, allowing VR applications to read the skeletal joints as trackers using any VRPN client. Additionally, the toolkit can also emulate keyboard input triggered by body posture and specific gestures. This allows the user add custom body-based control mechanisms to existing off-the-shelf games that do not provide official support for depth sensors.

FAAST is free to use and distribute for research and noncommercial purposes (for commercial uses, please contact us). If you use FAAST to support your research project, we request that any publications resulting from the use of this software include a reference to the toolkit .

  1.  Wii system
Nintendo completely redefined the nature of the competition with the release of its Wii system.  Comparing with xBox360 Nintendo's price is cheaper.

  • Sources
  1. J. D. Mulder, J. Jansen, and A. V. Rhijn, “An affordable optical  head  tracking  system  for  desktop  VR  /  AR systems,” in Proc.  workshop on Virtual environments 2003, pp. 215-223.