Configurable Input Devices for 3D Interaction using Optical Tracking

Abstract

Three-dimensional interaction with virtual objects is one of the aspects that needs to be addressed \nin order to increase the usability and usefulness of virtual reality. Human beings \nhave difficulties understanding 3D spatial relationships and manipulating 3D user interfaces, \nwhich require the control of multiple degrees of freedom simultaneously. Conventional interaction \nparadigms known from the desktop computer, such as the use of interaction devices as \nthe mouse and keyboard, may be insufficient or even inappropriate for 3D spatial interaction \ntasks. \nThe aim of the research in this thesis is to develop the technology required to improve 3D \nuser interaction. This can be accomplished by allowing interaction devices to be constructed \nsuch that their use is apparent from their structure, and by enabling efficient development of \nnew input devices for 3D interaction. \nThe driving vision in this thesis is that for effective and natural direct 3D interaction the \nstructure of an interaction device should be specifically tuned to the interaction task. Two \naspects play an important role in this vision. First, interaction devices should be structured \nsuch that interaction techniques are as direct and transparent as possible. Interaction techniques \ndefine the mapping between interaction task parameters and the degrees of freedom of \ninteraction devices. Second, the underlying technology should enable developers to rapidly \nconstruct and evaluate new interaction devices. \nThe thesis is organized as follows. In Chapter 2, a review of the optical tracking field is \ngiven. The tracking pipeline is discussed, existing methods are reviewed, and improvement \nopportunities are identified. \nIn Chapters 3 and 4 the focus is on the development of optical tracking techniques of rigid \nobjects. The goal of the tracking method presented in Chapter 3 is to reduce the occlusion \nproblem. The method exploits projection invariant properties of line pencil markers, and the \nfact that line features only need to be partially visible. \nIn Chapter 4, the aim is to develop a tracking system that supports devices of arbitrary \nshapes, and allows for rapid development of new interaction devices. The method is based on \nsubgraph isomorphism to identify point clouds. To support the development of new devices \nin the virtual environment an automatic model estimation method is used. \nChapter 5 provides an analysis of three optical tracking systems based on different principles. \nThe first system is based on an optimization procedure that matches the 3D device \nmodel points to the 2D data points that are detected in the camera images. The other systems \nare the tracking methods as discussed in Chapters 3 and 4. \nIn Chapter 6 an analysis of various filtering and prediction methods is given. These \ntechniques can be used to make the tracking system more robust against noise, and to reduce \nthe latency problem. \nChapter 7 focusses on optical tracking of composite input devices, i.e., input devices \n197 \n198 Summary \nthat consist of multiple rigid parts that can have combinations of rotational and translational \ndegrees of freedom with respect to each other. Techniques are developed to automatically \ngenerate a 3D model of a segmented input device from motion data, and to use this model to \ntrack the device. \nIn Chapter 8, the presented techniques are combined to create a configurable input device, \nwhich supports direct and natural co-located interaction. In this chapter, the goal of the thesis \nis realized. The device can be configured such that its structure reflects the parameters of the \ninteraction task. \nIn Chapter 9, the configurable interaction device is used to study the influence of spatial \ndevice structure with respect to the interaction task at hand. The driving vision of this thesis, \nthat the spatial structure of an interaction device should match that of the task, is analyzed \nand evaluated by performing a user study. \nThe concepts and techniques developed in this thesis allow researchers to rapidly construct \nand apply new interaction devices for 3D interaction in virtual environments. Devices \ncan be constructed such that their spatial structure reflects the 3D parameters of the interaction \ntask at hand. The interaction technique then becomes a transparent one-to-one mapping \nthat directly mediates the functions of the device to the task. The developed configurable interaction \ndevices can be used to construct intuitive spatial interfaces, and allow researchers to \nrapidly evaluate new device configurations and to efficiently perform studies on the relation \nbetween the spatial structure of devices and the interaction task