primary

5. PRIMARY USER INPUT INTERFACES
The majority of the types of interfaces discussed in this report are those that provide sensory inputs to the user, specifically, visual, auditory, and haptic inputs. It is important to look at the other side of the coin, that is, inputs from the user to a VE system. While trackers provide one type of input to a VE, this is usually a passive form of input as far as the user is concerned and not a means whereby the user can specify commands to the system. The devices discussed in this section provide the primary means for direct user interaction with VEs. They allow specifying input commands that serve, for example, to determine movement through the environment or effect virtual object manipulations. These devices are not limited to use with VEs, but are used with many computer-based applications that require interaction with 3-D objects, such as computer-aided design (CAD) applications. The form of the input command itself varies from naturalistic hand gestures, menu item selection, or object selection, to the operation of buttons with preset functions. Two categories of devices are considered in this section: (1) gloves and exoskeleton devices that can support the first two type of command form, and (2) 3-D pointing devices that support menu item and object selection and often provide a number of user-programmable buttons. The choice of which type of device is preferable in any particular application largely rests on the degree of naturalness desired for the interface. 5.1 Whole-Hand and Body Inputs

Sturman and Zeltzer (1993) define whole-hand inputs as “the full and direct use of the hand’s capabilities for the control of computer-mediated tasks... [providing an interface that] makes maximal use of the naturalness, dexterity, and adaptability of the human hand.” Since one of the primary goals of VEs is to enable natural methods of interaction, it is hardly surprising that whole-hand input in the form of hand gestures are one of the most commonly used methods for providing user inputs in the VE. Gestures are used to transmit messages relating to desired movement through the environment, and to select and manipulate objects, even objects as diverse as option menus and soda cans. While the advantages of gestures can seem clear in many VE applications, there has been little experimental evaluation of the use of whole-hand input compared to other forms of input. In one set of experiments, Sturman and Zeltzer (1993) compared the use of gestures to conventional input via a set of dials for the control of a six-legged walking robot. The gesture input was found superior for control of low-level walking and for high-level manipulation, whereas the conventional input gave best performance when steering the robot: the gesture input gave the

97

best performance when the required interaction mapped well to the human hand in terms of naturalness, adaptability, and dexterity, and when task characteristics (such as required degrees of freedom, resolution, and steadiness) mapped well to hand action capabilities. While the trackers previously discussed can provide information about absolute position of a user’s hand in space and palm orientation, use of the hand as an input device generally requires additional information. Gesture recognition is dependent on information about relative finger positions and this is determined by measuring the angles of joints in the fingers. Similarly, many applications that require knowledge of the movement of various user body parts require information about the joints that control those body parts. Before proceeding to look at specific products and current R&D in this area, this section presents applicable data on the range of motion of the human hand, arm, and shoulder, and human capabilities in sensing the positions of these joints. 5.1.1 The Human Hand and Arm Position Sense

As shown in Table 8, the range of motion provided by human joints varies quite widely. The human hand in particular is capable of great freedom of movement, providing 29 DOFs. Twenty-three of these DOFs are exhibited by the joints in the hand, and the remaining 6 DOFs by palm.
Table 8. Range of Motion for Hand, Arm, and Shoulder Jointsa Joint
Thumb

Motion
Palmar Adduction Radial Abduction Opposition MCP Flexion PIP Flexion

Range
90° 80-90° 90° 50° 80° ±15° 86-90° 22-45° 100-110° 91° 18° 105°

Joint
Shoulder

Motion
Abduction/Adduction Media/Lateral Rotation Horizontal Flexion/Extension Scapula Elevation/Depression Scapula Medial/Lateral Movement Scapula Rotation

Range
150-184° 130° 170° 10-12 cm 15 cm 60° 145° 155-180° 85/170° 56−60° 15° 45-55°

Digits

Abduction/Adduction Index MCP Flexion Index MCP Extension Index PIP Flexion 2nd finger MCP Flex. 2nd finger MCP Ext. 2nd finger PIP Flexion

Elbow Wrist

Elbow Flexion/Extension Forearm Supination/Pronation Flexion/Extension Radial/Ulnar Deviation Abduction Adduction

a. Adapted from (Greene and Heckman, 1994), (Livingstone, 1983), (Reynier, 1993), and (Sturman, 1992).

With respect to position sensing, Tan et al (1994) report that the just-noticeable-difference (JND) for the proximal-interphalangeal (PIP) and metacarpal-phalangeal (MCP) finger joints is 2.5°, at the wrist and elbow 2°, and decreases to 0.8° at the more proximal shoulder joint.

98

5.1.2

Commercially Available Devices

There are four glove-based devices for whole-hand tracking on the market, and these are described below. Two exoskeleton devices for hand and arm measurement are also commercially available. At the present time, only one body suit instrumented to measure the angles of various limbs is commercially available. Table 9 summarizes some of the key features of these devices. A glove product that is available in Japan but not, as yet, in the U.S. is Nissho Electronics Corporation’s SuperGlove. Another glove-based product is the PC PowerGlove, under development by the makers of the Mattel Power Glove, Abrams Gentile Entertainment, Inc., and intended to replace the original glove. The PC PowerGlove is scheduled to be released in the first quarter of 1996. It is designed for use with PC video games, and supports position and orientation tracking in 6 DOFs, finger position measurement, and tactile and sweat feedback. Additionally, some of the force feedback devices discussed in Section 6.2 have integrated joint position measurement capabilities. Virtual Technologies plans to bring a body suit to market in the near future, and Paradigm Shift is developing both a glove and a body suit interface device. Greenleaf Medical Systems has acquired the licensing rights to market VPL’s DataGlove and DataSuit for medical applications and are likely to acquire the remainder of VPL’s assets. This would broaden their rights for use of the glove and suit technology in other application areas and may lead to the development of some commercial products. 5.1.2.1 5th Glove

Fifth Dimension Technologies’ 5th Glove uses proprietary optical-fiber flexor technology sensors. Each finger of the glove is fitted with a sensor that measures the average flexure of the finger. In its latest release, the 5th Glove also includes a 2-axis tilt sensor that measures ±60° roll and pitch orientation of the user’s hand and can be mounted in either the horizontal or vertical direction. This new sensor allows the glove to emulate a mouse or a baseless joystick. For its physical structure, the glove uses stretch lycra material with the flexor sensors mounted on the fabric and the tilt sensor attached by velcro. A small electronics box is fastened to the glove and mounted on top of the wrist. A specification for the device is given in Figure 52. The device interface is serial RS-232 (3 wire) at 19.2 kbaud (full duplex). The number of gloves that may be supported simultaneously is limited by the number of serial ports. The interface package is supported by Windows and DOS-based software that enable installation, glove calibration, and graphical (using approximately 80 polygons) representation of a virtual hand. A gesture recognition program can be trained to identify certain hand positions using a least squares fit algorithm. A program called KineMusica that converts finger bend data to MIDI output and allows playing a variety of musical instruments is included with the package. Finally, a DOS application, with C++ source code, provides raw glove data and can be used to support development of device drivers for other applications. Additional software is provided with the latest release of the glove.

99

Table 9. Characteristics of Commercial Available Glove, Body Suit, and Hand/Arm Exoskeleton Input Devices Vendor
Fifth Dimension Technologies Glove Finger flexure, hand roll & pitch 8 bit ~350 g

Device Name

Device Type Measurements Weight

Sensor Resolution

Software Support
Windows & DOS interface package & drivers, C++ source code, gesture recognition, other VirtualHand Silicon Graphics interface, DOS demo interface. (GesturePlus package separate)

Price
$495

5th Glove

CyberGlove

Virtual Technologies, Inc.

Glove (18 sensor)

PIP, MCP finger joint angle, finger abduction, thumb opposition, palm arch, wrist flexion & abduction 3 flex sensors and abduction sensor per finger, thumb opposition, palm arch, wrist flexion and abduction 16 DOFs for fingers, 4 DOFs for thumb Identification of finger/thumb contacta Shoulder flexion/extension, shoulder ab/adduction, shoulder int/ext rotation, elbow flexion/extension, forearm supination/pronation PIP finger joint angle, 2 thumb joint angles, palm movement As 8 sensor version, plus: MCP finger joint angle, wrist flexation, adduction, and abduction Angles for all finger and thumb joints, wrist movements Custom designed for measurement at 32 sites Unavailable Unavailable N/A 0.1° 0.1° <15 oz 0.7 oz <1 lb (on arm)

0.5°

2.5 oz

$9,800

Glove (22 sensor)

$14,500

Dextrous HandMaster Fakespace, Inc. EXOS, Inc. Exoskeleton Glove (2)

EXOS, Inc.

Exoskeleton

Silicon Graphics interface Silicon Graphics & PC interface Silicon Graphics interface

Contact vendor $2,000 Contact vendor

100
T.C.A.S. Effects Ltd. Glove (11 sensor) Glove (8 sensor) Glove (16 sensor) T.C.A.S. Effects Ltd. Body suit

Pinch Glove

Position Exoskeleton ArmMaster

TCAS Glove

Unavailable

Unavailable

Support for display of data to screen or file, interfaces to some VE toolkits

Starts at $7,000

TCAS Data Wear

Support for display of data to screen or file, interfaces to some VE toolkits

Starts at $30,000

a. This system interprets contacts between any two or more fingers or thumbs (on both hands) as a particular gesture.

.

Update Rate Sampling Rate Sensor Resolution Latency Calibration Glove Sizes Glove Weight Interface

Specification Each sensor 200 Hz (max) For full hand, 200 samples/sec (max) Flex and tilt sensors, 8 bit 5 ms Once per user One size ~350 g RS-232

Photo courtesy of Fifth Dimension Technologies

Figure 52. 5th Glove

This new software includes both 16-bit and 32-bit DOS drivers, a diagnostic program, a Windows 32-bit DDL, upgradable to Windows ‘95. In addition, it includes a Microsoft mouse emulator that interprets the user tilting his hand as cursor movement commands. Drivers to support the use of the 5th Glove with the Sense8 WorldToolKit are available, and drivers for Vream, Inc.’s VRCreator and Avril are expected by the end of 1995. Fifth Dimension Technologies is developing a standard for usage of the 5th Glove. This standard is intended to ensure that a glove user, working with different applications, will not have to learn different sets of gestures for each application. It defines, for example, the gesture required to perform body and hand rotation to the right in a VE. Once stabilized, this standard will be provided with the 5th Glove as the “Recommended Implementation,” and the defined gestures will be supported as defaults for the gesture recognition program. Meanwhile, default hand gestures have been defined to emulate button clicks. The current price for the 5th Glove system is $495 for right-handed users, and $535 for left-handed users. Volume discounts are available. In addition to being available from the developers, Fifth Dimension Technologies in South Africa and 5DT (Europe) in Surrey, UK, these products are available in the US from General Reality Company. Fifth Dimension Technologies also offers an ultrasonic tracker (see Section 3.1.1.21) that can be used with the glove or as a head tracker. 5.1.2.2 CyberGlove

Virtual Technologies, Inc. markets the CyberGlove, an instrumented glove primarily designed for manipulation of 3-D objects in the company’s CyberCAD virtual design environment. The glove is constructed of a 80/20 Nylon/Lycra blend for flexibility, with mesh in the palm area and underside of fingers for ventilation. (Fingertips are left open to allow ease of keyboard typing or other physical object manipulation.) The CyberGlove uses

101

up to 22 sensors to measure joint angles: 3 flex sensors and an abduction sensor per finger, and sensors to measure thumb opposition, palm arch, and wrist flexion and abduction. The glove has a software programmable switch and LED in the wristband that can be used to control program input/output capability; preprogrammed functions include a time-stamp and readout of glove status. The wristband also provides mounting provisions for either a Polhemus or Ascension 6 DOF tracking sensor. Further details are given in Figure 53.

s

Specification Sampling Rate Constant or polled; update rate over 100 Hz for 18 sensors operating at 38.4 kbaud, 150 Hz at 115.2 kbaud; selection of reduced set of sensors increases data rate Sensor Resolution 0.5° Latency 5 msec for 18 sensors Calibration Once per user Glove Size 1 size Glove Weight 2.5 oz Interface RS-232, up to 115.2 kbaud Photo courtesy of Virtual Technologies, Inc.

Figure 53. 22-Sensor CyberGlove

The CyberGlove comes with VirtualHand software for a Silicon Graphics workstation which displays a 2500-polygon Gouraud-shaded graphic representation of the user’s hand and finger movements, or a lower resolution 325-polygon hand model. An executable version of this software that can be used for calibration and demonstration purposes is available for PCs. The CyberGlove system is supported under the Silicon Graphics version of WorldToolKit, with support for other versions of this VE toolkit to follow in the near future. It is also supported by Division, Inc. in some of their toolkit products. For an 18-sensor glove, the CyberGlove with interface unit and VirtualHand executable software is priced at $9,800; the custom 22-sensor CyberGlove is $14,500. An additional Virtual Technologies, Inc. software product to support use of the glove was released in Fall 1995. Called the GesturePlus, this package uses neural networks that users can train to recognize their own customized hand gestures (up to 255 different gestures are possible). These gestures are associated with user-selected symbols to allow their mapping to user-defined actions. The GesturePlus system comes with an interface unit that performs the necessary gesture recognition processing and provides a serial RS-232 interface for connection to a range of computer platforms. Its introductory price is $3,500. 5.1.2.3 Dextrous HandMaster

The Dextrous HandMaster is available from EXOS, Inc. It is a exoskeleton device that uses Hall Effect sensors to provide measurement of the joint flexion for four fingers and thumb. It can be used for providing motion commands in a VE or teleoperation envi102

ronment, or for recording finger motion. An interface to AT-compatible machines is provided. Further details are given in Figure 54. This product is made to order and price information is not available.

Specification DOF 4 fingers 16 Thumb 4 Joint Motions 120° Sensor Resolution 0.1° Sampling Rate 20 channels at up to 75 Hz, up to 1500 Hz for single channel Weight <15 oz Interface RS-422

Photo courtesy of EXOS, Inc.

Figure 54. Dextrous HandMaster

5.1.2.4

Pinch Glove

The most recently announced glove interface is the Pinch glove system introduced by Fakespace, Inc., based on a prototype developed by researchers at the University of Central Florida, Institute for Simulation and Training. Unlike the other glove interfaces discussed here, the Pinch glove system does not measure finger joint angles. Instead, gloves are worn on both hands and contact between any two or more fingers, or thumbs, completes a conductive path, allowing the definition of a variety of “pinch” gestures that an application developer can map actions against. Over 1,000 gestures are theoretically possible. The gloves are constructed of a stretchable fabric and contain an electrical sensor in each fingertip. Each glove has a back-of-hand mount to accommodate a spatial tracker. The user’s point of interaction in the VE is represented by a 3-D cursor. Further details are given in Figure 55. The interface system, called the Pinch Hand Gesture system, consists of gloves for the left and right hand, electronics interface, and controlling software for either PCs or Silicon Graphics workstations. The system is supported by Sense8’s WorldToolKit and support for other VE toolkits is under development. The price of the system is $2000. Additional, single gloves can be purchased at $100 each. 5.1.2.5 Position Exoskeleton ArmMaster

The EXOS, Inc. Position Exoskeleton ArmMaster (EAM I) is a pre-cursor of the Force Exoskeleton ArmMaster (see Section 6.2.2.2). It provides 5 DOF passive sensing of the upper and lower arm through a shoulder mechanism that provides 3 DOF sensing and an elbow mechanism that provides 2 DOF. These mechanisms are modular and are avail103

Specification Sampling Rate < 1 msec Calibration Not needed Glove Sizes 1 size Glove Weight 0.7 oz Interface RS-232

Photo courtesy of Fakespace, Inc.

Figure 55. Pinch Glove

able separately. The sensors employed in the device are precision conductive plastic potentiometers. Further details are given in Figure 56. This device is made to order and price information is not available.

Specification DOF Shoulder 3 Elbow 1 Forearm Sup/Pro 1 Joint Motions Shoulder flexion/extension Shoulder ab/adduction Shoulder int/ext rotation Elbow flexion/extension Forearm supination/pronation Sensor Resolution 0.1° (joint computation ~1.5°) Weight <1 lb on arm

120° 120° 90° 130° 150°

Photo courtesy of EXOS, Inc.

Figure 56. Position Exoskeleton ArmMaster

5.1.2.6

TCAS DATAWEAR

T.C.A.S. Effects Ltd. market a range of body tracking systems based on their patented conductive elastomer sensor technology. These systems are based on a washable body suit that is available in five sizes and consists of a jacket (with or without gloves) and pants that can be attached by three positional zippers and stud fastenings. Both a Lycra suit intended for use as an undergarment and a neoprene modular over-suit are available. The glove itself has standard configurations of 8, 11, or 16 sensors. The eight sensor version measures joint angles for the PIP finger joints, the two thumb joints, and palm movement. The 11 sensor glove adds measurement of the MCP finger joints and wrist flexation, abduction, and adduction movements. Finally, the 16 sensor version measures angles for all finger 104

and thumb joints, and wrist movements. Additional sensors are attached to the body suit according to the joints to be monitored. Another product, a rigid face mask, provides for monitoring facial expressions, including lip movements, using 7 to 12 sensors. The number of sensors that can be supported simultaneously is limited by the control unit, which currently provides up to 32 channels. An increase to 64 channels is expected in 1996. To reduce tethering requirements, each set of 8 channels is packaged into a single standard 25 core cable. The tethering limits the operational range of the bodysuit and glove to 10 m2, although extension cables can be added as necessary. The software support for TCAS DATAWEAR provides for screen display of collected data or saving the data to file. Interfaces to VE toolkits, such as WorldToolKit, and computer animation packages are being developed. The basic eight sensor glove is available for $7,000. Body suits are custom-developed, with a 32-sensor suit starting around $30,000. 5.1.3 Current Research and Development

By and large, there is little research on whole-hand and body tracking to discuss. Most of the ongoing research and development is regarded as highly proprietary and little information is publicly available. One effort for which detailed information could not be acquired, though not necessarily for reasons of privacy, was John Fairley’s development of a body suit. This work began at the University of Illinois at Urbana-Champaign, Advanced Digital Systems Laboratory, with the development of a bodysuit that uses fiber-optic cables to measure the bend or rotation of joints. The data is transmitted to a PC where user movements are modeled by a 3-D mannequin. In subsequent independent R&D, this bodysuit is being refined so that it requires less calibration, monitors additional joints, and is cheaper to produce. 5.1.3.1 Armstrong Laboratory

Researchers at the Virtual Environment Interface Laboratory (VEIL) at Armstrong Laboratory, Crew Systems Directorate, see VE technology as providing a flexible, multimodal medium that can support a broad range of adaptive interface concepts. Their work has two broad goals. One is to develop multi-modal, adaptive user interface concepts within a cooperative agent framework. Here the researchers are concerned with the cognitive design of interfaces that can be used to provide novel interaction channels for VEs. The other goal is to establish a quantitative relation between human performance (sensory, perceptual, psychomotor, and cognitive) in VEs and hardware, software, and model world properties of VE systems. The objective is to produce design trade-off nomographs that quantify the relation between various aspects of user performance and VE system factors that design engineers can use in the development of specific applications. To date, the majority of the work has focused on the second goal. Led by Dr. Robert Eggleston, the researchers at VEIL have already conducted several experiments. The objec105

tive of one of the first experiments was to determine whether a person using a virtual hand controller could achieve performance comparable to that obtainable with a physical control device. The virtual hand controller used was a standard issue Nomex flyers’ glove with attached Ascension Bird tracker, and the physical control device was a spring-loaded, return-to-center isotonic joystick. A single-axis control task was used as the experimental task, specifically, the Critical-Instability Tracking Task. The VE was provided by a singleseat cockpit simulator. The major finding of the experiment was that comparable performance can be obtained from the two device types (Eggleston, 1993). Subsequent experiments have looked at the impact of VE system delays on tracking performance, the difference between device types in recovering from tracking errors, and impact of the biomechanics of virtual controllers versus those of physical devices in catering for a range of body sizes. Information on the results of these other experiments is not publicly available at the present time. Currently, the researchers are engaged in a series of experiments that are designed to investigate different types of control movements using virtual and physical controllers. Other on-going work is concerned with identifying those particular characteristics of multimodal VE interfaces that enable performance similar to that obtainable in the real world. Since earlier work has indicated that human perceptual systems are not engaged in a VE in the same way that they are in the real world, this area of work is likely to include basic investigations into perception issues. Another area of current work is concerned with sensing whole-body movement. Here the researchers are exploiting the new techniques developed at MIT for electric field sensing. The general principle is that when capacitive coupling is established between a human actor and a generated electric field, because the coupling profile changes as a function of the location of the body parts, this profile can be used to track the movement of those parts. Researchers have developed a basic capability for this type of tracking and are currently engaged in further studies of the basic technology and its use. Work supporting the development of design trade-off nomographs will continue. Starting in 1996, however, increased attention will be paid to multi-modal, adaptive user interface concepts. 5.1.3.2 Georgia Institute of Technology

In a study completed in 1994, researchers at Georgia Institute of Technology, Graphics, Visualization and Usability Center, performed an experimental evaluation of Virtual Technologies’ CyberGlove, Model CG1801. The objective of this work was to determine the glove sensor characteristics and, on this basis, determine its suitability for personindependent gesture recognition. The three-part experiment investigated the level of sensitivity of the glove sensors, the performance in recognizing angles, and factors that affect accuracy of angle recognition. Sixteen subjects were selected who provided a range of hand sizes. As reported by Kessler, Hodges, and Walker (1994), the measurement range for each

106

joint varied considerably. For example, the average range for the subjects’ MCP joints varied from 99.5° to 104.6° across the four fingers, and the average range for the PIP joints varied from 96.8° to 118.2° across fingers; this data was collected prior to any glove calibration. After simple calibration that measured two extreme values of flexion for each finger joint to allow for a translation from discrete values to joint angles, collected data showed that the angle calculated did represent the actual angle of the joint. Given a set of possible angles (that are related to particular gestures), the reported angle was correctly classified 90% of the time, with some notable exceptions (thumb joints, abduction joints, and high values of joint flexion). Additional calibration based on data already collected for an individual, or a group’s data, served to increase angle recognition, although problems remained in recognizing finger abduction angles, and accuracy decreased as angles increased. Repeatability of the recognition was found to be dependent both on the joint and angle being measured. Device noise and hand size were not significant factors in recognition accuracy. Based on this evaluation, the researchers concluded that current technology supports only a limited gesture recognition ability. Future work may include an investigation of the effect that a glove-based interface has on the sense of presence experienced by VE users. 5.2 3-D Pointing Input

3-D input via a mouse-like joystick or trackball device is a dominant form of user input in both immersive and non-immersive VEs. Such devices are used as a means of navigation, object selection, and, in some cases, object manipulation. They provide no feedback to the user, other than that which is inherent in the device’s physical construction. They use acoustic, electromagnetic, inertial, mechanical, or optical transducers to convert a physical phenomenon, force or velocity, into a measurable signal. 5.2.1 Commercially Available Devices

There are fewer of these devices on the market than one might expect, and what products there are differ quite widely in physical form and functional capabilities. Six devices are reported below and their characteristics are summarized in Table 10. Information on additional products from Global Devices was not available. Two more devices that are essentially elements, or special cases, of more general tracking systems are discussed in Section 3.1; these are the Logitech 3D Mouse and the RPI Mouse-Sensor3D. 5.2.1.1 CyberWand

Specifically designed as a navigational device for VE applications, InWorld VR Inc.’s CyberWand is a handheld joystick. In its base format, the CyberWand provides 2 DOFs via a hat-shaped device that controls either left/right or forward/back movement, one of the four programmable buttons is used to specify which axis is being controlled. In general, the buttons can be used to achieve movement in 6 DOFs by specifying how movement of the hat sensor should be interpreted. Source code is provided for the CyberWand, allow107

Table 10. Characteristics of Commercially Available 3-D Point Input Devices
Product CyberWand Immersion Probe-MD Magellan 3D Controller RingMouse Vendor InWorld VR, Inc. Immersion Corporation Logitech, Inc. Kantek, Inc. Input 2-Da 3-D 2- or 3-D 3-D Device Type Handheld joystick (4 buttons) Desktop boom-based stylus Desk-based mouse (9 buttons) Finger ring (2 buttons) Software Support WorldToolKit Interface drivers AutoCAD Windows 3.1 and 95, DOS interface WorldToolKit, VREAM, Superscape WorldToolKit, AutoCAD, CADKey, Microstation Price $60-104 $1,995 $550 $120

Spaceball 2003

Spacetec IMC Corporation Spacetec IMC Corporation

3-D

Desk-based mouse (8 buttons) Desk-based mouse (2 buttons)

$1,195

Space Controller

3-D

$595

a. 3-D input available by attaching a Polhemus or Ascension tracking sensor.

ing users to develop their own device driver, although special drivers for Sense8’s WorldToolKit are included. In addition, a demonstration program is provided, example programs for WorldToolKit, and a calibration program for a ThrustMaster dual-port ACM card. With the ThrustMaster ACM card, the CyberWand is available for $99, by itself it costs $60. Specification information for this device is not available. An alternate version of the CyberWand provides special provisions for the attachment of a Polhemus or Ascension tracking sensor. The attachment of such a sensor provides true 6 DOF control for the CyberWand, leaving the four buttons open for other uses. In this case, the hat-sensor can be used to achieve large movements that are beyond the reach of the handheld CyberWand. This version of the CyberWand costs $104. 5.2.1.2 Immersion PROBE-MD

Immersion Corporation has recently released the Immersion PROBE-MD, a 3-D input device intended for use with VE, CAD, telerobotic, medical imaging, and graphics applications. The PROBE-MD is a mechanical arm controller with a stylus tip that can be freely moved in 6 DOFs, further details are provided in Figure 57. The price of PROBEMD is $1,995. A Developer’s Programming Library (in the C programing language) is available for PC, Mac, SGI, or Unix platforms for an additional $175. The probe’s interface is capable of supporting an additional input device. A digital foot pedal (standard or heavy duty), digital hand switch, or analog foot pedal are available for use with this spare channel.

108

Specification Sampling Rate 1000 Hz Spatial Resolution 0.025 in Angular Resolution 0.25° Maximum Reach 44 in Latency 1 ms Footprint 8 x 6.5 in (triangle) Interface RS-232

Photo courtesy of Immersion Corporation

Figure 57. Immersion PROBE-MD

5.2.1.3

Magellan 3D Controller and Space Controller

Logitech, Inc. is phasing out their CyberMan joystick, and has introduced the Magellan 3D Controller to replace it. An early version of this controller was used onboard the space shuttle Columbia for space-based telerobotics applications. The Magellan 3D Controller provides a spring-mounted puck for controlling movement and absolute measurement is achieved by a patented linear optical measuring system. The device can be used in either 2-D or 3-D mode. For 3-D application, the controller is operated by the user to position an object, while working on that object with an ordinary mouse. The controller provides nine user-programmable buttons, some of which can be used to adjust sensitivity and motion control (for example, only reporting the coordinate with the greatest magnitude, or only rotation coordinates). Device drivers are available for PC, IBM, Sun, HP and Silicon Graphics platforms. The PC version is additionally supported with an AutoCAD driver and demonstration software. Further details are given in Figure 58. The Magellan 3D Controller is available for $550. 5.2.1.4 RingMouse

Kantek, Inc. recently announced a new type of 3-D mouse, called the RingMouse. As its name suggests, this device takes the form of a ring worn on the user’s finger. The ring itself consists of an ultrasonic transmitter, held in position by a velcro strap. This transmitter sends both an infrared and ultrasonic pulse to a receiver that is mounted on top of a monitor, the delay between receiving these signals is used in determining the ring position. The system has a 3 ft tracking area and is wireless. The ring is powered by a long-life watch battery and an automatic sleep mode switches the power off if the ring is unused for a minute or so, until one of the buttons is pressed. The ring itself has two programmable buttons, intended to be pressed using the thumb. Since the RingMouse receiver plugs into the

109

Specification Cycle Time 60 ms Cap Moving Range 65 mm Translational Deflection ±1.5 mm Rotational Deflection ±4° Resolution 8 bits Dimensions 165 x 40 x 112 mm Weight 670 g Interface RS-232

Photo courtesy of Logitech, Inc.

Figure 58. Magellan 3D Controller

serial port, this device can be used in conjunction with a traditional mouse. Further details for the RingMouse are given in Figure 59.

Specification Sampling Rate 50 updates/sec Resolution 100 dpi Range Up to 3 ft Weight < 1/2 oz Interface RS-232

Photo courtesy of Kantek, Inc.

Figure 59. RingMouse

The software support for RingMouse includes an emulator for a traditional mouse. There is also a joystick emulator for use with DOOM and other DOS-based games, and special interfaces for the Dark Forces and Descent games. A 3-D hoop toss game is provided with the RingMouse. The price for RingMouse is approximately $120. 5.2.1.5 Spaceball 2003 and Space Controller

Spacetec IMC Corporation developed and market the Spaceball 2003 controller that is designed to support interactive control of 3-D models and VE navigation in simultaneous 6 DOFs. A patented sensing technology is embodied in a PowerSensor ball that can be pulled, pushed, or twisted using the fingertips to control movement. Eight buttons provide motion control filters, performing functions such as switching rotations on/off, adjusting sensitivity, and view reset. Further details are given in Figure 60. The price of Spaceball 2003 is $1,195. 110

The proprietary SpaceWare IMC interface software supports interfaces with many CAD and computer-aided manufacturing applications. Spaceball 2003 is also supported in a number of VE world building packages, including WorldToolKit, VREAM, and Superscape. A Software Developer’s Kit is available to provide the source code and information needed to integrate Spaceball support into custom applications.

Specification Max/Min Detectable Force ~4.5/0.1 lb Spatial Resolution 10 bits Max/Min. Detectable Torque ~5.3/0.1 lb/in Dimensions 9.1 x 3.5 x 4.9 in Weight 1.75 lb Interface RS-232, 9 pin

Photo courtesy of Spacetec IMC Corporation

Figure 60. Spaceball 2003

Specification Max/Min Detectable Force 0.1/4.6 lb Max/Min. Detectable Torque 0.1/5.3 lb/in Resolution 10 bits Dimensions 7.6 x 3 x 4.4 in Weight 1.25 lb Interface RS-232, 9 pin

Figure 61. SpaceController

A more recent product, SpaceController, provides a lower cost version of the Spaceball 2003. Here only two buttons are provided and these control pop-up menus that give access to the functionality of the Spaceball 2003 buttons. Further details for the SpaceController are given in Figure 61. Its price is $595.

An additional Spacetec IMC Corporation 3-D input product, the Spaceball Avenger, is intended only for use in PC video games and not discussed further. 5.2.2 Current R&D

Here, again, there are few research and development efforts to report. In part, this can be attributed to the fact that there are no major outstanding technical issues in the construction and operation of this type of 3-D input devices. However, data on the utility of the particular devices for specific types of manipulations is still needed. More generally, as the example of the work underway at the University of Toronto shows, there are many human factors and cognitive engineering concerns that remain to be addressed. 5.2.2.1 Digital Image Design Inc.

Digital Image Design Inc. is designing a 3-D input device that will be the successor to their Cricket device, shown in Figure 62, that is no longer under manufacture.

111

Photo courtesy of Digital Image Design, Inc.

Figure 62. Cricket

While still in early design stages, the new device is expected to be functionally similar to the Cricket. That is, it will be a device specifically designed to support VE applications that require substantial free-space manipulation, probably providing buttons to support an occasional need for 2-D operations (such as menu manipulation), object picking, and object grabbing. Current work is focusing on ergonomic issues, particularly in the areas of reducing the stress placed on a user’s hand. No dates for the expected release of the new product are yet available.

5.2.2.2

University of Toronto

For the past several years, researchers at the University of Toronto, Department of Industrial Engineering, have been investigating human factors issues concerned with 6 DOF input techniques for the manipulation of objects in 3-D environments. This work is being led by Dr. Shumin Zhai and Dr. Paul Milgram. The overall goal of the work is to determine critical factors for the design of 6 DOF input devices and their impact on human manipulation performance. A central aspect of the work has been the development of a model for classifying 6 DOF input devices along the human factors dimensions of mapping (position versus rate control), sensing mode (ranging from isotonic, through elastic, to isometric), and degree of integration (based on number of discrete controls to be manipulated). This model has served as the framework for a series of experimental studies. In the first experiment, the researchers compared isotonic-position, isotonic-rate, isometric-rate, and isometric-position control approaches (Zhai, 1993). A glove was used for the first two approaches and a spaceball for the third and fourth. In an experimental 6 DOF docking task presented via a non-immersive VE system, using eight subjects, the researchers found a strong interaction between sensing mode and mapping, and that isotonic-position and isometric-rate approaches gave the best performance. While the first experiment showed that the isotonic-position device was more direct, it was tiring to use. The isometric rate device was less fatiguing but provided little kinesthetic feedback to the user. Consequently, the objective of the second experiment was to assess an intermediate approach by comparing elastic-rate and isometric-rate control approaches. (The elastic-rate device developed by the researchers is called the EGG, further details about this device are given in Figure 63 below.) The same experimental task was used, this time with twenty-six subjects. Significant differences in performance were found only during early stages of learning with the devices, with the EGG outperforming the spaceball. The third experiment was designed to see whether larger differences would result 112

with a more challenging experimental task. Again using the EGG and spaceball, twenty six subjects were asked to perform a pursuit tracking task. As in the previous experiment, subjects’ performance with each device improved substantially with practice. Zhai and Milgram (1993) report that the elastic-rate EGG produced better scores, especially in the early learning stage.

Sampling Rate Movement Area Footprint Latency Interface

Specification <100 Hz ±20 mm translation, ±30° rotation 17 x 17 x 17 cm 50 ms RS-232

Figure 63. EGG

The data collected in the third experiment were also analyzed to determine the subjects’ spatial accuracy in the x, y, and z directions. The mean tracking error in each direction was found to decrease over time, but the mean error in the z direction was significantly greater than in the x direction through all experimental phases. The mean error in the y direction was comparable to that in the z for early training and subsequently greatly decreased to the level of the mean error in the x direction. Overall the mean error in the z direction was 20% and 40% greater than that in the y and x directions, respectively. These findings were the same for both types of devices. The researchers hypothesize that a reason for the superior performance in tracking in the x direction than the y direction might be a higher attentional resource priority for horizontal movement. A fourth experiment studied the issue of which joints and muscle groups should be used for 6 DOF manipulation. For this experiment, two isotonic-position control techniques were tested in a 6 DOF docking task. One technique utilized the user’s wrist, elbow, and shoulder, while the other technique additionally made use of the user’s fingers. The results showed that the participation of fingers significantly improved the task performance. In a final experiment to be reported, issues concerning the visual representation formats of users’ input control actions in relation to a target object were investigated. In a 3D dynamic target acquisition task, it was found that both binocular displays and partial occlusion through semi-transparency, a novel graphic technique, were beneficial. In particular, the use of semi-transparent surfaces appeared to enhance human performance in discrete tasks more than the classical stereoscopic viewing technique.

113

Currently, these researchers are investigating human behavior in coordinating hand movements in 3-D environments. This work includes developing new measures for correlations and determining how human coordination relates to different interface designs. 5.3 Summary and Expectations

In the case of glove-based devices, this is an area of current growth. All the glovebased devices on the market are relatively new products and several additional products are expected to be released in the near future. The motivation behind these devices is to allow users to manipulate a VE in a similar manner to that in which they would manipulate a real environment. The current set of commercially available products do provide this capability but in a limited manner. The limitations arise chiefly from the lack of sensory feedback to the user’s hand and the inability for fine discrimination between gestures. With respect to this last issue, technical improvements can be expected to occur in the next few years, but it seems likely that significant improvements in gesture recognition are more likely to result from a context-based approach for gesture recognition, a topic that does not seem to be receiving attention. Even so, the common use of gloves as a primary VE interface device is expected to continue. The alternative to glove-based devices, exoskeleton devices, are expensive and encumbering. While some products are on the market, these are built to order and intended to be tailored to particular applications where precise joint measurement is required. In the case of VEs, these applications will be the exception rather than the rule and it is highly unlikely that exoskeleton devices will come into widespread use. Other uses of these devices include various specialized medical applications, these also are unlikely to provide a large market demand. There are no particular technical challenges in the design and manufacture of 3-D input devices and there are several general-purpose devices available as commercial products. Are more devices needed, that is, do current products provide the necessary functionality and quality? There is no evidence that, in general, user needs are not being met. Consequently, while new devices may become available in the next few years, either as totally new offerings or replacements for existing products, there is no reason to expect that the overall situation will change and large numbers of 3-D input devices will appear. An area that is receiving some, but not enough attention, is consideration of the human factors issues in the use of these different types of devices. This concerns more than ergonomic design issues. Basic questions pertaining to the usability and appropriateness of different device types need to be answered. Such questions should include consideration of both the application and the characteristics of the other types of display that are being used, primarily visual and auditory displays. They should consider not only the requirements a device places on motor skills, but also any cognitive burden that takes human processing resources away from the primary application task. While there is not a lot of evidence to suggest that these issues will receive in-depth investigation, hopefully the next few years

114

will see more work in this area. Meanwhile, there is a lack of data on the comparative capabilities and usability of current devices that can help users in selecting one over another.

115

116