Following is the seminar report on “Harnessing Technology: An exploration of Haptic Interactivity”. It specially focuses on Haptics.
Abstract
Haptics is a recent enhancement to virtual environments allowing users to touch and feel the simulated objects they interact with. Current commercial products allow tactile feedback through desktop interfaces (such as the FeelIt mouse(tm) or the PHANToM arm(tm)) and dextrous tactile and force feedback at the fingertips through haptic gloves (such as the CyberTouch(tm) and the CyberGrasp(tm)).Haptics VR programming requires good physical modeling of user interactions, primarily through collision detection, and of object response, such as surface deformation, hard-contact simulation, slippage, etc. It is at present difficult to simulate complex virtual environments that have a realistic behavior.
This task is added by the recent introduction of haptics toolkits (such as Ghost(tm) or VPS(tm)).Current technology suffers from a number of limitations, which go beyond the higher product cost of haptic interfaces. These technical drawbacks include the limited workspace of desktop interfaces, the large weight of force feedback gloves, the lack of force feedback to the body, safety concerns, etc. Not to be neglected is the high bandwidth requirement of haptics, which is not met by current Internet technology. As a result, it is not possible at present to have a large number of remote participants interacting hapticly in a shared virtual space. in a haptics application, a user basically uses a haptics hardware device, such as a joystick, scissor, robotic arm, etc, to manipulate virtual objects such as poking solid shapes (squares, circles), cutting virtual animal tissue, and lifting and throwing objects which appear on the computer screen. While manipulating these virtual objects, a user receives force feedback from the hardware device, which creates a sensation of force or resistance to the user’s hand, making her feel as if she is touching and feeling “real” objects versus “virtual objects”.
Introduction
Haptics refers to sensing and manipulation through touch. Haptics (pronounced HAP-tiks) is the science of applying touch (tactile) sensation and control to interaction with computer applications. (The word derives from the Greek haptein meaning “to fasten.”) By using special input/output devices (joysticks, data gloves, or other devices), users can receive feedback from computer applications in the form of felt sensations in the hand or other parts of the body. In combination with a visual display, haptics technology can be used to train people for tasks requiring hand-eye coordination, such as surgery and space ship maneuvers. It can also be used for games in which you feel as well as see your interactions with images.
What is Haptics?
Haptics (pronounced HAP-tiks) is the scientific field that studies the sense of touch. In computing, Haptics is the science and art of applying touch sensation to human interaction with computers. A haptic device gives people a sense of touch with computer generated environments, so that when virtual objects are touched, they seem real and tangible. The word ‘haptics” derives from the Greek haptikos, from haptesthai, meaning “to grasp, touch, or perceive”, equiv. to hap(tein) to grasp, sense, perceive
Haptics involves the modality of touch and the sensation of shape and texture an observer feels when exploring a virtual object, such as a three-dimensional model of a porcelain vase from a museum, a tactile map, or a graphic designer’s rendering of an imaginary object. At IMSC we use haptic devices that provide force feedback to the user’s fingers, simulating the sensory experiences that the actual, physical object would generate if the user were to touch it. The image on the left shows a researcher calibrating the CyberGrasp, a whole-hand force-feedback glove that can be used to grasp virtual objects. A network of “tendons” transmits grasp forces back to the user’s fingers. A visitor to the Haptic Museum can use these devices locally at a desktop or remotely over the Internet.
Users with the same and even different haptic devices, such as a glove and a stylus, can not only manipulate and explore the same virtual objects over the Internet, but can also make realistic touch contact with one another.
Someone from a museum curatorial staff can interact with a student in a remote classroom and together they can jointly examine an ancient pot or bronze figure, haptics can be used to allow museum visitors to explore objects in ways that cannot be permitted in physical museums due to concerns about breakage and deterioration of the object surface.
Haptic Interactivity
Haptic devices allow users to feel virtual objects, describe the technology thus: “Force display technology works by using mechanical actuators to apply forces to the user. By simulating the physics of the user’s virtual world, we can compute these forces in real-time, and then send them to the actuators so that the user feels them”. Really means is a person using a haptic device can feel a simulation of a solid object.
There are several alternative devices available. The Wingman force-feedback mouse from Logitech is a simpler alternative to the PHANToM. It only provides 2 DOF (x and y dimensions, like a normal desktop mouse) but is much smaller and can be used as a replacement to a standard PC mouse.
History
One of the earliest forms of haptic devices is used in large modern aircraft that use servo systems to operate control systems. Such systems tend to be “one-way” in that forces applied aerodynamically to the control surfaces are not perceived at the controls, with the missing normal forces simulated with springs and weights. In earlier lighter aircraft without servo systems, as the aircraft approaches a stall the aerodynamic buffeting is felt in the pilot’s controls, a useful warning to the pilot of a dangerous flight condition. To replace this missing cue the angle of attack is measured and when it approaches the critical stall point a “stick shaker” (an unbalanced rotating mass) is engaged, simulating the effects of a simpler control system. This is an example of haptic feedback.
Haptic Perception
Haptics is a general term relating to the sense of touch. This is very broad and there are many component parts to the global sense of touch. Not all of these parts are well understood as there has been much less psychological research into touch than into the senses of hearing or vision.The word ‘haptic’ has grown in popularity with the advent of touch in computing. The human haptic system consists of the entire sensory, motor and cognitive components of the bodybrain system. It is therefore closest to the understood meaning of proprioceptive. Under this umbrella term, however, fall several significant distinctions. Most important of these is the division between cutaneous and kinesthetic information. The distinction becomes important however when we attempt to describe the technology. In brief, a haptic device provides position input like a mouse but also stimulates the sense of touch by applying output to the user in the form of forces.
System Design
Marrying haptics and holovideo permits us to render simple dynamic scenes in the user’s manipulatory space, the domain of real objects. Two separate modules comprise the computation which feeds the displays; a haptics module that performs force modeling, and the holovideo module whichpre-computes holograms and drives rapid local holographic display updates based on changes to the model. The haptics and hologram modules are organized by the Workspace Resource Manager (WRM) which is notified of geometry changes imparted to the spinning cylinder by the user’s hand, and requests hologram updates to local regions of the visual display where changes have occurred. The haptics and hologram modules rely upon separate and characteristically different representations of the cylinder, which are carefully spatially and metrically registered. From the point of view of the user, who is holding the stylus and pressing it into the holographic image, a single multimodal representation
Haptic Happenstance
Most interesting for health care (and probably overall) are machine and computer haptics, which have two main sub-components:
- Touching and feeling virtual objects
- Tele-manipulation: the ability to manipulate virtual and real objects by touch, including at a distance over a network.
The Technology Behind the Touch
The most mature type of machine haptics uses a single-point, force-feedback “stick”, which serves as a good proxy for many tools. Overall, haptics technology depends on two major types of technology and will evolve with them:
- Computational speed: Haptically displaying an object requires a motor to turn on and resist the user as he or she touches the object. This “force feedback” has to be within milliseconds, so this requires very fast computer calculations.
- Interfaces: Haptic interfaces are the equivalent for the sense of touch to video monitors for the sense of sight—they are the computer- and machine-generated way users perceive the world of touch. Another condition is to have haptic interfaces that have minimal impact on the user’s freedom of motion. Ideally, the user should be able to walk freely in a large working volume and have haptic feedback at any location in this volume. One haptic interface that gives the user more freedom of motion is the CyberPack® produced by Virtual, the CyberForce attaches to the back of the palm and produces both translating forces and as such the user can feel Object mechanical compliance (produced by the haptic glove), and the object weight and inertia (produced by the CyberForce arm).
The system is aimed at desktop usage, like CAD/CAM design or service training tasks, thus it is not applicable to large volume haptics. There are other force feedback arms that can be attached at the user’s wrist, such as the PHANToM Premium, or the Sarcos Master. However, these arms need to be grounded on the wall, or floor, which restricts the user’s freedom of motion to the workspace of the haptic arm.
Haptics Uptick: Where is Haptics Now?
There already is widespread use of haptic technologies interfacing to machines. So long as the device is tethered to sufficient electrical and computing power, sophisticated functions are already fairly mature.
- Gaming technology: The average gaming controller (for Playstation and Xbox as well as arcade games) already has joysticks or steering wheels that vibrate when you’ve run your “car” off the smooth road, or crashed your fighter jet.
- Cars: Given the increased complexity of modern cars and the increased distractions from the amount of (non-driving) stuff in them, many automakers are working on a haptic control interface for the peripherals like the stereo, heating, navigation system, etc. For the past two model years, the BMW 7 series has contained the iDrive (based on Immersion Corp’s technology), which uses a small wheel on the console to give haptic feedback so the driver can control those peripherals through menus on a video screen. Expect less awkward versions to show up in cheaper cars in three to five years.
- Going mobile: Later this year, Samsung will introduce cell phones that will vibrate in different ways depending on who’s calling—the vibrational equivalent of downloadable ring tones. This is an early precursor of more sophisticated information delivered haptically on mobile devices, but its spread will be dependent on miniaturization, battery life (as it does suck up the juice) and cost.
Representative Applications of Haptics
Ø Surgical Simulation and Medical Training
A primary application area for haptics has been in surgical simulation and medical training. Haptic device in a training simulation for palpation of subsurface liver tumors. They modeled tumors as comparatively harder spheres within larger and softer spheres. Realistic reaction forces were returned to the user as the virtual hand encountered the “tumors,” and the graphical display showed corresponding tissue deformation produced by the palpation. a PHANToM with four tips that mimic dental instruments; they can be used to explore simulated materials like hard tooth enamel or dentin. PHANToMs in a training simulation in which residents passed simulated needles through blood vessels, allowing them to collect baseline data on the surgical skill of new trainees, medical applications. subjects in a telementoring session did not profit from the addition of force feedback to remote ultrasound diagnosis.
Ø Museum Display
A few museums are exploring methods for 3D digitization of priceless artifacts and objects from their sculpture and decorative arts collections, making the images available via CD-ROM or in-house kiosks. Few museums have yet explored the potential of haptics to allow visitors access to three-dimensional museum objects such as sculpture, bronzes, or examples from the decorative arts. Haptic interfaces can allow fuller appreciation of three-dimensional objects Haptics raises the prospect of offering museum visitors not only the opportunity to examine and manipulate digitized three-dimensional art objects visually.
Ø Painting, Sculpting, and CAD
There have been a few projects in which haptic displays are used as alternative input devices for painting, sculpting, and computer-assisted design (CAD). PHANToM can be used by the visually impaired; line thickness varies with the user’s force on the fingertip thimble and colors are discriminated by their tactual profile.
Ø Visualization
Haptics has also been incorporated into scientific viualization. Both haptics and graphics displays are directed by the movement of the PHANToM stylus through haptically rendered data volumes.
Ø Military Applications
Haptics has also been used in aerospace and military training and simulations. There are a number of circumstances in a military context in which haptics can provide a useful substitute information source; that is, there are circumstances in which the modality of touch could convey information that for one reason or another is not available, not reliably communicated, nor even best apprehended through the modalities of sound and vision.
Ø Interaction Techniques
An obvious application of haptics is to the user interface, in particular its repertoire of interaction techniques, loosely considered that set of procedures by which basic tasks, such as opening and closing windows, scrolling, and selecting from a menu, are performed (Kirkpatrick & Douglas, 1999). Indeed, interaction techniques have been a popular application area for 2D haptic mice like the Wingman and I-Feel, which work with the Windows interface to add force feedback to windows, scroll bars, and the likesystem.
Ø Assistive Technology for the Blind and Visually Impaired
Most haptic systems still rely heavily on a combined visual/haptic interface. This dual modality is very forgiving in terms of the quality of the haptic rendering. This is because ordinarily the user is able to see the object being touched and naturally persuades herself that the force feedback coming from the haptic device closely matches the visual input. However, in most current haptic interfaces, the quality of haptic rendering is actually poor and, if the user closes her eyes, she will only be able to distinguish between very simple shapes (such as balls, cubes, etc.). Today there has been a modest amount of work on the use of machine haptics for the blind and visually impaired.
Ø Space application
A force-feedback system could be used for rehabilitation purposes in space applications, as it can simulate gravitational forces, thus preventing the bone and muscle loss that are the result of extended operations in space.
Given below are several more potential applications:
- Medicine: manipulating micro and macro robots for minimally invasive surgery; remote diagnosis for telemedicine; aids for the disabled such as haptic interfaces for the blind.
- Entertainment: video games and simulators that enable the user to feel and manipulate virtual solids, fluids, tools, and avatars.
- Education: giving students the feel of phenomena at nano, macro, or astronomical scales; “what if” scenarios for non-terrestrial physics; experiencing complex data sets.
- Industry: integration of haptics into CAD systems such that a designer can freely manipulate the mechanical components of an assembly in an immersive environment.
- Graphic Arts: virtual art exhibits, concert rooms, and museums in which the user can login remotely to play the musical instruments, and to touch and feel the haptic attributes of the displays; individual or co-operative virtual sculpturing across the internet.
New Dimension in Haptics
One of the primary techniques for the interpretation of scientific information is visualization. Ordinarily, a student uses his or her eyes to receive data and appropriate that data into a cognitive network of related information. This process requires skills such as pattern recognition, data filtering, the detection of critical elements of the data, and, most importantly, predictive generalization.
On the other hand, data filtering in a haptic environment can be quite difficult and generally requires a high signal-to- noise ratio. While it is certain that haptic input is necessary for blind students to interpret complex graphical images, the methods for the creation of haptic environments are often complex and confusing. Basically, haptic interfaces can be created in two ways: “on-line” and “off-line”.
Potential Problems with Haptics Devices in Museums
One of the main problems with devices such as the PHANToM is cost. As they are so expensive it is impractical to use them on a large scale. As discussed above, prices are falling so this may not be a problem in the future. It is therefore important to investigate the use of such devices now. Another problem is reliability and robustness. Most of the devices are fairly reliable and robust because they have been built for games or other demanding environments.
Some Limitations of Current Harnessing Technology
Even the best haptic devices are limited in some respects. One of the main limitations is that all contact is through a single point (like a single finger or a probe). There are no whole hand devices that yet provide high fidelity force-feedback. This limits the range of applications that haptic devices are currently good for.
A further problem is that cutaneous feedback is very limited in most haptic devices as they stimulate the sense of touch by applying output to the user in the form of forces and movement. Subtle surface textures are normally perceived cutaneously as tiny deformations in the surface of the skin. This is very difficult to do mechanically and most haptic devices do not do it at all. This limits the range of surface textures that can be displayed.
Todays and Tomorrow’s Enabling Technologies
Today we can use a Pentium PC and a Phantom to display (or “render”) and interact with simple haptic scenes. We have recently crossed an important threshold in haptic simulation realness by capitalizing on the design insights embodied in the Phantom and the inexpensive compute power offered by today’s processors. The computer industry’s aggressive development of graphics accelerators gives us the extremely fast geometric computation capabilities needed for haptic rendering. Greater computation capabilities, combined with expected advances in motor technology, further understanding of haptic psychophysics, and sensible product design, will make haptic technology an important adjunct to the way we work with computers. The current haptic interface paradigm embodied in the Phantom focuses on the simplest of interactions –forces at the fingertip or tool tip. There are other aspects of human haptic perception — such as pressure distribution, temperature and high-frequency vibration that could enhance the quality of interaction.
