Just like computer graphics, holograms are being applied as tools to solve individual
research, engineering, and presentation problems within several domains. Up until
today, however, these tools have been applied separately. The overall goal of our
project is to combine both technologies to create a powerful tool for science, industry
and education. We are currently investigating the possibility of integrating computer
generated graphics and holograms.
Our goal is to combine the advantages of conventional holograms (i.e. extremely high
visual quality and realism, support for all depth queues and for multiple
observers at no computational cost, space efficiency, etc.) with the advantages of
today’s computer graphics capabilities (i.e. interactivity, real-time rendering,
simulation and animation, stereoscopic and autostereoscopic presentation,
etc.).
Several engineering and computer science topics will be addressed throughout the
project: The potentials of different hologram types with respect to the project’s goal
have to be investigated. New three-dimensional displays that combine computer
graphics and holography will be engineered. New real-time rendering algorithms,
registration methods, and human–computer interaction techniques that are adequate
for the proposed metaphor will be developed.
The outcome will be a three-dimensional display concept whose application is
envisioned in areas such as scientific visualization (e.g., paleontology, pathology,
density, medicine, biomedicine, orthopedics or archeology), industrial simulation
(e.g., design, manufacturing and quality assurance), and education (e.g., medical
training or public museums).
Here are some of our initial results.
Using digital light to reconstruct the holographic image
The two basic hologram types—transmission and reflection—are both reconstructed
by illuminating them with spatially coherent light (i.e. using a point-source of light).
These two types have generated a number of variations. Although some holograms
can be reconstructed only with laser light, others can be viewed under white
light.
Rainbow holograms, one of the most popular types of white-light transmission
hologram, diffract each wavelength of the light through a different angle. This
lets viewers observe the recorded scene from different horizontal viewing
positions but also makes the scene appear in different colors when observed
from different vertical points of view. In contrast to rainbow holograms,
white-light reflection holograms can provide full parallax and display the recorded
scene in a consistent color (monochrome or multi-color) for different viewing
positions.
Conventional video projectors represent point sources that are well suited for viewing
white-light reflection or transmission holograms. Today’s high-intensity discharge
lamps of projectors can produce a very bright light. The main advantage for using
video projectors is that the reference wave used to reconstruct the hologram can be
digitized. Thus it is possible to control the amplitude and wavelength of each discrete
portion of the wavefront over time.
Figure 1 shows the projected reference wave in different states, and the resulting
holographic image of a monochrome white-light reflection hologram. A uniform
reference wave reconstructs the entire hologram uniformly. Selectively emitting light
in different directions allows us to create an incomplete reference wave that
reconstructs the hologram only partially. Local amplitude variations in the reference
wave result in proportional amplitude variations in the reconstructed object wave.
Variations in wavelength do not lead to useful effects in most cases due to the
wavelength dependency of holograms. But this is still a matter for further
investigations.
Partially reconstructing object waves
It is possible to reconstruct the object wave of a hologram only partially, leaving gaps
where graphical elements can be inserted.
Both reflection holograms (without an opaque backing layer) and transmission
holograms remain transparent if not illuminated. Thus, they can serve as optical
combiners—leading to very compact displays.
Real-time computer graphics can be integrated into the hologram from one side,
while illuminating it partially from the other side [1]. Thereby, rendering and
illumination are view-dependent and have to be synchronized.
If autostereoscopic displays are used to render 3D graphics registered to the
hologram, both holographic and graphical content appear three-dimensional within
the same space. If depth information of both is known, correct occlusion effects
between hologram and graphics can be generated.
Figure 2 shows a rainbow hologram of a dinosaur skull combined with graphical
representations of soft tissue and bones. If the holographic plate is illuminated
with a uniform light, the entire hologram is reconstructed. If the plate is
illuminated only at the portions not occluded by graphical elements, the
synthetic objects can be integrated by displaying them on the screen behind the
plate.
Light interaction
The reconstructed object wave’s amplitude is proportional to the reference wave’s
intensity. In addition to using an incomplete reference wave for reconstructing a
fraction of the hologram, intensity variations of the projected light permit local
modification of the recorded object wave’s amplitude.
Practically, this means that to create the illumination image which is sent out by the
projector, graphical shading and shadowing techniques are used to reconstruct the
hologram instead of illuminating it with a uniform intensity. To do this, the real
shading effects on the captured scenery caused by the real light sources
used for illumination during hologram recording, as well as the physical
lighting effects caused by the video projector on the holographic plate, must
both be neutralized. Next, the influence of a synthetic illumination must be
simulated [1].
Using conventional graphics hardware, it becomes possible not only to create
consistent shading effects, but also to cast synthetic shadows correctly from all
holographic and graphical elements onto all other elements.
The figures show the same rainbow hologram as above with 3D graphical elements
and synthetic shading effects. Shadows are cast correctly from the hologram onto the
graphics and vice versa. A virtual point-source of light was first located at
the top-left corner, and then moved to the top-right corner, in front of the
display.
Proof-of-concept prototypes
The desktop prototypes shown in figure
4 consist entirely of off-the-shelf
components, including either an autostereoscopic lenticular-lens sheet display with an
integrated head-finder for wireless user tracking, or a conventional CRT screen
with active stereo glasses, wireless infrared tracking, and a touch screen for
interaction.
Both prototypes use digital light projectors. A single PC with a dual-output graphics
card renders the graphical content on the screen and illuminates the holographic
plate on the video projector.
In both cases, the screen additionally holds further front layers—glass protection,
holographic emulsion, and optional mirror beam splitter (used for transmission
holograms only).
Interaction with the graphical content is supported with a mouse or a transparent
touch-screen mounted in front of the holographic plate.
Experiments with a digital multiplex hologram
Digital holography uses holographic printers to expose the photosensitive emulsion
with computer generated or captured images.
This results in conventional holograms with digital content rather than real scenery.
Pre-processed 2D and 3D graphics or digital photographs and movies can be printed.
This allows, for instance, the holographic recording of completely synthetic objects,
real outdoor scenes, and objects in motion—which is difficult and sometimes
impossible to achieve with optical holography.
Like optical holograms, digital holograms can be multiplexed. This allows us
to divide the viewing space and to assign individual portions to different
contents.
The content for digital holograms can easily be created by non-experts,
and the printing process is inexpensive. Usually a 3D graphical scene, a
series of digital photographs or a short movie of a real object is sufficient for
producing digital holograms. However, these digital holograms lack in the
quality (resolution, color appearance, sharpness, etc.) of conventional optical
holograms.
Figure 5 shows a digital color white-light reflection hologram of a car headlight. It
was generated by taking 360 perspective photographs from different angles (in 0.5∘
steps to cover a 110∘ total viewing zone plus two 35∘ clipping areas). The perspective
photographs were multiplexed into different sub-zones (40∘ = 80 images for the front
view + 2 × 35∘ = 140 images for the side and rear views + 2 × 12.5∘ = 50 images to
fill the partially visible clipping area outside the 110∘ total viewing zone
+ 2 × 22.5∘ = 90 images to fill the invisible clipping area outside the 110∘ total
viewing zone).
Consequently, three different partial views (front, rear, and side) can be
observed by moving within the total viewing zone of 110∘. After registering
the holographic plane and calibrating the projector, interactive graphical
elements, such as wire-frame or shaded CAD data can be integrated into the
hologram.
Holographic windows
The ability to control the reconstruction of a hologram’s object wave allows
integrating them seamlessly into common desktop-window environments.
If the holographic emulsion that is mounted in front of a screen is not illuminated, it
remains transparent. In this case the entire screen content is visible and an
interaction with software applications on the desktop is possible in a familiar
way.
The holographic content (visible or not) is always located at a fixed spatial position
within the screen/desktop reference frame. An application that renders the graphical
content does not necessarily need to be displayed in full-screen mode (as in the
examples above), but can run in a ‘windows’ mode—covering an arbitrary area on
the desktop behind the emulsion.
If the position and the dimensions of the graphics window are known, the
projector-based illumination can be synchronized to bind the reference wave to the
portion of the emulsion that is located directly on top of the underlying window.
Thereby, all the techniques that are described above (partial reconstruction and
intensity variations) are constrained to the window’s boundaries. The remaining
portion of the desktop is not influenced by the illumination, the graphical or the
holographic content.
In addition, the graphical content can be rendered in such a way that it remains
registered with the holographic content—even if the graphical window is moved or
resized.
This simple, but effective technique allows a seamless integration of holograms into
common desktop environments. It allows us to temporarily minimize the
“holographic window” or to align it over the main focus while working other
applications.
Figure 6 shows a holographic window in different states on a desktop together with
other applications. It displays an optical (monochrome) white-light reflection
hologram of a dinosaur skull with integrated graphical 3D soft tissues. A stereoscopic
screen was used in this case, because autostereoscopic displays (such as lenticular
screens or barrier displays) do not yet allow an undisturbed view on a non-interlaced
2D content.
Outlook
Holograms can store a massive amount of information on a thin holographic
emulsion. This technology can record and reconstruct a 3D scene with almost no loss
in quality. Moore’s law—which asserts that computing power doubles every 18
months—must be applied many times for graphical or electro-holographic
rendering techniques and displays in order to reach this quality at interactive
frame rates. A combination of interactive computer graphics and high-quality
holograms represents an alternative that can be realized today with off-the-shelf
consumer hardware. We believe that this concept can be beneficial for many
applications.
Archaeologists, for example, already use holograms to archive and investigate ancient
artifacts. Scientists can use hologram copies to perform their research without having
access to the original artifacts or settling for inaccurate replicas. They can combine
these holograms with interactive computer graphics to integrate real-time simulation
data or perform experiments that require direct user interaction, such as packing
reconstructed soft tissue into a fossilized dinosaur skull hologram. In addition,
specialized interaction devices can simulate haptic feedback of holographic and
graphical content while scientists are performing these interactive tasks. An entire
collection of artifacts will fit into a single album of holographic recordings, while a
light-box-like display such as that used for viewing x-rays can be used for
visualization and interaction.
The same applies to the biomedical domain that already uses digital volumetric
holograms produced from CT or MRI data of inner organs.
In the automotive industry, for instance, complex computer models of cars and
components often lack realism or interactivity. Instead of attempting to achieve high
visual quality and interactive frame rates for the entire model, designers could
decompose the model into sets of interactive and static elements. The system
could record physical counterparts of static elements in a hologram with
maximum realism, and release computational resources to render the interactive
elements with a higher quality and increased frame rate. Multiplexing the
holographic content also lets users observe and interact with the entire model from
multiple perspectives. Beside display holograms, holographic interferograms
used for non-destructive measurement and testing are yet another example
of industrial applications. Analogue interferograms that indicate motion,
vibration, or deformations of objects can be combined with digital simulation
data.
Augmenting holograms in museums with animated multimedia content lets exhibitors
communicate information about the artifact with more excitement and effectiveness
than text labels offer. Such displays can also respond to user interaction. Because
wall-mounted variations require little space, museums can display a larger number of
artifacts.
Figure 7 shows two illustrations of envisioned future applications: A wall-mounted
display in a museum environment, with a ceiling-mounted video projector replacing
conventional spotlights, and a desktop display that can be used in a light-box
fashion. A special input device allowing interaction, including haptic feedback of
holographic and graphical content.
The technical and scientific progress that is planned to be made during this project
can be organized into six linked goals:
-
Holograms
- Investigation of different hologram types and individual solutions
that combine them with computer graphics. Optimization of optical
properties during the recording process to achieve the best possible effects.
-
Displays
- Experiments with different optical setups that support the
integration of computer graphics into the different hologram types.
Experiments with different stereoscopic and autostereoscopic techniques.
Investigation of different display form factors that serve a variety of
applications.
-
Calibration and registration
- Development of fully- or semi-automated
methods that calibrate the display optics, extract auxiliary information
(such as depth) recorded in the hologram and register holographic and
graphical content.
-
Rendering
- Development of effective rendering and illumination algorithms
that support different hologram types, special effects, and a realistic and
consistent presentation of holographic and graphical content at interactive
frame rates.
-
Interaction
- Investigation of the potentials and limitations of existing
interaction techniques and devices in combination with interactive
holograms. Development of new interaction forms that are suited for the
different display approaches and potential application areas.
-
Demonstration and evaluation
- Implementation of demonstrators that
address different application areas. The effectiveness of the proposed
concept is evaluated by presenting the demonstrators to domain experts.
Acknowledgements
This project is supported by the Deutsche Forschungsgemeinschaft (DFG). The
experiments shown in figure 5 were supported by DaimlerChrysler AG Research
Technology. The content shown in figures 2, 3 and 6 was provided by Ohio
University’s department of Biomedical Sciences. The HoloGraphics project is
supported by the Deutsche Forschungsgemeinschaft (DFG).
References
[1] O. Bimber January 2004 Combining holograms with interactive
computer graphics IEEE Computer 85–91
