Chapter Five: Augmented Reality (AR)

This chapter Covers

◦ Augmented Reality(AR) and Features
◦ Application and architecture of AR
◦ Virtual Reality
◦ Mixed Reality
◦ The difference between AR, VR, and MR
 Augmented Reality (AR)
The fundamental idea of AR is to combine, or mix, the
view of the real environment with additional, virtual
content that is presented through computer graphics.
Its convincing effect is achieved by ensuring that the
virtual content is aligned and registered with the real
objects.
AR is a form of emerging technology that allows users
to overlay computer generated content in the real
world.
Augmented Reality (AR) as a real-time direct or
indirect view of a physical real-world environment that
has been enhanced/augmented by adding virtual
…
Fig 5.1 Examples of AR Technology
 Virtual Reality
 VR is fully immersive, which tricks your senses into thinking you’re
in a different environment or world apart from the real world.
 VR is a computer-simulated reality.
 Is the use of computer technology to create a simulated
enviromented.
 It refers to computer technologies using reality headsets to
generate realistic sounds, images and other sensations that
replicate a real environment or create an imaginary world.
 Advanced VR environment will engage all five senses (taste, sight,
smell, touch, sound), but it is important to say that this is not always
possible.
 Using VR devices such as HTC Vive, Oculus Rift or Google
Cardboard, users can be transported into a number of real-world
and imagined environments.
…
Fig 5.2 Examples of VR Technology
…cont
Most VR headsets are connected to a
computer (Oculus Rift) or a gaming console
(PlayStation VR) but there are standalone
devices (Google Cardboard is among the
most popular) as well.
Most standalone VR headsets work in
combination with smartphones – you insert a
smartphone, wear a headset, and immerse in
the virtual reality.
 Mixed Reality (MR)
 Mixed Reality (MR), sometimes referred to as hybrid reality, is
the merging of real and virtual worlds to produce new
environments and visualizations where physical and digital
objects co-exist and interact in real-time.
 It means placing new imagery within a real space in such a
way that the new imagery is able to interact, to an extent, with
what is real in the physical world we know
 For example, with MR, you can play a virtual video game, grab
your real-world water bottle, and smack an imaginary
character from the game with the bottle. Imagination and
reality have never been so intermingled.
 The key characteristic of MR is that the synthetic content and
the real-world content are able to react to each other in real-
time.
 …cont
 In mixed reality, you interact with and manipulate both
physical and virtual items and environments, using next-
generation sensing and imaging technologies.
 MR allows you to see and immerse yourself in the world
around you even as you interact with a virtual environment
using your own hands—all without ever removing your
headset.
 It provides the ability to have one foot (or hand) in the real
world, and the other in an imaginary place, breaking down
basic concepts between real and imaginary, offering an
experience that can change the way you game and work
today.
 …cont

Fig 5.3. Mixed Reality in Engineering and Medicine

…cont

Fig 5.4. Mixed reality in entertainment

 The difference between AR, VR,
and MR
 Augmented Reality(AR)
◦ AR overlays digital content on top of the real-world.
◦ It is the integration of digital information with the user's environment in
realtime.
◦ Numerous augmented reality apps and games can run on almost every
smartphone on the market.
 Virtual Reality(VR)
◦ VR is content which is 100% digital and can be enjoyed in a fully immersive
environment.
◦ which creates a totally artificial environment
◦ virtual reality programs require specialized VR headsets, noise-canceling
headphones, cameras to track room space and boundaries, and sometimes
even motion capture technology.
◦ VR tech: Oculus Rift, HTC Vive, and PlayStation VR.
…cont

Mixed Reality(MR)
◦ MR is a digital overlay that allows interactive
virtual elements to integrate and interact with the
real-world.
◦ Mixed reality hardware is still emerging and hasn’t
quite broken into the mainstream consumer
market, most likely due to the price.
◦ MR tech: Microsoft HoloLens and Magic Leap.
◦ Mixed reality applications sometimes require
exponentially more processing power and thus
require more powerful hardware.
 …cont

Figure 5.5 Paul Milgram's Reality-Virtuality (RV)

Continuum
…cont
 The architecture of AR Systems
The first Augmented Reality Systems (ARS) were usually
designed with a basis on three main blocks.
Infrastructure Tracker Unit
Processing Unit
Visual Unit.
 The Infrastructure Tracker Unit was responsible for
collecting data from the real world, sending them to the
Processing Unit, which mixed the virtual content with the
real content and sent the result to the Video Out module
of the Visual Unit. Some designs used a Video In, to
acquire required data for the Infrastructure Tracker Unit.
Fig 5.6 Augmented Reality Systems (ARS) standard
architecture
 …cont
 The
Visual Unit can be classified into two types of system,
depending on the followed visualization technology:
1. Video see-through: It uses a Head-Mounted Display (HMD) that employs a
video-mixing and displays the merged images on a closed-view HMD.
2. Optical see-through: It uses an HMD that employs optical combiners to
merge the images within an open-view HMD.
 HMDsare currently the dominant display technology in the AR field.
However, they lack in several aspects,
 Ergonomics
 high prices
 relatively low mobility due to their sizes and connectivity features.
◦ An additional problem involving HMD is the interaction with the real
environment, which places virtual interactive zones to the user, making the
collision with these zones hard due to the difficulty to interact with multiple
points in different depths.
Applications of AR Systems
Technology is ever-changing and ever-
growing. One of the newest developing
technologies is augmented reality (AR), which
can be applied to many different disciplines
such as
◦ Education
◦ Medicine
◦ Entertainment
◦ Military, etc.
 AR In education
Augmented reality allows flexibility in use that is attractive
to education. AR technology can be utilized through a
variety of mediums including desktops, mobile devices, and
smartphones.
◦ The technology is portable and adaptable to a variety of scenarios.
AR can be used to enhance content and instruction within
the traditional classroom, supplement instruction in the
special education classroom, extend content into the world
outside the classroom, and be combined with other
technologies to enrich their individual applications.
When using AR technology in the classroom, teachers can
create an authentic learning environment for students with
different learning styles.
…cont
Affordable learning materials
◦ posters, digital illustrations, physical models, prototypes are
very expensive and it’s impossible for schools to find enough
money to purchase all the supplementary materials they
would like to.
◦ Using AR technology allows for avoiding investments in
physical materials. Besides, students can get access to
learning materials and interact with them anytime.
Interactive lessons
◦ when AR technology is used in classrooms, students can view
models on their own smartphones and get a better idea of the
concepts they are studying. That increases engagements and
reinforces the learning.
…cont
Higher engagement
◦ when teachers integrate augmented reality into their
lectures, they attract the attention of their students and
make lessons more effective.
Higher retention
◦ using the AR app, students can get access to augmented
models that represent any real objects from a famous
monument or work of art to a molecule.
◦ Besides, students can get access to a website with
specific information. When learning with AR technology,
students use different senses and retain more knowledge
for a long time.
 …cont
Boost intellectual curiosity
◦ augmented reality makes students more
excited about learning certain subjects.
◦ Modern students were born in a digital era so
they will always be excited with innovative
technologies that can help them learn new
ideas and develop their critical thinking
skills.
Fig 5.7 AR in education
AR In Medicine
 Augmented reality has the potential to play a big role in
improving the healthcare industry.
 Only a few years since the first implementations of
augmented reality in medicine, it has already filled an
important place in doctors’ and nurses’ routine, as well as
patients’ lives.
 AR is enhancing medicine and healthcare towards more
safety and efficiency. For now, augmented reality has already
made significant changes in the following medical areas:
◦ surgery (minimally invasive surgery)
◦ education of future doctors
◦ Diagnostics
◦ AR tools may also aid to detect the signs of depression and other
mental illnesses by reading from facial expressions, voice tones, and
physical gestures.
 AR application in Medicine
 Describing symptoms
◦ Have you ever been in a situation when it was hard to
describe to the doctor what was bothering you? It is a
common problem for all us, the roots of which extend to
overreacting and lack of knowledge.
◦ And what is most important, it impacts on finding out the
accurate diagnosis. The first steps to find the solutions are
already made.
◦ To increase patients’ education, medical app AyeDecide is
using augmented reality to show the simulation of the
vision, harmed by the different diseases. It helps patients to
understand their conditions and describe correctly their
symptoms.
…cont

Fig 5.8.1 EyeDecide App

…cont
Nursing care
◦ About 40% of the first intravenous injections fail,
and this ratio is even higher in the case of children
and elderly patients.
◦ The AccuVein uses augmented reality to cope with
this negative statistic. A handheld scanner projects
on the skin and shows the patients’ veins.
◦ It increases the successful finding of the vein from
the first try in 3,5 times. That is why this invention
got the greatest recognition among the general
public and medical staff.
 …cont

Fig 5.8.2 Image by AccuVein

…cont
 Surgery
◦ In no sphere augmented reality does not have such practical application
as in the medicine, especially in surgery, where it literally helps to save
lives.
◦ Three dimensional reconstructions of organs or tumors will help
surgeons become more efficient at surgery operations.
◦ For example, spinal surgery, as usually, is a long and difficult process.
But with the use of AR, it can reduce the time, cut the risks and improve
the results.
◦ The Israeli startup Augmedics had created an augmented reality
headset for spine surgeons. This technology overlays a 3D model of the
CT-scan on the spine, so, the surgeon gets some kind of “X-ray” vision.
◦ Surgeons will be able to visualize bones, muscles, and internal organs
without even having to cut open a body.
…cont

Fig 5.8.3 Image by AUGmedics

…cont
Ultrasounds
◦ Some time ago ultrasound made a small revolution
in medicine. Today, it has another one chance to
make the same with using augmented reality.
◦ Already a few AR software companies developed
handy ultrasound scanner, which with the help of
smart glasses works as a traditional one.
…cont
Diabetes management
◦ In 2017, the number of people struggle with
diabetes reached up to 425 million adults
worldwide. And the amount of diagnosed people is
increasing every year.
◦ In 2014, Google revealed the plans for creating a
smart contact lens (Google Contact Lens), in which
the main function will be to measure the glucose
levels in the tears. It will help people with this
disease to live the life they used to, without
permanent worries about sugar level in the blood.
…cont
Navigation
◦ The using AR in navigation apps has already
become a “traditional” way. By pointing your
phone to the city landscape, you get the
information about nearby objects of your interest
(museums, hotels, shops, metro stations, etc.).
◦ The same way AR can be useful to provide
information about the nearest hospitals. For
example, the EHBO app helps to find the nearest
to you AEDs (automated external defibrillators).
…cont
Generally, AR provides the following benefits
to patients and healthcare workers:
◦ Reduce the risks associated with minimally
invasive surgery.
◦ Better informed decisions about the right
treatment and illness prevention.
◦ Make procedures more tolerable.
◦ Better aftercare
◦ Medical training and education.
◦ Assistance in medical procedures and routine
tasks.
 AR In Entertainment
 AR in games
◦ the AR games were praised for increasing physical activity in
people – you actually have to move around to find your
target, for example, Pokémon.
◦ At the same time, there are complaints that players could
cause various incidents and accidents being too engrossed in
the game. In any case, Pokémon GO has rightfully earned its
popularity and opened the world of AR games to us.

Fig 5.9 AR Gaming with

tony Godar
…cont
AR in music
◦ music is not only about listening to favorite tracks
and putting together playlists. When we like a
piece, we often want to find out more about its
background: the performers’ bios, the lyrics of the
song, the making of the recording or the music
video.
◦ Augmented reality can do all that and much more
providing complete information on the track or its
performer. Augmented reality can enhance live
performances by illustrating the story told by a
track or displaying the way it was created by the
…cont
AR on TV
◦ One way of integrating augmented reality in
television is adding supplementary information to
what is going on the TV screen – such as match
scores, betting options, and the like.
AR in eSports
◦ Competitive online gaming has become as
fascinating as real sports, and the technology is
following it closely with new solutions and unusual
implementations.
◦ Augmented reality turns eSports shows into
…cont
AR in the theater
◦ augmented reality can serve not only for
entertainment purposes but also for the purposes
of accessibility.
◦ The possibility to overlay virtual objects over the
real environment can be used, for example, for
subtitling in various theater shows.
◦ Now, many theaters use LED displays either to
provide subtitles for translation or to assist
hearing-impaired visitors.
◦ However, LED equipment is not available in each
Augmented reality tourism is the new dimension of the industry, where visitors
are experiencing the power of AR with a lot of interactivity, interest & joy.