IEEE transactions on visualization and computer graphics, 2022
Virtual Reality (VR) has the potential to support mobile knowledge workers by complementing tradi... more Virtual Reality (VR) has the potential to support mobile knowledge workers by complementing traditional input devices with a large three-dimensional output space and spatial input. Previous research on supporting VR knowledge work explored domains such as text entry using physical keyboards and spreadsheet interaction using combined pen and touch input. Inspired by such work, this paper probes the VR design space for authoring presentations in mobile settings. We propose PoVRPoint—a set of tools coupling penand touch-based editing of presentations on mobile devices, such as tablets, with the interaction capabilities afforded by VR. We study the utility of extended display space to, for example, assist users in identifying target slides, supporting spatial manipulation of objects on a slide, creating animations, and facilitating arrangements of multiple, possibly occluded shapes or objects. Among other things, our results indicate that 1) the wide field of view afforded by VR results...
Structure-from-motion and multi-view stereo can be used to create 3D models from image sequences,... more Structure-from-motion and multi-view stereo can be used to create 3D models from image sequences, but are not widely adopted by consumers. We study how to address two challenges of such systems for non-professional users: 1) their technical complexity, and, 2) the computational demand needed for processing. To this end, we embed an open-source pipeline in a scalable cloud environment and create a user interface aimed at non-professional users of photogrammetry systems. Finally, we evaluate both the cloud-based infrastructure and the user interface and demonstrate its usability.
Smartphones are a popular device class for mobile Augmented Reality but suffer from a limited inp... more Smartphones are a popular device class for mobile Augmented Reality but suffer from a limited input space. Around-device interaction techniques aim at extending this input space using various sensing modalities. In this paper we present our work towards extending the input area of mobile devices using front-facing device-centered cameras that capture reflections in the cornea. As current generation mobile devices lack high resolution front-facing cameras, we study the feasibility of around-device interaction using corneal reflective imaging based on a high resolution camera. We present a workflow, a technical prototype and a feasibility evaluation.
With the rising popularity of Augmented and Virtual Reality, there is a need for representing hum... more With the rising popularity of Augmented and Virtual Reality, there is a need for representing humans as virtual avatars in various application domains ranging from remote telepresence, games to medical applications. Besides explicitly modelling 3D avatars, sensing approaches that create person-specific avatars are becoming popular. However, affordable solutions typically suffer from a low visual quality and professional solution are often too expensive to be deployed in nonprofit projects. We present an open-source project, BodyDigitizer, which aims at providing both build instructions and configuration software for a high-resolution photogrammetry-based 3D body scanner. Our system encompasses up to 96 Rasperry PI cameras, active LED lighting, a sturdy frame construction and open-source configuration software. %We demonstrate the applicability of the body scanner in a nonprofit Mixed Reality health project. The detailed build instruction and software are available at this http URL.
We explore the use of hand postures to interact with stroke-based rendering (SBR) on touch-sensit... more We explore the use of hand postures to interact with stroke-based rendering (SBR) on touch-sensitive large displays. In contrast to traditional WIMP interfaces, we allow people to directly engage with and influence a rendering. Our system allows the creation of new stroke primitives as well as provides mechanisms to distribute and then manipulate them on the canvas. We offer a set of natural mappings from hand postures to rendering parameterizations. The resulting system allows an intuitive exploration of SBR without the need for traditional desktop interfaces.
We present a demonstration of HeadPhones (Headtracking + smart-Phones), a novel approach for the ... more We present a demonstration of HeadPhones (Headtracking + smart-Phones), a novel approach for the spatial registration of multiple mobile devices into an ad hoc multi-display environment. We propose to employ the user's head as external reference frame for the registration of multiple mobile devices into a common coordinate system. Our approach allows for dynamic repositioning of devices during runtime without the need for external infrastructure such as separate cameras or fiducials. Specifically, our only requirements are local network connections and mobile devices with built-in front facing cameras. This way, HeadPhones enables spatially-aware multi-display applications in mobile contexts.
An increasing number of consumer-oriented head-mounted displays (HMD) for augmented and virtual r... more An increasing number of consumer-oriented head-mounted displays (HMD) for augmented and virtual reality (AR/VR) are capable of finger and hand tracking. We report on the accuracy of off-theshelf VR and AR HMDs when used for touch-based tasks such as
2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)
Text entry is a challenge for Virtual Reality (VR) applications. In the context of immersive VR H... more Text entry is a challenge for Virtual Reality (VR) applications. In the context of immersive VR Head-Mounted Displays, text entry has been investigated for standard physical keyboards as well as for various hand representations. Specifically, prior work has indicated that minimalistic fingertip visualizations is an efficient hand representation. However, these representations typically require external tracking systems. Touch-sensitive physical keyboards allow for on-surface interaction, with sensing integrated into the keyboard itself. However, they have not been thoroughly investigated within VR. We close this gap by comparing text entry on a standard physical keyboard and a touch-sensitive physical keyboard in a controlled user study (n=26). Our results indicate that text entry using touch-sensitive physical keyboards can be as efficient as the fingertip visualization, but that results vary between experienced and inexperienced typists.
Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services - MobileHCI '12, 2012
Magic lens and static peephole interfaces are used in numerous consumer mobile phone applications... more Magic lens and static peephole interfaces are used in numerous consumer mobile phone applications such as Augmented Reality browsers, games or digital map applications in a variety of contexts including public spaces. Interface performance has been evaluated for various interaction tasks involving spatial relationships in a scene. However, interface usage outside laboratory conditions has not been considered in depth in the evaluation of these interfaces. We present findings about the usage of magic lens and static peephole interfaces for playing a find-and-select game in a public space and report on the reactions of the public audience to participants" interactions. Contrary to our expectations participants favored the magic lens over a static peephole interface despite tracking errors, fatigue and potentially conspicuous gestures. Most passersby did not pay attention to the participants and vice versa. A comparative laboratory experiment revealed only few differences in system usage.
Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services - MobileHCI '13, 2013
The development of mobile Augmented Reality application became increasingly popular over the last... more The development of mobile Augmented Reality application became increasingly popular over the last few years. However, many of the existing solutions build on the reuse of available standard metaphors for visualization and interaction without considering the manifold contextual factors of their use. Within this workshop we want to discuss theoretical design approaches and practical tools which should help developers to make more informed choices when exploring the design space of Augmented Reality interfaces in mobile contexts.
We present an efficient natural feature tracking pipeline solely implemented in JavaScript. It is... more We present an efficient natural feature tracking pipeline solely implemented in JavaScript. It is embedded in a web technologybased Augmented Reality system running plugin-free in web browsers. The evaluation shows that real-time framerates on desktop computers and while on smartphones interactive framerates are achieved.
In this paper we present several techniques to interactively explore representations of 2D vector... more In this paper we present several techniques to interactively explore representations of 2D vector fields. Through a set of simple hand postures used on large, touch-sensitive displays, our approach allows individuals to customdesign glyphs (arrows, lines, etc.) that best reveal patterns of the underlying dataset. Interactive exploration of vector fields is facilitated through freedom of glyph placement, glyph density control, and animation. The custom glyphs can be applied individually to probe specific areas of the data but can also be applied in groups to explore larger regions of a vector field. Re-positionable sources from which glyphs-animated according to the local vector field-continue to emerge are used to examine the vector field dynamically. The combination of these techniques results in an engaging visualization with which the user can rapidly explore and analyze varying types of 2D vector fields, using a virtually infinite number of custom-designed glyphs.
In this paper, we discuss the impact of tracking technology on user studies of mobile augmented r... more In this paper, we discuss the impact of tracking technology on user studies of mobile augmented reality applications. We present findings from several of our previous publications in the field, discussing how tracking technology can impact, influence and compromise experimental results. Lessons learned from our experience show that suitable tracking technology is a key requirement and a fundamental factor in the user experience of the application. Tracking technology should therefore be considered not only during ...
Virtual and Augmented Reality (VR and AR) have the potential to change information work. The abil... more Virtual and Augmented Reality (VR and AR) have the potential to change information work. The ability to modify the workers senses can transform everyday environments into a productive office, using portable head-mounted displays (HMDs) combined with conventional interaction devices, such as keyboards and tablets. While a stream of better, cheaper and lighter HMDs have been introduced for consumers in recent years, there are still many challenges to be addressed to allow this vision to become reality. This chapter summarizes the state of the art in the field of extended reality for knowledge work in everyday environments and proposes steps to address the open challenges.
The SmartReality project is a new Austrian national project dedicated to exploring how people can... more The SmartReality project is a new Austrian national project dedicated to exploring how people can access smarter information about the things around them through a combination of Augmented Reality, Web services and semantic technologies. This paper introduces the project and our initial thoughts concerning the current and necessary state of AR standards, interfaces and platforms.
This deliverable updates the scenarios and requirements described in D2.1.2 to include on-demand ... more This deliverable updates the scenarios and requirements described in D2.1.2 to include on-demand features identified during connectivity phase development, the usage by driving experiments and experiments funded through the open calls. The document provides a description of the functionality that needs to be developed within the expansion phase, to be included in the V2.0 release.
IEEE transactions on visualization and computer graphics, 2022
Virtual Reality (VR) has the potential to support mobile knowledge workers by complementing tradi... more Virtual Reality (VR) has the potential to support mobile knowledge workers by complementing traditional input devices with a large three-dimensional output space and spatial input. Previous research on supporting VR knowledge work explored domains such as text entry using physical keyboards and spreadsheet interaction using combined pen and touch input. Inspired by such work, this paper probes the VR design space for authoring presentations in mobile settings. We propose PoVRPoint—a set of tools coupling penand touch-based editing of presentations on mobile devices, such as tablets, with the interaction capabilities afforded by VR. We study the utility of extended display space to, for example, assist users in identifying target slides, supporting spatial manipulation of objects on a slide, creating animations, and facilitating arrangements of multiple, possibly occluded shapes or objects. Among other things, our results indicate that 1) the wide field of view afforded by VR results...
Structure-from-motion and multi-view stereo can be used to create 3D models from image sequences,... more Structure-from-motion and multi-view stereo can be used to create 3D models from image sequences, but are not widely adopted by consumers. We study how to address two challenges of such systems for non-professional users: 1) their technical complexity, and, 2) the computational demand needed for processing. To this end, we embed an open-source pipeline in a scalable cloud environment and create a user interface aimed at non-professional users of photogrammetry systems. Finally, we evaluate both the cloud-based infrastructure and the user interface and demonstrate its usability.
Smartphones are a popular device class for mobile Augmented Reality but suffer from a limited inp... more Smartphones are a popular device class for mobile Augmented Reality but suffer from a limited input space. Around-device interaction techniques aim at extending this input space using various sensing modalities. In this paper we present our work towards extending the input area of mobile devices using front-facing device-centered cameras that capture reflections in the cornea. As current generation mobile devices lack high resolution front-facing cameras, we study the feasibility of around-device interaction using corneal reflective imaging based on a high resolution camera. We present a workflow, a technical prototype and a feasibility evaluation.
With the rising popularity of Augmented and Virtual Reality, there is a need for representing hum... more With the rising popularity of Augmented and Virtual Reality, there is a need for representing humans as virtual avatars in various application domains ranging from remote telepresence, games to medical applications. Besides explicitly modelling 3D avatars, sensing approaches that create person-specific avatars are becoming popular. However, affordable solutions typically suffer from a low visual quality and professional solution are often too expensive to be deployed in nonprofit projects. We present an open-source project, BodyDigitizer, which aims at providing both build instructions and configuration software for a high-resolution photogrammetry-based 3D body scanner. Our system encompasses up to 96 Rasperry PI cameras, active LED lighting, a sturdy frame construction and open-source configuration software. %We demonstrate the applicability of the body scanner in a nonprofit Mixed Reality health project. The detailed build instruction and software are available at this http URL.
We explore the use of hand postures to interact with stroke-based rendering (SBR) on touch-sensit... more We explore the use of hand postures to interact with stroke-based rendering (SBR) on touch-sensitive large displays. In contrast to traditional WIMP interfaces, we allow people to directly engage with and influence a rendering. Our system allows the creation of new stroke primitives as well as provides mechanisms to distribute and then manipulate them on the canvas. We offer a set of natural mappings from hand postures to rendering parameterizations. The resulting system allows an intuitive exploration of SBR without the need for traditional desktop interfaces.
We present a demonstration of HeadPhones (Headtracking + smart-Phones), a novel approach for the ... more We present a demonstration of HeadPhones (Headtracking + smart-Phones), a novel approach for the spatial registration of multiple mobile devices into an ad hoc multi-display environment. We propose to employ the user's head as external reference frame for the registration of multiple mobile devices into a common coordinate system. Our approach allows for dynamic repositioning of devices during runtime without the need for external infrastructure such as separate cameras or fiducials. Specifically, our only requirements are local network connections and mobile devices with built-in front facing cameras. This way, HeadPhones enables spatially-aware multi-display applications in mobile contexts.
An increasing number of consumer-oriented head-mounted displays (HMD) for augmented and virtual r... more An increasing number of consumer-oriented head-mounted displays (HMD) for augmented and virtual reality (AR/VR) are capable of finger and hand tracking. We report on the accuracy of off-theshelf VR and AR HMDs when used for touch-based tasks such as
2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)
Text entry is a challenge for Virtual Reality (VR) applications. In the context of immersive VR H... more Text entry is a challenge for Virtual Reality (VR) applications. In the context of immersive VR Head-Mounted Displays, text entry has been investigated for standard physical keyboards as well as for various hand representations. Specifically, prior work has indicated that minimalistic fingertip visualizations is an efficient hand representation. However, these representations typically require external tracking systems. Touch-sensitive physical keyboards allow for on-surface interaction, with sensing integrated into the keyboard itself. However, they have not been thoroughly investigated within VR. We close this gap by comparing text entry on a standard physical keyboard and a touch-sensitive physical keyboard in a controlled user study (n=26). Our results indicate that text entry using touch-sensitive physical keyboards can be as efficient as the fingertip visualization, but that results vary between experienced and inexperienced typists.
Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services - MobileHCI '12, 2012
Magic lens and static peephole interfaces are used in numerous consumer mobile phone applications... more Magic lens and static peephole interfaces are used in numerous consumer mobile phone applications such as Augmented Reality browsers, games or digital map applications in a variety of contexts including public spaces. Interface performance has been evaluated for various interaction tasks involving spatial relationships in a scene. However, interface usage outside laboratory conditions has not been considered in depth in the evaluation of these interfaces. We present findings about the usage of magic lens and static peephole interfaces for playing a find-and-select game in a public space and report on the reactions of the public audience to participants" interactions. Contrary to our expectations participants favored the magic lens over a static peephole interface despite tracking errors, fatigue and potentially conspicuous gestures. Most passersby did not pay attention to the participants and vice versa. A comparative laboratory experiment revealed only few differences in system usage.
Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services - MobileHCI '13, 2013
The development of mobile Augmented Reality application became increasingly popular over the last... more The development of mobile Augmented Reality application became increasingly popular over the last few years. However, many of the existing solutions build on the reuse of available standard metaphors for visualization and interaction without considering the manifold contextual factors of their use. Within this workshop we want to discuss theoretical design approaches and practical tools which should help developers to make more informed choices when exploring the design space of Augmented Reality interfaces in mobile contexts.
We present an efficient natural feature tracking pipeline solely implemented in JavaScript. It is... more We present an efficient natural feature tracking pipeline solely implemented in JavaScript. It is embedded in a web technologybased Augmented Reality system running plugin-free in web browsers. The evaluation shows that real-time framerates on desktop computers and while on smartphones interactive framerates are achieved.
In this paper we present several techniques to interactively explore representations of 2D vector... more In this paper we present several techniques to interactively explore representations of 2D vector fields. Through a set of simple hand postures used on large, touch-sensitive displays, our approach allows individuals to customdesign glyphs (arrows, lines, etc.) that best reveal patterns of the underlying dataset. Interactive exploration of vector fields is facilitated through freedom of glyph placement, glyph density control, and animation. The custom glyphs can be applied individually to probe specific areas of the data but can also be applied in groups to explore larger regions of a vector field. Re-positionable sources from which glyphs-animated according to the local vector field-continue to emerge are used to examine the vector field dynamically. The combination of these techniques results in an engaging visualization with which the user can rapidly explore and analyze varying types of 2D vector fields, using a virtually infinite number of custom-designed glyphs.
In this paper, we discuss the impact of tracking technology on user studies of mobile augmented r... more In this paper, we discuss the impact of tracking technology on user studies of mobile augmented reality applications. We present findings from several of our previous publications in the field, discussing how tracking technology can impact, influence and compromise experimental results. Lessons learned from our experience show that suitable tracking technology is a key requirement and a fundamental factor in the user experience of the application. Tracking technology should therefore be considered not only during ...
Virtual and Augmented Reality (VR and AR) have the potential to change information work. The abil... more Virtual and Augmented Reality (VR and AR) have the potential to change information work. The ability to modify the workers senses can transform everyday environments into a productive office, using portable head-mounted displays (HMDs) combined with conventional interaction devices, such as keyboards and tablets. While a stream of better, cheaper and lighter HMDs have been introduced for consumers in recent years, there are still many challenges to be addressed to allow this vision to become reality. This chapter summarizes the state of the art in the field of extended reality for knowledge work in everyday environments and proposes steps to address the open challenges.
The SmartReality project is a new Austrian national project dedicated to exploring how people can... more The SmartReality project is a new Austrian national project dedicated to exploring how people can access smarter information about the things around them through a combination of Augmented Reality, Web services and semantic technologies. This paper introduces the project and our initial thoughts concerning the current and necessary state of AR standards, interfaces and platforms.
This deliverable updates the scenarios and requirements described in D2.1.2 to include on-demand ... more This deliverable updates the scenarios and requirements described in D2.1.2 to include on-demand features identified during connectivity phase development, the usage by driving experiments and experiments funded through the open calls. The document provides a description of the functionality that needs to be developed within the expansion phase, to be included in the V2.0 release.
Uploads
Papers by Jens Grubert