Which cultural needs does VR AR meet

Im / material Spaces - Virtual and Augmented Reality open up new approaches to cultural heritage

What is the difference between VR and AR? What can technology realistically achieve for cultural institutions? And what can she not do yet? Lectures by Franziska Ritter, Pablo Dornhege (DTHG digital) and Maren Demant (Invisible Room).

Augmented Reality, Virtual Reality, Mixed Reality: these have been the new trending terms not only in the tech industry, but also in the cultural sector. Wherever technology can be used to improve the visitor experience or simply to make it more exciting - for example with VR glasses in the museum - cultural institutions want to be at the forefront. Here, the understanding of what makes virtual reality, augmented reality and mixed reality different is often not even grown. So what is the difference between VR and AR? What can technology realistically achieve for cultural institutions so far? And what can she not do yet?

These questions - and the right answers - were the subject of the event "Im / material Spaces - Virtual and Augmented Reality Open Up New Approaches to Cultural Heritage", which was held on November 26th at the invitation of and in the rooms of the Technologiestiftung Berlin with the speakers * Inside Franziska Ritter and Pablo Dornhege (both research and teach at several universities and jointly lead the research project 'Im / material Theater Spaces') and Maren Demant (XR designer and founder of Invisible Room).

In his lecture on the basics of virtual and augmented reality, Pablo Dornhege explained the essence of virtual and augmented reality with the help of a glossary of terms and showed possible ways of handling the technologies in a media-friendly manner. The following guide summarizes the most important basics of all speaker contributions:

AR + VR in cultural institutions - a guide

The history of immersive technologies

Immersive technologies did not just emerge in recent years, but have been part of the repertoire of art, culture and science for centuries. Even the early Baroque theater, with its deep perspective stage design, pretended to be an expanded spatial reality that emerged solely in the mind of the viewer. Similar examples can be found again and again in the history of mankind: For example, artists of the Romantic era tried with huge panorama paintings to offer the viewer a 360-degree impression of certain scenes.

19th / 20th century - virtual journey with stereoscopy

At the turn of the 20th century, the imperial panorama made it possible for up to 25 people to “travel virtually” to distant places at the same time with automatically changing stereoscopic image series.

1950s - cinema with smells

In 1954, producer and cameraman Morton Heilig developed the Sensorama, which was to offer the first fully immersive cinema experience. To this end, Heilig experimented with a device that not only showed moving images, but was also supposed to emit smells that match the scenes.

1960s and 70s - art discovers 'artificial reality'

The "Sword of Damocles" developed by the computer technician Ivan Sutherland in 1968, which is generally considered the first, was further advanced Head-mounted display (HMD) applies. This was already able to fade digital images into the viewer's field of vision and thus create a hybrid form of digital and analog space. In the mid-1970s, computer artists like Myron Krueger were already experimenting with the term “artificial reality”. Krueger's artificial reality test laboratory called "Videoplace" at the University of Connecticut made it possible to interact with a computer-generated artificial reality by moving one's own body.

1980s - forerunners of the first VR glasses in today's sense

Modern forerunners of today's VR glasses were created in the mid-1980s and early 1990s in NASA's development laboratories, which used them to prepare astronauts for use in space. The first commercially available VR systems or virtual reality helmets were, for example, the Forte VFX1 headgear in 1994. The mass-market 3D graphics cards that came onto the market shortly afterwards accelerated this development even further.

The foundation stone for the current hype about immersive technologies and especially VR was then laid by the iPhone: With the high-resolution display of modern smartphones and the built-in inclination and acceleration sensors, the foundation for a quantum leap in terms of new virtual reality systems, such as for example the Oculus Rift, finally created.

Since virtual reality has been a household name and immersive technologies can also be purchased by ordinary consumers in stores, there has been pure chaos around the exact terms used around artificial reality. It should be mentioned that the most well-known terms - virtual reality, mixed reality and augmented reality - all serve to describe certain gradations of the virtualization of reality:

Virtual Reality (VR) is at one end of the spectrum: It describes the fully immersive simulation of the physical space, the complete immersion in a virtual environment that is perceived as real. In its purest form, the viewer can no longer distinguish VR from the physical manifestation of reality. Examples are interactive worlds of experience in which users can immerse themselves with virtual reality glasses.

In contrast to VR, the physical reality is meanwhile in the Augmented Reality (AR) “Only” superimposed with digital content. It is therefore closer to the end of the spectrum in terms of reality. An application example for AR are applications in the museum room, in which visitors * can film a painting on the wall using an iPad, over which several new digital layers are simultaneously superimposed on the device screen, such as earlier sketches of the work.

In the augmented virtuality Meanwhile, the real world is not enriched by digital layers, but the digital world is augmented by physical space. This is the case, for example, when the perception of a virtual space is supported by physical factors. If a user sees images of a campfire and a heating lamp simulates the heat emanating from the fire at the same time, then this is a form of augmented virtuality.

Terminal devices for the representation of virtual worlds

To make VR and AR experiences tangible, there have been a whole range of end devices so far. AR can already be experienced via smartphones, tablets and smart glasses. There are three different methods of anchoring digital elements in real space:

  • Marker-based systems, in which an image is recorded by a camera, the contrast points of which are automatically recognized and analyzed by the software
  • Markerless systemsin which - as with the IKEA apartment planner, for example - you pull 3D objects from a database and insert them into a digital representation of the room. Here the position of the digital objects is defined by the application and not by a marker.
  • Location based systemsin which virtual objects become visible at predetermined locations (for example by using a smartphone with GPS tracking).

Meanwhile, mobile VR systems such as VR glasses and VR headsets are by no means all the same. A distinction is normally made between so-called 3DOF and 6DOF systems, where "DOF" stands for "degrees of freedom":

  • At 3DOF systems like the Oculus "Go" or Google Cardboard the user can look around, that means looking up and down and tilting and turning your head, but not approaching the digital objects, as the system does not record its own position in space.
  • At 6DOF systems like the Oculus Rift or HTC Vive, in addition to the rotation of the perspective, there is also the Movement in space to. So you can not only approach virtual objects, you can also move around them.

There are also further differences in the mobility and display quality of the respective devices. While wired VR headsets usually offer higher computing power and can play more complex applications, freedom of movement is restricted with these devices. With wireless systems, full freedom of movement is guaranteed, but the image quality often suffers.

VR or AR in the cultural sector: what makes sense?

When developing VR and / or AR projects for cultural institutions, the focus should always be on what content should be conveyed with the technology. Instead of thinking in terms of technology, those responsible should first clarify why they want to use VR or AR at all.

In the decision-making process for special VR glasses or other devices, the focus should also be on the user. What are possibly the reservations of people who have not yet had contact with VR and AR? What is the best way to introduce beginners to the technology quickly and easily? And what are the circumstances so that the AR and VR systems can be fully effective? These questions should be considered when choosing the technology.

The extent to which practical hurdles can limit the successful use of AR and VR systems is often underestimated: wearing comfort, compatibility with classic prescription glasses and hygiene issues are often at the top of the list of concerns.

Not least because of unexpected difficulties like these, it can make sense for cultural institutions to rely on additional support in the care of the systems in addition to help with technological development in VR and AR projects. The presence of trained staff can relieve many new users of the fear of contact with the technology.

In addition, it makes sense for cultural institutions to plan and implement VR and AR projects on a smaller scale in order to slowly gain experience with the implementation and introduce users to the new technology - for example in the context of museums or in the theater.

Opportunities and risks of VR and AR in cultural institutions

In the context of the work of cultural institutions, the use of VR and AR can offer both opportunities and risks. Immersive technologies in the museum space, for example, have the potential to increase the visibility of exhibitions, to open up new sensual and epistemological perspectives for visitors and to contextualize collection objects using digital information. In addition, using VR and AR, users can interact with historical objects for the first time and perceive detailed information that would not be visible without technology - such as background details on exhibits that are displayed digitally.

At the same time, the arbitrary and unthought-out use of VR and AR can also disrupt the content of an exhibition and, in case of doubt, overwhelm visitors. Especially if it has not been clarified beforehand why immersive technologies should be used and what purpose they should fulfill within the program - beyond the pure technological aha effect. Not least because of this, it should be clarified at an early stage which role VR and AR stations should play in addition to the actual exhibits.

Is it about making the real exhibits more visible? Should the physical format take a back seat? Or should the immersive technologies form a completely separate part of the exhibition? All of these questions need to be answered in order to best exploit the potential of VR and AR for your own purposes. In any case, it should also be borne in mind that immersive technologies not only have a positive novelty value, but also, due to their short history within cultural institutions, sometimes also have a certain disruptive factor: For example, visitors can feel comfortable by wearing VR glasses Feeling separated from the rest of the audience or even reacting physically to the VR experience, for example with dizziness.

Opportunities:

  • Increased accessibility & visibility of collections
  • Interest & motivation through technology
  • New sensual and epistemological dimensions, e.g. enabling time travel, contextualization of collection objects, interaction with historical / original objects
  • Emotional activation of the visitors
  • Link options between analog and digital
  • Support of multiple information needs, e.g. detailed information accessible via various filters, options for multilingualism
  • Inclusion, e.g. through virtual reconstruction of exhibition areas whose architecture is not barrier-free

Risks:

  • Arbitrary staging can distract from the exhibit and its content
  • Distraction by technology / user guidance, the importance of storytelling + communication goals are often not given enough consideration in view of the technical possibilities
  • Competitive situation to the physical format, overload in the exhibition space
  • Intensive care, sometimes 1 to 1 care necessary
  • "Ban on overpowering" especially in the case of historical staging (Beutelsbach consensus)
  • Limited group suitability: VR is often an isolating medium
  • Data protection aspects in augmented reality with face recognition

Case studies

During the event, VR and AR projects were presented at nine stations. This was 100% very well received by the participants, as it offered a practical insight into the variety of technical possibilities.

Featured VR / AR projects

VR

Wadi al Helo. Storytelling-based, virtual trip to Wadi al Helo, World Heritage Site of the United Arab Emirates. Office for Cultural Heritage Sharjah, United Arab Emirates | American University Sharjah & Studio 105106 | 2018

The Städel Museum in the 19th century. Virtual reconstruction of the presentation of the collection in the 19th century, developed with scientists. Städel | 2016

Anne Frank House. A trip to Anne Frank's house at the time of her diary entries. Force Field / Anne Frank House | 2016

Hold the world. A virtual tour behind the scenes and into the collection of the Natural History Museum London. Sky UK Ltd. | 2018

Also interesting:

AR

Interactive sound map. This is what Berlin sounds like. Application that set the sound of Berlin to music by the Konzerthausorchester and transformed it into an interactive city map. APOLLO (Virtual Concert Hall and HTW) | 2019

The virtual quartet. World's first virtual string quartet. With the help of an application, four playing cards are brought to life interactively, each playing one or more quartet voices. APOLLO (Virtual Concert Hall and HTW) | 2019

Animalia Sum. Artistic work about a journey into the future. The effects of environmental conflicts and resource scarcity can be experienced from the perspective of animals. Bianca Kennedy & The Swan Collective | 2019

Also interesting:

  • Belvedere torso, Magdeburg University of Applied Sciences & Winckelmann Museum Stendal, 2017
  • AR in the picture gallery, Berlin State Museums as part of the museum4punkt0 project
  • Messengers of the AI, holographic performance by Dennis Rudolph, 2018

learn more

Literature / Articles / Links

Blogs and podcasts

Text: Kai Schnier

Presentation & communication

  • Presenting & communicating

    Digital cultural offers to schools! New synergies and technical hurdles

    How can digital cultural offers be linked to everyday school life? This question was the subject of the online “Breakfast Talk” on March 19, 2021.

  • Presentation & MediationDigital CollectionCulture MediationUI / UX

    New approaches to cultural assets: visualization of digital collections on the web

    Can extensive digital collections be explored at a glance? What are the hurdles when presenting cultural assets on the internet? Experience from our VIKUS project.

  • Methods & MindsetsPresenting & CommunicatingAccessibilityCulture MediationParticipation

    Real barriers in virtual space

    How do visually impaired people find their way around VR applications? What hurdles do you face in advance and how accessible are platforms like Mozilla Hubs & Co. actually?