VR Digital Twin System, For The Palace Museum

Developer, Interaction Designer

OVERVIEW

My Role
Developer
Interaction Designer

Platform
Oculus Quest 2

Tool
Unreal Engine 4
Autodesk Revit
Figma

Team
Solo

Time
Dec 2021 - Jun 2022

Client
The Palace Museum, Beijing, China

INTERACTION PROTOTYPEs

1.Present historical information orderly:

1st layer panel

2.Optimized VR UI’s Readability:

Found:

  • UI panel’s size: Size: if it exceeds the range of Quest 2, it would cause users to turn their heads, leading to dizziness.

  • UI panel’s distance: best to interact with at the distance of 1~1.5m.

  • UI panel’s color: reflection would affect readability if it white panel with black text.

Iteration:

  • Size: smaller than Quest 2’s FOV (100°× 55°).

  • Distance: 1m from user’s camera.

  • Color: white text on deep gray panel & black text on light gray panel.

2.Synchronize data in real-time, to fit clients’ existing platform:

If the data from the user's existing BIM platform can be automatically synchronized into UE4, it will facilitate a better integration of VR into their current workflow.

3.Load and change scene in menu:

ITERATION AFTER USER TEST

1.Use button, instead of keyboard:

Finding: During user testing, I found keyboard was so hard to interact with, especially to who were new to VR.

Iteration: I changed the editing interaction from TYPING to CHOICE, because pushing button is the most natural way to interact in VR.

A Virtual-Reality interactive visualization plaform for Chinese heritage architecture of The Palace Museum, which is the client.

Realized functions below to meet clients’ requirements:

2nd-3rd layer panel

The information structure analysis

PROCESS

Background:

Architectural heritage researchers from the Palace Museum needs a smart computer visualization platform based on Augmented-Reality or Virtual-Reality, to integrate, manage and visualize various measurement data and models of the relics, so that they could monitor them and make preservation plans in a more intuitive way, with higher efficiency.

Project Goals:

1.Visualize the relics’ data through an interactive 3D UI system.
Users want to browse the archive and measurement data of the relic’s different architectural components in the VR environment. An interactive 3D UI panel with multiple levels is needed to visualize the data.

2.The data are automatically synchronized from H-BIM platform in real time.
I should find a technical solution to synchronize the diverse data from the existing H-BIM* platform in real time. The textual data should be associated with the relic components’ models automatically. The solution will be used as a standard by later developers.
(*H-BIM: Heritage Building Information Modeling. It is a method for managing of the measurement data and models of the relics. It is broadly used by the architectural heritage preservation industry.)

3.Build the data hierarchy according to the H-BIM platform.
The data between VR platform and H-BIM should have a structural mapping based on the H-BIM platform’s data hierarchy. Thus, the information structure of the UI system should be the same as H-BIM platform.

4.Enable users to take quick notes for preservation plans in VR.
Users want to make instant preservation plans of the relics in the VR environment, without taking off the VR headsets. The data modification should be synchronized automatically to the related Excel table in the H-BIM platform.

Sketch Prototype:

Figma prototype:

UE4 Blueprint Realization

VR Interactive System For Heritage Protection, for The Palace Museum.

Interaction Designer, VR Prototyper (UE4)

Shuozhi Shen

Jun 2022