In this document we describe the workflows used in the creation of Virtual and Augmented Reality digital twins. These workflows include: The generation of 3D avatars, scene reconstruction of real-life environments and motion capture.
Avatar creation
An avatar is a virtual representation of a player or non-player character in a virtual environment. Avatar creation ranges from basic humanoid placeholder models to hyper-realistic 3D scans of real humans using laser scanners or photogrammetry. Human avatars can be animated using motion capture of real humans.
Scene reconstruction
A digital representation of real world locations, objects, buildings and facades can be created using several methods, such as laser scanning or photogrammetry. For specific use cases a hybrid approach can be used in which a digital model is created from scratch using point-cloud or photographic reference data.
Scene reconstruction using LIDAR
Laser scanning (LIDAR) uses light to measure the distance to an object or environment. The LIDAR outputs a cloud of points in 3D space (point cloud data, PCD). Those points can then be converted into a solid surface via one of several techniques, such as delaunay triangulation or marching cubes.
The following sample shows a classroom scanned using the Trimble X7 LIDAR scanner. The left-most image is the original point-cloud data acquired from the scanner. The point-cloud contains more than 10 million colored points.
The middle and right images show a mesh reconstruction of the scene using the marching cubes algorithm. Marching cubes is a primitive but robust method of generating watertight meshes from complex geometry.
Scene reconstruction using photogrammetry
The following video demonstrates the usage of a marching cubes algorithm to reconstruct a complex environment (Wasa Innovation Center).
Scene reconstruction using photogrammetry
Using photogrammetry a 3D model is created by extracting features from a large number of color photographs. The object is photographed from many different overlapping perspectives. A sophisticated model extracts common features in the photographs and places them relative to each other in 3D space. A mesh can be reconstructed by comparing the different depths between the common features of the photographs. Finally, the photographs themselves can be mapped on the resulting mesh to generate a photo-realistic texture.
Laser scanning + CAD reconstruction
The final option is the manual creation of Virtual Reality and Augumented Reality 3D models using computer-assisted design tools, such as 3D modelling software. By overlapping real-world scans such as point clouds or photogrammetry meshes, a fully digital model can be drawn on top of the reference data to quickly produce a sharper and cleaner model based on real world geometry.
