AUGMENTED REALITY NAUTILUS OBSERVATORY (ARNO)

Built upon the Pacific Research Platform (PRP) network, UC San Diego’s Clarke Center for Human Imagination and the Qualcomm Institute are developing server-side rendering tools and deployment of Unreal-based game experience to scale the Metaverse AR/VR hardware needs into a distributed system, enabling a campus-wide XR test bed as part of developing a global Extended Reality Network Operations Center (XRNOC) networking visualization system. XRNOC represents the third generation of big data visualization, realized through persistent, procedurally rendered simulations produced from live data at global scale via Epic Game’s Unreal Engine.

Learn more about Nautilus

"Nautilus is a HyperCluster for running containerized Big Data Applications. It is using Kubernetes for managing and scaling containerized applications and Rook for automating Ceph data services."

For more information on this pioneering research and how to support it, please contact us at info@imagination.ucsd.edu

How to run Holographic Remote Streaming on Windows

• Grab the latest x64-Release Package https://gitlab.nrp-nautilus.io/ar-noc/ARNoc-Alpha/-/releases
• Run the Holographic Remoting app on the Hololens and take note of the IP.
• Edit HoloLensRemoting=<Hololens IP> inside arnocMain_HLRemotingJoinServer.bat
• Run batch script to join dedicated server and connect with Holographic Remoting.

How to get started developing (Windows)

Project Setup

0. Install Git and Git LFS
   a. "You'll only need to run git lfs install once. Once initialized for your system, Git LFS will bootstrap itself automatically when you clone a repository containing Git LFS content."

1. Setup Unreal Engine 5.0.1

2. Install Visual Studio 2019
   a. C++ profiling tools
   b. C++ AddressSanitizer (optional)
   c. Windows 10 SDK (10.0.18362 or Newer)

3. Currently the project will fail to clone due to issues with GitLFS. In order to sync with the project we use the following commands:

   • Linux shell:

     a. Download the project as a tarball (.tar.gz) from the repo. https://gitlab.nrp-nautilus.io/ar-noc/ARNoc-Alpha/-/archive/main/ARNoc-Alpha-main.tar.gz

       mkdir ARNoc-Alpha
       git clone --bare https://gitlab.nrp-nautilus.io/ar-noc/ARNoc-Alpha.git ARNoc-Alpha
       mkdir ARNoc-Alpha/.git
       mv ARNoc-Alpha/* ARNoc-Alpha/.git
       tar -xvf ARNoc-Alpha-main.tar.gz
       mv arnoc-alpha-main/* ARNoc-Alpha/
       Check that all files from arnoc-alpha-main/ have been moved to ARNoc-Alpha/ (i believe .gitignore, .gitattributes, etc will need to moved manually)
       cd ARNoc-Alpha
       git init
       git pull
       git reset --hard

After this git status should show that your project is up-to-date with git to push/pull as before.

CAUTION: If you are using an older snapshot compressed .zip or .tar.gz of the project it may contain content that has been removed. This will cause git status to show untracked files that needed to be staged. Remove those files from your project until git status is clean.

4. git submodule update --init --recursive

5. Generate Visual Studio Project files

6. Open arnocMain.sln in Visual Studio

7. Set as Startup Project

8. Set toolbar to desired target <Development Editor> <Platform (i.e Win64)>. Right click arnocMain and Build.

9. Open the arnocMain.uproject in Unreal Engine 5 editor.

Unreal Engine

0. Getting Started with Unreal Engine
1. Blueprint Learning Path
2. User Interfaces with UMG Quickstart
3. Cesium for Unreal Tutorials

ARNO Project Tutorials

• Using a Revit Survey Point geolocation with Cesium for Unreal
• Google Earth to Cesium ion to Cesium for Unreal workflow
• Snapping Revit Models onto 3D Tile Twin

Unreal Project Structure

A continuation of the ue4-data-viz, ue4-earth, and ue4-data-h2 repositories, integrated and extended to Unreal Engine 4.26. As of SP22 the project was migrated to UE5 and began to leverage the CollabViewer and HololensViewer Unreal Project templates to quickly bootstrap a foundation for Cross-Platform Desktop/VR/AR/PixelStreaming Multiplayer across Windows and Linux. Android and Mac are not supported. In addition, the Redpoint Games EOS plugin was chosen to expediate development of online services commonly found in multiplayer games. However, it's not yet implemented.

CollabViewer

The Collab Viewer Templates join multiple people together in a shared experience of the same 3D content. They are intended to make it easier and quicker for your team to review and communicate about designs in realtime, so that you can identify problems and iterate on the content more efficiently.

HololensViewer

The HoloLens Viewer Template is an adaptation of the Collab Viewer Template that works on the Microsoft HoloLens 2. You can use it to see your 3D content overlaid on your actual surroundings in the HoloLens viewer. You can also interact with your models, moving them around and annotating them in real-world space.

EOS Online Subsystem

Connect players with their friends across all platforms and all stores using Epic’s free, shared account services. Players can log in using accounts from PlayStation, Xbox, Switch, Facebook, Google, Apple, Steam, and Epic.

Access free services including voice chat, achievements, matchmaking, live ops, and more. Game services support all platforms, all stores, and all identity services—including your own or Epic’s—on the engine of your choice.

Level: NOC_withCesium (Multiplayer)

Purpose: Initial Environment to test Cross-Platform Multiplayer Framework along with UI/UX support for Desktop/VR/AR/PixelStreamer users. A Collaborative space that persistently updates several data visualization outlined below. The _withCesium derivative focus on integrating Project Anywhere XR into the NOC.

G2

see DataViz_README.md

Netbox

Cesium

Level: Earth (Persistent Level)

Purpose: Create the Cesium Plugin Features (e.g World Terrain, Gamemode, Sunlight, etc)

World Composition with streaming sublevels is used with cesium to Build Global Scenes with Georeferenced Sublevels for Earth (Persistent Level).

For the XRNOC Application a sublevel will generally encapsulate a Revit Datasmith Architectural Asset (e.g AtkinsonsHall, Geisel, SDSC, etc) that is streamed in/out as described by the "Georeferenced Sublevels" tutorial above. Within the Datasmith Model several Netbox Sites may be procedurally generated via Event BeginPlay inside BP_NetboxGeneration_Cesium.

Sublevels: Sites (Streaming Sublevels)

Purpose: Modulerize Implementation of new Facilities and provide functionality for asynchronous level streaming during runtime.

Atkinson Hall (Sublevel)

San Diego Supercomputer Center (SDSC) (Sublevel)

Geisel Library (Sublevel)

Design and Innovation Building (Sublevel)

Franklin Antonio Hall (Sublevel)

Visual Arts Facility (Sublevel)

Mandeville Center (Sublevel)

Navigating Cluster Visualization States

Level 1 (Earth View):

Level 2 (Regional View):

Level 3 (County View):

Level 4 (Ground View):

Level 5 (Data Center View):

Blueprints

BP_Tracegraph_Cesium

Purpose: Visualization of network trace routes as a graph on a 3D globe Earth. Meant to be an overview of the entire network and highlight points of interest for admins to look in to.

All data is loaded via REST APIs at runtime. See datautils > kubeless > ue4-data.py.

• Request is sent to the datautils kubeless function
• Receives a JSON containing the trace graph in a nice format for parsing in Unreal
• In the request body, aggregate can be set to 'none', 'location', or 'router' -- this will collapse the graph so that all nodes in the same e.g. location will be collapsed into a single node. Useful to reduce visual clutter
• All of this stuff, like aggregation, is being worked on in the datautils repo
• VaRest is used to perform these requests and parse the JSON.

The trace graph nodes are then procedurally generated inside of Unreal.

• For every node (both primary and hop IPs), a unique actor is spawned and added to an internal hash map variable
• This allows future reference or modification of the actor by specifying an IP address.
• The glyphs representing each node point orthogonal to the planet surface

The trace graph edges are then procedurally generated inside of Unreal.

• For every edge, get the actors for the two connected IPs from the hash map we built earlier
• We can use those actors and get their positions (in local coordinates), then spawn a spline between those positions
• These splines connect as a straight line -- passing through the earth or wrap around the earth.
• You can edit the spline length variable inorder to change the number spline points there are for each traceroute.
  • Increasing spline length will help to make the wrap smoother, but will take more processing.
  • Currently the best setting for spline lengths are lengths 1(pass through earth) and 2(wrap around earth). The tangent at the midpoint is altered give the route more bend, so increasing the spline length without taking out the tangent change will create a handle bar shape.
• Using the data attributes, we color the edge red if the MTU is less than 9000 and green otherwise