Tech Stack

Localization With Vision

The key technology to connect agents, real world, virtual worlds and digital content is the localization in the physical space.

Zeno uses cutting-edge spatial computing platforms designed for city-scale spatial mapping and vision-based localization service and infrastructure. It offers a variety of data capture, e.g. using various sensors like Lidar, 360-degree cameras, and RGB cameras on mobile device or XR headsets. The platform features a robust cloud-based visual world model and computing system capable of processing multi-gigabytes of raw sensor data daily for large areas (city-scale / global zones) indexed for quick spatial queries, or instant processing for small areas (room-scale / anchored zones) in parallel with high throughput. It's designed to be self-learning, improving with both high-quality and third-party data. The service can handle hundreds of spatial queries per second if scaled, offering precise 6-DOF localization results, shared spatial anchor creation, rapid 3D visual reconstruction, instant semantic segmentation and other scene understandings. It's highly salable and enables applications in AR gaming, navigation, advertising, or productivity tools.

On the client end, Zeno's goal is to adapt as many as agents' beacons out there, thus only vision signal with IMU sensors on device are essential. This should work with large portion of existing smart phones, tablets, XR headsets, robots, and different kinds of devices that are portable with cameras and IMU sensors.

With that, Zeno's client localizes the beacon in the physical reality by using vision and IMU signals with Zeno's cloud services and on-device models. It answers with a precise 6-DOF pose (position and orientation) to the application mostly real-time. Applications then discover virtual worlds and contents overlaid on the physical surroundings, and bring the blended immersive experience to the users, or guide the robot or agent bots to perform there tasks.

Robot Dog

Drone

Human with XR

System Infrastructure

Zeno platform is a full stack system, from data layer to application layer. Here illustrates how Zeno construct and weave different components together to create the platform system and supporting applications.