096 - Advanced visual INS for high-end handheld devices

Vision-based navigation (e.g., SLAM, visual INS), and its combination with other PNT sensors, is at the core of the positioning engine of many autonomous systems; the prospects for further improvement for autonomous vehicles has been studied in various NAVISP activities (e.g. EL1-026). Meanwhile, powerful vision and graphical processing units are introduced in handheld devices in combination with more advanced vision sensors (e.g., multiple cameras, stereoscopic capabilities, advanced GPU, etc.). These capabilities can support the tailoring of the navigation engines developed for vehicles (concepts and technologies) for seamless navigation of handheld devices in support of high-end applications such augmented reality, guidance of visually impaired person and reliable APNT solutions for 1st responders. The adaptation could further consider the combination of advanced Visual INS with positioning sensors available in handheld devices. Furthermore, it could contemplate the growth of remote sensing with 5G (and soon 6G) signals, in support of situational awareness for navigation, derived from bi-static radar sensing. The objective of the activity is to demonstrate advanced navigation engine for handheld devices, combining vision-based navigation and visual INS, derived from multiple cameras onboard the device, with other PNT sensors available in high-end handheld devices. The design will include an additional sensing element (e.g. 5G or WIFI) to calibrate out the drift from a straightforward combined INS and visual odometry approach. 

The tasks to be performed shall include:

• Survey of the state-of-the-art (e.g. Visual INS, SLAM solutions for Autonomous Vehicle, vision-related technologies of high-end handheld devices) and definition of preferred use cases and associated performance targets 
• Definition and design of the solution for the proof of concept, starting from trade-offs and tailoring to handheld capabilities of various implementations options derived from technologies developed for automotive and robotic domain, followed by incremental and agile development (innovative tailoring, optimisation), leveraging the ability to perform small scale field trials during design and development phases using available hardware and evaluation platforms.
• Proof of concept in real world conditions (but not necessarily utilizing real time processing) 
  
  The main outputs of the activity will consist of: 
• Survey of state of the art on Visual INS and SLAM processing for handheld devices 
• Breadboard and test report proving the concept of advanced navigation engine for handled combining vision-based INS with other PNT sensors.
• Roadmap for commercialization, including potential NAVISP EL2 activities