Evolved-5G is an H2020 research project funded by the European Commission. Its aim is to set up an ecosystem to facilitate access to 5G network functionalities for third-party developers. Over and above the academic aspects (scientific publications, conference participation, etc.), the project’s aim was to set up procedures and tools for the professional world interested in 5G. After 3 years of research, a global budget of 7.9 million euros and fruitful collaborations, the EVOLVED-5G project will come to an end at the end of 2023. Immersion takes this opportunity to look back at the advances made in Augmented Reality remote assistance and 5G.
Remote assistance
Immersion invested in this project to explore the benefits of this technology for one of its key application areas: remote assistance. Indeed, what could be better than Augmented Reality (AR) or Mixed Reality (MR) to guide a person in difficulty from a distance? Immersive technologies make it possible to replace cumbersome paper documentation with step-by-step guidance to facilitate users’ work.
This is what we did at Renault Trucks a few years ago with Hololens 2 headsets.
In the case of remote assistance, an expert will be able to see his or her interlocutor’s workspace in real time and guide him or her remotely. However, this raises a big question mark over the network, which needs to be particularly high-performance to enable smooth collaboration between the two users.
The benefits of 5G
This is where the benefits of 5G come in. A 5G network offers levels of latency and bandwidth compatible with the requirements of real-time AR collaboration. It also offers interesting additional mechanisms, such as network Quality of Service (QoS) monitoring. In other words, it’s possible to be informed in real time of the status of the network and the extent of any problems that may arise.
Achievements: Immersion’s Network App
5G networks are therefore relevant for some of our use cases. How can we link our AR remote assistance application to these network functionalities? EVOLVED-5G provides a clear answer: Network Apps. In other words, applications that act as intermediaries between a business application (referred to as a vertical application) and the key components of the 5G network (5G Core). Immersion has designed and developed its own Network App. This Flask-based Python application receives as input high-level requests from our AR remote assistance application. It interprets these requests and triggers the corresponding function calls. More specifically, using the SDK developed during the project, it uses standardized network APIs behind the scenes: CAPIF and NEF (Network Exposure Function).
This enables it to make low-level requests to the 5G network with parameters tailored to the initial request (via the QosMonitoring API). The network will accept the request, create a subscription and start making regular reports to the Network App about users. The Network-App will then simply translate these reports into simplified information to be sent to the remote assistance application.
This first feature is already very interesting, as it enables a vertical application to easily exchange and make requests to the 5G network. However, the added value of Immersion’s Network App goes one step further. It also includes a module that suggests to the vertical application how to adapt to the current quality of service. Thus, if the 5G network is slightly degraded, the Network App may propose to temporarily interrupt the synchronization of virtual objects shared between the two users, in order to reserve available bandwidth for audio and video streams. In the event of severe network disruption, only audio will remain active until quality of service returns to normal.
The benefits of this Network App feature are twofold. Firstly, it offers flexibility to the developer of the vertical application, who is not only informed of the state of the network, but is also offered more or less critical adaptations. Secondly, these adaptations are transparent to end-users. There’s no need for them to intervene, and everything can be automated to adapt smoothly to network disruptions, keeping collaboration as efficient as possible.
Real-life testing
EVOLVED-5G was the ideal opportunity to carry out real-life tests with a 5G network. In particular, Immersion made several visits to the Malaga campus to test the performance of its remote assistance application coupled with the Network App. To do this, we already had to investigate how to connect the Hololens 2 headset to a 5G network. Our application needed real-time aspects (Time Sensitive Networking or TSN), which require special infrastructures. The Network App also needed to be deployed on the University of Malaga’s Kubernetes infrastructure. This was an opportunity for the R&D team to get to grips with technologies such as Docker.
The tests carried out showed not only the feasibility of our approach, but also its relevance. Although it was necessary to adjust the architecture of the TSN part so that the Hololens could connect to it, 5G performance was on target for our AR application.
All that remained was to pass the two EVOLVED-5G pipelines (Validation and Certification) in order to offer our Network App on the project’s MarketPlace.
Here is the video on the Evolved 5G project
Watch the video at the beginning of the Evolved 5G project
BONUS: We’ve won a European project on 6G, due to start in January 2024. This will be an opportunity for us to test and validate its immersive applications in different types of services and deployments, subject to various threats and attacks.
