Digitale Schiene Deutschland and NVIDIA collaborate on a digital twin of the rail network
In order to be able to drive fully automatically, sensor technology in conjunction with AI must be able to record and interpret the track environment and obstacles in the future. The number of possible situations in railway operations is very high and cannot be fully recorded in practice. With an exact digital twin of the rail network, it is possible to simulate a wide variety of situations in any number and to train the AI in a targeted manner — detached from the availability of real railway tracks or vehicles.
For fully automated driving at the highest level of automation, GoA4 (driverless driving), sensor-based perception systems for the detection of the track environment as well as objects on the track are of central importance. In practice, these systems consist of a combination of different sensors, such as stereo or infrared cameras, lidars and radars, mounted on the front of the vehicle. In combination with intelligent software, these systems must also be able to interpret what they detect sensorially. In fully automated driving, they must ultimately take on complex tasks, such as route monitoring during train journeys and intelligent decision-making in the event of irregular occurrences. Only then can the traction unit be enabled to react independently to special incidents, such as the initiation of emergency braking. For the Sensors4Rail project, Digitale Schiene Deutschland has been testing the performance of such systems very intensively since 2019 with a multiple-unit train specially equipped for this purpose (see Fig. 1).
While the development of sensor technology is already advanced, the development of AI-based functions is still in its infancy. The number of possible situations that will have to be mastered by an AI in the railway operation of the future is very high: they range from everyday to very rare events in a wide variety of weather, daytime and seasonal conditions and at any location in the German rail network with its approx. 33,000 km length. In order to develop such an AI for environment perception, very large amounts of data are needed. These can be obtained on the one hand by recording real sensor data and on the other hand by simulating artificially generated data. The use of simulated data is essential, as it is impossible in practice to cover all conceivable events and special cases with recordings.
For the simulation, it is necessary to create an exact digital twin of the environment of the railway lines and stations. This makes it possible to provide data for AI development based on an extensive catalogue of scenarios in a wide variety of forms. The decisive factor here is a very high degree of fidelity to reality of the simulated data. Detached from the availability of the real railway lines, the railway sector can then let the AI for a fully automated system learn virtually and purposefully in the digital twin, i.e. significantly faster than in the vehicle. The learning speed of the AI is essentially only limited by the available computing power.
For this project, Digitale Schiene Deutschland worked with NVIDIA’s state-of-the-art technology to enable the railway sector to accelerate the development and training of AI. NVIDIA provides specialized computing systems for training AI models (NVIDIA DGX systems) and accelerating photorealistic simulations (NVIDIA OVX) powered by NVIDIA Omniverse, a highly developed real-time 3D simulation platform for the creation and operation of digital twins. Digitale Schiene Deutschland is responsible for the actual design, the application, as well as for ensuring and applying a qualified process for the creation of the digital twin in cooperation with DB Systel and other development partners specialised in simulation, mapping, 3D modelling and computer graphics.
The basis of the digital twin is a high-resolution digital map that contains very precisely measured geometries of the real routes. During the construction of the digital twin, this map is supplemented with 3D models, textures and material properties. The result is a digital, photorealistic image of the real tracks (see Fig. 2). This contains tracks that run through cities and landscapes, as well as many other details such as precisely reproduced stations, (railway) buildings, infrastructure elements and vegetation. Furthermore, the future sensor set of the trains and the sensor models are exactly replicated in the digital twin, so that physically correct measurement data can be collected in the virtual world for training the AI. The permanent storage and provision of the simulated data is carried out via a so-called data factory, the construction of which has already begun in the first half of 2022.
The next development goal is to make the simulation available in the data factory. The overall development will be implemented in several steps, which will be interlocked with the staggered digitisation of the railway lines.
Currently, the simulation is being built on the basis of the measurement and map data obtained in the Sensors4Rail project, so that a first photorealistic digital twin of a larger subnetwork in Germany is available for training automated driving functions (see Fig. 4).
In the Hanseatic City of Hamburg, between the stations Berliner Tor and Hamburg Bergedorf (the test track of the Sensors4Rail project), numerous irregular events will thus soon take place around the track. But fortunately only purely virtually and with the intention of making the rail traffic of the future more efficient and reliable!
