Sensors4Rail

Sensors4Rail started as a joint development and innovation cooperation between the Digitale Schiene Deutschland sector initiative and industrial partners from various sectors, who were recruited for the project as part of a tender in 2019. In the project consortium, Deutsche Bahn was responsible for project management, providing the train and track as well as the work packages IT security, train engineering and train conversion. Siemens Mobility was responsible for the localization and integration test work packages. Bosch Engineering and MicroVision collaborated on the environment perception work package and HERE Technologies was responsible for the digital map work package. Deutsche Bahn was also responsible for testing, integration test and data management during the long-term test operation from 2022.

Video: The Sensors4Rail Project

System overview

The Sensors4Rail system is based on a train from the second series of the 472 series (472 061), which was equipped with modern sensor technology and powerful network and computer units. DB Systemtechnik acted as the rail transport company for the test and demonstration runs in Hamburg and provided both test managers and train drivers.

In addition to the train and the hardware installed on it, an internal DB cloud environment (DB modular cloud) is part of the Sensors4Rail system. This is necessary for localizing the train. For this purpose, important operational data is collected at high frequency (e.g. position data or obstacles on the track) and made available via a cloud interface. In this project, a prioritized LTE/4G connection from the partner Vodafone ensured communication between the cloud and the train. The data can be used to process disruptions more quickly and optimize train scheduling. This makes rail traffic smoother, more reliable and increases line capacity. Not shown is the separately developed, infrastructure-side Data Factory of Digitale Schiene Deutschland, which records the data collected in the project and makes it available for analyses and further projects.

Project implementation and train integration

An important prerequisite for the trial operation of the Sensor4Rail system was the non-intrusiveness of existing systems, such as the train control or control and safety technology. The train driver therefore remained fully responsible for observing and carrying out the journey in this first test step. In the future, however, technical systems will take over route monitoring and hazard assessment. A particular challenge in the Sensors4Rail project was the train integration of the new hardware components. In addition to sensors such as cameras, radar (radio detection and ranging) and Lidar (light detection and ranging), these also include high-performance server systems with several GPUs (graphics processing units), a data storage unit and the use of a 10 Gbit network for data transmission between the components - many of which are being used in rail operations for the first time.

In total, around 4.3 km of cable was laid over a train length of 60 m for the Sensors4Rail prototype, as well as 45 special hardware components and 3,500 small parts. These figures impressively show what was necessary to demonstrate an integrated system of environment perception, localization and digital map on a suburban train in real time for the first time.

Train front of the Sensors4Rail project

The sensors for environment perception are clearly visible at the front of the train: four radars, four cameras (three of which are in the visible spectral range and one in the mid-infrared range) and six Lidar sensors. The inertial measurement unit for determining the train's own movement, which is installed on the inside next to the Lidar sensors, is not visible.

Inside of the train

Inside the train are the high-performance server systems with several graphics processing units (GPUs), a data memory and a 10Gbit network for data transmission between the components - many of which are being used in rail operations for the first time.

By integrating the three subsystems environment perception, localization and digital map, the following six use cases were prototypically implemented and tested in the Sensors4Rail project:

1. Landmark detection: Detection of landmarks and comparison with the digital map to determine the exact position of the front of the train. 
    
2. Train detection: Detection and classification of trains on adjacent tracks so that they are not identified as a danger or obstacle.
    
3. Person detection: Detection, classification and positioning of people on the platform in order to be able to react in an emergency.
    
4. Track detection: Detection of own track and adjacent tracks as well as a comparison with the digital map in order to be able to position objects relative to the track.
    
5. Consolidated train front position: Merger of all localization and route information into a consolidated position on the track for high-precision real-time localization - without additional infrastructure elements
    
6. Track clearance detection: Monitoring of the volume inside and outside the clearance limits of your own track and the adjacent track enables you to react to unknown obstacles.

Continuous test operation and data transfer to the "Data Factory" of Digitale Schiene Deutschland

In the second phase of the project, which ran from January 2022 to June 2023, the test area was extended to other routes in Hamburg's rapid transit line network. This provided valuable insights into the performance of the functions in different scenarios, environmental conditions and route sections. The raw data from the sensors serves as the basis for the continuous further development of the system through repeated training of the neural networks used. They are required to prepare artificial intelligence for driverless, fully automated driving (GoA4). This sensor data was fed into the "Data Factory" of Digitale Schiene Deutschland together with functional data from the system and additional diagnostic data.

In addition to the project objectives already outlined, the focus was also on the live demonstration of the project results at the ITS World Congress 2021 in Hamburg. The particular challenge here was to visualize the highly technical content in such a way that the functionality of sensor-based perception systems could be understood and at the same time experienced by trade fair visitors. Based on a modern graphics environment, a virtual image of reality was created and fed with live data from the Sensor4Rail system.

Find out more about the project presentation at the ITS World Congress in the video:

On April 6, 2023, the Sensors4Rail test train "Santana" was transferred from Hamburg to the Havelländische Eisenbahn (HVLE) in Berlin Spandau and then decommissioned. This step marked the successful completion of the 15-month test and data collection in the Hamburg S-Bahn network and thus the final phase of the project. After running for a total of four years, Sensors4Rail, one of the most successful research and development projects of Digitale Schiene Deutschland to date, has come to an end. The findings gathered from the individual project phases and the data from the test runs are now being incorporated into the specification and development of the next generation of sensor-based environment detection, localization and digital maps. Together, these functionalities form an important basis for fully automated rail travel in the rail sector.

You can find more information about the completion of the project and the findings here.

Building on the findings of Sensors4Rail and Digital S-Bahn Hamburg, the AutomatedTrain research and development project was launched in July 2023. Over the next three years, the project will test the fully automated provisioning and parking of trains.