Sensors4Rail tests sensor-based perception systems in rail operations for the first time
Significant step towards fully automated driving
Thanks to new sensor-based perception systems, trains can now analyse their surroundings and location very precisely. This means they can help to monitor train journeys. Combined with automatic train operation (ATO) technology (see Digital S-Bahn Hamburg), these systems are an important component of fully automated driving. Additionally, essential operational data is collected at high frequency helping to detect disruptions more quickly and trigger a faster response. Thus, train scheduling is optimised making rail traffic smoother and more reliable. Ultimately, this increases capacity of existing train lines.
Sensors4Rail was Digital Schiene Deutschland’s (DSD) first central partnership project – designed to test how environmental perception and train localisation technology interacts with digital maps. To do this, state-of-the-art sensors and powerful computing technology were integrated into an S-Bahn Hamburg test train, and the corresponding software was developed.
This system provides the most accurate possible real-time information about the train's surroundings and position. Through smartly combining this information, it’s possible to detect static and dynamic obstacles on and next to the track, as well as assess hazards. This plays a crucial role in developing assistance systems and preparing for fully automatic operation.
To determine a train’s position to the centimetre, high-precision digital HD maps and modern satellite technology are used. The train locates itself using landmarks in the track environment: such as poles carrying overhead cables, platform edges and other prominent objects. The software compares the landmarks detected with landmarks stored in the HD map and determines the position via the global navigation satellite system (GNSS) and vehicle geometry. This turns the map into a ‘digital twin’ of the rail and its surroundings.
In addition, essential operational data – such as positioning information and obstacles on the track – is collected at high frequency and made available via the cloud. This data can then be used in other use cases, for example to process disruptions more quickly and optimise train scheduling. This makes rail traffic smoother and more reliable and increases track capacity.
Behind the scenes: technology and use cases
This is the front of the Sensors4Rail test train, showing its environmental perception sensors: four radars, four cameras (three in the visible spectrum range and one in the mid-infrared) and six Lidar sensors. There’s also an inertial measurement unit (not visible) that assesses the vehicle’s own movement, installed inside the train, next to the Lidar sensors.
Inside, there are also high-performance server systems with several graphics processing units (GPUs), a data storage unit and a 10 GB network that transmits data between components – many of which are being used in rail operations for the first time.
The numbers alone illustrate the complexity: a total of around 4.3 km of cables were laid for the Sensors4Rail prototype, with a train length of 60 m. Meanwhile, 45 special hardware components and 3,500 small parts were installed.
A DB-internal cloud environment (DB Modular Cloud) that includes a digital map is also part of the system.
This image shows the hardware and software used in the test vehicle. The train uses built-in sensors to record the environment and simultaneously receives GNSS signals. Communication between the cloud and the train is provided by a prioritised LTE/4G connection from partner, Vodafone.
By integrating the three subsystems – environmental perception, localisation and digital maps – the following use cases were implemented and tested in the Sensors4Rail project:
- Landmark detection: detection of landmarks and comparison with digital maps for accurate positioning of the front of the train.
- Train detection: detection and classification of trains on neighbouring tracks to avoid their classification as a danger or obstacle.
- People detection: detection, classification and positioning of people on the platform to ensure a quick response in the event of an emergency.
- Track detection: detection of own track and neighbouring tracks, and comparison with digital maps to position objects relative to the track.
- Consolidated position of train front: fusion of all localisation and track information into a consolidated track position for highly accurate real-time localisation – without the need for additional infrastructure elements.
- Track clearance detection: monitoring of the volume inside and outside the clearance profile of the train’s own track and neighbouring track, to enable fast response to unknown obstacles.
A greater understanding of the Sensors4Rail project
What are the goals of this collaborative project and which partners were involved in the initial phase? Learn more about Sensors4Rail in this video:
Phase 1: from theory to track
To realise the Sensors4Rail project, cutting-edge technology from different industries needed to be integrated into the railway system. In the first phase, DSD worked together with four partners from complementary industrial sectors:
● Siemens Mobility supported the vehicle integration, and was responsible for system tests and positioning the front of the train via a modern odometry unit that integrated the GNSS, vehicle odometry and landmark positions.
● Partners Bosch Engineering (radar, long- and mid-range infrared and stereo camera) and Ibeo Automotive Systems (Lidar sensors) were responsible for realising environmental perception and object detection.
● HERE Technologies provided 3D HD maps to enable centimetre-precise location of the train on the integrated DB track topography.
● Deutsche Bahn was responsible for project management, vehicle engineering, approval, safety and providing vehicles and infrastructure.
The premiere: ITS World Congress 2021
Sensors4Rail was launched at the Intelligent Transport Systems (ITS) World Congress 2021 in Hamburg. A test vehicle equipped with sensors ran for 23 kms along the S21 line between Hamburg-Berliner Tor and Bergedorf in special journeys – testing sensor-based perception systems in operation for the first time.
Passing with flying colours: around 400 trade fair attendees experienced live animation of the special journeys on video screens in the newly equipped train. At the same time, a moderated livestream was broadcast to several hundred trade fair visitors at the DB stand at the Hamburg Exhibition Centre. The unique challenge here was to present such technical content in such a way that sensor-based perception systems, including subsystems and tested functionality, could be experienced and understood.
Testing, conversion, premiere: Sensors4Rail project phase 1
How was this made possible? A virtual image of reality was created, with modern graphics, and fed with live data from the Sensors4Rail system. The visualisation was based on track and station data drawn from high-precision surveying of the line. The special journeys were also streamed live on the internet and moderated separately – and in this way made accessible to a large number of interested parties beyond the trade fair.
Want to virtually experience a Sensors4Rail test drive? Climb on board and watch the recorded livestream.
Phase 2: testing in day-to-day operation
Sensors4Rail is entering its next phase. Numerous research runs are planned from 2022-23. In this second phase, the focus will be on data acquisition. By expanding the test area to other routes in the Hamburg S-Bahn network, valuable insights can be gained into how the functionalities perform in different scenarios and route sections.
In addition, the focus is on increasing data diversity. This includes adding new test routes as well as recording raw sensor data in a wide range of weather conditions and times of day. Gathering this raw data enables continuous development of the system through repeated training of its neural networks. A detailed analysis on the correlation between the amount of test/training data and the performance of the AI system is necessary – to assess how much work will be needed to develop, test and introduce AI functionality for operational use. Phase 2 is comparable to the first test drives of sensor-equipped cars for autonomous driving on the road – however, on the tracks the driver will continue to have full control of the vehicle. The Sensors4Rail system runs along ‘silently’ and learns.
Digitale Schiene Deutschland is delighted that this cross-industry cooperation can continue into its second phase. All partners from phase 1 will also be on board in phase 2 – and will continue contributing their expertise to further the introduction of sensor-based perception systems in the rail network.
A successful project comes to an end
On April 6, the time arrived: The Sensors4Rail test vehicle "Santana" was successfully transferred from Hamburg to Berlin Spandau to Havelländische Eisenbahn (HVLE) and subsequently decommissioned. With this step, the 15-month test and data collection in the network of the S-Bahn Hamburg and thus the last phase of the project was successfully completed.
After four years, Sensors4Rail, one of the most successful research and development projects of Digitale Schiene Deutschland so far, has come to an end. The collected findings from the individual project phases as well as the data from the test runs are now being incorporated into the specification and development of the next generation of sensor-based environment recognition, localization and digital map. Together, these functionalities for the rail sector form an important basis for fully automated driving on rails.
More information about the project completion and collected findings can be found here.
