Development of Low-cost Train Patrol Support Method Using Smartphone

Abstract

Regional railway companies are facing difficult business conditions. However, railway facilities and rolling stock inspections and maintenance are still required to ensure safe and stable train operations. This study introduces the development of a smartphone-based train patrol support application as a low-cost track condition management method that can be introduced even by regional railway companies. Test measurements were made using the application on a commercial line. We also investigated possible uses of the measurement data. Results showed that acceleration data are effective for train vibration management, and forward view video data are effective for understanding track conditions during desktop reviews.

1. Introduction

Railway operators are facing a tough business environment as the number of railway users and railway employees decreases due to the declining birthrate and an aging population. This trend is particularly noticeable for regional railway operators, with approximately 90% of regional railway operators having current account deficits [1]. Furthermore, many regional railway operators also face the deterioration of their railway vehicles and facilities [1]. Notwithstanding, railway operators still need to inspect and maintain railway facilities and railway vehicles to operate railway vehicles safely and reliably.

To support operators, railway digital transformation (DX), a business transformation in the railway industry using digital technology, has been promoted over the past few years. Measures include, for example, the use of mobile information terminals such as smartphones (hereinafter referred to as “smartphones”). Smartphones are equipped with many built-in sensors and general-purpose products. They are also easy to procure and relatively low-cost compared to dedicated devices. Finding a way to use these devices to support railway maintenance and management, could lead to lower maintenance costs. Making full use of digital technology could also help save labor and provide tools to facilitate maintenance and management work that has traditionally relied on visual inspections and experience. Smartphones for maintenance and management of public infrastructure is already actively being promoted in the road sector. For example, they are already used by road patrols and for road surface condition management [2].

In the railway industry, several studies have been conducted on the use of smartphones for track maintenance and management [3, 4]. However, there are no examples yet of widespread implementation to support train patrols. This study addresses this gap. Using train patrol requirements set out in the Maintenance Standards for Railway Structures (Track Part) [5], we investigated a method designed to support train patrols using various sensors on smartphones, developed a dedicated train patrol support application software, and report in this paper the results of trials on an actual railway line.

2. Train patrol method using smartphone

2.1 Roles and issues of train patrols in the Maintenance Standards for Railway Structures (Track Part)

The “Maintenance Standards for Railway Structures (Track Part) [5]” (hereinafter referred to as the “Maintenance Standards”) issued by the Director-General of the Railway Bureau of the Ministry of Land, Infrastructure, Transport and Tourism (MLIT) in January 2007, states that track patrols should “gather an overall understanding of the condition of the tracks.” It also states that track patrols should conduct this work “on foot, by train, or by track motor car, etc.”

Furthermore, the “Guidance of Maintenance Standards for Railway Structures (Track Part) [6]” (hereinafter referred to as the “Guidance of the Maintenance Standards”) published by the Railway Technical Research Institute in March 2007 gives examples of what train patrols should look for. For example, “the presence or absence of abnormal train vibrations, the presence or absence of creaking noises, and the presence or absence of disruption to train operations.”

In most cases, actual train patrols are conducted by railway staff who ride at the front of a commercial train, visually and physically checking a wide range of items, including the examples mentioned above, and record them in a field notebook. In addition, some railway operators conduct train patrols during extremely hot periods to detect signs of significant track irregularity, which is labor intensive. In this context, there is a need to consider ways to save labor and improve train patrol efficiency.

2.2 Overview of developed train patrol support application software

We set out to develop a low-cost method of supporting train patrols that can be introduced in regional railways, taking into account the roles of train patrols as defined in the aforementioned Maintenance Standards and the Guidance of the Maintenance Standards as well as their actual working conditions and status. Specifically, the method uses a smartphone installed with a dedicated application software (hereinafter referred to as the “app”) developed for supporting train patrols. Digital data is used to facilitate the work of train patrols. For example, videos or images are used for the wide range of items that require visual checking, and acceleration data is used to check train vibrations traditionally checked by body sensory information.

Figure 1 shows the measurement screen of the train patrol support app (Train Patroller) [7]. This app runs on a smartphone equipped with Apple's iOS and is designed for easy operation. Currently, railway staff install the smartphone on board the railway vehicle during train patrols and start and stop measurements.

Fig. 1　Measurement screen of train patrol support app

Table 1 shows the main measurement items of this app. A GPS receiver built into the smartphone enables measurement of moving speed, latitude, and longitude; motion sensors allow measurement of three-axis acceleration and angular velocity, a rear camera can be used to record video, and audio is recorded through a microphone. There are three measurement modes: “Vibration,” “Vibration & Video,” and “Video,” and measurement items can be selected depending on the purpose. For video, the highest setting allows measurement at 60 fps/4K resolution. To reduce blurring in video measurement mode, iPhone 14 or later is recommended.

Table 1　Main measurement items of train patrol support app

Sensor	Measurement item	Measurement mode			Sampling, etc. (file format)
		Vibration	Vibration & video	Video
GPS receiver	Moving speed	Yes	Yes	Yes	1 Hz (text format)
	Latitude/Longitude	Yes	Yes	Yes
Motion sensor	Three-axis acceleration	Yes	Yes	-	100 Hz (text format)
	Three-axis angular velocity	Yes	Yes	-
Camera (rear camera)	Video	-	Yes	Yes	10/20/30/60 fps VGA/HD/FullHD/4K (mp4 format)
Micro-phone	Sound	Yes	Yes	Yes	16 kHz (m4a format)

2.3 Method for installation on railway vehicles

Figure 2 shows how to install the smartphone on a railway vehicle. Figure 2 (a) illustrates measurement of vehicle acceleration for managing train vibration. The Guidance of the Maintenance Standards [6] states that “measurement should be taken on the floor on the front bogie at the very front of the train or the rear bogie at the very rear of the train. To ensure accurate measurement, the accelerometer should be placed on a horizontal surface and straight in the train direction.” However, in practice this position is not suitable for making measurements because the GPS reception sensitivity is low, and in commercial trains, the area directly above the bogie is the passenger compartment.

Fig. 2　Example of smartphone installation on railway vehicle

Figure 2 (b) illustrates the proposed alternative. The smartphone is installed on the windshield surface inside the driver’s cab at the front of the vehicle. By installing it this way, the GPS increases in reception sensitivity and the smartphone is able to capture the forward view video as well. It has been confirmed that the acceleration when the smartphone is installed at this position is about 5 to 15% higher than when it is installed directly above the bogie, although there is variation depending on the vehicle type [8]. In addition, by using a fixture, it is possible to set the smartphone with an angle of dip, making it possible to adjust the field of view of the forward view video depending on the purpose of inspection. Since the installation angle of the smartphone, including angle of dip, affects not only the field of view of the forward view video but also the evaluation of train vibration, it can be checked using the “spirit level” function shown in Fig. 1 when installing the smartphone, and is automatically recorded when measurement starts, and can be used to correct train vibration, which will be described later. Furthermore, the “device temperature” display makes it possible to monitor the internal temperature of the smartphone and prevent measurement abnormalities caused by thermal runaway.

2.4 Method for processing measurement data

The data acquired by the developed train patrol support app is saved in a common file format as shown in Table 1, so it can be displayed and processed by various software. Figure 3 shows an example of data processing procedure. First, the measurement data is imported into a PC using Apple's media player “iTunes” in a Windows OS environment. Next, since it is known that there are minute sampling fluctuations in the acceleration and angular velocity data, and GPS-related data recorded by the smartphone, the data is converted into intermediate data using dedicated software “Train Patroller Resampler” to correct these fluctuations [8]. Then, the track maintenance management database system “LABOCS ”[9] is used to perform filtering, time/distance conversion, kilometer distance assignment, and creation of a significant value list. This makes it possible to display charts in the LABOCS waveform viewer. LABOCS is a signal processing software developed by the Railway Technical Research Institute that specializes in processing track-related data, and has been adopted by many railway operators. On the other hand, the video data can be viewed as is in a general-purpose video viewer, or it can be viewed by overlaying subtitle information created in LABOCS and assigning kilometer distances. In addition, it is possible to create a bird’s-eye image by projective transformation processing [10] and view it with a general-purpose image viewer.

Fig. 3　Example of how measurement data is processed

3. Test implementation on commercial line

3.1 Overview of railway line and measurement method

The developed train patrol support app was installed on an Apple smartphone, iPhone 14 Pro, and test measurements were performed on an actual railway line. The test line was a single-track, electrified line with mostly ballast track. The annual passing tonnage on the test section was approximately 7 million tons, and the maximum operating speed on the line was 130 km/h.

Figure 4 shows the installation of a smartphone on a commercial train (express train). In this example, three smartphones were used for comparison and verification. Smartphones A and B were installed on the glass surface of the vehicle front door, and smartphone C was installed on the floor of the vehicle front area. Smartphone A was fixed directly to the glass surface with double-sided tape at an angle of dip 0° to check the situation of the track and its surrounding wide area. Smartphone B was fixed at an angle of dip 28° using a commercially available suction cup with sufficient rigidity to check the situation of the track. The forward view video was acquired with a resolution of 4K (3840 × 2160) and a frame rate of 30 fps. Smartphone C was also fixed directly to the floor with double-sided tape for comparison. It should be noted that since the test vehicle area directly above the bogie was inside the passenger compartment, measurement with the smartphone fixed on the bogie was not performed.

Fig. 4　Example of smartphone installation on a commercial train

3.2 Example of using acceleration to manage train vibration

Acceleration data acquired by the train patrol support app can be used to manage train vibration. Figure 5 shows an example of track irregularity waveforms measured in the same section as the train vibration measurement under the conditions shown in Fig. 4 at a similar time. The acceleration is shown as a waveform processed with an 8 Hz low-pass filter based on the Guidance of the Maintenance Standards [5]. It can be seen from the figure that vertical acceleration tends to be large in areas with large longitudinal level, and lateral acceleration tends to be large in areas with large alignment. Moreover, comparing the waveforms of smartphones A, B, and C, it can be seen that there is no clear difference in the waveforms depending on the installation position, installation method, and angle of dip of the smartphone. In addition to the horizontal alignment, the figure also shows the yaw angular velocity measured simultaneously by the smartphone for reference. The phase of the horizontal alignment and the yaw angular velocity are well matched, which shows that the time-distance conversion and kilometer-distance calculation method shown in Fig. 3 have sufficient kilometer-distance calculation accuracy for practical use.

Fig. 5　Example of waveforms for acceleration and track irregularity

Figure 6 shows an example of the power spectral density (PSD) of acceleration calculated using the data of the train section shown in Fig. 5. The average speed of the train in the analyzed section is approximately 100 km/h. Figure 6 (a) shows that the PSD of the vertical acceleration is almost the same for smartphone A and smartphone C, so the influence of the installation position is considered to be small. On the other hand, when comparing smartphone A and smartphone B, the PSD of smartphone B is smaller overall. We infer that this is due to the fact that the observation axis of the acceleration of the smartphone was shifted from the vertical direction because smartphone B was fixed with an angle of dip as shown in Fig. 4. Next, Fig. 6 (b) shows that the PSD of the horizontal acceleration is almost the same for smartphone A and smartphone B, so the influence of the angle of dip is considered to be small. On the other hand, when comparing smartphone A and smartphone C, some differences are seen. This is thought to be due to the influence of the difference in the installation position as shown in Fig.4.

Fig. 6　Comparison of acceleration PSDs (for section with speed of about 100 km/h)

Figure 7 shows an example of a comparison of the total amplitude of acceleration calculated using data including the section shown in Fig. 5. In this figure, the peaks of the total amplitude of smartphone C with 1.5 m/s² or more are extracted and plotted for both vertical and lateral acceleration. Figure 7 (a) shows that the total amplitude of vertical acceleration is roughly the same for smartphone C and smartphone A. Comparing smartphone C and smartphone B, the total amplitude of smartphone B tends to be about 15% smaller than that of smartphone C. We infer that this is due to the influence of the angle of dip of smartphone B. Next, Fig. 7 (b) shows that the total amplitude of lateral acceleration is generally smaller and has a large variation compared to vertical acceleration, but smartphone A and smartphone B tend to be about 3 to 4% smaller than smartphone C. We infer that this is due to the characteristics of the vehicle structure and the influence of the smartphone installation position.

Fig. 7　Comparison of total acceleration amplitudes

Therefore, for the vertical acceleration of smartphone B, which differed significantly from smartphone C, we considered correcting the vertical acceleration by vector synthesis of X-axis acceleration and Z-axis acceleration using the angle of dip of smartphone B. Figure 8 shows a comparison between the vertical acceleration of smartphone C and the vertical acceleration corrected by vector synthesis of the acceleration of smartphone B. As can be seen from the figure, the difference between the two is almost eliminated, and it was found that, even if a smartphone is installed with an angle of dip, it can be used to manage vertical vibration. In addition, when using the acceleration data obtained by this method for train vibration inspection, we consider that it is possible to correct the difference between the acceleration measured at the front of the train and directly above the bogie by performing a comparative measurement when introducing this method.

Fig. 8　Vertical acceleration corrected by vector synthesis

3.3 Example of using forward view video to check track conditions

The forward view video data acquired by the train patrol support app can be used for desktop track patrols. For example, it can be used to check the track condition at points where train vibration exceeds the standard value. Figure 9 shows an example of a forward view image of a mud pumping area extracted from a forward view video acquired under the conditions shown in Fig. 4. The upper part of the image in Fig. 9 (a) shows the kilometer distance, train speed, and total amplitude of vertical and horizontal vibrations as subtitle information. It can be seen that the train was traveling at about 98 km/h in the relevant section. The yellow trapezoidal dashed line in each figure is the approximate region of projective transformation. The forward view image of Smartphone A installed with an angle of dip 0° shown in Fig. 9 (a) is suitable for checking the overall track conditions. On the other hand, the forward view image of smartphone B installed with an angle of dip 28° shown in Fig. 9 (b) has a field of view too narrow to check the entire track, but the area around the track is magnified making it possible to check the track condition in more detail than in Fig. 9 (a).

Fig. 9　Example of forward view video of mud pumping area

Figure 10 shows the results of projective transformation of each of the forward view images shown in Fig. 9 to create a bird’s-eye image. Each figure also shows an enlarged image focusing on the same rail fastening system. The overhead image of Smartphone A shown in Fig. 10 (a) includes the track periphery, and the area of the track occupies a small portion of the entire image. On the other hand, the bird’s eye image of smartphone B shown in Fig. 10 (b) has a wide area of the entire image occupied by the track, and the image distortion is smaller than that of Fig. 10 (a), so the details of the track, such as the rail fastening system, can be inspected.

Fig. 10　Example of bird's-eye image created by projective transformation of forward view image

It is recommended to set the angle of dip of the smartphone appropriately depending on the range and object to be checked. Forward view videos, forward view images extracted from them, and bird’s-eye images obtained by projective transformation of these videos have the potential to be used not only to support train patrols, but also to complement or replace foot patrols.

The analysis time required to extract a forward view image from a forward view video and obtain a bird’s-eye image by projective transformation depends on the specifications of the PC used. In this study, we used a general-purpose PC to extract images of all frames of the video, which took approximately three times the video playback time.

4. Conclusions

In this study, we developed a train patrol app that utilizes various sensors of a smartphone and investigated a method of using this app to support train patrols, and conducted trials on an actual railway line. The findings are as follows:

・Considering the requirements for train patrols stipulated in the Maintenance Standards (Track Part), we developed a train patrol support app that runs on a smartphone. The developed train patrol support app is easy to operate and can measure train speed, acceleration, angular velocity, forward view video, etc. by utilizing various sensors and cameras built into the smartphone. At the highest setting, the video can record the measurement results at 60 fps/4K resolution.

・Using the developed app, we carried out test measurements during train patrols on a commercial line, and examined possible uses of the obtained data. We found that acceleration data can be used to manage train vibrations by correcting acceleration, even when the smartphone is fixed with an angle of dip. Similarly, for forward view video data, we developed a method to add kilometer distance, acceleration values, etc. to forward view videos as subtitle information, and carried out projective transformation of the forward-facing images extracted from the video to create a bird’s-eye image, demonstrating that images of the track as seen from directly above can be reproduced. These results are expected to make it easier to carry out desktop checks of track conditions.

Finally, although the technology described in this paper has reached a certain level of practical application as a train patrol support tool, there is still room for further research, such as automatic measurement functions, automatic track anomaly detection technology, and the development of a server system for data processing and viewing. We are currently working on developing these technologies and plan to report on them in due course.

Acknowledgment

We would like to express our gratitude to all the railway operators who offered their cooperation in the trial of the train patrol support method proposed in this study on commercial lines.

References

• [1]  Railway Bureau of Ministry of Land, Infrastructure, Transport and Tourism, “地域鉄道の現状,” https://www.mlit.go.jp/tetudo/content/001573729.pdf (Reference data: July 1, 2025) (in Japanese).
• [2]  Zhao, B., Nagayama, T. and Xue, K., “Road profile estimation, and its numerical and experimental validation, by smartphone measurement of the dynamic responses of an ordinary vehicle,” Journal of Sound and Vibration, Vol. 457, pp. 92-117, 2019.
• [3]  Shinoda, N., Tsunashima, H., Matsumoto, A., Sato, Y. and Ogata, S., “Railway track management based on vibration data of in-service train measured by a mobile device,” Transactions of the JSME (in Japanese), Vol. 88, No. 911, No. 21-00375, 2022 (in Japanese).
• [4]  Ikeuchi, M. and Tanaka, K., “Development of track maintenance system using smartphones,” Japanese Railway Engineering, Vol. 63, No. 2, pp. 8-10, 2023.
• [5]  Ministry of Land, Infrastructure, Transport and Tourism, “Maintenance Standards for Railway Structures (Tracks Part) ,” 2007 (in Japanese).
• [6]  Railway Technical Research Institute, “Guidance of Maintenance Standards for Railway Structures (Tracks Part) ,” 2007 (in Japanese).
• [7]  Tanaka, H., Zhao, B., Su, D. and Nagayama, T., “Study on practical application of low-cost train patrol support method for railway track maintenance using a smartphone,” Journal of AI data science, JSCE, Vol. 5, No. 1, pp. 56-65, 2024 (in Japanese).
• [8]  Tanaka, H., Su, D. and Nagayama, T., “Fundamental study of low-cost track condition monitoring by using smartphones and tablets,” Journal of railway engineering, JSCE, No. 22, pp. 39-46, 2018 (in Japanese).
• [9]  Yoshimura, A., Yoshida, Y. and Kamiyama, M., “Development of database system for railway track maintenance management, micro LABOCS for Windows and its features,” QR (Quarterly Report of RTRI), Vol. 38, No. 2, pp. 102-106, 1997.
• [10]  Nagamine, N., Goda W., Maeda, R., Tsubokawa, Y., Kato, S. and Itoi, K., “Inspection method of track facilities using image analysis taken from front of trains,” QR (Quarterly Report of RTRI), Vol. 64, No. 3, pp. 205-210, 2023.

Authors

	Hirofumi TANAKA, Dr.Eng. Manager, Track Geometry & Maintenance Laboratory, Track Technology Division Research Areas: Track Condition monitoring, On-board Measurement, Track Maintenance
	Boyu ZHAO, Ph.D. CEO, SMARTCITY RESEARCH INSTITUTE Co., Ltd. Research Areas: Track Monitoring, Inverse Analysis, Structural Dynamics
	Di SU, Ph.D. Associate Professor, Department of Civil Engineering, The University of Tokyo Research Areas: Structural Simulation, Dynamics of Transportation Infrastructures, Reliability Design and Risk Analysis
	Tomonori NAGAYAMA, Ph.D. Professor, Department of Civil Engineering, The University of Tokyo Research Areas: Bridge and Vehicle Dynamics, Inverse Analysis, Infrastructure Monitoring