Deep learning-based tornado vortex detection through the BRIDGE program: advancing technology and multidisciplinary applications

Abstract

This paper describes the development of the “Collaboration with Startups for Localized Severe Weather Countermeasures: Building a Real-time Disaster Prevention Field using AI” project, which is positioned as part of the BRIDGE program. The core technology used in this BRIDGE project is based on collaborative work between the Meteorological Research Institute and East Japan Railway Company, which involves using deep learning to automatically detect low-level rotational airflows associated with wind gusts in high-resolution radar data. The key objectives of the BRIDGE project are to enhance the existing deep learning models for accurate tornado vortex detection, expand the application scope beyond railway operations to broader sectors by integrating GPS location data, and foster industry- academia-government collaboration, including partnerships with startups, for efficient technology development and practical implementation. The paper outlines the principles of observing tornado vortices using Doppler radar, the construction of deep learning models for detecting tornado vortex patterns, and the processing flow and application examples in train operation control, building upon our previous work presented in Kusunoki et al. (2022). It also provides an overview of the BRIDGE program and the positioning of this BRIDGE project within it, highlighting the industry- academia-government collaboration system and the involvement of startup companies. The initial results of the project are presented, including the development of advanced deep learning models for tornado vortex detection, comparing their performance against the VGG model which we previously developed, and the efforts towards building a real-time disaster prevention information dissemination system integrated with GPS data. The paper concludes by discussing the expected future developments, academic insights, and societal impacts of this research, which aims to strengthen resilience against localized meteorological disasters while contributing to the advancement of tornado research.

1. BACKGROUND AND PURPOSE OF THE STUDY

Meteorological disasters pose a significant problem with the potential to severely impact societal safety and economic activities. Examining recent instances of meteorological disasters reveals a variety of forms including landslides due to heavy rains, storm surges and violent winds caused by typhoons, and tornadoes among others, each causing loss of life and severe damage to social infrastructure. Thus, appropriate responses to meteorological disasters are crucial for protecting the lives and property of citizens and for the realization of a sustainable society. In particular, localized and rapidly developing wind gusts, such as tornadoes, are extremely difficult to predict, making rapid detection and real-time information provision essential. These localized and rapidly developing weather events can cause extensive damage in a short period, necessitating early detection and improved forecasting accuracy to prompt evacuation and take appropriate action. Therefore, developing effective countermeasure technologies for these meteorological disasters is considered a critical task for enhancing societal safety. This project, “Collaboration with Startups for Localized Severe Weather Countermeasures: Building a Real-time Disaster Prevention Field using AI,” is part of the BRIDGE program, which aims to bridge the gap between R&D and an ideal society (details in Chapter 3). This project, hereafter referred to as the BRIDGE project, emphasizes collaboration with startups specializing in IT, meteorological data analysis, and disaster prevention.

The clouds that generate tornadoes and bring about wind gust damages are cumulonimbus clouds. Cumulonimbus clouds, characterized by strong upward air currents, develop vertically and have a horizontal spread of approximately 10 km, with a lifespan of about 10 minutes to one hour. Tornadoes, being smaller in spatial scale than cumulonimbus clouds and rapidly developing within a short time, are difficult to accurately predict in terms of location and timing.

Disaster prevention information aimed at capturing tornadoes employs two primary methods:

The first method focuses on predicting the likelihood of tornado occurrence over a broad area, providing a broader probabilistic forecast for the entire country. The second method, on the other hand, aims to immediately detect localized, low-level rotational airflows within cumulonimbus clouds that are more directly related to tornado formation. This is particularly useful for specific applications such as railway operation control.

The first method, the Tornado Occurrence Probability Nowcast, issued by the Japan Meteorological Agency¹⁾, estimates the possibility of tornado occurrence over a broad area and time frame. This method covers the entire country, including areas where radar resolution is insufficient to directly detect wind gusts. It expresses the likelihood of a tornado occurring (or already occurring) in two stages. The forecast considers the development level of cumulonimbus clouds and the presence of mesocyclones (cyclonic rotations within the clouds, several kilometers in diameter) as signals of potential tornado occurrence. It should be emphasized that a mesocyclone is a distinct phenomenon from a tornado, and not all mesocyclones are accompanied by tornadoes. It also incorporates surrounding atmospheric conditions to calculate the occurrence probability on a 10 km grid for up to one hour ahead (10 to 60 minutes in the future). When the occurrence probability reaches level 2, a tornado advisory is issued for the region.

The second method, jointly developed by the Meteorological Research Institute and East Japan Railway Company (JR East), uses Doppler radar to frequently scan and automatically detect wind gusts occurring within cumulonimbus clouds within a relatively close range to the radar. This approach focuses on detecting much smaller-scale, low-level rotational airflows directly associated with wind gusts, including tornadoes. It employs deep learning to automatically identify characteristic vortex signatures in high-resolution Doppler radar data within a 60 km range²⁾. These signatures, typically about 1 km in diameter, are more directly linked to tornado formation and occur closer to the ground. This method enables real-time detection, tracking, and trajectory prediction of low-level rotational airflows that may lead to tornadoes. These vortex signatures, which we refer to as “tornado vortices” throughout this paper, provide important information for predicting the occurrence of tornadoes. A detailed explanation of the relationship between tornado vortices and actual tornadoes, considering the limitations of radar observations, is provided in Chapter 2.

The deep learning models developed in this research are trained to identify the specific Doppler velocity patterns associated with tornado vortices, which are typically about 1 km in diameter, as described in the following sections. By focusing on detecting these specific patterns, the technology enables the accurate identification of tornado vortices, which are particularly hazardous and require prompt warning and response.

This paper describes the deep learning technology used in the latter method. Chapter 2, an expanded and revised English version of our prior work on fundamental deep learning techniques for tornado vortex detection, originally published as a technical commentary in Japanese by the Japan Society of Wind Engineering³⁾, presents a comprehensive overview of the foundational concepts that underpin the further advancements discussed in this paper. This chapter has been enhanced with additional explanations and details to provide a more thorough understanding of the underlying principles. The content from the original publication is used in accordance with the Japan Society of Wind Engineering. First, the principles and examples of capturing patterns of low-level rotational airflows (including not only strong vortices, or tornadoes, that cause wind damage, but also weaker vortex-like airflows) with Doppler radar are discussed.

Subsequently, the developed deep learning model is described, along with examples of its application to train operation control. Moreover, we believe that this initiative not only contributes to enhancing the safety of railway operations but also significantly aids in reducing meteorological disaster risks in other sectors of social infrastructure. This paper introduces this new project under the BRIDGE program, which aims to develop an advanced tornado vortex detection system using deep learning techniques. This BRIDGE project builds upon previous collaborative research between the Meteorological Research Institute and East Japan Railway Company on using deep learning to detect tornado vortex signatures from Doppler radar data. The key objectives of this BRIDGE project are:

1. To improve the existing deep learning models for accurate tornado vortex detection by incorporating advanced architectures and techniques.

2. To expand the application scope beyond railway operations to broader sectors by integrating GPS location data and enabling real-time dissemination of localized severe weather information.

3. To encourage industry-academia-government collaboration, including partnerships with startups, for efficient technology development and practical implementation.

This paper outlines the background, objectives, collaboration framework, and initial outcomes of the project in its first year. The document specifically focuses on the development of advanced deep learning models for tornado vortex detection, comparing their performance against the VGG model, which we previously developed, and the efforts towards building a real-time disaster prevention information dissemination system integrated with GPS data. Details regarding the specific hyperparameter settings, architectural choices, training procedures, and performance metrics used for the deep learning models are provided in Section 4.

By advancing tornado vortex detection capabilities and enabling targeted information delivery, this initiative aims to strengthen societal resilience against localized meteorological disasters while contributing to academic insights through large-scale pattern analysis.

2. DEEP LEARNING TECHNIQUE FOR DETECTING TORNADO VORTEX PATTERNS USING DOPPLER RADAR

(1) Principle of observing tornado vortices with Doppler radar

Doppler radar transmits electromagnetic beams from an antenna and observes the intensity of precipitation (reflectivity) and the radial component of wind velocity (Doppler velocity) based on the signal reflected from precipitation particles carried by the wind. By keeping the antenna elevation angle constant and rotating the antenna azimuthally at every 15 to 30 seconds, a cross-sectional scan of the cloud is obtained, enabling the observation of the internal structure of the cloud and its changes. Although Doppler radar can observe farther ranges, the spatial resolution becomes coarser, imposing a limitation on capturing the flow patterns of tornado vortices. Therefore, the detection target is generally limited to a relatively nearby range, for example, within a 60 km radius of the radar.

While Doppler radar is capable of detecting tornado vortices within this range, it’s important to understand the limitations and interpretations of these observations. The radar typically detects signatures of tornado vortices that are about 1 km in diameter. It is imperative, however, to recognize the difference between these radar-detected signatures and the actual tornado phenomenon. The diameter of a tornado, which is a violently rotating column of air extending from the base of a cumulonimbus cloud to the ground, is considered to be in the range of tens of meters to hundreds of meters based on evidence of tornado damage⁴⁾. Due to the limited spatial resolution of the radar, these signatures may not always represent a direct observation of the tornado itself. In other words, the tornado vortex signatures may be capturing the broader rotation of the air around the tornado. Nevertheless, these vortex patterns represent the small-scale rotational airflows directly related to the tornado’s rotation, and provide important information for predicting the occurrence of tornadoes.

Figure 1 shows an example of a Doppler velocity pattern observed by Doppler radar associated with a tornado vortex ⁵⁾. This is a case observed on January 25, 2008, when a vortex that formed over the Sea of Japan made landfall near the coast, causing a strong wind gust with a maximum wind speed of 28.8 ms^-1 and a pressure drop. The adjacent orange (enclosed by the upper semicircle) and green (enclosed by the lower semicircle) regions are observed because the Doppler velocity exhibits a pair of maximum (positive) and minimum (negative) values corresponding to the tornado vortex, with the air moving away from and toward the radar, respectively (Figure 2). The detection of tornado vortices by Doppler radar primarily involves using deep learning to identify the characteristic pair of maximum and minimum Doppler velocity pairs, hereinafter referred to as the “vortex pattern,” associated with tornado vortices, which are typically about 1 km in diameter.

(2) Construction of the deep learning model

In deep learning (supervised learning), a large amount of “example and answer” data, specifically positive (correct) and negative (incorrect) examples, is required as training data to learn the features (quantities representing characteristics that serve as cues for detection) for the model.

To enhance the model’s performance and generalization ability, we incorporated techniques to address the variability in tornado characteristics. One such consideration is the rotation direction of tornadoes. While tornadoes in Japan predominantly rotate counterclockwise, a certain percentage exhibit clockwise rotation⁶⁾. To address this, our deep learning model employs a data augmentation technique to accommodate both clockwise and counterclockwise rotating tornadoes. This technique involves pre-identifying the rotation direction of the tornado in the input data and flipping the images horizontally if the tornado rotates clockwise. This process creates a unified dataset consisting of counterclockwise rotating tornadoes and flipped clockwise rotating tornadoes, which is then used to train the deep learning model. During evaluation, the model’s output is interpreted based on the original rotation direction of the input data. This approach enables us to effectively utilize data from both types of tornadoes, potentially enhancing the model’s generalization ability. Furthermore, it eliminates the need for a complex model structure that explicitly accounts for rotation direction, allowing us to build a simpler and more efficient model.

In the deep learning approach described in this paper, the Doppler velocity patterns corresponding to tornado vortices and those not corresponding to tornado vortices serve as positive and negative examples, respectively⁷⁾. Figure 3 shows examples of the training data, where the upper row (a) represents positive examples, and the lower row (b) represents negative examples. The positive examples exhibit patterns considered to correspond to tornado vortices, while the negative examples are those that coincidentally resemble tornado vortices but are not. Specifically, Figure 3 (b) provides two such negative examples: the left panel illustrates a case of artificial noise, characterized by the diagonally elongated red pattern, likely caused by radar signals reflecting off non-meteorological targets like buildings or terrain features. The right panel exemplifies a velocity aliasing artifact, a common issue in Doppler radar data processing. The alternating red and blue streaks, representing rapidly switching positive and negative velocities along the data row, indicate a failure in the unfolding algorithm to correctly account for velocity values exceeding the radar’s Nyquist velocity⁸⁾. The deep learning model is expected to identify the patterns shown in the positive examples as tornado vortices and appropriately exclude the non- representative patterns shown in the negative examples. The training data was compiled by utilizing archived Doppler radar data from past observations, with experts manually annotating each pattern as either corresponding to a tornado vortex or not, assigning positive or negative labels accordingly (a process referred to as “annotation”).

The experts employed the following criteria to identify tornado vortex patterns:

1. Artificial patterns, such as noise or artifacts, as shown in Figure 3, were easily identified and labeled as negative.

2. For patterns resembling tornado vortices, the experts examined the temporal continuity of the pattern by referring to the Doppler velocity data from preceding and subsequent time steps. Patterns that exhibited continuous movement in a particular direction were considered more likely to be tornado vortices.

3. Patterns that appeared only briefly without temporal continuity were often judged to be random atmospheric features resembling tornado vortices and were labeled as negative.

The prepared training data consists of approximately 37,000 samples (approximately 16,000 positive and 21,000 negative examples) of Doppler velocity data observed primarily by JR East’s Doppler radar located along the Sea of Japan coast in Sakata City, Yamagata Prefecture, during the winter seasons from December 2016 to March 2017. Using these training data, a convolutional neural network model was constructed.

(3) Processing flow and application example

Doppler radar can obtain observation data every 15 to 30 seconds per rotation. From the obtained observation data, the pairs of maximum and minimum Doppler velocity (Figure 4) corresponding to tornado vortices are rapidly and accurately detected within the target range (within a 60 km radius of the radar). This method processes Doppler radar data in near real-time to provide immediate and quantitative information. Before applying our deep learning model, a preliminary screening using mathematical algorithms is conducted to identify potential tornado vortex candidates. This screening process examines whether the Doppler velocity pattern exhibits a structure similar to a Rankine vortex⁹⁾, which is a theoretical model for the airflow distribution in a vortex. However, as natural vortices do not always perfectly conform to the ideal Rankine vortex pattern, the criteria for this screening are relaxed to avoid excluding potential tornado vortices. Even patterns that appear artificial, such as those shown in Figure 3, can be selected as candidates for input to the subsequent deep learning model.

After this preliminary screening, our deep learning model focuses on classifying these Doppler velocity patterns as either positive (vortex) or negative (non-vortex) examples, as illustrated in Figure 3. This model, along with the preceding mathematical algorithm used to identify potential vortex candidates from the Doppler velocity field, was jointly developed by the Meteorological Research Institute and JR East, and is disclosed in Patent No. 6756889¹⁰⁾. The preceding mathematical algorithm calculates the center location, diameter, and maximum tangential wind speed for each vortex candidate, providing additional information for analysis.

As will be introduced later, this technique is being used for train operation control against wind gusts during winter along the Sea of Japan coast.

The method of controlling train operations based on Doppler radar-detected wind gusts was introduced by JR East in 2017. Starting in 2020, the deep learning technique described in this section, which utilizes a convolutional neural network model to automatically detect tornado vortices from Doppler velocity data, has been used in step [2] of the following train operation control procedure (Figure 5). Before outlining the steps, we will first focus on the key process of calculating maximum wind speed and predicting the trajectory in step [3]. In step [3], the detected tornado vortex is tracked every 30 seconds, and the maximum wind speed caused by the vortex is calculated. The trajectory for the next 10 minutes is predicted based on the vortex’s position information over the past 10 minutes. A warning area, encompassing the most likely path of the tornado vortex, is defined around the predicted movement path. This area accounts for prediction uncertainties using distance and directional error parameters (2σ confidence interval) determined from past tornado events. Note that a deep learning model is not used in this step.

The four steps involved in this procedure are as follows:

[1] Doppler radar observation: Observe the lower atmosphere within a 60 km radius at 30-second intervals.

[2] Tornado vortex detection: Detect tornado vortices from the obtained Doppler velocity data using deep learning.

[3] Calculation of maximum wind speed and trajectory prediction: Track the tornado vortices at 30-second intervals, calculate the maximum wind speed caused by the vortices based on the obtained information.

[4] Train operation control: Stoppages are instructed appropriately based on the wind alerts.

3. OVERVIEW OF THE BRIDGE PROGRAM

(1) Positioning of the BRIDGE program and the BRIDGE project

In 2023, as part of the Integrated Innovation Strategy¹²⁾, we launched a new research project, building upon the achievements of the previous “Public/Private R&D Investment Strategic Expansion Program (PRISM) ¹³⁾.” This BRIDGE project is being implemented under the “BRIDGE: programs for Bridging the gap between R&D and the Ideal society (Society 5.0) and Generating Economic and social value ¹⁴⁾ (hereinafter referred to as the “BRIDGE program”).

The aforementioned Train operation control is currently implemented on certain rail lines in Yamagata, Akita, and Niigata Prefectures on the Sea of Japan side, specifically during the winter season. This pioneering operational implementation during winter on the Sea of Japan side was prompted by reports of wind gust damage caused by disturbances over the Sea of Japan, which have been considered a potential cause of train derailment disasters¹⁵⁾.

On the other hand, in Japan, tornadoes are more frequently reported along the Pacific coast during the warmer seasons from summer to autumn rather than in winter³⁾. Since these occur in densely populated areas with major transportation and lifeline infrastructure, the detection of tornadoes during the warmer seasons is also an important research topic we are pursuing.

While sharing the common characteristic of rotating airflow in the lower atmosphere, the morphological characteristics of warm-season tornadoes may differ from those occurring in the winter on the Sea of Japan side due to differences in their formation mechanisms.

Specifically, warm-season tornadoes often originate from cumulonimbus clouds with intense convection, such as supercells, which are large and powerful thunderstorms characterized by a rotating updraft. Winter-season tornadoes typically form from relatively weak convective cumulonimbus clouds or lines of clouds that develop as a result of the Siberian cold air mass passing over the relatively warm Sea of Japan, which supplies warm, moist air at lower levels.

This difference in parent storm characteristics can lead to variations in the appearance of tornadoes on Doppler radar imagery. For example, warm-season tornadoes, due to the influence of dry air entrainment and intense convection, may exhibit distorted or incomplete rotation patterns on radar. Winter-season tornadoes often show clearer and more distinct rotation patterns.

These differences are expected to manifest as variations in the feature quantities used for deep learning models. Specifically, this difference poses a challenge for developing deep learning models that can accurately detect tornadoes across different seasons and weather conditions. A key challenge is therefore to effectively incorporate these differences into deep learning models.

To address the challenge of developing deep learning models capable of accurately detecting tornadoes across different seasons and weather conditions, including warm-season tornadoes, the Meteorological Research Institute initiated research prior to the BRIDGE program.

Specifically, prior to the BRIDGE program, the Meteorological Research Institute had been conducting research using data from the Doppler Radar for Airport Weather (hereafter referred to as DRAW) as part of the “Public/Private R&D Investment Strategic Expansion Program (PRISM)” from 2019 to 2022. The DRAW is a radar system installed by the Japan Meteorological Agency at nine major airports across the country to ensure the safe operation of aircraft by obtaining low-level wind shear information. It has an observation range of 120 km, an azimuth resolution of 0.7°, provides independent data points every 150 m, and performs high-frequency scans at the lowest elevation angle of 0.7° every 60 seconds, making it suitable for developing deep learning models for detecting warm-season tornadoes. The PRISM project addressed the crucial research challenge of detecting tornadoes during warmer seasons, which pose significant risks to densely populated areas with major transportation and lifeline infrastructure. Furthermore, it successfully demonstrated the feasibility and societal value of deep learning-based tornado detection for enhancing railway safety, particularly during the winter season.

Building upon these achievements of the PRISM project, the BRIDGE project was initiated to develop a more versatile system applicable to various weather conditions and sectors beyond railway operations.

Specifically, the BRIDGE program aims to further advance the deep learning-based tornado vortex detection technology developed through the collaborative research between the Meteorological Research Institute and JR East, as well as the efforts to implement research on tornadoes during warmer seasons conducted under PRISM. The program has set “priority issues” such as developing new business environments, promoting business creation by startups, and fostering human resources, with the goal of promoting measures to link innovative technologies to the resolution of societal issues. Additionally, the application of deep learning to the detection of low-pressure rotations (mesocyclones) with diameters of a few kilometers within clouds, as handled by the tornado occurrence probability nowcast introduced in Chapter 1, is expected to contribute to improving the accuracy of such nowcasts.

Figure 6 illustrates the timeline and relationship between the PRISM and BRIDGE projects.

Figure 7 illustrates an overview of this BRIDGE project. While JR East Japan has been operating an automatic tornado vortex detection and prediction system using deep learning technology co-developed with the Meteorological Research Institute within its own closed network, this BRIDGE project aims to further improve the technology based on the results of PRISM and expand its application to weather radar data nationwide. Importantly, this BRIDGE project also integrates GPS location data, aiming to enable real-time delivery of disaster prevention information for deployment in various fields beyond the railway sector.

Concretely, the project is developing technology to accurately detect and predict various localized severe weather events by utilizing deep learning on publicly available weather radar data distributed nationwide. This disaster prevention information is expected to be utilized not only by railway operators but also by road operators, power companies, private weather service providers, and businesses across various sectors for disaster prevention and mitigation efforts in their respective operational activities.

A key feature of this BRIDGE project is the construction of a system to generate and distribute real-time disaster prevention information by combining this detection and prediction information with GPS location data. As described in Section 2(3), our tornado vortex detection process can not only determine whether or not a tornado is present, but also identify the location of the tornado vortices within the radar coverage area. By integrating this detection process with GPS data from users’ smartphones, we aim to provide personalized real- time alerts to individuals located within the predicted path of a tornado vortex.

This integration enables us to predict the potential risk to individuals based on their current location and the projected trajectory of the tornado vortex, including whether they are within the predicted path of the vortex, the estimated time until the vortex reaches their location, and potential evacuation routes to safety.

(2) Industry-academia-government collaboration system and collaboration with startup companies

This BRIDGE project is being promoted through broad collaboration between industry, academia and government. The Meteorological Research Institute serves as a hub, driving the project in one go from core research and development to commercialization by companies including startups. Some of the research and development is contracted out to external organizations such as startups and universities, building a system that maximizes the strengths of each party.

Particularly strong emphasis is placed on collaborating with startup companies, with the goal of pursuing flexible and swift development. Startups have the advantages of being able to quickly respond to social needs and easily incorporate new ideas. On the other hand, the Meteorological Research Institute and large corporations can provide insights backed by years of experience and the safety and reliability required to implement technologies in society. By leveraging these mutual strengths, open innovation is expected to be promoted.

For instance, it is already partnering with AI startups like Laboro.AI Inc. to develop advanced deep learning models for tornado vortex detection (detailed in Chapter 4, sections (1) and (2)). Furthermore, collaborations are planned with startups specializing in information dissemination to develop user-friendly mobile applications for real- time weather alerts. These startups are expected to contribute their expertise in developing deep learning models, integrating GPS data, building real- time information dissemination systems, and creating solutions for disaster prevention. By leveraging the startups’ agility, innovative ideas, and ability to quickly respond to societal needs, the project aims to build a comprehensive real-time disaster prevention field using AI.

This BRIDGE project also incorporates mechanisms and matching funds for participating companies to connect research and development outcomes to practical application and commercialization. The aim is to enable the swift practical application of developed technologies and solution of social issues.

Collaboration with other disaster prevention- related projects is also considered, targeting deployment in broader fields. Addressing meteorological disasters requires the complex involvement of various factors, so it is extremely important for diverse entities to integrate their knowledge and expertise.

As described above, this BRIDGE project features organic collaboration between industry, academia and government to efficiently promote everything from technological development to social implementation.

4. INITIAL RESULTS

As mentioned earlier, a localized tornado vortex detection system utilizing Doppler radar has been implemented to control train operations during winter along the Sea of Japan coast through collaborative research between JR East and the Meteorological Research Institute. Unlike the JR East system, which focused on train operation control along specific rail lines, the system developed in the BRIDGE project needs to be designed with higher versatility to cater to a wide range of general users. Specifically, this system is designed to deliver real-time information customized to users’ locations by incorporating recent advancements in deep learning models and integrating GPS data.

In this way, while building upon the previous achievements, a key characteristic of the BRIDGE project is the pursuit of a more universal disaster information dissemination system. Therefore, the development efforts initially concentrated on applying advanced deep learning models and integrating them with GPS data. The following sections outline these two initial results.

(1) Development of deep learning models for tornado vortex detection

a) Data preparation and VGG model ¹⁶⁾

From the Airport Weather Doppler Radar (DRAW) data covering the period between 2010 and 2019 at nine airports, we extracted Doppler velocity fields and annotated them following essentially the same procedure as described in Chapter 2 to create training data for deep learning models that detect tornadoes. Experts manually examined each pattern, assigning positive labels to those corresponding to tornado vortices and negative labels to those that did not. The dataset was divided into the following:

Training and Validation Data: 12,737 samples (of which 2,280 are positive samples). Test Data: 894 samples (of which 398 are positive samples).

While the dataset exhibits an imbalance between positive and negative samples, we did not employ specific techniques like data augmentation to artificially balance the class distribution. This decision is based on the characteristics of the Doppler velocity patterns and the operational context of tornado vortex detection. Positive examples, representing tornado vortex patterns, exhibit limited variation in their shape. On the other hand, negative examples, encompassing various non-vortex patterns, are more diverse. To enhance the model’s ability to distinguish true tornado vortices from other atmospheric features and minimize false alarms, we prioritized increasing the number of negative examples rather than artificially balancing the dataset. This approach aligns with the safety-critical nature of the application, where unnecessary train stoppages due to false positives can have significant operational and economic consequences.

Using this data, we re-trained the Visual Geometry Group (VGG) model developed under the PRISM program.

b) Development of new deep learning models

While VGG is a convolutional neural network (CNN) model widely used for image recognition tasks, new deep learning models have been developed specifically for tornado vortex detection in recent years. Table 1 provides an overview of the deep learning models utilized in this research.

We developed and evaluated three types of models: CNN, Neural Architecture Search (NAS), and Vision Transformer (ViT). For the CNN model, we adopted MobileNetV3¹⁷⁾, which employs 1x1 convolutions and depth-wise separable convolutions to reduce computational requirements and model size while maintaining accuracy. It incorporates Squeeze-and-Excitation modules and Global Average Pooling, which enable efficient integration of channel-wise and spatial information, thereby achieving high representational performance while mitigating overfitting. Unlike the Vision Transformer models, MobileNetV3 does not include the self-Attention mechanism.

For the NAS model, we used EfficientNetV2¹⁸⁾, which applies neural architecture search to optimize the CNN architecture and employs a progressive learning approach to scale up the network effectively. This model strikes a balance between accuracy and computational efficiency. As for the ViT model, we utilized SwinTransformerV2¹⁹⁾, which is based on the Vision Transformer (ViT) architecture and incorporates the self-Attention mechanism to capture long-range dependencies more effectively compared to traditional CNN models. This allows ViT-based models to focus on relevant features more effectively than traditional CNN-based models.

Having selected these models for their unique advantages, we proceeded with their implementation and training. We utilized ImageNet pre-trained models available in the Torchvision library, specifically VGG16²⁰⁾, EfficientNetV2-Small²¹⁾, MobileNetV3-Small²²⁾, and SwinTransformerV2- Tiny²³⁾. For training, we used a batch size of 32 and employed early stopping, terminating training when the validation loss did not improve for 50 epochs. We used the Stochastic Gradient Descent (SGD) optimizer with a Step Linear Regression (StepLR) learning rate scheduler, setting the initial learning rate to 0.005 and a decay rate of 0.7 with a step size of 1. The cross-entropy loss function was used for optimization. Model performance was evaluated using recall, precision, F1-score, and area under the receiver operating characteristic curve (AUC). The deep learning models were implemented using PyTorch version 2.1.1 on a computer running Ubuntu 20.04 with 64GB of memory and an NVIDIA A30 GPU with 24GB of memory. While this research did not explicitly tune the precision- recall trade-off for different use cases, it is important to note that the precision and recall requirements may differ between railway operations and general user applications. The flexible architectures of EfficientNetV2 and SwinTransformerV2 models potentially allow for adjusting this balance according to specific needs. This adaptability could be a valuable capability for deploying the models across diverse scenarios, ranging from railway systems to broader societal use cases requiring varied precision- recall profiles.

c) Performance evaluation and comparison

Figure 8 shows the recall and precision scores of EfficientNetV2, SwinTransformerV2, and MobileNetV3, compared to the benchmark VGG model from the previous PRISM program.

Specifically, EfficientNetV2 achieved a precision of 86.3% and a recall of 76.1%, while SwinTransformerV2 achieved a precision of 89.5% and a recall of 75.1%. SwinTransformerV2 exhibited a higher precision, indicating a greater accuracy in correctly identifying positive cases. However, EfficientNetV2 showed a slightly better recall, suggesting its ability to capture a larger proportion of actual positive cases. The results indicate that in terms of the area under the precision-recall curve (AP), EfficientNetV2 (AP=0.885) and SwinTransformerV2 (AP=0.917) outperformed the previously reported VGG model (AP=0.877). MobileNetV3 also achieved a higher AP of 0.892 compared to VGG. While both models showed improvements in certain metrics, it’s important to acknowledge that the performance differences are relatively small. It is plausible that VGG, with further optimization of hyperparameters or training data, could achieve comparable or even better performance than these newer models.

Despite these relatively small performance differences, the new models have shown the ability to achieve comparable or potentially improved accuracy compared to VGG. This finding suggests that these advanced models can serve as viable alternatives to VGG, potentially offering advantages in terms of computational efficiency or ease of implementation.

These advanced techniques, both capable of capturing long-range dependencies and focusing on relevant features, have proven effective in various computer vision tasks and seem to translate well to the domain of tornado vortex detection. Further exploration of these advanced deep learning techniques may lead to even greater accuracy improvements in the future.

d) Performance on CPUs

Figure 9 compares the inference time (latency) of the deep learning models on CPUs and GPUs. Note that this study focuses on evaluating the performance of the newer models on CPUs, and the previously reported VGG model is not included in this comparison, as a direct comparison with the VGG model’s previously reported performance would not be appropriate, given that the computational environments between the previous study and the current research differ.

The horizontal axis represents the models, and the vertical axis represents the inference time (in milliseconds). The results show that MobileNetV3, designed for efficient inference on CPUs, achieves reasonable performance even when deployed on CPUs. This capability opens up possibilities for deploying tornado vortex detection models on a wide range of devices, including edge devices and mobile platforms, enabling real-time risk assessments and prompt response actions in various scenarios.

In summary, this research has developed advanced deep learning models that achieve comparable or even better performance compared to traditional approaches in accurately detecting tornado vortices. By utilizing techniques and optimizing for computational efficiency, these models enable practical implementation in various settings, contributing to improved disaster prevention capabilities across society.

Furthermore, by incorporating elements such as progressive learning, shifted windows, and efficient architectures, these models can achieve high performance while maintaining computational efficiency, enabling deployment on various platforms, including edge devices and mobile applications.

This work on this research project, including the development of new deep learning models (b), performance evaluation and comparison (c), and performance on CPUs (d), was conducted in collaboration with researchers from Laboro.AI Inc., a startup company, with two of their researchers serving as co-authors on this paper.

(2) Efforts toward real-time disaster prevention information dissemination

In this BRIDGE project, while building upon the conventional railway wind gust detection system, we are also focusing on disseminating real-time disaster prevention information to the general public. It is crucial to provide accurate meteorological risk information to people on the move, various modes of transportation, and business establishments. By utilizing GPS location information and implementing the system on mobile devices such as smartphones, it becomes possible to issue alerts and present remaining evacuation times based on the user’s current location. The conventional railway wind gust detection system was designed to address the impact on train operations along fixed rail lines. In contrast, the new system leveraging GPS location information and the deep learning-based tornado vortex detection models represents a novel approach aimed at serving the general public and various mobile entities.

While based on the technology cultivated in the railway domain, it also caters to the emerging needs brought about by the proliferation of smartphones. The application to mobile entities holds the potential to significantly contribute to the mitigation of wind gust-related disasters. By expanding from fixed facilities to mobile applications, this crucial initiative enables the dissemination of wind gust risk information across various societal settings, making it an extremely useful approach.

a) Precise dissemination through integration with GPS location information

This BRIDGE project aims to rapidly and effectively provide information on localized severe weather phenomena observed by meteorological radar to general users. In this research project, a real- time dissemination system has been constructed that combines wind gust information, such as tornadoes detected by meteorological radar, with smartphone location information (GPS).

Figure 10 illustrates the API configuration of this system, demonstrating how information is exchanged via APIs. In this system, the wind gust detection information obtained from meteorological radar is stored in a database on the server. Meanwhile, the smartphone app transmits the user’s location information and settings. On the server side, these data are combined to generate optimal weather information for each user, which is then delivered via push notifications or displayed within the app. In this manner, the system is configured to disseminate optimal disaster prevention information to users by storing the tornado information observed by meteorological radar in a database and integrating it with the user’s location information. Figure 11 illustrates the specific flow of information dissemination. Information on tornado vortices detected by meteorological radar is pushed as notifications or alert displays to the user’s smartphone, based on their current location. The app under development aims to display the real-time 2km to 3km square alert range for tornado occurrences 5-10 minutes in advance and send warning push notifications to users within that range, providing advance notice and evacuation time to people, transportation, and businesses in the affected area. Furthermore, by utilizing the smartphone’s GPS functionality, the system can disseminate information only within the designated range, which is expected to help reduce excessive alerts and server load.

b) Implementation on smartphone Apps and other platforms

Figure 12 shows an example of an alert screen displayed on the developed smartphone app when the user’s current location falls within the predicted path of a tornado vortex. If a tornado vortex is imminent near the user’s current location, an alert corresponding to the wind speed is displayed. Color- coded circles indicate the position and size of the tornado vortex, designed to promptly encourage users to take evacuation actions. By integrating GPS and radar information, it has become possible to visualize accurate disaster prevention information for users and prompt timely action. Furthermore, a questionnaire survey targeting general users was conducted, receiving 327 responses. The results showed that many people had high expectations for the app’s functionality, and the real-time provision of tornado information could enhance their sense of crisis and lead to avoidance actions.

As described above, this research project has implemented a real-time dissemination system integrating meteorological data and GPS location information, establishing a mechanism for providing information to general users through smartphone apps. This will enable a prompt response to localized severe weather phenomena, contributing to the overall enhancement of disaster prevention capabilities across society.

5. CONCLUSION - FUTURE DEVELOPMENTS AND EXPECTED EFFECT

Before discussing the expected effects and future developments, we provide a brief summary of the project achievements outlined in chapters 1-4. Specifically, we developed advanced deep learning models that achieve comparable or even better performance compared to traditional approaches for detecting tornado vortices from radar data. Additionally, a real-time disaster prevention information dissemination system was implemented by integrating the detection system with GPS location data and smartphone applications. The research outcomes described in these initial results, including the development of advanced deep learning models and the real-time disaster prevention information dissemination system integrating GPS data, form the foundation for the expected future developments and impacts outlined in the following conclusion. Building on these outcomes, this concluding chapter considers the potential benefits and applications across various sectors, as well as the scientific contributions to tornado research.

(1) Widespread Societal Implementation for Improved Responsiveness to Meteorological Disasters

The tornado detection technology utilizing deep learning developed in this BRIDGE project has the potential for diverse applications beyond the railway field. Further research will be conducted to explore how this technology can contribute to a safer society across a number of sectors. Specifically, it can be applied across various transportation infrastructure, such as highways and ports, as well as by lifeline service providers like electric and telecom companies. Furthermore, deployment is envisioned for private companies that need tornado countermeasures, such as event organizers, construction site managers, and industrial facilities operators. These sectors are highly susceptible to wind gust impacts, and the ability to rapidly detect and respond to tornado vortices can significantly enhance safety and operational continuity. By leveraging GPS location data, it might enhance construction site safety by enabling timely evacuation and work stoppages for personnel operating cranes, scaffolding, or working at heights during a tornado threat. Furthermore, future applications could include strengthening lifeline infrastructure resilience by supporting electric and telecom companies in quickly identifying impacted areas and prioritizing restoration efforts, like power line repairs, after a tornado event. Additionally, this technology also holds the potential to directly benefit the general public by providing real-time severe weather information to people in areas at risk of tornado impact, enhancing overall public safety and disaster preparedness. Widespread social implementation of this technology is anticipated to dramatically improve societal resilience and recovery capacity against localized meteorological risks. This technology, through its widespread implementation across diverse sectors, is expected to enhance immediate responsiveness to localized meteorological disasters and contribute to improving national resilience. By promoting the widespread adoption of its outcomes, the BRIDGE project aims to simultaneously achieve disaster risk mitigation and strengthen social infrastructure.

While this BRIDGE project currently focuses on tornado vortex detection, the underlying technology, which combines deep learning with meteorological radar data, has the potential to be applied to other localized severe weather phenomena. For example, downbursts⁹⁾, characterized by strong downdrafts and diverging outflow winds, can also pose significant risks to transportation and infrastructure. While downbursts, unlike tornado vortices, are characterized by diverging rather than rotating airflows, their patterns may still be detectable in Doppler velocity data using deep learning models. However, directly applying our current system, designed to track and predict the path of horizontally moving tornado vortices, to downbursts, which involve vertically descending airflows, would be challenging. Future research will explore the feasibility of adapting our deep learning models and system design to detect and characterize downbursts, potentially expanding the scope of our real-time disaster prevention system to encompass a wider range of meteorological threats.

In addition to localized severe weather events, our deep learning approach could also contribute to the analysis and prediction of stationary line-shaped precipitation systems (SLPSs), which are massive, long-lasting clusters of thunderstorms, often spanning tens to hundreds of kilometers, that can produce prolonged heavy rainfall, sometimes for several hours. As defined in prior studies ^{24) 25)}, while not fitting the definition of a rapidly developing localized event, SLPSs exhibit characteristic features in radar images, such as their elongated shape and distinct internal rainfall intensity patterns. Future research will investigate the feasibility of applying our deep learning models, trained on tornado vortex patterns, to identify and characterize SLPSs. This could involve adapting the model architecture and training data to recognize the unique features of SLPSs. Successful application of this technology could lead to improved forecasting of SLPS formation, movement, and rainfall intensity, thereby enhancing disaster preparedness and mitigation efforts for heavy rainfall events.

(2) Expected academic insights

The main purpose of the tornado vortex detection technology using deep learning developed in this BRIDGE project is to provide real-time disaster prevention information. However, in addition to such practical value for disaster prevention, this technology also has great potential to significantly contribute to the academic understanding of tornado phenomena by serving as a tool to rapidly analyze tornado vortex patterns. Previously, detailed analysis for each individual case has been necessary to elucidate the occurrence and development mechanisms of tornadoes. Carrying out such detailed analysis of large-scale data has been difficult due to time and human resource constraints. However, by utilizing this technology, tornado vortex patterns can be automatically and rapidly extracted with high precision from massive amounts of data, enabling efficient investigation of tornado occurrence characteristics. This is expected to yield new insights on the diversity of tornadoes in Japan, including occurrence frequency, geographical distribution, seasonality, development processes, and relationships with environmental conditions. Furthermore, by applying this technology to radar data globally, major advances can be anticipated in academic tornado research, including grasping the climatological reality of global tornado occurrence, monitoring occurrence and development processes, and developing forecast models.

The introduction of a machine learning approach also raises expectations for new discoveries and awareness beyond existing knowledge. Data-driven analytical techniques may reveal overlooked features and relationships. This technology paves the way for the world’s first comprehensive understanding of the climatological reality of tornado phenomena globally. Thus, as a powerful “tool” to rapidly analyze past archive data, rather than just a disaster prevention tool, this technology is expected to have major scientific impact by enabling detailed understanding of tornado occurrence mechanisms. Both for real- time disaster mitigation and gaining academic insights, it is anticipated to greatly contribute to advancements in meteorological science.

Acknowledgments

This research project was conducted with the support of the ‘Basic Research Promotion System in the Transport Field,’ the ‘Public/Private R&D Investment Strategic Expansion Program (PRISM),’ and the ‘BRIDGE: Programs for Bridging the Gap Between R&D and the Ideal Society (Society 5.0) and Generating Economic and Social Value.’

References

• 1)  Japan Meteorological Agency, What is the Tornado Probability Nowcast?, https://www.jma.go.jp/jma/kishou/know/toppuu/tornado3-1.html, 2010. [in Japanese].
• 2)  Fujiwara, C. and Suzuki, H.: Development of train operation control method against wind gusts using Doppler radar, JR EAST Tech. Rev., Vol. 66, pp.40-43, 2021 [in Japanese].
• 3)  Kusunoki, K., Ishitsu, N., Adachi, T., Suzuki, O., Arai, K., Fujiwara, C. and Suzuki, H.: Development of tornado detection technique with Doppler radar using deep learning. Wind Engineers, JAWE, 47(3), 218-223, 2022. https://doi.org/10.5359/jawe.47.218 [in Japanese].
• 4)  Niino, H., Fujitani, T. and Watanabe, N.: A statistical study of tornadoes and waterspouts in Japan from 1961 to 1993, J. Climate, Vol. 10, pp. 1730-1752, 1997.
• 5)  Kusunoki, K., Arai, K., Kato, R. and Fujiwara, C.: Observations of the intensity and structure changes within a winter tornadic storm during landfall over the Japan Sea area, IEEJ Transactions on Fundamentals and Materials, Vol. 137, pp. 141-146, 2017.
• 6)  Kusunoki, K., Arai, K., Inoue, H. Y. and Fujiwara, C.: Doppler radar observations of a wintertime anticyclonic misocyclone associated with surface wind gust on the coast of the Sea of Japan, SOLA, Vol. 15, pp. 234-237, 2019.
• 7)  Ishitsu, N., Kusunoki, K., Adachi, T., Arai, K., Inoue, H. Y., Fujiwara, C. and Suzuki, H.: Detection of tornadic vortex from Doppler velocity field using convolutional neural networks, Proc.10th European Conf. on Severe Storms, 4-8 November 2019, Kraków, Poland.
• 8)  Holleman, I. and Beekhuis, H.: Analysis and correction of dual PRF velocity data, J. Atmos. Oceanic Technol., Vol. 20, pp. 443-453, 2003.
• 9)  Markowski, P. and Richardson, Y.: Mesoscale Meteorology in Midlatitudes, John Wiley & Sons, 658 pp., 2011.
• 10)  Japanese Patent No. 6756889, Vortex Detection Device, Vortex Detection Method, Program and Trained Model, 2020. https://www.j-platpat.inpit.go.jp/c1801/PU/JP-6756889/15/ja [in Japanese].
• 11)  East Japan Railway Company, implementation of Train operation control using AI-based wind gust detection method, https://www.jreast.co.jp/press/2020/20201006_ho03.pdf, 2020. [in Japanese].
• 12)  Cabinet Office, Government of Japan, Integrated Innovation Strategy 2023. https://www8.cao.go.jp/cstp/tougosenryaku/togo2023_honbun_eiyaku.pdf, 2023.
• 13)  Public/Private R&D Investment Strategic Expansion Program (PRISM), https://www8.cao.go.jp/cstp/panhu/prism2021_e/2021.html, 2021.
• 14)  Programs for Bridging the gap between R&D and the IDeal society (society 5.0) and Generating Economic and social value (BRIDGE), https://www8.cao.go.jp/cstp/english/society5_0/index.html, 2023.
• 15)  Inoue, H. Y., Kusunoki, K., Kato, W., Suzuki, H., Imai, T., Takemi, T., Bessho, K., Nakazato, M., Hoshino, S., Mashiko, W., Hayashi, S., Fukuhara, T., Shibata, T., Yamauchi, H. and Suzuki, O.: Finescale Doppler Radar Observation of a Tornado and low-level misocyclones within a winter storm in the Japan Sea coastal region, Mon. Wea. Rev., Vol. 139, pp. 351-369, 2011.
• 16)  Simonyan, K. and Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Inter. Conf. on Learning Representations (ICLR), 2015.
• 17)  Howard, A., Sandler, M., Chu, G., Chen, L., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V. and Le, Q. V.: Searching for MobileNetV3, Proc. IEEE/CVF Inter. Conf. on Computer Vision (ICCV), pp. 1314-1324, 2019.
• 18)  Tan, M. and Le, Q.: EfficientNetV2: Smaller Models and Faster Training, Proc. 38th Inter. Conf. on Machine Learning (ICML), 2021.
• 19)  Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., Wei, F. and Guo, B.: Swin Transformer V2: Scaling Up Capacity and Resolution, Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2022.
• 20)  Torchvision Library. “Models and pre-trained weights - VGG - vgg16.” https://pytorch.org/vision/stable/models/generated/torchvision.models.vgg16.html#torchvision.models.vgg16, Accessed: 2024-08-07.
• 21)  Torchvision Library. “Models and pre-trained weights - EfficientNetV2 - efficientnet_v2_s.” https://pytorch.org/vision/stable/models/generated/torchvision.models.efficientnet_v2_s.html#torchvision.models.efficientnet_v2_s, Accessed: 2024-08-07.
• 22)  Torchvision Library. “Models and pre-trained weights - MobileNetV3 - movilenet_v3_small.” https://pytorch.org/vision/stable/models/generated/torchvision.models.mobilenet_v3_small.html#torchvision.models.mobilenet_v3_small, Accessed: 2024-08-07.
• 23)  Torchvision Library. “Models and pre-trained weights - SwinTransformer - swin_v2_t.” https://pytorch.org/vision/stable/models/generated/torchvision.models.swin_v2_t.html#torchvision.models.swin_v2_t, Accessed: 2024-08-07.
• 24)  Min, K.-S., Tsuboki, K., Yoshioka, M. K., Moroda, Y. and Kanada, S.: Formation Mechanism of a Stationary Line-Shaped Precipitation System in the Kinki District, Japan - Case Study on 1 September 2015 Event -, J. Meteor. Soc. Japan, Vol. 99, pp. 357-377, 2021.
• 25)  Hirockawa, Y., Kato, T., Tsuguti, H. and Seino, N.: Identification and classification of heavy rainfall areas and their characteristic features in Japan, J. Meteor. Soc. Japan, Vol. 98, Special Edition on Extreme Rainfall Events in 2017 and 2018, pp. 835-857, 2020. https://doi.org/10.2151/jmsj.2020-043.