Enhanced Monitoring for Offshore Wind Turbines using Digital Twin and Artificial Intelligence
2024 Volume 5 Issue 1 Pages 1-6
Details
2024 Volume 5 Issue 1 Pages 1-6
Japan’s extreme environmental conditions have some potential challenges in the offshore wind development, and it requires enhancing the monitoring and maintenance practices. This study proposed an offshore wind turbine remote monitoring and advanced visual inspection by integrating digital twin technology, virtual reality, drones, and artificial intelligence (AI). An overview of Ramboll’s digital enabled asset management (DEAM) approach was elaborated, and its current advantages such as possible structure’s lifetime extension, failure mechanism detection and prevention, understanding the structure’s actual dynamic properties, environment monitoring, etc. In addition to that, two deep learning models that aimed to identify the OWT component and segment damages were trained, this might help to reduce the manpower, equipment cost, and lessen the visual inspection time. Furthermore, the challenges and conceptual possible integration of AI to drones during visual inspection were elaborated. The proposed method will include risk assessment and drone flight planning optimization in future studies.
In 2030, the Ministry of Economy, Trade and Industry forecasted that the renewable energy supply in Japan will be 22 to 24 percent. For wind energy, it is expected to increase to approximately 10 GW by 2023 and 30-45 GW by 20401).
In line with the increase of offshore wind power generation facilities, reliable preventive measures in inspection and maintenance should be implemented all throughout the lifetime of the structure. Consideration of Japan’s severe natural environment phenomena such as earthquakes, strong winds, waves, and lightning strikes must be taken in account1).
Several wind turbine damages have been recorded for the past years due to earthquakes, liquefaction, and typhoons. After the 2011 Tohoku Earthquake, a wind turbine’s tower located in Kashima city was tilted by around 1.8 degrees, the damage was related to the turbine’s foundation2) 3). Furthermore, the typhoon that occurred in Okinawa Prefecture last 2003 caused buckling of on-shore wind turbine towers, damages to blades and nacelles, and collapse of foundation4) 5). After this accident, the JSCE guideline (Guidelines for design of wind turbine support structures and foundations) were published in 2010, to help improving the monitoring and maintenance of offshore wind turbines (OWT) and to prevent such accidents due to Typhoon to happen.
To further improve the current methodology of OWT operation and maintenance, Japanese government agencies such as NEDO in collaboration with METI released the green innovation fund to encourage innovative technological development towards carbon neutrality in 20506). One of the project features tackled about the innovative offshore wind operation andmaintenance project which aims to enhance the current inspection/monitoring procedures for OWT.
Several approaches are currently under development such as the utilization of digital technology for preventive maintenance, development of digital twin platform, and application of AI technology.
(1) O&M Regulations and Challenges
In Japan, the maintenance of offshore wind power generation facilities guidelines are specified in the Unified Commentary standard of 20201). The laws and regulations with regards to the examination standards are based on the Electricity Business Act (Law No. 170 of 1964), the Port and Harbour Act (Law No. 218 of 1950), and the law concerning the promotion of use of sea areas for the development of renewable energy generation facilities for ocean use (Law No. 89 of 2018).
Several challenges are expected during the operation and maintenance of OWT such as:
- In the event of a sudden operation shutdown, the offshore wind facilities cannot be accessed immediately depending on the weather and sea conditions.
- In case that major components are broken such as blade failure, the replacement requires self elevationg platform vessels for transportation.
Therefore, it is recommended to conduct continuous maintenance by a combination of monitoring remotely and scheduled inspections to prevent major failure occurrence, monitor each components, etc. SCADA and CMS systems are common methods for remote monitoring, in which sensors are attached to different locations of the offshore wind turbine. Furthermore, it is recommended to adopt new technologies to improve the inspection methods. In addition to that, aside from remote sensing, the periodic and unscheduled on-site inspection is needed. The inspection cycle, inspection method, specific equipment, and each component were elaborated in Section 2.3.3 of Unified Commentary1).
For the inspection method specifically on the rotor nacelle assembly, tower, transition piece, and monopile, majority are done by visual inspection. Some of the items to be visually inspected for Monopile foundation aerial part based on Unified Commentary Table 2.3.3.21) are as follows:
- Corrosion, cracking, or steel damage on the transition piece (TP) and monopile (MP)
- Corrosion protection covering damage
- Tilt of the entire monopile structure
- Lighting fixtures of ancillary equipment Advanced technologies and innovations are encouraged to be studied to improve the current methods.
(2) Application of Digital Twin, AI, and Drones to OWT Visual Inspection and Risk Assessment
Digital twin can help to evaluate the structural integrity of the structure with higher accuracy due to the data gathered in the actual environment compared to the assumed parameters during the design phase. Augustyn et. al. (2021), evaluated the structural reliability of the offshore wind turbine substructure by utilizing the digital twin information. The potential lifetime extension of the structure was evaluated by assessing the accumulated fatigue damages from the updated soil stiffness and wave loading7).
Emergency maintenance is expected in case that anomaly is detected in the digital twin data, as well as periodic inspection and maintenance. Current researchers are implementing the usage of drones and artificial intelligence to enhance the inspection method. The use of unmanned aerial vehicles (UAV) / drones for remote inspection of wind turbines significantly shorten the time of detecting defects, increases the personnel’s safety, and helps in transporting equipments required in dangerous inspection works.
Shihavuddin et. al. (2019) proposed a wind turbine surface damage detection method with the aid of drones and deep learning. The proposed approach suggests the damage location and type of the wind turbine blade parts. The trained Inception-ResNet-v2 architecture inside the faster R-CNN obtained an 81.10% mean average precision (mAP) for the four considered damages8).
For risk assessment, Wu et. al (2022) trained an artificial neural network (ANN) model to assess the accessibility of off-shore wind turbines by crewtransfer vessels to ensure the safety of the work personnel and increase work efficiency. The factors considered were significant wave height, wave period, wind speed, and wind direction9).
This study proposed an improved OWT inspection and monitoring method which consists of three steps as shown in Fig.1. The first step consists of the OWT digital twin and virtual reality platform. This area was already developed by Ramboll and applied to several projects in Europe, the overview will be discussed in the next chapter.
In case there is a need of on-site visual inspection for scheduled / unscheduled maintenance or inspection, the second and third step was proposed which is the conceptual focus of this research.
The second step is the usage of drones with integration of trained AI models for real-time component detection, damage segmentation, risk detection, etc. This will help the engineer/drone pilot to identify the inspection priority and inspection route. The preliminary progress of this study such as detection of OWT and corrosion segmentation can be applied in the initial inspection during scheduled or unscheduled maintenance, which will be further discuss in chap.4.
After the collection of images, the collected data (images, videos, etc.) will be uploaded to the cloud/server, which can be accessed remotely. The AI model result will help to decrease the time required for the engineer to check each image and detect damages manually. The engineer will then perform a quality assurance check and proceed to the evaluation of the detected damages and components. The inspection data will be uploaded to the digital twin and virtual reality platform in each specific component or location. The inspection report, maintenance progress, and other valuable data can be included in step 1. The UAV flight path planning is not yet included in this study.
Ramboll is continuously developing the digital enabled asset management (DEAM) which includes digital twin and virtual reality platforms for operation and maintenance of off-shore wind structures. This study presents the overview of the offshore wind turbine (OWT) digital twin which is a virtual representation of the real structure with digital simulation of its environment throughout it’s lifetime, from installation over operation to decommissioning. Sensors were installed at various locations of the actual OWT and the real-time data reflected on the digital twin supports the operation and maintenance (O&M) of the physical asset and related condition-based decision-making. Fig.2 shows the digital twin interface and Fig.3 shows the accessibility.
The digital twin aims to conduct real-time structure monitoring which will be advantageous to:
- Increase workforce safety
- Reduce on-site work time
- Reduce the cost of inspection
- Minimize the downtime by preventing failures
- Possible structure’s lifetime extension
- Detection of failure mechanism
- Improved understanding of performance and dynamic behavior
- Etc.
To support the condition-based operation and maintenance and enable risk evaluation, the digital twin includes:
- Screening (Set up of simulation model, etc.)
- Model Calibration (validate design basis, etc.)
- Uncertainty Assessment
- Realtime monitoring (lifetime management, etc.
(1) Sensors
The OWT monitoring concept with digital twin involves various sensors to gather important informations for damage detection, model updating etc. Some of these sensors are as follows:
- Accelerometers: input to vibration monitoring.
- Inclinometers: to capture the static inclination of the structure.
- Strain gauges: used to calculate the fatigue stresses. Optional, since fatigue can also be calculated from combined information of other sensors.
In offshore wind, a major interface is located between the transition piece (TP) and tower, where accelerometers and strain gauges are often installed. Furthermore, SCADA (Supervisory Control and Data Acquisition) data is used as input for data analysis.
(2) Virtual Reality
RamView360 as shown in Fig.4, is the virtual reality platform that was developed by Ramboll to provide a visualization of the following:
- Live forecast of weather data
- 3D model of the structure with each components
- Inclusion of component information
- Inclusion of structure inspection report
- Safety Manuals on specific components
- Etc.
The virtual reality platform enables to incorporate the valuable sensor data and inspection reports on different specific components or locations within the structure.
Offshore wind turbines are located in extreme environment condition due to strong wind and waves. Common challenges for the deployment of drone’s in OWT visual inspection includes:
- weather conditions (wind, wave, etc.)
- requires highly specialized personnel to operate
- available of the suitable equipment and vessels to go near the OWT (depends on the planning)
- optimum flight path planning
- high risk of drone falling into the sea (in case of mechanical failure)
This study only presents the conceptual applicability of drones with the integration of AI in OWT visual inspection. Further studies and actual deployment is recommended.
Artificial intelligence application in OWT commonly involves detecting defects from inspection images, anomaly detection from sensor data, risk assessment, etc. This study focuses on component detection and damage segmentation by training a YOLOv8 model.
YOLOv8 model10) is capable of various vison AI tasks such as real-time object detection, segmentation, tacking, classification, and pose estimation. Furthermore, the weights are suitable to be incorporated in various hardware platforms. This study trained two YOLOv8 models that can detect the OWT and segment corrosion damages as discussed in the following paragraphs.
(1) Trained YOLOv8 OWT Detection Result
For the second step of the proposed method as discussed in Chapter 2, the drone inspection can be reinforced with trained deep learning models. Some applications are the detections of the OWT, classification of specific components, identification of critical areas, etc., which can help in the inspection path planning of the drone. Compared object/damage detection for other structure such as bridges and buildings, the harsh offshore environmental / weather conditions and the dynamic behavior (rotating blades) of the turbine serves as a challenge.
As a preliminary result of this study, considering 50 images of OWT, with 80% training and 20% validations, the trained YOLOv8 model can detect the wind turbine with a mean average precision of 0.78 percent after 10 epochs. The main advantage of this is that wind turbines have few main components which makes it easy for classification. Furthermore, increasing the training data can increase the accuracy to some extent.
Fig.5 shows the OWT sample raw image, annotated image, and the trained model OWT detection result. The validation mean average precision (mAP) and losses are shown in Fig.6.
(2) YOLOv8 for Corrosion Segmentation
For the transition piece (TP) and monopile (MP), critical damages to be identified during visual inspection include the presence of corrosion, cracking, damage to steel, and damage to corrosion protection. As a preliminary progress of this study, a deep learning model was trained to segment the corrosion damages along the transition piece (TP). 50 TP inspection images were collected, the corrosion areas were annotated, and the data was splitted into 80% training and 20% validation. After 10 epochs, the sample corrosion segmentation validation result was visualized in Fig.7. The trained AI model can be incorporated into the drone during the actual inspection or use it to segment corrosion damages after the inspection images are collected.
This study proposes an enhanced offshore wind turbine remote monitoring and visual inspection by utilization of different technologies such as digital twin, virtual reality, artificial intelligence, and drones.
The overview of Ramboll’s digital twin technology and virtual reality was briefly discussed. The digital enabled asset management (DEAM) of Ramboll is currently implemented and tested in several projects in Europe. The main advantages includes the real-time structural monitoring, defect detection, increased personnel’s safety, and further understanding of the dynamic bahaviours of the structure. Furthermore, the data gathered from various sensors installed in the actual structure will be used in the analysis and can possibly extend the structure’s lifespan.
For the aritificial intelligence and drone application, two deep learning models were trained which aim to detect the OWT structure and segment corrosion damages along the transition piece. Further improvements will be implemented by increasing the components and damage types in training. This approach will help to lessen the manpower and time, reduce equipment, and accelerate the inspection time. However, quality assurance on the AI result and further evaluation of the damage should be done by the inspector/engineer. The challenges of drone deployment were tackled, and further studies on flight path planning and risk assessment will be conducted. The overall proposed method will be further improved.
