Evaluation of Multiple Optimized AIGC Approaches in Low-Altitude Drone-based Riparian Waste Monitoring
2026 年 7 巻 1 号 p. 7-22
詳細
2026 年 7 巻 1 号 p. 7-22
Recently, LMM (large multimodal model, which can“understand”image, text or other types of modals) has been applied in camera-based riparian environmental monitoring (4K camera or drones). Concurrently, LMM can also assist Image-based Generative AI to generate AIGC (Artificial Intelligence Generated Content) using prompt engineering (i.e., users can drive the computers to provide the prompts for image generation purposely during the conversation with LMM). In this research, the authors designed multiple experiments to apply LMMs (i.e., Tencent YuanBao-interfaced DeepSeek and Hunyuan) to generate the prompts using prompt engineering. Based on the different prompt outputs, AIGC-based waste images were generated in Stable Diffusion. Correspondingly, generated images were trained in YOLOv8x models with the assistance of the annotations derived from previous trained AIGC waste model. Based on the needs of low-altitude waste monitoring, the authors tested the trained YOLOv8x models using benchmark dataset (LAD-RWB, Low-Altitude Drone Riparian Waste Benchmark). In conclusion, LMM-derived prompt-based AIGC have the possibility of applying in the low-altitude drone waste monitoring with over 0.8 precision (0.426 recall and 0.491 mAP50) in detecting the waste under the riparian background using a tiny test dataset (16 images). LMM can provide reasonable prompts for Stable Diffusion image generations.
Until now, camera-based riparian environmental monitoring is always a challenging task that needs professional personnel to understand the on-site situation in a short time. Simultaneously, training the young labors to become professional personnel always needs and time costs, which cannot solve the current tasks immediately. These years, camera platforms (smartphone1), 2), 4K camera3), 4) and drones 5)-7)) have provided powerful supports for the professional personnel to understand the platform-surround situations. Nevertheless it still needs human eyes to observe and judge. Correspondingly, the new technologies are in considerate to make effort to improve the current situation.
Currently instead of human, Artificial Intelligence using computer vision (You Look Only Once, YOLO series) can already deliver reasonable observation results1)-7). Correspondingly, data for training is still not enough under some specific scenes or backgrounds8). Following, image generation model, i.e., Stable Diffusion (SD) in this research9)-10), has not only shown its ability in the practical projects and also already increased the possibility of generating the images (i.e., AI-Generated Content, AIGC) for data augmentation, or in some cases, AIGC can solve the practical tasks solely. The reason is that AIGC can provide similar features as the practical world, which includes but not limited to color, shape and texture. Consistently, how to provide the necessary real-world-similar features in the generated images using Stable Diffusion, which is becoming more and more attractive theme to the researchers. At this moment, Large Multimodal Model (LMM) provided a solution that can drive image generation algorithm to provide the necessary real-world-similar features in the generated images. In recent years, LMMs have been applied in multiple scenes (i.e., Tencent 11)-interfaced DeepSeek12) and Hunyuan13) in this research)14)-20). Based on the results, systematic prompts used in the conversations can help LMMs to understand the purpose and main content derived from the users clearly21)-25).
Considering of YOLO-, AIGC- and LMM-related potential in riparian environmental monitoring, the authors should develop the workflow including the details, i.e., how to apply these new technologies in the practical applications (i.e., waste detection) systematically. As depicted in Fig.1, there are 3 sections included as follows, DeepSeek and Hunyuan, individually:
・ Section-1, training“AIGC Waste Dataset”6) in YOLOv5 to become the annotation generator in this research and annotating the AIGC images in Section-2. AIGC Waste Dataset is a SD-derived synthesis dataset that includes four types of objects in the riparian environment, i.e., card board, bicycle, plastic bottle and plastic bag;
・ Section-2, using LMM (DeepSeek R1 or Hunyuan T1) to generate prompts for Stable Diffusion image generation. During the process of the generation, one LAD-RWB (Low Altitude Drone Riparian Waste Benchmark Dataset, LAD-RWB8) in the following content) sample selected from Section-3 should be used in the img2img (an image generation function in Stable Diffusion) as reference. And the authors collected the AIGC Dataset (images and annotations) ready for the model training in Section-3;
・ Section-3, training AIGC Dataset in Section-2 and saving as Trained Model, then using LAD-RWB Dataset for Evaluation Metric on Trained Model, including P (Precision), R (Recall), mAP50 and mAP50-95.
(1) YOLOv5 (Waste Dataset, Annotator)
As shown in Table 1, the authors showed the parameters settings of the trained YOLOv5 model using AIGC Waste Dataset for generating the annotations of AIGC Datasets. As shown in Fig.2, there are 5 steps in the flow chart of generating the AIGC images6):
1. Selecting a Stable Diffusion checkpoint, i.e., v1-5-pruned-emaonly.safetensors [6ce0161689];
2. Using prompts (Objects, i.e., bikes, cardboard, PET bottles and plastic bags; Resolution, i.e., high resolution, 8k camera; View angle, i.e., bird view, UAV view; Area, i.e., riparian area) in txt2img;
3. Adjusting the Width/Height of image generations (i.e., 608/608 pixels);
4. Adjusting the Batch count/size of image generations (i.e., 100/1);
5. Starting the generation.
As shown in Fig.3 & 4, the authors showed results and label distribution statistics of Annotator. And correspondingly, as shown in Table 2, annotators should infer the AIGC images using the mentioned parameters for image sizes and confidence value.
(2) LMM
After preparing the trained model as the annotator, the authors should start to use LMM to understand and describe the provided image. As shown in Fig.5, it is a selected sample image from a dataset called “LAD-RWB: Low Altitude Dataset Riparian Waste Benchmark”. Noteworthy, image generation needs the information from different perspectives, including but not limited to color, texture, scene, and photography style. Next, as shown in Table 3 (3-Steps AIGC Prompts Engineering Method or 3-Steps Method), the authors applied prompt engineering to drive the LMM to extract the necessary information from the sample image. 3-Steps Method include following 3 steps:
1. Understanding the image that the prompts will not overcome the scope;
2. Adjusting the prompt output and re-describing the image to bias the output;
3. Summarizing and categorizing the prompts for clear understanding.
With the assistance of 3-Steps Method, as shown in Table 4, LMMs extracted the necessary information in the shape of prompts from the provided sample image using DeepSeek R1 and Hunyuan T1, individually. Table 4 (down-side) dictated the comparisons of prompts in different categories, which can be modified based on different purposes in the next step practical application.
(3) Stable Diffusion
As shown in Fig.6, the authors applied the prompts in fine-tuning Stable Diffusion WebUI online version txt2img function (https://stabledifffusion.com/) to generate the reference images. Correspondingly, as shown in Table 5 & 6, the authors depicted the parameter setting for computer and Stable Diffusion WebUI local version img2img function, individually.
Fig.7 includes 2 grid images which visualized the comprehensive view of the generated images. Because of different LMMs, the photography types of the following grid images are different: DeepSeek R1 used murky water with more bottles than yellow bags in a cloudy weather, Hunyuan T1 used natural sunlight with balanced waste contributions. With the assistance of the annotator, all the AIGC images shown in Fig.7 should be annotated using the mentioned annotator derived from AIGC Waste Dataset.
(4) YOLOv8x (DeepSeek R1)
Based on the purpose and costume, Fast-speed and balanced one-stage model YOLOv8x-det26) was chosen as the model for training the AIGC Datasets. 5 types of models (size changes from small to large, i.e., n, s, m, l and x) and 3 types of task modes (cls, classification; det, detection and seg, segmentation) can be chosen in this model. Following the parameters setting and image numbers shown in Table 7 & 8, the model was trained. Fig.8 & 9 depicted the detailed process of training/validating and labels distribution statistics. Fig.10 is a batch image sample of label (left-side) and corresponding prediction (right-side).
(5) YOLOv8x (HunYuan T1)
As same as DeepSeek R1-based Trained Model-1, Trained Model-2 followed the parameters setting and image numbers shown in Table 9 & 10 to train YOLOv8x model. Fig.11 & 12 depicted the detailed process of training/validating and labels distribution statistics. Fig.13 also showed a batch image sample of label (left-side) and corresponding prediction (right-side). Tencent YuanBao is an AI platform that provides access to various models, including both DeepSeek and Hunyuan. We accessed both LMMs through the YuanBao interface for consistency.
After collecting the mentioned 2 trained models, the authors tried to use parts of a benchmark dataset (16 images, 342 × 342 pixels, 100-m flight height) to evaluate them, individually. As shown in Table 11 & 12, the authors depicted the index used for the evaluations, i.e., Recall, Precision, mAP50 and mAP50-95 using 512 pixels image size and 0.001 confidence value. Fig.14 showed the original, label and prediction samples of the batch images used for corresponding index between Trained Model-1 and -2 for the Evaluation Metric. Trained Model-1 derived from DeepSeek R1-generated prompts has advantages in all the evaluation index. Noteworthy, as shown in Fig.15, the detected waste objects derived from Trained Model-1 have much higher confidence value comparing with Trained Model-2 different objects (apparent plastic bag, plastic foams and yellow plastic bag in Sample-1, Sample-2 & -4 and Sample-3, individually) under different backgrounds.
Fig.16 showed the objects that cannot be detected by both Trained Models, plastic bottle is too small for the detection and yellow plastic bag is not easy-to-detected between the background and its outline. Considering of the instances limitation derived from the image numbers, as shown in Table 14, the authors increased the images step by step (800 images each step). By increasing the image numbers, mAP50 has been changed from 0.491 to 0.613, i.e., to some degree, increasing the diversity and quantity of the AIGC Dataset can improve the detection accuracy.
In this research, the authors applied 3-Steps Method in LMMs to extract the necessary information from one specific image, and understanding the image systematically, which included but not limited to object, texture, color, scene and photography style. After summarizing and categorizing the mentioned prompts using prompt engineering, the extracted information can be used for image generation directly. And with the assistance of DeepSeek R1-based AIGC, Trained Model-1 showed its advantages comparing with Trained Model-2 (Hunyuan T1) in all cases of Evaluation Metric in the low-altitude drone-view waste monitoring.
Considering of the reasons, Trained Model-1 was trained using more instances than Trained Model-2 (around 12000 and 4000, individual), which is realistic. As shown in Table 4, the prompts derived from DeepSeek R1 are recommended to “generate imagery focused on pollution severity metrics ” , which lead to the image generations of waste or litter pollutions in the riparian area. And Hunyuan T1 generated the prompts“mid-air perspective showing multiple plastic bottles (clear/translucent, varied sizes) partially buried in soil” and “realistic texture details on litter and ground ” for much clearer individual objects- and background-related descriptions. Based on the differences between the prompts, Trained Model-1 derived from DeepSeek R1 is much appropriate to the benchmark dataset (16 images, 342 × 342 pixels, 100-m flight height) in this research.
Noteworthy, the authors also tried local txt2img function in Stable Diffusion directly to generate the images. As shown in Fig.17, although riparian-related features have been generated correctly, waste-based objects have not been generated as same as img2img function, even the waste-objects-related prompts have already been included. Worth mentioning, if the reference images in img2img have already been generated using fine-tuned model, the image generations are more controllable. Because the existing reference images can be judged and adjusted firstly, with the assistance of reference images, img2img function is more stable than txt2img in the image generation tasks (which was mainly based on the human judgment).
The systematic novel pipeline (LMM → AIGC → Annotator → Detector) demonstrates a path toward generating large-scale, task-specific synthetic training data with minimal human intervention, starting from just a single reference image and a conversational interface with an LMM. This approach is particularly valuable for applications where collecting and manually annotating thousands of real-world images is prohibitively expensive or logistically challenging.
LMM-derived prompt-based AIGC have the possibility of applying in the low-altitude drone waste monitoring (LAD-RWB-based) with over 0.8 precision in detecting the waste under the riparian background. After increasing the image numbers, mAP50 has been improved from under 0.5 to over 0.6. LMM can provide reasonable prompts for Stable Diffusion image generations using novel 3-Steps Method. YOLOv5 “annotator” was derived from real-world dataset called Nisha484, which can be considerate as a trustable annotator. And in conclusion, to some degree, codifying systematic biases and false boxes was ignored. But when generating the synthetic data, the generated content was limited in the specific objects, in the authors’ opinion, which can be accepted. And in the previous works, comparing with the real-world general dataset, specific-designed synthetic data annotated by the trained annotator had much higher accuracy in the specific waste detection metric. And the automatic annotator should be used or considerate as a speed-up rough annotation provider for any types of objects. After the rough annotation, manual corrections are necessary for the higher accuracy. Rather than the higher accuracy, faster speed for annotating the real-world data with controllable errors is our target.
Even the method of using LMM as the prompt generator for Stable Diffusion, has been proved its possibility, the limitation of the dataset amount for test still exists in this research. Because all the AIGC data for training were generated based on the test data sample, in this research the authors only consider the limited test dataset. The small test set size is a major limitation that affects the statistical reliability and generalizability of our findings, the results in this research are indicative rather than conclusive. This research is basically discussing the possibility of applying the LMM to generate the necessary data using prompt engineering, and it is a proof of concept instead of practical application (or not yet).
The method applied in this research focused on the waste monitoring task observed from close-distance drone-view. The prompt-engineering derived from researchers can applied in the Edge AI devices27)-31) to generate dataset including both images and annotations automatically. And the application of edge AI devices can support the indoor operators to reduce the workload. And researchers can connect prompt engineering with Large Language Models32)-38), Text-To-Speech39)-47) and Automatic-Speech-Recognition48)-51) together in the drone-platform Edge AI devices without any other unnecessary operation just using human voice.
The method in this research was designed for low-altitude drone-view waste monitoring. Prompts should be accumulated based on the 3-Steps Method and applied under more practical engineering scenes, particularly drone-platformed applications.
Excepting the mentioned content, there are also several bulleted points that can deepen the research discovery, as follows:
・ Waste Specificity-related Prompts;
・ Environmental Tone-related Prompts;
・ Structural Philosophy-related Prompts;
・ Scenes-related Prompts;
・ Technical Intent-related Prompts;
・ Negative Prompts;
We will discuss specific factors contributing to the gap, such as the lack of complex occlusions, unrealistic textures, lighting inconsistencies in AIGC images, and the inability to generate very small or heavily degraded objects (as seen in Fig.16 ).
We will discuss potential mitigation strategies, including domain adaptation techniques, using more diverse reference images for img2img, incorporating negative prompts more effectively to avoid unrealistic features, and the potential of using real background images with synthetic waste overlays.
We will provide a quantitative analysis of the failure cases, for instance, by reporting the average size (in pixels) of missed objects compared to detected ones, and discussing the impact of IoU thresholds on the low recall.
This work was supported by JSPS KAKENHI Grant Number 25K07936.
