IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences
Online ISSN : 1745-1337
Print ISSN : 0916-8508
Regular Section
Semantic Guided Infrared and Visible Image Fusion
Wei WUDazhi ZHANGJilei HOUYu WANGTao LUHuabing ZHOU
Author information
JOURNAL RESTRICTED ACCESS

2021 Volume E104.A Issue 12 Pages 1733-1738

Details
Abstract

In this letter, we propose a semantic guided infrared and visible image fusion method, which can train a network to fuse different semantic objects with different fusion weights according to their own characteristics. First, we design the appropriate fusion weights for each semantic object instead of the whole image. Second, we employ the semantic segmentation technology to obtain the semantic region of each object, and generate special weight maps for the infrared and visible image via pre-designed fusion weights. Third, we feed the weight maps into the loss function to guide the image fusion process. The trained fusion network can generate fused images with better visual effect and more comprehensive scene representation. Moreover, we can enhance the modal features of various semantic objects, benefiting subsequent tasks and applications. Experiment results demonstrate that our method outperforms the state-of-the-art in terms of both visual effect and quantitative metrics.

Information related to the author
© 2021 The Institute of Electronics, Information and Communication Engineers
Previous article
feedback
Top