Integration of Text and Image Analysis for Flood Event Image Recognition

Flood event monitoring plays an important role for emergency management. With the fast growth of social media, a large number of images and videos are uploaded and searched on the internet during disasters, which can be used... [ view full abstract ]

Flood event monitoring plays an important role for emergency management. With the fast growth of social media, a large number of images and videos are uploaded and searched on the internet during disasters, which can be used as “sensors” for improving efficiency of emergency management. This work proposes a novel framework in which the rich information available from social media is incorporated with image analysis to enhance image retrieval for disaster management. The text associated with images of flooding events was used to extract prominent words associated with flooding. The image features are represented by a histogram of visual words obtained using the Bag-of Words (BoW) model. The text and image analysis were integrated at the feature level, in which the text features were conjoined directly with image features. The proposed approach was evaluated based on two flood event corpuses obtained from the US Federal Emergency Management Agency media library and from public Facebook pages and groups related to flood and flood aid (in German). The experimental results demonstrate the improved performance of image recognition after incorporating the text features, which suggests the potential to enhance emergency management.