Multi-modal Asymmetric Autoencoders for Massive Photo Collection Applications

There has been an abundant use of applications where many photos obtained from camera-equipped devices can be leveraged and exploited to enable emerging services, e.g., mobile crowdsourcing. These systems usually collect a large data stream of images coming from different heterogeneous sources (e.g, IoT devices and humans) in an inadvertent way. Due to the limitations and challenges related to communication bandwidth, storage, and processing capabilities, it is unwise to transfer unselectively all the photos since most of them often either contain duplicate information, are inaccurate, or are just falsely submitted. In this paper, we propose to design a smart image selection procedure using an asymmetric multi-modal neural network autoencoder to select a subset of photos that has high utility coverage for multiple incoming streams. The proposed system enables selecting high context data from an evolving picture stream and ensures relevance. The approach uses the photo's metadata such as geo-location and timestamps along with the pictures' semantics to decide which photos can be submitted and which ones must be discarded. Simulation results for two different multi-modal autoencoder architectures indicate that a mixed asymmetric stacked autoencoder approach can yield better results outperforming the concatenated input autoencoder while leveraging user-side rendering to reduce bandwidth consumption and computational overhead.