Prof. Hanjalic is seen internationally as a scientific authority in the field of Multimedia Information Retrieval (MIR). His contribution to this field focuses on providing efficient, effective, intuitive and (socially) responsible access to and interaction with information stored in large (multimedia) data collections in a variety of use scenarios involving both search and recommendation. His work has had high and lasting impact along several MIR research directions, some of which he pioneered himself.
Already his early work (1999-2002) on automated low- and high-level video segmentation and key-frame based video representation resulted in publications that have served as standard references in the field for many years. With his paper ‘Video and image retrieval beyond the cognitive level: the needs and possibilities’ published at the IS&T/SPIE Storage and Retrieval for Media Databases 2001 conference, Prof. Hanjalic launched the research direction on affective multimedia content analysis and retrieval. At that time, the MIR field focused primarily on how to bridge the semantic gap between what audiovisual features to measure in images and video and their interpretation in terms of the depicted objects and scenes. Contrary to this, the main idea of Prof. Hanjalic was to focus on bridging the affective gap and in this way enable computers to automatically analyze the audiovisual content of a video in terms of the affective states (feelings, emotions) it elicits in viewers. This can enable image and video indexing based on affective clues and expand the possibilities for video search and recommendation towards better personalization. Later on, he was the first to propose an integral framework for affective video content representation and modeling (IEEE Transactions on Multimedia, 2005), and the first one to approach this modeling in the 2D affect space consisting of the arousal and valence dimensions. Modelling a video as a curve in the 2D affect space reveals the expected affective reactions of a user to the video content over time. This significantly broadens the possibilities for browsing, categorizing and retrieving video content and improves the modelling generalization, as compared to a typical classification-based approach using predefined events and emotions as labels. The pioneering role and impact of this work have played a key role in an explosive growth of this research direction that has developed into one of the largest and most active subfields of MIR. In 2016, Prof. Hanjalic was named IEEE Fellow for his contribution to the field of multimedia information retrieval.
In recent years, Prof. Hanjalic and his research group have shifted their focus more and more towards the challenges in the development of recommender systems. A number of seminal works and awards (ACM Recommender Systems Grand Challenge winner 2010, ACM Recommender Systems Best Paper Award 2012, >10000 downloads of the paper in ACM Computing Surveys 2014) gave the group a significant international visibility in this domain. The research group of Prof. Hanjalic continues to build on these successes and to pursue challenges in the recommender systems domain, focusing on the issues of bias, fairness and privacy as the critical systems development criteria, and with the aim of making recommender systems operate in a socially responsible manner.