Frans Oliehoek

Dr. Frans A. Oliehoek (1981) is an Associate Professor in the Department of Intelligent Systems, Delft University of Technology, where he focuses on interactive learning and decision making.  He received his Ph.D. in Computer Science (2010) and M.Sc. Artificial Intelligence (2005) both from the University of Amsterdam (UvA). He subsequently did postdocs at MIT (2010-2012) and Maastricht University (2012-2013), where he was appointed as a (non-permanent) Assistant Professor (2013-2014). In 2014 he moved back to UvA supported by an NWO VENI Fellowship and in parallel he took up a position as Lecturer at the University of Liverpool, where he was promoted to Senior Lecturer in 2017.

Frans’ research interests lie in the intersection of machine learning, AI and game theory. He is considered an expert in the field of decision making under uncertainty, with emphasis on multiagent systems. He organized several workshops on Multiagent Sequential Decision Making Under Uncertainty and Multiagent Reinforcement Learning and taught tutorials on Decision Making under Uncertainty at AAMAS and the European Agent Systems Summer School. He received the best PC-member award at AAMAS 2012, and was awarded a number of research grants, including a prestigious €1.5M ERC Starting Grant for his project "INFLUENCE: Influence-based Decision-making in Uncertain Environments" which started February 2018.

Contact Information:

Dr. Frans Oliehoek

INFLUEnce-based decision-making in uNCertain Environments (INFLUENCE)

Project description:

Decision-theoretic sequential decision making (SDM) is concerned with endowing an intelligent agent with the capability to choose the 'best' actions, i.e., those that that optimize the agent's performance on its task. SDM techniques have the potential to revolutionize many aspects of society, and recent successes, e.g., agents that learn to play Atari games and beat master Go players, have sparked renewed interest in this field.

However, despite these successes, fundamental problems of scalability prevent SDM methods from addressing other problems with hundreds or thousands of state variables. To overcome this barrier, INFLUENCE will develop a new class of influence-based SDM methods that address scalability issues by using novel ways of abstraction.

For instance, when we think of controlling traffic lights in an entire city, an intersection’s local problem is manageable, but the influence that the rest of the network exerts on it is complex. The key idea is that by using (deep) machine learning methods, we can learn sufficiently accurate representations of such influence to facilitate near-optimal decisions.

This project will aim to develop novel decision making methods and demonstrate their scalability on (at least) two simulated challenge domains: control of traffic lights in large area (e.g. an entire city), and robotic order picking in a large-scale autonomous warehouse.

If successful, INFLUENCE will produce a range of influence-based SDM algorithms that can, in a principled manner, deal with a broad range of large and complex problems, thus making an important step towards realizing the promise of autonomous agent technology.

PhD / postdoc opportunities:

The project research team will be led by Dr. F.A. Oliehoek and comprise 3 PhD students and 2 postdocs. These members will focus on novel machine learning techniques to learn approximate influences and novel methods for planning and reinforcement learning that exploit such representations.

Graduation project opportunities:

There is a large number of research questions closely related to this project that would be suitable for MSc or BSc graduation projects. For instance, I would be happy to supervise projects that would use the 'SUMO' traffic simulator ( These could for instance focus on:

-implementing and comparing current state-of-the-art scheduling approaches for traffic light control
-using coordinated reinforcement learning for large-scale urban traffic light control
-assessing the impact of improved sensors for traffic light control

Contact Information:

For more information, please contact:
Dr. Frans Oliehoek