Masters Projects

The normal prerequisite for all projects below is that you are a student doing your master's in computer science at Delft, including at least the course Advanced Algorithms (or Artificial Intelligence Techniques) and one other course in our group (e.g. the course Algorithms for Intelligent Decision Making gives a very good preparation for doing your final research project). If you are doing your master's at Delft in another subject, please additionally send us a motivation why you would like to undertake your thesis with our group.

Below you can find a representative sample of projects. The first table lists mostly applied projects (often with an external collaboration or internship), and the second table more fundamental projects and their source (typically industry or a scientific collaborator). If you are interested in one of these projects and meet the criteria above, please get in touch with the respective contact person in the Algorithmics group. If you meet the criteria but do not have a specific project or supervisor in mind, please get in touch with the group secretary Kim Roos.


Applying AI Algorithms to Real-World Problems

Many exciting master's projects can be defined by applying state-of-the-art planning, scheduling, and coordination algorithms to real-world problems.

Topic

Collaborator

Contact

Interpretable and Constrained Machine Learning in Protein Engineering

Dr Stas Mazurenko from Masaryk University (Brno, Czechia)

Demirovic
Employee Scheduling using Integer Programming and Column Generationpossible internship at KLMDemirovic

Power to Heat – Optimize use of industrial electric boiler

possible internship at Uniper

Spaan

Agent-based airport surface traffic planning under uncertainty

Aerospace Engineering

Spaan

Traffic flow optimization

TU Delft Transport Institute

Spaan

Fleet management at DEAL services

DEAL services, R. Negenborn (3mE)

Spaan

AI-based traffic light control

possible internship at Siemens (Zoetermeer)

Spaan

Developing a Driving Simulator Environment for Studying Mixed TrafficTransport & Planning (CEG)Spaan

Optimization of grid investments

possible internship at Alliander

Spaan/Yorke-Smith

Planning algorithms for energy grids

possible internship at Alliander

Spaan/Yorke-Smith

Optimizing supply temperatures in district heating grids

possible internship at WithTheGrid

de Weerdt

Decentralized optimization of DC microgridspossible internship at DC Opportunitiesde Weerdt

Quality personal transport for elderly and disabled people

possible internship at Transvision

de Weerdt

Transfer learning in policies for scheduling maintenance

possible internship at NS

de Weerdt

Improving and using lower bounds in multi-machine scheduling

possible internship at NS

de Weerdt

Human-system collaboration for maintenance planningpossible internship at NSde Weerdt/Yorke-Smith
Optimization for energy transition decisions in cities (improving CEGOIA)possible internship at CE Delftde Weerdt

Effect and design of a market for flexibility in congested networks

possible internship at AgroEnergy

de Weerdt

Using data science and machine learning for solving operations research optimization modelspossible internship at AIMMSde Weerdt
Optimizing wind farms dynamicallypossible internship at Whifflede Weerdt

Shift and personnel rostering with constraints

possible internship at KLM

Yorke-Smith

Applied optimisation with commercial datapossible internship at ORTECYorke-Smith
Hybrid optimisation for electroplate assembly linespossible internship at leading supplierYorke-Smith

Comparing centralised and decentralised optimisation for dynamic pickup and delivery

KU Leuven

Yorke-Smith

Planning for safety and security of critical industrial facilitiesP. van Gelder (TPM)Yorke-Smith

Deep learning for combinatorial optimisation in practice

TU Eindhoven

Yorke-Smith

Optimisation or machine learning in container port operations

possible internship at TBA Group

Yorke-Smith

Engineer dispatching using hybrid machine learning

possible internship at Macomi

Yorke-Smith

Simulation of self-organising logistics chainsinternship at TNO (The Hague)Yorke-Smith
Multi-agent reinforcement learning for autonomous logisticsinternship at TNO (The Hague)Yorke-Smith

Expertise matchmaking

possible internship at The Missing Middle

Yorke-Smith

Carrier demand and supply matching algorithm

possible internship at UTURNYorke-Smith
Autonomous greenhouse growing: data-driven agricultureInholland Delft, D. Tax (Pattern Recognition, EWI)Yorke-Smith
Autonomous greenhouse growing: multiple coordinated optimisationpossible internship at Blue RadixYorke-Smith

Agent-based modelling of housing market evolution

I. Nikolic (TPM)

Yorke-Smith

Fundamental Challenges in AI Algorithms

A scientifically-important master's project can come from addressing a fundamental algorithmic challenge.

Topic

Collaborator

Contact

Representative Solutions for Multi-Objective Optimisation

Dr Nicolas Schwind from The National Institute of Advanced Industrial Science and Technology (AIST) in Tokyo, JapanDemirovic

Machine Learning for Algorithm Selection

Demirovic

Compact and Interpretable Machine Learning

Demirovic

Advancing the Reasoning Engine for Combinatorial Optimisation

Demirovic

Optimal Decision Tree Learning for Controller Design

Dr Anna Lukina from the Institute of Science and Technology (IST Austria)Demirovic

Machine Learning for Fluid Simulations

Demirovic

Routing Taxis with Dynamic Pickup Locations

Dr Adi Botea (Eaton, Ireland)Demirovic

AI for the Romanian Puzzle Challenge: Constraint Optimisation for Large Problems

Dr Adi Botea (Eaton, Ireland)

Demirovic

Advanced Flow Propagators

Demirovic

Multi-objective mechanism design

Bosman

Evolutionary Algorithms

Bosman

Predict + optimiseVUBDemirovic/Yorke-Smith

Matching uncertain demand and supply in smart electricity networks

E. Walraven (TNO)

Spaan

Reinforcement learning for transportation, traffic and logistics

E. Walraven (TNO)

Spaan

Planning under uncertainty

Spaan

Multi-party multi-issue negotiation versus matching

C. Jonker (Interactive Intelligence, EWI)

de Weerdt

Voting with incomplete preferences

University of Southampton

de Weerdt

Auctions for capacity and tenders with uncertainty

A. Stawska

de Weerdt

Learning to searchNS / L. Bliekde Weerdt
Machine-learning-based state space simplification in sequential decision-making modelsG. Neustroevde Weerdt
Long versus short-term decision makingG. Neustroevde Weerdt
Decision diagrams for optimizationNSde Weerdt

Robust local search using scenario addition

NS

de Weerdt

Online (order acceptance and) schedulingL. He (NUDT)de Weerdt

Experimental comparison of algorithms for (temporal) planning

L. Planken

de Weerdt/Yorke-Smith

Acquiring CSP models with interactive machine learning

J. Shah (MIT)

Yorke-Smith

Machine learning coupled with state-of-the-art OR algorithms

TU Eindhoven

Yorke-Smith

Reinforcement learning in Mixed Integer ProgrammingK. Aardal (Maths, EWI)Yorke-Smith
Training neural networks with discrete optimisation solvers
Yorke-Smith

Data mining output of agent-based simulation models

I. Nikolic (TPM)

Yorke-Smith

Unit testing for functional agent-based languagesE. Chappin, I. Nikolic (TPM)Yorke-Smith

From organisational norms to constraint optimisation

V. Dignum (Umeå)

Yorke-Smith

Optimization under culture

G.J. Hofstede (Wageningen)

Yorke-Smith

Scheduling with disjunctions and preferences

B. Venable (Tulane)

Yorke-Smith

Approximate negotiation in international tradeC. Jonker (Interactive Intelligence, EWI)Yorke-Smith

AI for the Romanian Puzzle Challenge: Constraint Optimisation for Large Problems

The goal is to develop an AI that can outperform humans on a competitive complex crossword puzzle.

Conventional out-of-the-box AI techniques were previously not successful on this problem because of the size and constrained nature of the problem. The top performers are all humans, and fully automated algorithms have not shown human-level performance.

From a technical perspective, the student is expected to develop novel optimisation techniques to handle constrained problems when the specification of the problem is large.  The most important rules of the puzzle are as follows. Each year, the competition organisers select a theme and provide a list of 300-400 Romanian words. The goal is to place as many words from the thematic list as possible into a 13x13 grid. For each word and letter, the competitor is awarded a point, i.e., if a letter is used by two words, two points are assigned. In addition to the words, 26 black squares must be placed on the grid. Other words from the Romanian dictionary may be used, along with any two-letter word, but these do not award points. A list of words from the dictionary are available, and the student may focus primarily on developing the algorithm. The search space is large and there are thousands of words that need to be considered, resulting in a challenging AI problem with many open questions.

The project requires good algorithmic skills. Students not familiar with C/C++ programming are expected to develop their skills on the way. The project is in collaboration with Dr Adi Botea (Eaton, Ireland).

Routing Taxis with Dynamic Pickup Locations

The aim is to develop algorithms to route vehicles to pick up clients that are moving towards their destination. In a standard setting, upon requesting a taxi, the client waits for the taxi at a predefined location. However, it would be more convenient if the client could start moving towards their destination and have the taxi pick them up on the way. Other similar problems arise in a number of applications, e.g., computer games.

From a technical perspective, this is a challenging problem. Conventional multi-agent path finding considers stationary targets, but in this case the targets are moving. The agents must predict the movements of the clients and collaboratively agree on the allocation, but must also dynamically react if the movements of the clients diverge from the expectation.

The project requires good algorithmic skills. Students not familiar with C/C++ programming are expected to develop their skills on the way. The project is in collaboration with Dr Adi Botea (Eaton, Ireland).

Machine Learning for Fluid Simulations

The goal of the project is to investigate the use of machine learning to simulate complex natural phenomena, such as the movement of water or fire.

The movement of natural phenomena, e.g., water or fire, can be described using a set of differential equations, i.e., the Navier-Stokes equations. Solving these equations is a computationally intensive task and in some cases may require hours or days even for relatively small scenarios. Reducing the computation time is an important problem, e.g., in engineering applications or computer games. A possible approach is to use machine learning to learn the dynamics of fluids and provide a fast alternative that is reasonably accurate.

The problem requires familiarity with machine learning algorithms, e.g., neural networks. Knowledge of (numerical methods for) differential equations or computer visualisation techniques is a plus but not necessary. 

Optimal Decision Tree Learning for Controller Design

The aim of this sub-project is to replace the traditional heuristic algorithms used in controller design with optimal decision trees. This includes extending the algorithms to make more informed algorithmic decisions based on the guarantees provided with the optimal decision tree algorithm.

Controllers are algorithms that make automated decisions about a system. Designers of modern automated technologies often employ machine-learning techniques for learning controllers, e.g., reinforcement learning. Unfortunately, interpreting the policies obtained by machine learning is often difficult, which may raise safety concerns and lead in general to a mistrust in the system. The aim of this project is to simplify the machine learning policies using simpler models that are easier to understand for humans while retaining the comparable performance. By using formal reasoning and optimisation techniques, guarantees about the performance can be derived, which may lead to more robust and safer learned controller, but also paves the way for improving current algorithms through intelligent decision making.

The project requires good algorithmic skills and basic familiarity with reinforcement learning. Students not familiar with C/C++ programming are expected to develop their skills on the way. Knowledge of controllers is not necessary. The project is in collaboration with Dr Anna Lukina from the Institute of Science and Technology (IST Austria).

Advancing the Reasoning Engine for Combinatorial Optimisation

The aim is to advance state-of-the-art optimisation algorithms by analysing and improving the core reasoning component of these algorithms.

Algorithms for (Maximum) Satisfiability and constraint programming have been shown to be state-of-the-art for a wide range of problems, e.g., timetabling, scheduling. These methods are based on backtracking search. Each time a conflict is encountered during search, the reasoning engine is invoked. It analyses the conflict and provides an explanation to avoid making the same mistake in the future. The conflict analysis procedure is a key component of the algorithm as it results in a substantial reduction in the search space. While the reasoning mechanism has been shown to be effective, its behaviour is not entirely understood.

The goal of the project is set in two stages. In the first stage, the task is to extend the reasoning mechanism to provide human-interpretable explanations of the conflicts. This would provide more insight in the inner workings of the optimisation algorithm, something that is not entirely understood even by experts. In the second stage, based on the previous step, develop novel algorithmic approaches to improve the reasoning. The focus of the project is on timetabling and scheduling applications.

The project requires good algorithmic skills. Students not familiar with C/C++ programming are expected to develop their skills on the way.

Compact and Interpretable Machine Learning

The aim is to develop novel machine algorithms that are easy for humans to interpret.

Machine learning has shown to be a valuable tool in many applications. Unfortunately, it often results in a black-box decision maker, e.g., it may be difficult to understand exactly how the machine learning algorithm such as a neural network is making decisions. This is particularly important when making decisions about humans, e.g., socially sensitive contexts. To ensure a safe, trustable, and robust use of machine learning in practice, it is necessary to demystify the black-box and provide human-interpretable explanations.

In this project, the aim is to develop novel algorithms that will compute the optimal models, e.g., smallest model according to a predefined criterion. The resulting algorithm is expected to trade computational time, which is typically not critical during training, for higher accuracy and interpretable models.

The project requires good algorithmic skills. Students not familiar with C/C++ programming are expected to develop their skills on the way. Basic familiarity with machine learning is a plus, but not necessary.

Machine Learning for Algorithm Selection

The goal is to develop a machine learning approach to select the best algorithm for a given instance.

Combinatorial optimisation problems are ubiquitous throughout society. Examples of such problems include scheduling, routing, timetabling, and other forms of management decision problems. Unfortunately, many of these problems are NP-hard and no algorithm can efficiently solve every instance of the given problem. In practice, algorithms are developed for specific classes of instances. However, once a new instance is given, it is not easy to determine which algorithm to use even for experts. A possible approach to address this issue is to use machine learning to predict the best algorithm. This may lead to two main points: 1) more effective solving of combinatorial optimisation problems in practice, and 2) insight in which certain algorithms perform well, which may result in the improvement of combinatorial optimisation algorithms. In the project, the student will consider a large set of diverse optimisation algorithms and unite them through developing a robust algorithm selection approach.

Basic familiarity with machine learning and algorithms is a plus, but not necessary. The primary focus is on algorithm selection with machine learning, but should the student choose to later improve an optimisation algorithm based on their findings (optional), the student will need to develop their C/C++ on the way.

 

Representative Solutions for Multi-Objective Optimisation

The aim is to develop optimisation algorithms that can quickly compute a small but meaningful set of optimal solutions when given multiple competing objectives. 

The goal of conventional single-objective optimisation is to find the best solution according to a given metric. In contrast, multi-objective optimisation aims to optimise two or more generally competing objective functions. For instance, in supply-chain management, the goal is to simultaneously minimise delivery time and delivery cost. An “ideal” option generally does not exist and trade-offs must be made. Since it is often difficult for the user to determine beforehand the importance of each objective, multi-objective optimisation often involves computing a large number of solutions, each with its own unique trade-off. However, this raises two issues: 1) time complexity, as the number of nondominated solutions in general can be infinite for continuous problems and exponentially large for discrete problems, and 2) lack of decisiveness, e.g., it may be difficult for the user to decide amongst the overwhelming number of options. To combat these issues, this project focusses on the computation of a small, “relevant” subset of all optimal solutions called the representative set, which provides meaningful trade-offs between the objectives. The expected result is an algorithm that can quickly compute the representative set and make multi-objective optimisation easier to use in practice.

The project requires good algorithmic. Knowledge on multi-objective optimisation is not necessary. The project is in collaboration with Dr Nicolas Schwind from The National Institute of Advanced Industrial Science and Technology (AIST) in Tokyo, Japan.

Advanced Flow Propagators

The aim of the project is to develop novel algorithms to solve constrained flow problems. Ideally, the algorithms will allow a generic specification of constraints, allowing its application to a wide range of problems without needing to customise the flow algorithm for each application individually. The flow algorithms are envisioned either as a stand-alone solver or part of a constraint programming solver.  

Maximum flow problems involve finding a flow through a flow network to maximise the given objectives, typically to ensure the maximum flow possible. There are many variations of the problem with a wide range of applications, for example scheduling, nurse rostering, and circulation problems.

In real-life application, additional constraints may be imposed in addition to the flow problem, e.g., the flow problem may be one component in a large optimisation problem. In these cases, it is necessary to ensure the flow algorithm can interact and respect other constraints imposed by the user. One possible way to realise this is to incorporate the flow algorithms within a constraint programming framework, which allows users to specify arbitrary constraints on the problem. This gives rise to the challenge of developing an algorithm that can easily adapt throughout the search, effectively solving thousands or millions of similar flow problems within minutes.

The project requires good algorithmic skills. Students not familiar with C/C++ programming are expected to develop their skills on the way.

Interpretable and Constrained Machine Learning in Protein Engineering

The aim of the project is to develop novel machine learning algorithms specifically for applications in protein engineering (bioinformatics).

An outstanding challenge for machine learning in bioinformatics is to derive models that not only represent the data well, but also allow easy interpretation and respect biological properties. As an example, anti-symmetry is a desirable property, which states that when all features from a predefined set change their signs to the opposite, the predictor should return the opposite label in classification or opposite sign in regression. Currently ad-hoc approaches are employed that do not guarantee these types of constraints. This makes interpreting the results and applying them difficult. The student is expected to implement novel machine learning algorithms that overcome these issues with a focus on real-world protein data.

The project requires good algorithmic skills. Students not familiar with C/C++ programming are expected to develop their skills on the way. Knowledge of biology is not necessary. The project is in collaboration with Dr Stas Mazurenko from Masaryk University (Brno, Czechia).

Employee Scheduling using Integer Programming and Column Generation

The aim of the project is to solve staff scheduling problems using advanced optimisation techniques based on integer programming and column generation.

Staff scheduling problems arise whenever there is the need for efficient management and distribution of workforce over periods of time. A wide range of different institutions can benefit from an optimised staff schedule, including hospitals, airlines, security personnel, transportation services, and virtually any organisation that deals with a large number of employees. Finding the ideal workforce roster is, however, not an easy task, and even a basic variant of this problem belongs to the class of NP-hard problems. Nevertheless, these problems are of great importance and the task is to find an efficient algorithm.

The project requires good algorithmic skills. Familiarity with integer programming solvers is a plus, but not necessary. Depending on preference of the student, in collaboration with Dr Yorke-Smith, the project can be connected to shift and staff scheduling problems at KLM.

Optimizing supply temperatures in district heating grids

A district heating grid is used to heat buildings and to provide them with hot tap water. This heat is produced centrally at one or more producers and then transported as hot water to consumers through insulated pipelines. Minimizing production costs while fulfilling the energy demand is a trade off between mass flow and supply temperature. A higher mass flow results in higher pumping costs, and higher supply temperature leads to higher energy losses during transport. The optimization has some challenging non-linearities, most notably the variable temperature propagation time.

This project is in collaboration (or possible internship) with Withthegrid. Withthegrid is a young and fast growing company that focuses on improving energy grids. It does this by designing and delivering internet of things monitoring devices and developing optimisation software. Current customers include Stedin, Eneco and Ennatuurlijk. Our office is currently based in an energy production facility giving a proper industrial look&feel. During your Master thesis/internship you will be working closely with the team and discuss on a daily basis with the engineering team members. Lunches are together and once a month there are Friday afternoon beers with the team. 

Contact: Mathijs de Weerdt

Quality personal transport for elderly and disabled people

Currently the quality of personal transport for elderly and disabled people is very low. The main reason is the heavy competition for the three-year long contracts with governmental institutions. What if people can choose their preferred company per ride? What if people can choose between different pick-up times for their transport?

In this project you will test multiple methods for assigning transportation jobs to taxi companies, such that the preferences and quality for the end-users increases without increasing the costs for the government too much.

Contact: Mathijs de Weerdt

Improving maintenance of trains at NS

NS Maintenance, part of the national Dutch Railway group
(NS group) is responsible for the tender, maintenance and revision of rolling stock, i.e., trains. Trains to be maintained are delivered by NS Travelers, also part of the NS group.

Arrival of trains (release times of maintenance jobs) 
is quite unpredictable. NS Travelers decides exact
 supply of rolling stock at the very last moment and delays  in the schedule occur on a daily basis. The exact duration of maintenance jobs is only known after
 initial inspection jobs, therefore there is significant uncertainty in the operational scheduling; Also, maintenance is performed by a number of teams at different locations, and
 each team constructs its own operational schedule. Furthermore, shunting at service sites, and crew assignment further complicate the problem to be solved.

Several master's thesis projects are possible with an internship at NS in Utrecht, on the following topics:

  • Design a fast scheduling algorithm for the train unit shunting and 
maintenance problem such that rescheduling 
is possible in real-time;
  • Compute capacity of maintenance locations (nodes);
  • Find a method to decompose the maintenance scheduling problem 
into a number of independent subproblems;
  • Use machine learning methods to improve exact methods for solving the combined scheduling and shunting problem;
  • Apply transfer learning to extend policies for smaller shunting yards to larger ones;
  • Improve the current simulated annealing approach by using GOMEA.
  • Take preferences of maintenance engineers, global cost objectives and preferences of other parties into account when scheduling;
  • To determine whether found schedules are any good, we would like to compare the costs to an optimal solution. Typically, however, optimal solutions cannot be found. However, we can have lower bounds (an optimal solution to a relaxed problem). Earlier work has shown how to obtain these for an abstract problem. Next steps are to use these in finding solutions, and to extend these to the realistic setting.
  • Study collaboration between human planner and AI-based decision support system, to take the algorithmic advances closer to practice for NS/NedTrain.

Contact: Mathijs de Weerdt, Bob Huisman

Hybrid optimisation for electroplate assembly lines

Electroplate assembly is an industrial example of a scheduling problem called the hoist scheduling problem. This is an important and complex real world problem. In it we must decide which hoist (robot) works on which track, handles which jobs, and at what times. This project investigates a hybrid approach based on a combination of MIP, CP and possibly ML/RL which you will define and explore. You will experiment on real world data from an industrial partner.

Contact: Neil Yorke-Smith

Comparing centralised and decentralised optimisation for dynamic pickup and delivery

You have a fleet of vehicles. They deliver items to some customers, and they collect items from some customers. You know some of the pickup and delivery jobs in advance. Other jobs arise dynamically as the day progresses. How do you route your vehicles and (re)assign jobs to them?

One approach is centralised: you have a single planning system with all the data, it makes and updates the schedule, and it tells the drivers what to do. Another approach is decentralised: the drivers each have a planning system with limited knowledge, and these planning agents communicate and develop each vehicles' schedule independently. Which approach is best in which situations?

This project is a collaboration with KU Leuven. Contact: Neil Yorke-Smith

Planning for safety and security of critical industrial facilities

How do we design an industrial facility for safety and security? Or how do we retrofit an existing facility to mitigate safety risks and improve response to incidents? How do we ensure fail-safe/fail-secure for the most critical facilities? This thesis will study how to optimize an industrial facility layout for safety, using a combination of bayesian networks, constraint-based models and machine learning.


This project is a collaboration with TPM. Contact: Neil Yorke-Smith

Deep learning for combinatorial optimisation in practice

Recently, deep learning techniques have been tried for combinatorial optimisation problems. For example, travelling salesperson, graph colouring, vehicle routing, train shunting. We want to use the pre-computation and function approximation power of deep neural networks on real and simulated data, in order to help solve real-world combinatorial optimisation more accurately, more quickly, or with better solution quality. With this project you can explore this cutting edge topic, with data from real logistics problems.

This project is a collaboration with TU/e. Contact: Neil Yorke-Smith

Autonomous greenhouse growing: data-driven agriculture

Data science in horticulture, is still in its infancy, has the potential to fight the direct relationship between world hunger and food wastage. This project applies machine learning and data-driven algorithms to the growing of greenhouse tomatoes. Data is available from the World Horti Center (Naaldwijk).

This project is a collaboration with InHolland Delft University, and Pattern Recognition (EWI). Contact: Neil Yorke-Smith

Autonomous greenhouse growing: multiple coordinated optimisation

Algorithms can fill in the gap of the shortage of human expertise in greenhouse agriculture. Blue Radix is a data-driven service provider and innovation leader in the international horticulture market. Their Crop Controller manages crop strategy, climate strategy, climate control in real time. This project seeks to combine the optimisation of the modules of the Crop Controller into a single coherent AI optimisation system.

This project is a collaboration with Blue Radix. Contact: Neil Yorke-Smith

Agent-based modelling of housing market evolution

What factors are important for the residential housing market?  How do buyers' and sellers' behaviours affect prices?  What should you do to win a bidding war?  Should the government change its policy about mortgage rules and taxes?  This project studies the Dutch housing market using agent-based simulation.

This project is a collaboration with TPM. Contact: Neil Yorke-Smith

Simulation of self-organising logistics chains

In het project SOLport doen we onderzoek naar zelforganiserende logistiek in de haven. We onderzoeken onder welke omstandigheden en voor welk type ketens welke vorm van aansturing (centraal versus decentraal, of een tussenvorm) het meest geschikt is. Daarnaast wordt nagegaan wat dit betekent voor de betrokken partijen in de logistiek en wat de voor- en nadelen ervan zijn.

Decentrale aansturing op basis van lokale intelligentie en autonomie van eenheden in de logistieke keten is nog nieuw waardoor er nog veel vragen spelen rond deze vorm van aansturing. Dit onderzoek richt zich dan ook met name op de potentie van decentrale aansturing (zelforganisatie) of mengvormen en de meerwaarde ten opzichte van centrale aansturing.

Een onderdeel is het uitvoeren van een praktisch experiment waarin wordt gekeken welke zelforganiserende elementen kunnen worden toegepast in de praktijk, hoe dit werkt en wat het oplevert. Op basis van een praktijkcase zal een simulatie worden ontwikkeld om verschillende vormen van aansturing met elkaar te vergelijken.

Jouw (afstudeer)stage bestaat uit:

  • Het ontwikkelen van een simulatiemodel op basis van data over een logistieke keten uit de praktijk.
  • Het doorrekenen en analyseren van enkele scenario’s met verschillende vormen van aansturing (centraal versus decentraal).
  • Het beantwoorden van jouw onderzoeksvraag over zelforganiserende logistiek in de praktijk. Deze onderzoeksvraag zullen we in overleg met jou formuleren.

This project is a collaboration with TNO The Hague. Contact: Neil Yorke-Smith

Multi-agent reinforcement learning for autonomous logistics

In logistics, cargo needs to be transported from location A to location B by a limited number of resources (trucks, ships, etc.). To efficiently distribute cargo over resources, human planners have to dynamically assign cargo while dealing with many challenges. For truck planners these include for example strict cut-off times, driver fatigue, matching chassis to containers, etc. The goal: deliver cargo at the customer’s site at exactly the right time with minimal cost. The purpose of this internship is determining whether it is possible to forego a central planner and create a self-organising logistics system where intelligent individual trucks make decisions themselves.

Recent advances in cooperative multi-agent reinforcement learning show large promise for reaching self-organisation in logistics. By considering the trucks as communicating, independent agents that determine among themselves the best suitable candidates to transport a specified container, self-organisation can be accomplished in a highly scalable and efficient way.

The expected results of the internship are a proof of concept of a cooperative multi-agent reinforcement solution of autonomous logistics based on an available simulator of real-world data. 

This project is a collaboration with TNO The Hague. Contact: Neil Yorke-Smith

Matching uncertain demand and supply in smart electricity networks

Integration of renewable energy in power systems is a potential source of uncertainty, because renewable generation is variable and may depend on changing and highly uncertain weather conditions. An example is power generated by wind turbines. Although renewable wind energy is clean and cheap, it may be intermittent and its availability is uncertain and difficult to predict. To reduce peak power consumption and to mitigate the effects of uncertain renewable power supply, electricity usage can be deferred in time such that demand and supply are balanced. An example is the charging process of an electric vehicle, which often does not have to be charged immediately, as long as the available power in the battery is sufficient to reach a destination. In our group we develop efficient planning techniques to automatically coordinate deferrable loads, such that electricity is used when renewable supply is available.

Various MSc projects can be formulated in this domain. Examples include, but are not limited to, coordination of charging electric vehicles, scheduling household appliances, electricity usage within network constraints and predicting electricity demand and supply. We also have an existing collaboration with the Electrical Sustainable Energy department of the EEMCS faculty.

Contact: Erwin Walraven, Matthijs Spaan

Planning under uncertainty

Planning under uncertainty is a technique for enabling agents to successfully plan their decisions in domains with stochastic transitions and partial observability, e.g., because of noisy sensors. In the Artificial Intelligence community the Partially Observable Markov Decision Process (POMDP) and related multiagent models have become popular choices to address these hard planning problems. In the Algorithmics group a considerable body of expertise, algorithms and software is present regarding these methods. This allows for defining challenging projects focused on improving state-of-the-art algorithms with a high potential for publications.

Contact: Matthijs Spaan

Reinforcement learning for transportation, traffic and logistics

Reinforcement learning is a branch of machine learning focusing on agents that are able to automatically learn how they should act in their environment, and has been applied for coordination of multiagent systems in several real-world domains. For instance, reinforcement learning has been used for intelligent control of multiple traffic lights in the urban area, as well as optimization of traffic flow on highways. Other examples in the area of transportation and logistics include air traffic management and automated unloading of ships in a harbor. Typically, a real-world problem is modeled as reinforcement learning problem and then evaluated through realistic simulations.

MSc projects may focus on the application of reinforcement learning algorithms to realistic domains, but also allow for more fundamental research related to reinforcement learning and decision making under uncertainty. In our group expertise and software are present for reinforcement learning in transportation, traffic and logistics domains. However, students may also choose their own problem domain to define an interesting thesis project.

Contact: Erwin Walraven, Matthijs Spaan

Developing a Driving Simulator Environment for Studying Mixed Traffic

The automotive industry jointly with many software companies and automotive suppliers are boosting their efforts to transform the future of transportation by making self-driving cars become a reality. Testing the interactions between automated and human driven vehicles in real life is complex as limited number of automated vehicles drive on our road network. Therefore, testing these interactions in a driving simulator environment in the first stage is more realistic. However, a suitable driving simulator environment which can simulate human driven vehicles and automated vehicles does not exist. Your aim in this thesis is to develop the driving simulator environment, including the integration of existing behavioural models for humans as well as automated vehicles from the literature into the driving simulator environment.

Contact: Matthijs Spaan, Haneen Farah (Transport & Planning, CEG)

Acquiring CSP models with interactive machine learning

Constraint Programming is known as a powerful approach to combinatorial optimisation problems.  Part of the "Holy Grail" of computing is for the user to state the problem and the computer to solve it.  Progress over the last 30 years in CP means the computer can, for many problems, solve a CP model without human intervention.  However, acquiring a suitable CP model still requires human expertise.  This thesis project explores the recent uses of machine learning to acquire improved CP models.

Contact: Neil Yorke-Smith

Machine learning coupled with state-of-the-art OR algorithms

Reinforcement Learning (RL) has revolutionised what computers can do: game playing, speech translation, car driving, making artwork.  Given their success, deep RL techniques are now being applied to combinatorial optimisation problems, such as vehicle routing [1].  These kind of problems are important in business, science and policy making.  Traditionally their study has been called Operations Research (OR) [2].

The aim of this project is understand how RL can be used as an alternative and as a complement to already successful OR models and methods.  Relevant questions could for example be: 

  • What is the latest work on deep neural networks to solve OR problems?
  • How can RL and OR/AI work in synergy?
  • How can machine learning help to acquire models, especially constraint-based models?
  • How can machine learning help to solve models, especially constraint-based models?
  • What are the limitations of deep RL for combinatorial optimisation?

Contact: N. Yorke-Smith

[1] Wouter W. M. Kool, Max Welling: Attention Solves Your TSP. CoRR abs/1803.08475 (2018)
[2] Leo Liberti, Thierry Marchant, Silvano Martello: Twelve surveys in operations research. Annals of OR 240(1): 3-11 (2016)
[3] Michele Lombardi, Michela Milano, Andrea Bartolini: Empirical decision model learning. Artificial Intelligence 244: 343-367 (2017)
[4] Andrea Galassi, Michele Lombardi, Paola Mello, Michela Milano: Model Agnostic Solution of CSPs via Deep Learning: A Preliminary Study. CPAIOR 2018: 254-262
[5] Christian Bessiere, Frédéric Koriche, Nadjib Lazaar, Barry O'Sullivan: Constraint acquisition. Artificial Intelligence 244: 315-342 (2017)

Reinforcement learning in Mixed Integer Programming

This project is a specific instance of the project area Machine learning coupled with state-of-the-art OR algorithms. You will look at the solving process of modern MIP solvers such as Gurobi. There is recent work on injecting ML into specific aspects of MIP solving [1,2,3,4]: you will look at using RL, possibly in combination with other forms of learning.

This project is a collaboration with Mathematics. Contact: Neil Yorke-Smith

[1] Andrea Lodi, Giulia Zarpellon: On learning and branching: A survey. TOP 25(2): 207-236 (2017)
[2] Martina Fischetti, Andrea Lodi, Giulia Zarpellon: Learning MILP Resolution Outcomes Before Reaching Time-Limit. CPAIOR 2019: 275-291
[3] Elias Boutros Khalil, Pierre Le Bodic, Le Song, George L. Nemhauser, Bistra N. Dilkina: Learning to Branch in Mixed Integer Programming. AAAI 2016: 724-731
[4] Maxime Gasse, Didier Chételat, Nicola Ferroni, Laurent Charlin, Andrea Lodi: Exact Combinatorial Optimization with Graph Convolutional Neural Networks. arXiv:1906.01629. 2019

Predict + optimise

In the past, machine learning and combinatorial optimisation components of a solving pipeline are viewed as independent black-boxes: predict the input parameters and then optimise. However, conventional ML metrics, such as mean-square error, are not necessarily indicative for the outcome of the optimisation.

The study of how to better predict + optimise started in 2018. There are three approaches: a) direct methods which interact with the optimisation problem during ML training by considering a relaxed version of the combinatorics, b) semi-direct methods which take into account features of the optimisation problem but do not directly interact during training, and c) indirect methods which ignore the optimisation problem (standard ML approaches). This project can explore either a direct or a semi-direct method. There is a good potential to publish an academic paper.

This project is a collaboration with Vrije Universiteit Brussel.  Contact: Emir Demirovic, Neil Yorke-Smith

Training neural networks with discrete optimisation solvers

Fischetti & Jo (2018) model a simple class of neural network as an integer program. You will build on that initial study to see what classes of neural networks can be modelled. What is feasible? Which discrete optimisation solvers? How deep can we go? Complexity analysis?

Contact: Neil Yorke-Smith

[1] Matteo Fischetti, Jason Jo: Deep neural networks and mixed integer linear optimization. Constraints 23(3): 296-309 (2018)

Unit testing for functional agent-based languages

NetLogo is a functional multi-paradigm programming language for agent-based social simulation. NetLogo is widely used is the social sciences. It is open source, based on Scala. NetLogo has some features for static and dynamic code analysis and a simple profiler. It has no features for unit testing or advance code analysis. This project studies the requirements, feasibility and implementation of a prototype for NetLogo.

This project is a collaboration with TPM faculty. Contact: Neil Yorke-Smith


More Information

The topics listed above are only a sample of possible projects; fully up-to-date information can be obtained by contacting our staff members:

If you are currently not studying in Delft but would like to study here, you will find TU Delft a stimulating academic and social environment. To apply for a MSc place, please read and follow the formal guidelines of Delft University of Technology.