Vlado Menkovski bio photo

Vlado Menkovski

Assistant Professor at Eindhoven University of Technology.

Email Google Scholar

Project Description: Transfer learning [1,2,3] is a machine learning method where a model developed for a task is reused as the starting point for a model on a second task. For instance, the accelerating, steering and braking logic learned while driving a car can be transferred to learn driving a truck. By transfer learning machines can learn to adjust to the real world and new tasks in an efficient method. Transfer learning is particularly important for deploying machine learning models into business. Models are usually trained on a specific collection of data which are generated under certain conditions. What these models lack is the ability to generalize to conditions that are different from the ones encountered during training. By transfer learning a machine is able to generalize itself in conditions it hasn’t encountered earlier and ensure its robustness and accuracy in real world applications.

Recently deep reinforcement learning [4] has been adopted to solve the train shunting problem for a service site at NS with great success. Instead of training a Deep Q Network from scratch for each service site, this project aims to explore effective transfer learning methods to transfer the model or part of the model learned on one service site to another service site; given the assumption that these two service sites would have similar but not identical yard layouts.

[1] M. Madden and T. Howley. “Transfer of experience between reinforcement learning environments with progressive difficulty”. Artificial Intelligence Review, 21:375–398, 2004.

[2] T. Perkins and D. Precup. ”Using options for knowledge transfer in reinforcement learning”. Technical Report UM-CS-1999-034, University of Massachusetts, Amherst, 1999.

[3] B. Price and C. Boutilier. ”Implicit imitation in multiagent reinforcement learning”. In International Conference on Machine Learning, 1999.

[4] E. Peer, V. Menkovski, Y. Zhang and W-J. Lee. “Shunting trains with deep reinforcement learning”. The 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC2018), 2018.

Company Description: NS is the major railway operator in the Netherlands and operates one of the most dense rail network in the world.