Vlado Menkovski bio photo

Vlado Menkovski

Assistant Professor at Eindhoven University of Technology.

Email Google Scholar

Project Description: The Dutch Railways (NS) have a lot of trains. Most of them are in use during peak hours. However at night and off peak hours they have to deal with surplus of this ‘rolling stock’. Simply leaving them on the main rail network is not an option (because it is used for freight trains for instance), and therefore they are stored in a shunting yard. On these yards also maintenance activities take place.

The Train Unit Shunting Problem is a problem the NS faces that encompasses matching incoming trains to outgoing train services, parking the trains on tracks in such order that they can leave when they need to (and no other train is blocking their path), routing the trains to their right positions and planning service activities like cleaning and maintenance tasks. Since the NS is expanding their fleet, it gets increasingly complex to create plans to fit all trains on available tracks, and find feasible solutions.

An simplified example of a (small) shunting yard is shown below.

example of a shunting yard

Recently deep reinforcement learning [1] has been adopted to solve the train shunting problem for a service site at NS with great success. Instead of training a Deep Q Network from scratch for each service site, this project aims to explore effective transfer learning methods to transfer the model or part of the model learned on one service site to another service site; given the assumption that these two service sites would have similar but not identical yard layouts.

For our Train Unit Shunting Problems, we have a instance generator to generate planning instances containing incoming trains, departure trains, and service tasks according to certain pre-defined distribution. Currently the instance generator is based on mixed integer programming to ensure the generated instances fit the pre-defined distribution better. However, the mixed integer programming is a time-consuming optimization procedure.

We would aim to study the possibility of using Generative Adversarial Network (GAN)[2] to generate planning instances. Since the generation of instances is less relevant to the physical layout of service sites, we think this GAN-based instance generator will be generic, fast and universal.

[1] E. Peer, V. Menkovski, Y. Zhang and W-J. Lee. “Shunting trains with deep reinforcement learning”. The 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC2018), 2018.

[2] Goodfellow, Ian, et al. “Generative adversarial nets.” Advances in neural information processing systems. 2014.

Company Description: NS is the major railway operator in the Netherlands and operates one of the most dense rail network in the world.