Plenary Lecture
Multi-Armed Bandits in: Healthcare, Machine Learning and Scheduling
Professor Michael N. Katehakis
Management Science and Information Systems Department
Rutgers University
NJ, USA
E-mail: mkatehakis@gmail.com
Abstract: First, we provide a survey of the basic multi-armed bandit models and problems with emphasis on modern data-driven analytics, e.g. allocation of scarce resources, matching patients to treatments, clinical trials, and scheduling.
Then, we present solutions to two key open problems: The first deals with the situation in which a decision to engage a bandit (process) is subject to a commitment of a, possibly stochastic, number or duration of activations before a change to a different process is possible. It is shown that these activation commitments are equivalent to a more general depreciation model for which optimal policies can be constructed using propitiously defined restart in state indices. The second problems deals with the model in which outcomes from different bandits are normally distributed with unknown means and unknown variances, for which the regret increase rate can be minimized by sequential 'upper confidence bounds' based policies.
Lastly, we discuss three new challenging problems involving 'maximizing rewards under multiple simultaneous activation', 'minimizing total deployment costs', and maximizing rewards from 'cumulative payout models'.
Brief Biography of the Speaker: Michael N. Katehakis is a Professor in the Management Science and Information Systems Department at Rutgers University.
He is known for his work on Markov decision processes, Stochastic Models, data-driven analytics and their application to queuing, reliability and service systems. He has served on many panels (NSF, IEEE, conferences), as judge (for the 2013 and 2014 INFORMS Innovative Applications in Analytics Award, the 2013 INFORMS Interactive Sessions Award, and the Jacob Wolfowitz Prize 1994-2006), and editorial boards including the `Annals of Operations Research', the `Naval Research Logistics', `Operations Research Letters', and the `Probability in the Engineering and Informational Sciences'.
His contributions to the profession have been recognized by INFORMS with an INFORMS Fellow award. He is an Elected member of the International Statistical Institute (ISI) and a Senior Member of the Institute of Electrical and Electronics Engineers (IEEE).
In addition to Rutgers he has taught at Stanford, Columbia, SUNY at Stony Brook and at the University of Athens, and at the University of Crete in Greece. Besides to research and teaching, he works with firms in a number of industries on analytics and process improvement projects and he has held industry positions at Bell-Labs and at Brookhaven National Lab.
More information and his full c.v.is available at: http://www.rci.rutgers.edu/~mnk/