Code for our AJCAI 2020 paper:
"Online Semi-Supervised Learning in Contextual Bandits with Episodic Reward"
by Baihan Lin (Columbia).
For the latest full paper: https://arxiv.org/abs/2009.08457
All the experimental results can be reproduced using the code in this repository. Feel free to contact me by doerlbh@gmail.com if you have any question about our work.
Abstract
We considered a novel practical problem of online learning with episodically revealed rewards, motivated by several real-world applications, where the contexts are nonstationary over different episodes and the reward feedbacks are not always available to the decision making agents. For this online semi-supervised learning setting, we introduced Background Episodic Reward LinUCB (BerlinUCB), a solution that easily incorporates clustering as a self-supervision module to provide useful side information when rewards are not observed. Our experiments on a variety of datasets, both in stationary and nonstationary environments of six different scenarios, demonstrated clear advantages of the proposed approach over the standard contextual bandit. Lastly, we introduced a relevant real-life example where this problem setting is especially useful.
Language: Matlab
Platform: MacOS, Linux, Windows
by Baihan Lin, Dec 2019
If you find this work helpful, please try out the models and cite our works. Thanks!
@inproceedings{lin2020semi,
title={Online Semi-Supervised Learning in Contextual Bandits with Episodic Reward},
author={Lin, Baihan},
booktitle={Australasian Joint Conference on Artificial Intelligence},
year={2020},
organization={Springer}
}
- Matlab
This work is related the following work by the same author and shares codes with their repositories. Feel free to check out and contact the primary author Baihan Lin (doerlbh@gmail.com) for any questions. Thank you!
- https://github.com/doerlbh/ABaCoDE (Lin et al., "Contextual Bandit with Adaptive Feature Extraction", ICDMW 2018)
- https://github.com/doerlbh/MiniVox (Lin & Zhang, "VoiceID on the fly: A Speaker Recognition System that Learns from Scratch", InterSpeech 2020)
- https://github.com/doerlbh/mentalRL (Lin et al., "A Story of Two Streams: Reinforcement Learning Models from Human Behavior and Neuropsychiatry", AAMAS 2020; Lin et al., "Split Q Learning: Reinforcement Learning with Two-Stream Rewards", IJCAI 2019)
- https://github.com/doerlbh/dilemmaRL (Lin et al., "Online Learning in Iterated Prisoner's Dilemma to Mimic Human Behavior", arXiv 2020)