Becoming the Multi-armed Bandit