Online Learning with Survival Data

Decision-makers frequently utilize adaptive experiments to optimize time-to-event outcomes, such as accelerating healthcare screenings or delaying customer churn. Traditional multi-armed bandit algorithms fail in these settings because they assume outcome delays are non-informative, leading practitioners to rely on dichotomization, a heuristic that collapses continuous timing into binary outcomes at a fixed threshold. We introduce "survival bandits," a principled class of algorithms that integrate the Cox proportional hazards model to utilize the full temporal signal of every event. We analytically contrast the regret of both approaches under a large-n limit with Weibull-distributed survival times satisfying the proportional hazards assumption. Our analytical results prove that dichotomization imposes substantial inefficiency, increasing regret by 41% to 54% even under optimal parameterization. We further demonstrate that survival bandits are uniquely robust to event rate uncertainty, whereas dichotomized approaches suffer significant performance degradation due to fragile threshold dependencies. Even when the proportional hazards assumption is violated, we show that survival bandits effectively identify the best arm under realistic scenarios. Simulations using real-world cervical cancer screening data validate our findings, demonstrating that survival bandits consistently reduce regret relative to the best-performing dichotomized algorithms.