[2008.07146] Open Bandit Dataset and Pipeline: Towards Reali

last revised 26 Oct 2021 (this version， Yusuke Narita View a PDF of the paper titled Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation， there has been growing research interest in this field. There is，194 KB) [v3] Sat， v5)] Title: Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation Authors: Yuta Saito， by Yuta Saito and 3 other authors View PDFHTML (experimental) Abstract: Off-policy evaluation (OPE) aims to estimate the performance of hypothetical policies using data generated by a different policy. Because of its huge potential impact in practice， Computer Science > Machine Learning arXiv:2008.07146 (cs) [Submitted on 17 Aug 2020 (v1)， no real-world public dataset that enables the evaluation of OPE， 6 Feb 2021 18:11:53 UTC (1， however， Megumi Matsutani， 26 Oct 2021 08:57:39 UTC (1。

ZOZOTOWN. Our dataset is unique in that it contains a set of multiple logged bandit datasets collected by running different policies on the same platform. This enables experimental comparisons of different OPE estimators for the first time. We also develop Python software called Open Bandit Pipeline to streamline and standardize the implementation of batch bandit algorithms and OPE. Our open data and software will contribute to fair and transparent OPE research and help the community identify fruitful research directions. We provide extensive benchmark experiments of existing OPE estimators using our dataset and software. The results open up essential challenges and new avenues for future OPE research. Comments: Accepted at NeurIPS2021 Datasets and Benchmarks Track Subjects: Machine Learning (cs.LG) ; Machine Learning (stat.ML) Cite as: arXiv:2008.07146 [cs.LG] (or arXiv:2008.07146v5 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2008.07146 Focus to learn more arXiv-issued DOI via DataCite Submission history From: Yuta Saito [view email] [v1] Mon，。

17 Aug 2020 08:23:50 UTC (1，673 KB) [v2] Sun，703 KB) [v4] Mon。

15 Nov 2020 14:39:36 UTC (2， 27 Sep 2021 21:16:28 UTC (1， we present Open Bandit Dataset， a public logged bandit dataset collected on a large-scale fashion e-commerce platform，848 KB) [v5] Tue， Shunsuke Aihara，121 KB) ， making its experimental studies unrealistic and irreproducible. With the goal of enabling realistic and reproducible OPE research。