Tenth Workshop on Data Management for End-to-End Machine Learning (DEEM).

Bojan Karlas https://orcid.org/0000-0002-6462-3579 https://www.wikidata.org/entity/Q130991767 Liane Vogel Bojan Karlas Stefan Grafberger Matteo Interlandi Tenth Workshop on Data Management for End-to-End Machine Learning (DEEM). 608-609 2026 SIGMOD Companion https://doi.org/10.1145/3788853.3802504 conf/sigmod/2026c db/conf/sigmod/sigmod2026c.html#VogelKGI26 Liane Vogel Bojan Karlas Stefan Grafberger Matteo Interlandi Proceedings of the Data Management for End-to-End Machine Learning, DEEM 2026, Bengaluru, India, 31 May 2026- 5 June 2026 DEEM@SIGMOD ACM 2026 979-8-4007-2668-2 https://doi.org/10.1145/3807899 db/conf/deem/deem2026.html Bojan Karlas Babak Salimi Sebastian Schelter Navigating Data Errors in Machine Learning Pipelines: Identify, Debug, and Learn. 4523-4529 2025 ICDE https://doi.org/10.1109/ICDE65448.2025.00344 conf/icde/2025 db/conf/icde/icde2025.html#KarlasSS25 streams/conf/icde Barrie Kersbergen Olivier Sprangers Bojan Karlas Maarten de Rijke Sebastian Schelter Scalable Data Debugging for Neighborhood-based Recommendation with Data Shapley Values. 441-450 2025 RecSys https://doi.org/10.1145/3705328.3748049 conf/recsys/2025 db/conf/recsys/recsys2025.html#KersbergenSKRS25 Bojan Karlas Babak Salimi Sebastian Schelter Navigating Data Errors in Machine Learning Pipelines: Identify, Debug, and Learn. 813-820 2025 SIGMOD Conference Companion https://doi.org/10.1145/3722212.3725636 conf/sigmod/2025c db/conf/sigmod/sigmod2025c.html#KarlasSS25

Luis Oala Manil Maskey Lilith Bat-Leah Alicia Parrish Nezihe Merve Gürel Tzu-Sheng Kuo Yang Liu 0018 Rotem Dror Danilo Brajovic Xiaozhe Yao Max Bartolo William Gaviria Rojas Ryan Hileman Rainier Aliment Michael W. Mahoney Meg Risdal Matthew Lease Wojciech Samek Debojyoti Dutta Curtis G. Northcutt Cody Coleman Braden Hancock Bernard Koch Girmaw Abebe Tadesse Bojan Karlas Ahmed Alaa 0001 Adji Bousso Dieng Natasha F. Noy Vijay Janapa Reddi James Zou 0001 Praveen K. Paritosh Mihaela van der Schaar Kurt Bollacker Lora Aroyo Ce Zhang 0001 Joaquin Vanschoren Isabelle Guyon Peter Mattson DMLR: Data-centric Machine Learning Research - Past, Present and Future. 2024 (5):1-27 1 J. Data-centric Mach. Learn. Res. https://data.mlr.press/assets/pdf/v01-5.pdf db/journals/dmlr/dmlr1.html#OalaMBPGKLDBYBRHAMRLSDN24 streams/journals/dmlr

Bojan Karlas David Dao Matteo Interlandi Sebastian Schelter Wentao Wu 0001 Ce Zhang 0001 Data Debugging with Shapley Importance over Machine Learning Pipelines. 2024 ICLR https://openreview.net/forum?id=qxGXjWxabq conf/iclr/2024 db/conf/iclr/iclr2024.html#KarlasDIS0024 Mark Mazumder Colby R. Banbury Xiaozhe Yao Bojan Karlas William Gaviria Rojas Sudnya Frederick Diamos Greg Diamos Lynn He Alicia Parrish Hannah Rose Kirk Jessica Quaye Charvi Rastogi Douwe Kiela David Jurado David Kanter Rafael Mosquera Will Cukierski Juan Ciro Lora Aroyo Bilge Acun Lingjiao Chen Mehul Raje Max Bartolo Evan Sabri Eyuboglu Amirata Ghorbani Emmett D. Goodman Addison Howard Oana Inel Tariq Kane Christine R. Kirkpatrick D. Sculley Tzu-Sheng Kuo Jonas W. Mueller Tristan Thrush Joaquin Vanschoren Margaret Warren Adina Williams Serena Yeung Newsha Ardalani Praveen K. Paritosh Ce Zhang 0001 James Y. Zou Carole-Jean Wu Cody Coleman Andrew Y. Ng Peter Mattson Vijay Janapa Reddi DataPerf: Benchmarks for Data-Centric AI Development. 2023 NeurIPS http://papers.nips.cc/paper_files/paper/2023/hash/112db88215e25b3ae2750e9eefcded94-Abstract-Datasets_and_Benchmarks.html conf/nips/2023 db/conf/nips/neurips2023.html#MazumderBYKRDDH23 Sebastian Schelter Stefan Grafberger Shubha Guha Bojan Karlas Ce Zhang 0001 Proactively Screening Machine Learning Pipelines with ARGUSEYES. 91-94 2023 SIGMOD Conference Companion https://doi.org/10.1145/3555041.3589682 https://www.wikidata.org/entity/Q130991768 conf/sigmod/2023c db/conf/sigmod/sigmod2023c.html#SchelterGGK023 Maurice Weber Xiaojun Xu Bojan Karlas Ce Zhang 0001 Bo Li 0026 RAB: Provable Robustness Against Backdoor Attacks. 1311-1328 2023 SP https://doi.org/10.1109/SP46215.2023.10179451 conf/sp/2023 db/conf/sp/sp2023.html#WeberXKZL23

Luis Oala Manil Maskey Lilith Bat-Leah Alicia Parrish Nezihe Merve Gürel Tzu-Sheng Kuo Yang Liu 0018 Rotem Dror Danilo Brajovic Xiaozhe Yao Max Bartolo William Gaviria Rojas Ryan Hileman Rainier Aliment Michael W. Mahoney Meg Risdal Matthew Lease Wojciech Samek Debojyoti Dutta Curtis G. Northcutt Cody Coleman Braden Hancock Bernard Koch Girmaw Abebe Tadesse Bojan Karlas Ahmed M. Alaa Adji Bousso Dieng Natasha F. Noy Vijay Janapa Reddi James Zou 0001 Praveen K. Paritosh Mihaela van der Schaar Kurt Bollacker Lora Aroyo Ce Zhang 0001 Joaquin Vanschoren Isabelle Guyon Peter Mattson DMLR: Data-centric Machine Learning Research - Past, Present and Future. 2023 abs/2311.13028 CoRR https://doi.org/10.48550/arXiv.2311.13028 db/journals/corr/corr2311.html#abs-2311-13028

Bojan Karlas Data Systems for Managing and Debugging Machine Learning Workflows. 2022 ETH Zurich, Zürich, Switzerland https://hdl.handle.net/20.500.11850/554603 https://doi.org/10.3929/ethz-b-000554603 https://www.base-search.net/Record/a6918c5b213969301dc3f3a9cb0026c612af051646e93e82d29d9308b05d1e9c base-search.net (ftethz:oai:www.research-collection.ethz.ch:20.500.11850/554603)

Fotis Psallidas Yiwen Zhu Bojan Karlas Jordan Henkel Matteo Interlandi Subru Krishnan Brian Kroth K. Venkatesh Emani Wentao Wu 0001 Ce Zhang 0001 Markus Weimer Avrilia Floratou Carlo Curino Konstantinos Karanasos Data Science Through the Looking Glass: Analysis of Millions of GitHub Notebooks and ML.NET Pipelines. 30-37 2022 51 SIGMOD Rec. 2 https://doi.org/10.1145/3552490.3552496 db/journals/sigmod/sigmod51.html#PsallidasZKHIKK22

Sebastian Schelter Stefan Grafberger Shubha Guha Olivier Sprangers Bojan Karlas Ce Zhang 0001 Screening Native Machine Learning Pipelines with ArgusEyes. 2022 CIDR https://vldb.org/cidrdb/2022/screening-native-machine-learning-pipelines-with-arguseyes.html conf/cidr/2022 db/conf/cidr/cidr2022.html#SchelterGGSK022 Sabri Eyuboglu Bojan Karlas Christopher Ré Ce Zhang 0001 James Zou 0001 dcbench: a benchmark for data-centric AI systems. 9:1-9:4 2022 DEEM@SIGMOD https://doi.org/10.1145/3533028.3533310 conf/deem/2022 db/conf/deem/deem2022.html#EyubogluKR0022 Zhuolin Yang Zhikuan Zhao Boxin Wang Jiawei Zhang 0013 Linyi Li 0001 Hengzhi Pei Bojan Karlas Ji Liu 0002 Heng Guo 0001 Ce Zhang 0001 Bo Li 0026 Improving Certified Robustness via Statistical Learning with Logical Reasoning. 2022 conf/nips/2022 NeurIPS http://papers.nips.cc/paper_files/paper/2022/hash/e1b248453bca182b6138b8c14a75340d-Abstract-Conference.html db/conf/nips/neurips2022.html#YangZWZLPKLGZL22 Sebastian Schelter Stefan Grafberger Shubha Guha Olivier Sprangers Bojan Karlas Ce Zhang 0001 schelterlabs/arguseyes. 2022 January https://doi.org/10.5281/zenodo.7661608 Zenodo streams/repo/zenodo

Bojan Karlas David Dao Matteo Interlandi Bo Li 0026 Sebastian Schelter Wentao Wu 0001 Ce Zhang 0001 Data Debugging with Shapley Importance over End-to-End Machine Learning Pipelines. 2022 abs/2204.11131 CoRR https://doi.org/10.48550/arXiv.2204.11131 db/journals/corr/corr2204.html#abs-2204-11131

Mark Mazumder Colby R. Banbury Xiaozhe Yao Bojan Karlas William Gaviria Rojas Sudnya Frederick Diamos Greg Diamos Lynn He Douwe Kiela David Jurado David Kanter Rafael Mosquera Juan Ciro Lora Aroyo Bilge Acun Sabri Eyuboglu Amirata Ghorbani Emmett D. Goodman Tariq Kane Christine R. Kirkpatrick Tzu-Sheng Kuo Jonas Mueller 0001 Tristan Thrush Joaquin Vanschoren Margaret Warren Adina Williams Serena Yeung Newsha Ardalani Praveen K. Paritosh Ce Zhang 0001 James Zou 0001 Carole-Jean Wu Cody Coleman Andrew Y. Ng Peter Mattson Vijay Janapa Reddi DataPerf: Benchmarks for Data-Centric AI Development. 2022 abs/2207.10062 CoRR https://doi.org/10.48550/arXiv.2207.10062 db/journals/corr/corr2207.html#abs-2207-10062

Cédric Renggli Luka Rimanic Nezihe Merve Gürel Bojan Karlas Wentao Wu 0001 Ce Zhang 0001 A Data Quality-Driven View of MLOps. 11-23 2021 44 IEEE Data Eng. Bull. 1 http://sites.computer.org/debull/A21mar/p11.pdf db/journals/debu/debu44.html#RenggliRGK0021

Mohammad Reza Karimi Nezihe Merve Gürel Bojan Karlas Johannes Rausch Ce Zhang 0001 Andreas Krause 0001 Online Active Model Selection for Pre-trained Classifiers. 307-315 2021 AISTATS http://proceedings.mlr.press/v130/reza-karimi21a.html conf/aistats/2021 db/conf/aistats/aistats2021.html#KarimiGKR0021 Leonel Aguilar Melgar David Dao Shaoduo Gan Nezihe Merve Gürel Nora Hollenstein Jiawei Jiang 0001 Bojan Karlas Thomas Lemmin Tian Li 0005 Yang Li 0106 Susie Xi Rao Johannes Rausch Cédric Renggli Luka Rimanic Maurice Weber Shuai Zhang 0007 Zhikuan Zhao Kevin Schawinski Wentao Wu 0001 Ce Zhang 0001 Ease.ML: A Lifecycle Management System for Machine Learning. 2021 CIDR https://vldb.org/cidrdb/2021/easeml-a-lifecycle-management-system-for-machine-learning.html conf/cidr/2021 db/conf/cidr/cidr2021.html#MelgarDGGHJKL0L21

Cédric Renggli Luka Rimanic Nezihe Merve Gürel Bojan Karlas Wentao Wu 0001 Ce Zhang 0001 A Data Quality-Driven View of MLOps. 2021 abs/2102.07750 CoRR https://arxiv.org/abs/2102.07750 db/journals/corr/corr2102.html#abs-2102-07750

Bojan Karlas Peng Li 0062 Renzhi Wu Nezihe Merve Gürel Xu Chu 0002 Wentao Wu 0001 Ce Zhang 0001 Nearest Neighbor Classifiers over Incomplete Information: From Certain Answers to Certain Predictions. 255-267 2020 14 Proc. VLDB Endow. 3 http://www.vldb.org/pvldb/vol14/p255-karlas.pdf https://doi.org/10.5555/3430915.3442426 db/journals/pvldb/pvldb14.html#KarlasLWGC0020

Bojan Karlas Matteo Interlandi Cédric Renggli Wentao Wu 0001 Ce Zhang 0001 Deepak Mukunthu Iyappan Babu Jordan Edwards Chris Lauren Andy Xu Markus Weimer Building Continuous Integration Services for Machine Learning. 2407-2415 2020 KDD https://doi.org/10.1145/3394486.3403290 conf/kdd/2020 db/conf/kdd/kdd2020.html#KarlasIR0ZBELXW20

Zhuolin Yang Zhikuan Zhao Hengzhi Pei Boxin Wang Bojan Karlas Ji Liu 0002 Heng Guo 0001 Bo Li 0026 Ce Zhang 0001 End-to-end Robustness for Sensing-Reasoning Machine Learning Pipelines. 2020 abs/2003.00120 CoRR https://arxiv.org/abs/2003.00120 db/journals/corr/corr2003.html#abs-2003-00120

Maurice Weber Xiaojun Xu Bojan Karlas Ce Zhang 0001 Bo Li 0026 RAB: Provable Robustness Against Backdoor Attacks. 2020 abs/2003.08904 CoRR https://arxiv.org/abs/2003.08904 db/journals/corr/corr2003.html#abs-2003-08904

Bojan Karlas Peng Li 0062 Renzhi Wu Nezihe Merve Gürel Xu Chu 0002 Wentao Wu 0001 Ce Zhang 0001 Nearest Neighbor Classifiers over Incomplete Information: From Certain Answers to Certain Predictions. 2020 abs/2005.05117 CoRR https://arxiv.org/abs/2005.05117 db/journals/corr/corr2005.html#abs-2005-05117

Mohammad Reza Karimi Nezihe Merve Gürel Bojan Karlas Johannes Rausch Ce Zhang 0001 Andreas Krause 0001 Online Active Model Selection for Pre-trained Classifiers. 2020 abs/2010.09818 CoRR https://arxiv.org/abs/2010.09818 db/journals/corr/corr2010.html#abs-2010-09818

Cédric Renggli Frances Ann Hubis Bojan Karlas Kevin Schawinski Wentao Wu 0001 Ce Zhang 0001 Ease.ml/ci and Ease.ml/meter in Action: Towards Data Management for Statistical Generalization. 1962-1965 2019 12 Proc. VLDB Endow. 12 http://www.vldb.org/pvldb/vol12/p1962-renggli.pdf https://doi.org/10.14778/3352063.3352110 https://www.wikidata.org/entity/Q127232350 db/journals/pvldb/pvldb12.html#RenggliHKSWZ19

Chen Yu 0009 Bojan Karlas Jie Zhong Ce Zhang 0001 Ji Liu 0002 AutoML from Service Provider's Perspective: Multi-device, Multi-tenant Model Selection with GP-EI. 2829-2838 2019 AISTATS http://proceedings.mlr.press/v89/yu19e.html conf/aistats/2019 db/conf/aistats/aistats2019.html#YuKZZL19 Cédric Renggli Bojan Karlas Bolin Ding Feng Liu Kevin Schawinski Wentao Wu 0001 Ce Zhang 0001 Continuous Integration of Machine Learning Models with ease.ml/ci: Towards a Rigorous Yet Practical Treatment. 2019 SysML https://proceedings.mlsys.org/paper_files/paper/2019/hash/4284d31e68c0a4a39dcdad167ac4bd72-Abstract.html conf/mlsys/2019 db/conf/mlsys/mlsys2019.html#RenggliKDLS0Z19 Vojislav Dukic Sangeetha Abdu Jyothi Bojan Karlas Muhsen Owaida Ce Zhang 0001 Ankit Singla Is advance knowledge of flow sizes a plausible assumption? 565-580 2019 NSDI https://www.usenix.org/conference/nsdi19/presentation/dukic conf/nsdi/2019 db/conf/nsdi/nsdi2019.html#DukicJKOZS19

Cédric Renggli Bojan Karlas Bolin Ding Feng Liu Kevin Schawinski Wentao Wu 0001 Ce Zhang 0001 Continuous Integration of Machine Learning Models with ease.ml/ci: Towards a Rigorous Yet Practical Treatment. 2019 abs/1903.00278 CoRR http://arxiv.org/abs/1903.00278 db/journals/corr/corr1903.html#abs-1903-00278

Fotis Psallidas Yiwen Zhu Bojan Karlas Matteo Interlandi Avrilia Floratou Konstantinos Karanasos Wentao Wu 0001 Ce Zhang 0001 Subru Krishnan Carlo Curino Markus Weimer Data Science through the looking glass and what we found there. 2019 abs/1912.09536 CoRR http://arxiv.org/abs/1912.09536 db/journals/corr/corr1912.html#abs-1912-09536

Bojan Karlas Ji Liu 0002 Wentao Wu 0001 Ce Zhang 0001 Ease.ml in Action: Towards Multi-tenant Declarative Learning Services. 2054-2057 2018 11 Proc. VLDB Endow. 12 http://www.vldb.org/pvldb/vol11/p2054-karlas.pdf https://doi.org/10.14778/3229863.3236258 db/journals/pvldb/pvldb11.html#KarlasLWZ18

Vojislav Dukic Sangeetha Abdu Jyothi Bojan Karlas Muhsen Owaida Ce Zhang 0001 Ankit Singla Network Scheduling in the Dark. 516 2018 SoCC https://doi.org/10.1145/3267809.3275458 conf/cloud/2018 db/conf/cloud/socc2018.html#DukicJKOZS18

Chen Yu 0009 Bojan Karlas Jie Zhong Ce Zhang 0001 Ji Liu 0002 Multi-device, Multi-tenant Model Selection with GP-EI. 2018 abs/1803.06561 CoRR http://arxiv.org/abs/1803.06561 db/journals/corr/corr1803.html#abs-1803-06561

Hamza Harkous Rameez Rahman Bojan Karlas Karl Aberer The Curious Case of the PDF Converter that Likes Mozart: Dissecting and Mitigating the Privacy Risk of Personal Cloud Apps. 123-143 2016 2016 Proc. Priv. Enhancing Technol. 4 https://doi.org/10.1515/popets-2016-0032 db/journals/popets/popets2016.html#HarkousRKA16

Hamza Harkous Rameez Rahman Bojan Karlas Karl Aberer The Curious Case of the PDF Converter that Likes Mozart: Dissecting and Mitigating the Privacy Risk of Personal Cloud Apps. 2016 abs/1608.05661 CoRR http://arxiv.org/abs/1608.05661 db/journals/corr/corr1608.html#HarkousRKA16

Girmaw AbebeGirmaw Abebe Tadesse Karl Aberer Bilge Acun Ahmed Alaa 0001Ahmed M. Alaa Rainier Aliment Newsha Ardalani Lora Aroyo Deepak Mukunthu Iyappan Babu Colby R. Banbury Max Bartolo Lilith Bat-Leah Kurt Bollacker Danilo Brajovic Lingjiao Chen Xu Chu 0002 Juan Ciro Cody Coleman Will Cukierski Carlo Curino David Dao Gregory Frederick DiamosGreg Diamos Sudnya Frederick Diamos Adji B. DiengAdji Bousso Dieng Bolin Ding Rotem Dror Vojislav Dukic Debojyoti Dutta Jordan Edwards K. Venkatesh Emani Sabri EyubogluEvan Sabri Eyuboglu Avrilia Floratou Shaoduo Gan Amirata Ghorbani Emmett D. Goodman Stefan Grafberger Shubha Guha Heng Guo 0001 Nezihe Merve Gürel Isabelle Guyon Braden Hancock Hamza Harkous Lynn He Jordan Henkel Ryan Hileman Nora Hollenstein Addison Howard Frances Ann Hubis Oana Inel Matteo Interlandi Jiawei Jiang 0001 David Jurado Sangeetha Abdu Jyothi Tariq Kane David Kanter Konstantinos Karanasos Mohammad Reza Karimi Barrie Kersbergen Douwe Kiela Hannah KirkHannah Rose Kirk Christine R. Kirkpatrick Bernard Koch Andreas Krause 0001 Subru Krishnan Brian Kroth Tzu-Sheng Kuo Chris Lauren Matthew Lease Thomas Lemmin Bo Li 0026 Linyi Li 0001 Peng Li 0062 Tian Li 0005 Yang Li 0106 Feng Liu Ji Liu 0002 Yang Liu 0018 Michael W. Mahoney Manil Maskey Peter Mattson Mark Mazumder Leonel Aguilar Melgar Rafael Mosquera Jonas Mueller 0001Jonas W. Mueller Andrew Y. Ng Curtis G. Northcutt Natasha F. Noy Luis Oala Muhsen Owaida Praveen K. Paritosh Alicia Parrish Hengzhi Pei Fotis Psallidas Jessica Quaye Rameez Rahman Mehul Raje Susie Xi Rao Charvi Rastogi Johannes Rausch Christopher Ré Vijay Janapa Reddi Cédric Renggli Maarten de Rijke Luka Rimanic Megan RisdalMeg Risdal William Gaviria Rojas Babak Salimi Wojciech Samek Mihaela van der Schaar Kevin Schawinski Sebastian Schelter D. Sculley Ankit Singla Olivier Sprangers Tristan Thrush Joaquin Vanschoren Liane Vogel Boxin Wang Margaret Warren Maurice Weber Markus Weimer Adina Williams Carole-Jean Wu Renzhi Wu Wentao Wu 0001 Andy Xu Xiaojun Xu Zhuolin Yang Xiaozhe Yao Serena Yeung-LevySerena Yeung Chen Yu 0009 Ce Zhang 0001 Jiawei Zhang 0013 Shuai Zhang 0007 Zhikuan Zhao Jie Zhong Yiwen Zhu James Zou 0001James Y. Zou