{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T06:38:26Z","timestamp":1781246306985,"version":"3.54.1"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2023,6,19]],"date-time":"2023-06-19T00:00:00Z","timestamp":1687132800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2022YFB4500302"],"award-info":[{"award-number":["2022YFB4500302"]}]},{"name":"Scientific Research Program of National University of Defense Technology","award":["ZK20-03"],"award-info":[{"award-number":["ZK20-03"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62025208"],"award-info":[{"award-number":["62025208"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004761","name":"Natural Science Foundation of Hunan Province of China","doi-asserted-by":"crossref","award":["2022JJ40555"],"award-info":[{"award-number":["2022JJ40555"]}],"id":[{"id":"10.13039\/501100004761","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Storage"],"published-print":{"date-parts":[[2023,8,31]]},"abstract":"<jats:p>\n            This article systematically studies 99 distributed performance bugs from five widely deployed distributed storage and computing systems (Cassandra, HBase, HDFS, Hadoop MapReduce and ZooKeeper). We present the\n            <jats:italic>TaxPerf<\/jats:italic>\n            database, which collectively organizes the analysis results as over 400 classification labels and over 2,500 lines of bug re-description. TaxPerf is classified into six bug categories (and 18 bug subcategories) by their root causes; resource, blocking, synchronization, optimization, configuration, and logic. TaxPerf can be used as a benchmark for performance bug studies and debug tool designs. Although it is impractical to automatically detect all categories of performance bugs in TaxPerf, we find that an important category of blocking bugs can be effectively solved by analysis tools. We analyze the cascading nature of blocking bugs and design an automatic detection tool called\n            <jats:italic>PCatch<\/jats:italic>\n            , which (i) performs program analysis to identify code regions whose execution time can potentially increase dramatically with the workload size; (ii) adapts the traditional happens-before model to reason about software resource contention and performance dependency relationship; and (iii) uses dynamic tracking to identify whether the slowdown propagation is contained in one job. Evaluation shows that PCatch can accurately detect blocking bugs of representative distributed storage and computing systems by observing system executions under small-scale workloads.\n          <\/jats:p>","DOI":"10.1145\/3580281","type":"journal-article","created":{"date-parts":[[2023,1,18]],"date-time":"2023-01-18T11:28:10Z","timestamp":1674041290000},"page":"1-33","update-policy":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Performance Bug Analysis and Detection for Distributed Storage and Computing Systems"],"prefix":"10.1145","volume":"19","author":[{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0002-8963-2535","authenticated-orcid":false,"given":"Jiaxin","family":"Li","sequence":"first","affiliation":[{"name":"National University of Defense Technology, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0001-6450-8485","authenticated-orcid":false,"given":"Yiming","family":"Zhang","sequence":"additional","affiliation":[{"name":"National University of Defense Technology and Xiamen University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0002-0757-4600","authenticated-orcid":false,"given":"Shan","family":"Lu","sequence":"additional","affiliation":[{"name":"University of Chicago, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0003-3680-8450","authenticated-orcid":false,"given":"Haryadi S.","family":"Gunawi","sequence":"additional","affiliation":[{"name":"University of Chicago, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0001-8811-0084","authenticated-orcid":false,"given":"Xiaohui","family":"Gu","sequence":"additional","affiliation":[{"name":"North Carolina State University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0002-0740-9373","authenticated-orcid":false,"given":"Feng","family":"Huang","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0001-9743-2034","authenticated-orcid":false,"given":"Dongsheng","family":"Li","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,6,19]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Apache HBase Project. (n. d.). Retrieved January 29 2023 from https:\/\/2.zoppoz.workers.dev:443\/http\/hbase.apache.org."},{"key":"e_1_3_2_3_2","unstructured":"Apache ZooKeeper Project. (n. d.). Retrieved January 29 2023 from https:\/\/2.zoppoz.workers.dev:443\/http\/zookeeper.apache.org."},{"key":"e_1_3_2_4_2","unstructured":"HDFS Architecture. (n. d.). Retrieved January 29 2023 from https:\/\/2.zoppoz.workers.dev:443\/http\/hadoop.apache.org\/common\/docs\/current\/hdfs_design.html."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/93542.93576"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/945445.945454"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3064176.3064186"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/1869459.1869519"},{"key":"e_1_3_2_9_2","unstructured":"Apache. MapReduce-4576. (n. d.). Retrieved January 29 2023 from https:\/\/2.zoppoz.workers.dev:443\/https\/issues.apache.org\/jira\/browse\/MAPREDUCE-4576."},{"key":"e_1_3_2_10_2","unstructured":"Mona Attariyan Michael Chow and Jason Flinn. 2012. X-ray: Automating root-cause diagnosis of performance anomalies in production software. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201912) . USENIX Association 307\u2013320."},{"key":"e_1_3_2_11_2","first-page":"237","volume-title":"Proceedings of the 9th Symposium on Operating Systems Design and Implementation (OSDI\u201910)","author":"Attariyan Mona","year":"2010","unstructured":"Mona Attariyan and Jason Flinn. 2010. Automating configuration troubleshooting with dynamic information flow analysis. In Proceedings of the 9th Symposium on Operating Systems Design and Implementation (OSDI\u201910). USENIX Association, 237\u2013250."},{"key":"e_1_3_2_12_2","unstructured":"Michael Chow David Meisner Jason Flinn Daniel Peek and Thomas F. Wenisch. 2014. The mystery machine: End-to-end performance analysis of large-scale Internet services. In Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201914) . USENIX Association 217\u2013231."},{"key":"e_1_3_2_13_2","unstructured":"Tyson Condie Neil Conway Peter Alvaro Joseph M. Hellerstein Khaled Elmeleegy and Russell Sears. 2010. MapReduce online. In Proceedings of the 7th USENIX Conference on Networked Systems Design and Implementation (NSDI\u201910) . USENIX Association 21."},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/2254064.2254076"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/2815400.2815409"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/2660193.2660210"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/2670979.2670987"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/2509136.2509539"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/1453101.1453111"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/1287624.1287681"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-02658-4_7"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/1806596.1806630"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/2670979.2670986"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/1963559.1963560"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/2038916.2038934"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568232"},{"key":"e_1_3_2_27_2","unstructured":"IBM. Main Page - WalaWiki. (n. d.). Retrieved January 29 2023 from https:\/\/2.zoppoz.workers.dev:443\/http\/wala.sourceforge.net\/wiki\/index.php\/Main_Page."},{"key":"e_1_3_2_28_2","unstructured":"jboss javassist. Javassist. (n. d.). Retrieved January 29 2023 from https:\/\/2.zoppoz.workers.dev:443\/http\/jboss-javassist.github.io\/javassist\/."},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/2254064.2254075"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/1250734.1250755"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/1882291.1882297"},{"key":"e_1_3_2_32_2","doi-asserted-by":"crossref","unstructured":"Avinash Lakshman and Prashant Malik. 2010. Cassandra - a decentralized structured storage system. ACM SIGOPS Operating Systems Review 44 2 (2010) 35\u201340.","DOI":"10.1145\/1773912.1773922"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/359545.359563"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/2872362.2872374"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3190508.3190552"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2950309"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3037697.3037735"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/2555243.2555244"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/2815400.2815415"},{"key":"e_1_3_2_40_2","unstructured":"Madanlal Musuvathi Shaz Qadeer Thomas Ball Gerard Basler Piramanayagam Arumuga Nainar and Iulian Neamtiu. 2008. Finding and reproducing heisenbugs in concurrent programs. In Proceedings of the 8th USENIX conference on Operating Systems Design and Implementation (OSDI\u201908) . USENIX Association 267\u2013280."},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/2465351.2465366"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/109625.109640"},{"key":"e_1_3_2_43_2","doi-asserted-by":"crossref","unstructured":"Adrian Nistor Po-Chun Chang Cosmin Radoi and Shan Lu. 2015. CARAMEL: Detecting and fixing performance problems that have non-intrusive fixes. In Proceedings of the 37th International Conference on Software Engineering (ICSE\u201915) Volume 1 IEEE Press 902\u2013912.","DOI":"10.1109\/ICSE.2015.100"},{"key":"e_1_3_2_44_2","doi-asserted-by":"crossref","unstructured":"Adrian Nistor Linhai Song Darko Marinov and Shan Lu. 2013. Toddler: Detecting performance problems via similar memory-access patterns. In Proceedings of the International Conference on Software Engineering (ICSE\u201913) . IEEE Press 562\u2013571.","DOI":"10.1109\/ICSE.2013.6606602"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/2737924.2737966"},{"key":"e_1_3_2_46_2","unstructured":"Oracle. HPROF: A heap\/cpu profiling tool. (n. d.). Retrieved January 29 2023 from https:\/\/2.zoppoz.workers.dev:443\/http\/docs.oracle.com\/javase\/7\/docs\/technotes\/samples\/hprof.html."},{"key":"e_1_3_2_47_2","unstructured":"Kai Shen Ming Zhong and Chuanpeng Li. 2005. I\/O system performance debugging using model-driven anomaly characterization. In Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies (FAST\u201905) Volume 4 USENIX Association 23."},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/2660193.2660234"},{"key":"e_1_3_2_49_2","unstructured":"Christopher Stewart Ming Zhong Kai Shen and Thomas O\u2019Neill. 2006. Comprehensive depiction of configuration-dependent performance anomalies in distributed server systems. In Proceedings of the Second conference on Hot topics in System Dependability (HotDep\u201906) . USENIX Association 1."},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2010.63"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3173162.3173206"},{"key":"e_1_3_2_52_2","unstructured":"Yang Wang Manos Kapritsos Lara Schmidt Lorenzo Alvisi and Mike Dahlin. 2014. Exalt: Empowering researchers to evaluate large-scale storage systems. In Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation (NSDI\u201914) . USENIX Association 129\u2013141."},{"key":"e_1_3_2_53_2","unstructured":"Mark Weiser. 1981. Program slicing. In Proceedings of the 2013 International Conference on Software Engineering (ICSE\u201981) . 439\u2013449."},{"key":"e_1_3_2_54_2","unstructured":"Alexander Wert Jens Happe and Lucia Happe. 2013. Supporting swift reaction: Automatically uncovering performance problems by systematic experiments. In Proceedings of the 2013 International Conference on Software Engineering (ICSE\u201913) . IEEE Press 552\u2013561."},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/2483760.2483784"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/1542476.1542523"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/1806596.1806617"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/1629575.1629587"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2931070"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/2541940.2541968"},{"key":"e_1_3_2_61_2","unstructured":"Piotr Zalewski and Jinwoo Hwang. IBM thread and monitor dump analyze for Java. (n. d.). Retrieved January 29 2023 from https:\/\/2.zoppoz.workers.dev:443\/https\/www.ibm.com\/developerworks\/community\/groups\/service\/html\/communityview?communityUuid=2245aa39-fa5c-4475-b891-14c205f7333c."},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/2254064.2254074"},{"key":"e_1_3_2_63_2","unstructured":"Ennan Zhai Ruichuan Chen David Isaac Wolinsky and Bryan Ford. 2014. Heading off correlated failures through independence-as-a-service. In Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201914) . USENIX Association 317\u2013334."},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/1181775.1181786"},{"key":"e_1_3_2_65_2","unstructured":"Xu Zhao Kirk Rodrigues Yu Luo Ding Yuan and Michael Stumm. 2016. Non-intrusive performance profiling for entire software stacks based on the flow reconstruction principle. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201916) . USENIX Association 603\u2013618."}],"container-title":["ACM Transactions on Storage"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/dl.acm.org\/doi\/10.1145\/3580281","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/dl.acm.org\/doi\/pdf\/10.1145\/3580281","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:19Z","timestamp":1750182559000},"score":1,"resource":{"primary":{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/dl.acm.org\/doi\/10.1145\/3580281"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,19]]},"references-count":64,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,8,31]]}},"alternative-id":["10.1145\/3580281"],"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.1145\/3580281","relation":{},"ISSN":["1553-3077","1553-3093"],"issn-type":[{"value":"1553-3077","type":"print"},{"value":"1553-3093","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,19]]},"assertion":[{"value":"2022-04-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-12-29","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}