{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T16:00:23Z","timestamp":1773331223239,"version":"3.50.1"},"reference-count":53,"publisher":"Wiley","issue":"3","license":[{"start":{"date-parts":[[2024,6,10]],"date-time":"2024-06-10T00:00:00Z","timestamp":1717977600000},"content-version":"vor","delay-in-days":9,"URL":"https:\/\/2.zoppoz.workers.dev:443\/http\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100014013","name":"UK Research and Innovation","doi-asserted-by":"publisher","award":["EP\/S023356\/1"],"award-info":[{"award-number":["EP\/S023356\/1"]}],"id":[{"id":"10.13039\/100014013","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Computer Graphics Forum"],"published-print":{"date-parts":[[2024,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In natural language processing (NLP), text classification tasks are increasingly fine\u2010grained, as datasets are fragmented into a larger number of classes that are more difficult to differentiate from one another. As a consequence, the semantic structures of datasets have become more complex, and model decisions more difficult to explain. Existing tools, suited for coarse\u2010grained classification, falter under these additional challenges. In response to this gap, we worked closely with NLP domain experts in an iterative design\u2010and\u2010evaluation process to characterize and tackle the growing requirements in their workflow of developing fine\u2010grained text classification models. The result of this collaboration is the development of SemLa, a novel Visual Analytics system tailored for 1) dissecting complex semantic structures in a dataset when it is spatialized in model embedding space, and 2) visualizing fine\u2010grained nuances in the meaning of text samples to faithfully explain model reasoning. This paper details the iterative design study and the resulting innovations featured in SemLa. The final design allows contrastive analysis at different levels by unearthing lexical and conceptual patterns including biases and artifacts in data. Expert feedback on our final design and case studies confirm that SemLa is a useful tool for supporting model validation and debugging as well as data annotation.<\/jats:p>","DOI":"10.1111\/cgf.15098","type":"journal-article","created":{"date-parts":[[2024,6,10]],"date-time":"2024-06-10T14:42:46Z","timestamp":1718030566000},"update-policy":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Visual Analytics for Fine\u2010grained Text Classification Models and Datasets"],"prefix":"10.1111","volume":"43","author":[{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0001-7562-414X","authenticated-orcid":false,"given":"M.","family":"Battogtokh","sequence":"first","affiliation":[{"name":"King's College Londonz  United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0003-1521-6616","authenticated-orcid":false,"given":"Y.","family":"Xing","sequence":"additional","affiliation":[{"name":"King's College Londonz  United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0009-0003-8493-4563","authenticated-orcid":false,"given":"C.","family":"Davidescu","sequence":"additional","affiliation":[{"name":"ContactEngine  United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0002-6257-876X","authenticated-orcid":false,"given":"A.","family":"Abdul\u2010Rahman","sequence":"additional","affiliation":[{"name":"King's College Londonz  United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0002-0926-2061","authenticated-orcid":false,"given":"M.","family":"Luck","sequence":"additional","affiliation":[{"name":"King's College Londonz  United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0003-2875-6793","authenticated-orcid":false,"given":"R.","family":"Borgo","sequence":"additional","affiliation":[{"name":"King's College Londonz  United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2024,6,10]]},"reference":[{"key":"e_1_2_12_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2023.3326569"},{"key":"e_1_2_12_3_2","doi-asserted-by":"crossref","unstructured":"BrooksM. AmershiS. LeeB. DruckerS. M. KapoorA. SimardP.: FeatureInsight: Visual support for error\u2010driven feature ideation in text classification. In2015 IEEE Conference on Visual Analytics Science and Technology (VAST)(2015) pp.105\u2013112. doi:10.1109\/VAST.2015.7347637. 2","DOI":"10.1109\/VAST.2015.7347637"},{"key":"e_1_2_12_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/978\u20103\u2010031\u201050396\u20102_23"},{"issue":"30","key":"e_1_2_12_5_2","first-page":"993","article-title":"Latent dirichlet allocation","volume":"3","author":"Blei D. M.","year":"2003","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_12_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030342"},{"key":"e_1_2_12_7_2","unstructured":"CasanuevaI. Tem\u010dinasT. GerzD. HendersonM. Vuli\u0107I.: Efficient intent detection with dual sentence encoders. InProceedings of the 2nd Workshop on NLP for Conversational AI(July2020) pp.38\u201345. doi:10.18653\/V1\/2020.NLP4CONVAI\u20101.5. 1 2 7 8"},{"key":"e_1_2_12_8_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i10.21292"},{"key":"e_1_2_12_9_2","unstructured":"DevlinJ. ChangM.\u2010W. LeeK. ToutanovaK.: BERT: Pre\u2010training of deep bidirectional transformers for language understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics(Minneapolis Minnesota June2019) pp.4171\u20134186. doi:10.18653\/v1\/N19\u20101423. 7"},{"key":"e_1_2_12_10_2","doi-asserted-by":"crossref","unstructured":"DeYoungJ. JainS. RajaniN. F. LehmanE. XiongC. SocherR. WallaceB. C.: ERASER: A benchmark to evaluate rationalized NLP models. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics(Online July2020) pp.4443\u20134458. doi:10.18653\/v1\/2020.acl\u2010main.408. 3","DOI":"10.18653\/v1\/2020.acl-main.408"},{"key":"e_1_2_12_11_2","doi-asserted-by":"crossref","unstructured":"DemszkyD. Movshovitz\u2010AttiasD. KoJ. CowenA. NemadeG. RaviS.: GoEmotions: A dataset of fine\u2010grained emotions. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics(Online July2020) pp.4040\u20134054. doi:10.18653\/v1\/2020.acl\u2010main.372. 8","DOI":"10.18653\/v1\/2020.acl-main.372"},{"key":"e_1_2_12_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2934654"},{"key":"e_1_2_12_13_2","doi-asserted-by":"crossref","unstructured":"EberleO. ChalkidisI. CabelloL. BrandlS.: Rather a nurse than a physician \u2010 contrastive explanations under investigation. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing(Singapore Dec.2023) BouamorH. PinoJ. BaliK. (Eds.) pp.6907\u20136920. doi:10.18653\/v1\/2023.emnlp\u2010main.427. 1 8","DOI":"10.18653\/v1\/2023.emnlp-main.427"},{"key":"e_1_2_12_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2944182"},{"key":"e_1_2_12_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.visinf.2023.08.002"},{"key":"e_1_2_12_16_2","doi-asserted-by":"crossref","unstructured":"GomezO. HolterS. YuanJ. BertiniE.: ViCE: visual counterfactual explanations for machine learning models. InProceedings of the 25th International Conference on Intelligent User Interfaces(New York NY USA 2020) IUI '20 p.531\u2013535. doi:10.1145\/3377325.3377536. 3","DOI":"10.1145\/3377325.3377536"},{"key":"e_1_2_12_17_2","doi-asserted-by":"crossref","unstructured":"GomezO. HolterS. YuanJ. BertiniE.: AdViCE: Aggregated visual counterfactual explanations for machine learning model validation. In2021 IEEE Visualization Conference (VIS)(2021) pp.31\u201335. doi:10.1109\/VIS49827.2021.9623271. 3","DOI":"10.1109\/VIS49827.2021.9623271"},{"key":"e_1_2_12_18_2","unstructured":"GrootendorstM.: BERTopic: Neural topic modeling with a class\u2010based TF\u2010IDF procedure.arXiv preprint(2022). doi:10.48550\/arXiv.2203.05794. 6"},{"key":"e_1_2_12_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030350"},{"key":"e_1_2_12_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2843369"},{"key":"e_1_2_12_21_2","doi-asserted-by":"crossref","unstructured":"JacoviA. GoldbergY.: Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness? InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics(Online July2020) pp.4198\u20134205. doi:10.18653\/v1\/2020.acl\u2010main.386. 3","DOI":"10.18653\/v1\/2020.acl-main.386"},{"key":"e_1_2_12_22_2","doi-asserted-by":"crossref","unstructured":"JeonH. KuoY.\u2010H. AupetitM. MaK.\u2010L. SeoJ.: Classes are not clusters: Improving label\u2010based evaluation of dimensionality reduction.IEEE Transactions on Visualization and Computer Graphics(2023). doi:10.48550\/arXiv.2308.00278. 5","DOI":"10.1109\/TVCG.2023.3327187"},{"key":"e_1_2_12_23_2","doi-asserted-by":"crossref","unstructured":"JacoviA. SwayamdiptaS. RavfogelS. ElazarY. ChoiY. GoldbergY.: Contrastive explanations for model inter\u2010pretability. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing(Online and Punta Cana Dominican Republic Nov.2021) pp.1597\u20131611. doi:10.18653\/v1\/2021.emnlp\u2010main.120. 2 3","DOI":"10.18653\/v1\/2021.emnlp-main.120"},{"key":"e_1_2_12_24_2","unstructured":"JainS. WallaceB. C.: Attention is not explanation. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics(Minneapolis Minnesota June2019) pp.3543\u20133556. doi:10.18653\/v1\/N19\u20101357. 3"},{"key":"e_1_2_12_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.2981456"},{"key":"e_1_2_12_26_2","doi-asserted-by":"crossref","unstructured":"LiebeL. BaumJ. Sch\u00fctzeT. CechT. ScheibelW. D\u00f6llnerJ.: unCover: Identifying AI generated news articles by linguistic analysis and visualization. InProceedings of the 15th International Joint Conference on Knowledge Discovery Knowledge Engineering and Knowledge Management \u2010 KDIR(2023) pp.39\u201350. doi:10.5220\/0012163300003598. 2","DOI":"10.5220\/0012163300003598"},{"key":"e_1_2_12_27_2","doi-asserted-by":"crossref","unstructured":"LiJ. ChenX. HovyE. JurafskyD.: Visualizing and understanding neural models in NLP. InProceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics(San Diego California June2016) pp.681\u2013691. doi:10.18653\/v1\/N16\u20101082. 2","DOI":"10.18653\/v1\/N16-1082"},{"key":"e_1_2_12_28_2","doi-asserted-by":"publisher","DOI":"10.1007\/978\u2010981\u201015\u20109323\u20109_15"},{"key":"e_1_2_12_29_2","unstructured":"LundbergS. M. LeeS.\u2010I.: A unified approach to interpreting model predictions. InProceedings of the 31st International Conference on Neural Information Processing Systems(Red Hook NY USA 2017) NIPS'17 Curran Associates Inc. p.4768\u20134777. URL:https:\/\/2.zoppoz.workers.dev:443\/https\/dl.acm.org\/doi\/10.5555\/3295222.3295230. 2"},{"key":"e_1_2_12_30_2","doi-asserted-by":"crossref","unstructured":"LuoQ. LiuL. LinY. ZhangW.: Don't miss the labels: Label\u2010semantic augmented meta\u2010learner for few\u2010shot text classification. InFindings of the Association for Computational Linguistics: ACL\u2010IJCNLP 2021(Online Aug.2021) pp.2773\u20132782. doi:10.18653\/v1\/2021.findings\u2010acl.245. 2","DOI":"10.18653\/v1\/2021.findings-acl.245"},{"key":"e_1_2_12_31_2","doi-asserted-by":"crossref","unstructured":"LarsonS. MahendranA. PeperJ. J. ClarkeC. LeeA. HillP. KummerfeldJ. K. LeachK. LaurenzanoM. A. TangL. MarsJ.: An evaluation dataset for intent classification and out\u2010of\u2010scope prediction. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP\u2010IJCNLP)(Hong Kong China Nov.2019) pp.1311\u20131316. doi:10.18653\/v1\/D19\u20101131. 9","DOI":"10.18653\/v1\/D19-1131"},{"key":"e_1_2_12_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3495162"},{"key":"e_1_2_12_33_2","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14733"},{"key":"e_1_2_12_34_2","doi-asserted-by":"publisher","DOI":"10.1038\/44565"},{"key":"e_1_2_12_35_2","doi-asserted-by":"crossref","unstructured":"LertvittayakumjornP. SpeciaL. ToniF.: FIND: Human\u2010in\u2010the\u2010loop debugging deep text classifiers. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing(Stroudsburg PA USA Oct.2020) pp.332\u2013348. doi:10.18653\/v1\/2020.emnlp\u2010main.24. 1 2 6","DOI":"10.18653\/v1\/2020.emnlp-main.24"},{"key":"e_1_2_12_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2022.3184186"},{"key":"e_1_2_12_37_2","doi-asserted-by":"crossref","unstructured":"LiuH. YinQ. WangW. Y.: Towards explainable NLP: A generative explanation framework for text classification. InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics(Florence Italy July2019) pp.5570\u20135581. doi:10.18653\/v1\/P19\u20101560. 3","DOI":"10.18653\/v1\/P19-1560"},{"key":"e_1_2_12_38_2","doi-asserted-by":"crossref","unstructured":"MekalaD. GangalV. ShangJ.: Coarse2Fine: Finegrained text classification on coarsely\u2010grained annotated data. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing(Online and Punta Cana Dominican Republic Nov.2021) pp.583\u2013594. doi:10.18653\/v1\/2021.emnlp\u2010main.46. 2","DOI":"10.18653\/v1\/2021.emnlp-main.46"},{"issue":"86","key":"e_1_2_12_39_2","first-page":"2579","article-title":"Visualizing data using t\u2010sne","volume":"9","author":"Maaten L. v. d.","year":"2008","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_12_40_2","doi-asserted-by":"publisher","DOI":"10.21105\/joss.00861"},{"key":"e_1_2_12_41_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2018.07.007"},{"key":"e_1_2_12_42_2","doi-asserted-by":"crossref","unstructured":"NguyenD.: Comparing automatic and human evaluation of local explanations for text classification. InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics(New Orleans Louisiana June2018) pp.1069\u20131078. doi:10.18653\/v1\/N18\u20101097. 3","DOI":"10.18653\/v1\/N18-1097"},{"key":"e_1_2_12_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2023.103476"},{"key":"e_1_2_12_44_2","doi-asserted-by":"crossref","unstructured":"RossA. Marasovi\u0107A. PetersM.: Explaining NLP models via minimal contrastive editing (MiCE). InFindings of the Association for Computational Linguistics: ACL\u2010IJCNLP 2021(Online Aug.2021) pp.3840\u20133852. doi:10.18653\/v1\/2021.findings\u2010acl.336. 3","DOI":"10.18653\/v1\/2021.findings-acl.336"},{"key":"e_1_2_12_45_2","doi-asserted-by":"crossref","unstructured":"RibeiroM. T. SinghS. GuestrinC.: \u201cWhy should i trust you?\u201d: Explaining the predictions of any classifier. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining(New York NY USA 2016) KDD '16 p.1135\u20131144. doi:10.1145\/2939672.2939778. 1 2","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_2_12_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3051315"},{"key":"e_1_2_12_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2012.213"},{"key":"e_1_2_12_48_2","doi-asserted-by":"crossref","unstructured":"SureshV. OngD.: Not all negatives are equal: Label\u2010aware contrastive loss for fine\u2010grained text classification. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing(Online and Punta Cana Dominican Republic Nov.2021) pp.4381\u20134394. doi:10.18653\/v1\/2021.emnlp\u2010main.359. 1 2","DOI":"10.18653\/v1\/2021.emnlp-main.359"},{"key":"e_1_2_12_49_2","doi-asserted-by":"crossref","unstructured":"SahuG. RodriguezP. LaradjiI. AtighehchianP. VazquezD. BahdanauD.: Data augmentation for intent classification with off\u2010the\u2010shelf large language models. InProceedings of the 4th Workshop on NLP for Conversational AI(Dublin Ireland May2022) pp.47\u201357. doi:10.18653\/v1\/2022.nlp4convai\u20101.5. 2","DOI":"10.18653\/v1\/2022.nlp4convai-1.5"},{"key":"e_1_2_12_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2934629"},{"key":"e_1_2_12_51_2","unstructured":"SundararajanM. TalyA. YanQ.: Axiomatic attribution for deep networks. InProceedings of the 34th International Conference on Machine Learning(2017) ICML'17 JMLR.org p.3319\u20133328. URL:https:\/\/2.zoppoz.workers.dev:443\/https\/dl.acm.org\/doi\/10.5555\/3305890.3306024. 2"},{"key":"e_1_2_12_52_2","doi-asserted-by":"crossref","unstructured":"VigJ.: A multiscale visualization of attention in the transformer model. InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations(Florence Italy July2019) pp.37\u201342. doi:10.18653\/v1\/P19\u20103007. 1 2","DOI":"10.18653\/v1\/P19-3007"},{"key":"e_1_2_12_53_2","doi-asserted-by":"crossref","unstructured":"YanY. TaoY. JinS. XuJ. LinH.: An interactive visual analytics system for incremental classification based on semi\u2010supervised topic modeling. In2019 IEEE Pacific Visualization Symposium (PacificVis)(2019) pp.148\u2013157. doi:10.1109\/PacificVis.2019.00025. 2","DOI":"10.1109\/PacificVis.2019.00025"},{"key":"e_1_2_12_54_2","doi-asserted-by":"crossref","unstructured":"ZhangX. XuanX. DimaA. SextonT. MaK.\u2010L.: LabelVizier: Interactive validation and relabeling for technical text annotations. In2023 IEEE 16th Pacific Visualization Symposium (PacificVis)(2023) pp.167\u2013176. doi:10.1109\/PacificVis56936.2023.00026. 2","DOI":"10.1109\/PacificVis56936.2023.00026"}],"container-title":["Computer Graphics Forum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/cgf.15098","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,14]],"date-time":"2024-06-14T12:31:06Z","timestamp":1718368266000},"score":1,"resource":{"primary":{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/onlinelibrary.wiley.com\/doi\/10.1111\/cgf.15098"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6]]},"references-count":53,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,6]]}},"alternative-id":["10.1111\/cgf.15098"],"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.1111\/cgf.15098","archive":["Portico"],"relation":{},"ISSN":["0167-7055","1467-8659"],"issn-type":[{"value":"0167-7055","type":"print"},{"value":"1467-8659","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6]]},"assertion":[{"value":"2024-06-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e15098"}}