{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T05:15:30Z","timestamp":1763442930976,"version":"3.45.0"},"reference-count":26,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2025,11,15]],"date-time":"2025-11-15T00:00:00Z","timestamp":1763164800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"ZENITH Research and Leadership Career Development Fund","award":["ID23.01"],"award-info":[{"award-number":["ID23.01"]}]},{"name":"Swedish Foundation for Strategic Research","award":["ID24-0087"],"award-info":[{"award-number":["ID24-0087"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Federated representation learning (FRL) is a promising technique for learning shared data representations that capture general features across decentralized clients without sharing raw data. However, there is a risk of sensitive information leakage from learned representations. The conventional differential privacy (DP) mechanism protects the privacy of the whole data by randomizing (adding noise or random response) at the cost of deteriorating learning performance. Inspired by the fact that some data information may be public or non-private and only sensitive information (e.g., race) should be protected, we investigate the information-theoretic protection on specific sensitive information for FRL. To characterize the trade-off between utility and sensitive information leakage, we adopt mutual information-based metrics to measure utility and sensitive information leakage, and propose a method that maximizes the utility performance, while restricting sensitive information leakage less than any positive value \u03f5 via the local DP mechanism. Simulation demonstrates that our scheme can achieve the best utility\u2013leakage trade-off among baseline schemes, and more importantly can adjust the trade-off between leakage and utility by controlling the noise level in local DP.<\/jats:p>","DOI":"10.3390\/e27111163","type":"journal-article","created":{"date-parts":[[2025,11,17]],"date-time":"2025-11-17T10:24:58Z","timestamp":1763375098000},"page":"1163","update-policy":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Utility\u2013Leakage Trade-Off for Federated Representation Learning"],"prefix":"10.3390","volume":"27","author":[{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0009-0007-7122-1723","authenticated-orcid":false,"given":"Yuchen","family":"Liu","sequence":"first","affiliation":[{"name":"School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0002-0313-7788","authenticated-orcid":false,"given":"Onur","family":"G\u00fcnl\u00fc","sequence":"additional","affiliation":[{"name":"Lehrstuhl f\u00fcr Nachrichtentechnik, Technical University Dortmund, 44227 Dortmund, Germany"},{"name":"Information Theory and Security Laboratory (ITSL), Link\u00f6ping University, 581 83 Link\u00f6ping, Sweden"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuanming","family":"Shi","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0002-4383-9995","authenticated-orcid":false,"given":"Youlong","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,11,15]]},"reference":[{"key":"ref_1","unstructured":"McMahan, B., Moore, E., Ramage, D., Hampson, S., and Arcas, B.A.Y. (2017, January 20\u201322). Communication-efficient learning of deep networks from decentralized data. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1561\/0400000042","article-title":"The algorithmic foundations of differential privacy","volume":"9","author":"Dwork","year":"2014","journal-title":"Found. Trends\u00ae Theor. Comput. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Song, S., Chaudhuri, K., and Sarwate, A.D. (2013, January 3\u20135). Stochastic gradient descent with differentially private updates. Proceedings of the 2013 IEEE Global Conference on Signal and Information Processing, Austin, TX, USA.","DOI":"10.1109\/GlobalSIP.2013.6736861"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Bassily, R., Smith, A., and Thakurta, A. (2014, January 18\u201321). Private empirical risk minimization: Efficient algorithms and tight error bounds. Proceedings of the 2014 IEEE 55th Annual Symposium on Foundations of Computer Science, Philadelphia, PA, USA.","DOI":"10.1109\/FOCS.2014.56"},{"key":"ref_5","unstructured":"Geyer, R.C., Klein, T., and Nabi, M. (2017). Differentially private federated learning: A client level perspective. arXiv."},{"key":"ref_6","unstructured":"McMahan, H.B., Ramage, D., Talwar, K., and Zhang, L. (2017). Learning differentially private recurrent language models. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3454","DOI":"10.1109\/TIFS.2020.2988575","article-title":"Federated learning with differential privacy: Algorithms and performance analysis","volume":"15","author":"Wei","year":"2020","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Abadi, M., Chu, A., Goodfellow, I., McMahan, H.B., Mironov, I., Talwar, K., and Zhang, L. (2016, January 24\u201328). Deep learning with differential privacy. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria.","DOI":"10.1145\/2976749.2978318"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Agrawal, D., and Aggarwal, C.C. (2001, January 21\u201323). On the design and quantification of privacy preserving data mining algorithms. Proceedings of the Twentieth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, Santa Barbra, CA, USA.","DOI":"10.1145\/375551.375602"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Calmon, F.P., Makhdoumi, A., and M\u00e9dard, M. (2015, January 14\u201319). Fundamental limits of perfect privacy. Proceedings of the 2015 IEEE International Symposium on Information Theory (ISIT), Hong Kong, China.","DOI":"10.1109\/ISIT.2015.7282765"},{"key":"ref_11","unstructured":"Tripathy, A., Wang, Y., and Ishwar, P. (2017). Privacy-Preserving Adversarial Networks. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Sreekumar, S., and G\u00fcnd\u00fcz, D. (2019, January 7\u201312). Optimal Privacy-Utility Trade-off under a Rate Constraint. Proceedings of the 2019 IEEE International Symposium on Information Theory (ISIT), Paris, France.","DOI":"10.1109\/ISIT.2019.8849330"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2060","DOI":"10.1109\/TIFS.2023.3262112","article-title":"Bottlenecks CLUB: Unifying Information-Theoretic Trade-Offs Among Complexity, Leakage, and Utility","volume":"18","author":"Razeghi","year":"2023","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"du Pin Calmon, F., and Fawaz, N. (2012, January 1\u20135). Privacy against statistical inference. Proceedings of the 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton), Monticello, IL, USA.","DOI":"10.1109\/Allerton.2012.6483382"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"G\u00fcnd\u00fcz, D., Gomez-Vilardebo, J., Tan, O., and Poor, H.V. (2013, January 10\u201315). Information theoretic privacy for smart meters. Proceedings of the 2013 Information Theory and Applications Workshop (ITA), San Diego, CA, USA.","DOI":"10.1109\/ITA.2013.6503006"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Rodr\u00edguez-G\u00e1lvez, B., Thobaben, R., and Skoglund, M. (2021, January 17\u201321). A variational approach to privacy and fairness. Proceedings of the 2021 IEEE Information Theory Workshop (ITW), Kanazawa, Japan.","DOI":"10.1109\/ITW48936.2021.9611429"},{"key":"ref_17","unstructured":"Hamman, F., and Dutta, S. (2023, January 28). Demystifying local and global fairness trade-offs in federated learning using information theory. Proceedings of the International Conference on Machine Learning 2023, Honolulu, HI, USA."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kang, J., Xie, T., Wu, X., Maciejewski, R., and Tong, H. (2022, January 17\u201320). Infofair: Information-theoretic intersectional fairness. Proceedings of the 2022 IEEE International Conference on Big Data (Big Data), Osaka, Japan.","DOI":"10.1109\/BigData55660.2022.10020588"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ghassami, A., Khodadadian, S., and Kiyavash, N. (2018, January 17\u201322). Fairness in supervised learning: An information theoretic approach. Proceedings of the 2018 IEEE International Symposium on Information Theory (ISIT), Vail, CO, USA.","DOI":"10.1109\/ISIT.2018.8437807"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"793","DOI":"10.1137\/090756090","article-title":"What can we learn privately?","volume":"40","author":"Kasiviswanathan","year":"2011","journal-title":"SIAM J. Comput."},{"key":"ref_21","unstructured":"Asuncion, A., and Newman, D. (2007). UCI Machine Learning Repository, University of California."},{"key":"ref_22","unstructured":"Dieterich, W., Mendoza, C., and Brennan, T. (2016). COMPAS Risk Scales: Demonstrating Accuracy Equity and Predictive Parity, Northpointe Inc."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.jpdc.2019.09.010","article-title":"Variational approach for privacy funnel optimization on continuous data","volume":"137","author":"Nan","year":"2020","journal-title":"J. Parallel Distrib. Comput."},{"key":"ref_24","unstructured":"Creager, E., Madras, D., Jacobsen, J.H., Weis, M., Swersky, K., Pitassi, T., and Zemel, R. (2019, January 9\u201315). Flexibly fair representation learning by disentanglement. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_25","unstructured":"Louizos, C., Swersky, K., Li, Y., Welling, M., and Zemel, R. (2015). The variational fair autoencoder. arXiv."},{"key":"ref_26","first-page":"28","article-title":"Achieving Fairness through Separability: A Unified Framework for Fair Representation Learning","volume":"Volume 238","author":"Dasgupta","year":"2024","journal-title":"Proceedings of the 27th International Conference on Artificial Intelligence and Statistics"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/www.mdpi.com\/1099-4300\/27\/11\/1163\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T05:13:17Z","timestamp":1763442797000},"score":1,"resource":{"primary":{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/www.mdpi.com\/1099-4300\/27\/11\/1163"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,15]]},"references-count":26,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2025,11]]}},"alternative-id":["e27111163"],"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.3390\/e27111163","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2025,11,15]]}}}