{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T11:38:38Z","timestamp":1783597118164,"version":"3.55.0"},"reference-count":54,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2024,11,19]],"date-time":"2024-11-19T00:00:00Z","timestamp":1731974400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["RGPIN-2020-05375"],"award-info":[{"award-number":["RGPIN-2020-05375"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2024,12,19]]},"abstract":"<jats:p>Intrinsic image decomposition aims to separate the surface reflectance and the effects from the illumination given a single photograph. Due to the complexity of the problem, most prior works assume a single-color illumination and a Lambertian world, which limits their use in illumination-aware image editing applications. In this work, we separate an input image into its diffuse albedo, colorful diffuse shading, and specular residual components. We arrive at our result by gradually removing first the single-color illumination and then the Lambertian-world assumptions. We show that by dividing the problem into easier sub-problems, in-the-wild colorful diffuse shading estimation can be achieved despite the limited ground-truth datasets. Our extended intrinsic model enables illumination-aware analysis of photographs and can be used for image editing applications such as specularity removal and per-pixel white balancing.<\/jats:p>","DOI":"10.1145\/3687984","type":"journal-article","created":{"date-parts":[[2024,11,19]],"date-time":"2024-11-19T15:46:04Z","timestamp":1732031164000},"page":"1-12","update-policy":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":30,"title":["Colorful Diffuse Intrinsic Image Decomposition in the Wild"],"prefix":"10.1145","volume":"43","author":[{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0002-0800-1118","authenticated-orcid":false,"given":"Chris","family":"Careaga","sequence":"first","affiliation":[{"name":"Simon Fraser University, Burnaby, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0002-1495-0491","authenticated-orcid":false,"given":"Ya\u011f\u0131z","family":"Aksoy","sequence":"additional","affiliation":[{"name":"Simon Fraser University, Burnaby, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,11,19]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proc. ICCV.","author":"Barron Jonathan T.","year":"2015","unstructured":"Jonathan T. Barron. 2015. Convolutional Color Constancy. In Proc. ICCV."},{"key":"e_1_2_1_2_1","volume-title":"Barron and Jitendra Malik","author":"Jonathan","year":"2015","unstructured":"Jonathan T. Barron and Jitendra Malik. 2015. Shape, Illumination, and Reflectance from Shading. IEEE Trans. Pattern Anal. Mach. Intell. (2015)."},{"key":"e_1_2_1_3_1","volume-title":"Proc. ECCV.","author":"Baslamisli Anil S.","year":"2018","unstructured":"Anil S. Baslamisli, Thomas T. Groenestege, Partha Das, Hoang-An Le, Sezer Karaoglu, and Theo Gevers. 2018a. Joint Learning of Intrinsic Images and Semantic Segmentation. In Proc. ECCV."},{"key":"e_1_2_1_4_1","volume-title":"Proc. CVPR.","author":"Baslamisli Anil S.","year":"2018","unstructured":"Anil S. Baslamisli, Hoang-An Le, and Theo Gevers. 2018b. CNN based learning using reflection and retinex models for intrinsic image decomposition. In Proc. CVPR."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601206"},{"key":"e_1_2_1_6_1","volume-title":"Intrinsic Decompositions for Image Editing. Comput. Graph. Forum 36, 2","author":"Bonneel Nicolas","year":"2017","unstructured":"Nicolas Bonneel, Balazs Kovacs, Sylvain Paris, and Kavita Bala. 2017. Intrinsic Decompositions for Image Editing. Comput. Graph. Forum 36, 2 (2017)."},{"key":"e_1_2_1_7_1","article-title":"Intrinsic Image Decomposition via Ordinal Shading","volume":"43","author":"Careaga Chris","year":"2023","unstructured":"Chris Careaga and Ya\u011f\u0131z Aksoy. 2023. Intrinsic Image Decomposition via Ordinal Shading. ACM Trans. Graph. 43, 1, Article 12 (2023), 24 pages.","journal-title":"ACM Trans. Graph."},{"key":"e_1_2_1_8_1","volume-title":"Proc. SIGGRAPH Asia.","author":"Careaga Chris","year":"2023","unstructured":"Chris Careaga, S. Mahdi H. Miangoleh, and Ya\u011f\u0131z Aksoy. 2023. Intrinsic Harmonization for Illumination-Aware Compositing. In Proc. SIGGRAPH Asia."},{"key":"e_1_2_1_9_1","volume-title":"Proc. ECCV.","author":"Chen Xi","year":"2024","unstructured":"Xi Chen, Sida Peng, Dongchen Yang, Yuan Liu, Bowen Pan, Chengfei Lv, and Xiaowei Zhou. 2024. Intrinsic Anything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illumination. In Proc. ECCV."},{"key":"e_1_2_1_10_1","volume-title":"Proc. CVPR.","author":"Das Partha","year":"2022","unstructured":"Partha Das, Sezer Karaoglu, and Theo Gevers. 2022. PIE-Net: Photometric Invariant Edge Guided Network for Intrinsic Image Decomposition. In Proc. CVPR."},{"key":"e_1_2_1_11_1","volume-title":"Proc. ECCV.","author":"Dille Sebastian","year":"2024","unstructured":"Sebastian Dille, Chris Careaga, and Ya\u011f\u0131z Aksoy. 2024. Intrinsic Single-Image HDR Reconstruction. In Proc. ECCV."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2012.03137.x"},{"key":"e_1_2_1_13_1","volume-title":"Int. J. Comput. Vision","author":"Garces Elena","year":"2022","unstructured":"Elena Garces, Carlos Rodriguez-Pardo, Dan Casas, and Jorge Lopez-Moreno. 2022. A Survey on Intrinsic Images: Delving Deep Into Lambert and Beyond. Int. J. Comput. Vision (2022)."},{"key":"e_1_2_1_14_1","volume-title":"A multi-illuminant synthetic image test set. Color Research & Application","author":"Hao Xiangpeng","year":"2020","unstructured":"Xiangpeng Hao and Brian Funt. 2020. A multi-illuminant synthetic image test set. Color Research & Application (2020)."},{"key":"e_1_2_1_15_1","volume-title":"Proc. NeurIPS.","author":"Janner Michael","year":"2017","unstructured":"Michael Janner, Jiajun Wu, Tejas Kulkarni, Ilker Yildirim, and Joshua B Tenenbaum. 2017. Self-Supervised Intrinsic Image Decomposition. In Proc. NeurIPS."},{"key":"e_1_2_1_16_1","doi-asserted-by":"crossref","unstructured":"Kevin Karsch Kalyan Sunkavalli Sunil Hadap Nathan Carr Hailin Jin Rafael Fonte Michael Sittig and David Forsyth. 2014. Automatic scene inference for 3d object compositing. ACM Trans. Graph. (2014).","DOI":"10.1145\/2602146"},{"key":"e_1_2_1_17_1","volume-title":"Proc. ICCV.","author":"Kim Dongyoung","year":"2021","unstructured":"Dongyoung Kim, Jinwoo Kim, Seonghyeon Nam, Dongwoo Lee, Yeonkyung Lee, Nahyup Kang, Hyong-Euk Lee, ByungIn Yoo, Jae-Joon Han, and Seon Joo Kim. 2021. Large Scale Multi-Illuminant (LSMI) Dataset for Developing White Balance Algorithm Under Mixed Illumination. In Proc. ICCV."},{"key":"e_1_2_1_18_1","volume-title":"Proc. CVPR.","author":"Kim Hyeongwoo","year":"2013","unstructured":"Hyeongwoo Kim, Hailin Jin, Sunil Hadap, and Inso Kweon. 2013. Specular reflection separation using dark channel prior. In Proc. CVPR."},{"key":"e_1_2_1_19_1","volume-title":"Proc. CVPR.","author":"Kocsis Peter","year":"2024","unstructured":"Peter Kocsis, Vincent Sitzmann, and Matthias Nie\u00dfner. 2024. Intrinsic Image Diffusion for Single-view Material Estimation. Proc. CVPR."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00312"},{"key":"e_1_2_1_21_1","first-page":"409","article-title":"EDEN: Multimodal Synthetic Dataset of Enclosed Garden Scenes. In Proc. WACV. Louis Lettry, Kenneth Vanhoey, and Luc van Gool. 2018. Unsupervised Deep Single-Image Intrinsic Decomposition using Illumination-Varying Image","volume":"37","author":"Le Hoang-An","year":"2021","unstructured":"Hoang-An Le, Partha Das, Thomas Mensink, Sezer Karaoglu, and Theo Gevers. 2021. EDEN: Multimodal Synthetic Dataset of Enclosed Garden Scenes. In Proc. WACV. Louis Lettry, Kenneth Vanhoey, and Luc van Gool. 2018. Unsupervised Deep Single-Image Intrinsic Decomposition using Illumination-Varying Image Sequences. Comput. Graph. Forum 37, 7 (2018), 409--419.","journal-title":"Sequences. Comput. Graph. Forum"},{"key":"e_1_2_1_22_1","volume-title":"Proc. ICCV.","author":"Li Yixuan","year":"2023","unstructured":"Yixuan Li, Lihan Jiang, Linning Xu, Yuanbo Xiangli, Zhenzhi Wang, Dahua Lin, and Bo Dai. 2023. Matrixcity: A large-scale city dataset for city-scale neural rendering and beyond. In Proc. ICCV."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00255"},{"key":"e_1_2_1_24_1","volume-title":"Proc. ECCV.","author":"Li Zhengqi","year":"2018","unstructured":"Zhengqi Li and Noah Snavely. 2018a. CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering. In Proc. ECCV."},{"key":"e_1_2_1_25_1","volume-title":"Proc. CVPR.","author":"Li Zhengqi","year":"2018","unstructured":"Zhengqi Li and Noah Snavely. 2018b. Learning Intrinsic Image Decomposition from Watching the World. In Proc. CVPR."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00218"},{"key":"e_1_2_1_27_1","volume-title":"Proc. CVPR.","author":"Li Zhen","year":"2022","unstructured":"Zhen Li, Lingli Wang, Xiang Huang, Cihui Pan, and Jiaqi Yang. 2022. PhyIR: Physics-based Inverse Rendering for Panoramic Indoor Images. In Proc. CVPR."},{"key":"e_1_2_1_28_1","volume-title":"Proc. CVPR.","author":"Liu Yunfei","year":"2020","unstructured":"Yunfei Liu, Yu Li, Shaodi You, and Feng Lu. 2020. Unsupervised Learning for Intrinsic Image Decomposition from a Single Image. In Proc. CVPR."},{"key":"e_1_2_1_29_1","volume-title":"NIID-Net: Adapting Surface Normal Knowledge for Intrinsic Image Decomposition in Indoor Scenes","author":"Luo Jundan","year":"2020","unstructured":"Jundan Luo, Zhaoyang Huang, Yijin Li, Xiaowei Zhou, Guofeng Zhang, and Hujun Bao. 2020. NIID-Net: Adapting Surface Normal Knowledge for Intrinsic Image Decomposition in Indoor Scenes. IEEE Trans. Vis. Comp. Graph. (2020)."},{"key":"e_1_2_1_30_1","volume-title":"CRefNet: Learning Consistent Reflectance Estimation With a Decoder-Sharing Transformer","author":"Luo Jundan","year":"2023","unstructured":"Jundan Luo, Nanxuan Zhao, Wenbin Li, and Christian Richardt. 2023. CRefNet: Learning Consistent Reflectance Estimation With a Decoder-Sharing Transformer. IEEE Trans. Vis. Comp. Graph. (2023)."},{"key":"e_1_2_1_31_1","volume-title":"Proc. ECCV.","author":"Ma Wei-Chiu","year":"2018","unstructured":"Wei-Chiu Ma, Hang Chu, Bolei Zhou, Raquel Urtasun, and Antonio Torralba. 2018. Single Image Intrinsic Decomposition Without a Single Intrinsic Image. In Proc. ECCV."},{"key":"e_1_2_1_32_1","volume-title":"Proc. CVPR.","author":"Maralan Sepideh Sarajian","year":"2023","unstructured":"Sepideh Sarajian Maralan, Chris Careaga, and Ya\u011f\u0131z Aksoy. 2023. Computational Flash Photography through Intrinsics. Proc. CVPR."},{"key":"e_1_2_1_33_1","volume-title":"Proc. CVPR.","author":"Meka Abhimitra","year":"2018","unstructured":"Abhimitra Meka, Maxim Maximov, Michael Zollhoefer, Avishek Chatterjee, Hans-Peter Seidel, Christian Richardt, and Christian Theobalt. 2018. LIME: Live Intrinsic Material Estimation. In Proc. CVPR."},{"key":"e_1_2_1_34_1","unstructured":"Abhimitra Meka Mohammad Shafiei Michael Zollhoefer Christian Richardt and Christian Theobalt. 2021. Real-time Global Illumination Decomposition of Videos. ACM Trans. Graph. (2021)."},{"key":"e_1_2_1_35_1","volume-title":"Proc. SIGGRAPH.","author":"Miangoleh S. Mahdi H.","year":"2024","unstructured":"S. Mahdi H. Miangoleh, Mahesh Reddy, and Ya\u011f\u0131z Aksoy. 2024. Scale-Invariant Monocular Depth Estimation via SSI Depth. In Proc. SIGGRAPH."},{"key":"e_1_2_1_36_1","volume-title":"Proc. ICCV.","author":"Murmann Lukas","year":"2019","unstructured":"Lukas Murmann, Michael Gharbi, Miika Aittala, and Fredo Durand. 2019. A Multi-Illumination Dataset of Indoor Object Appearance. In Proc. ICCV."},{"key":"e_1_2_1_37_1","volume-title":"Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer","author":"Ranftl Ren\u00e9","year":"2020","unstructured":"Ren\u00e9 Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. 2020. Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer. IEEE Trans. Pattern Anal. Mach. Intell. (2020)."},{"key":"e_1_2_1_38_1","volume-title":"Nathan Paczan, Russ Webb, and Joshua M. Susskind.","author":"Roberts Mike","year":"2021","unstructured":"Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M. Susskind. 2021. Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding. In Proc. ICCV."},{"key":"e_1_2_1_39_1","volume-title":"Proc. ICCV.","author":"Sengupta Soumyadip","year":"2019","unstructured":"Soumyadip Sengupta, Jinwei Gu, Kihwan Kim, Guilin Liu, David W. Jacobs, and Jan Kautz. 2019. Neural Inverse Rendering of an Indoor Scene From a Single Image. In Proc. ICCV."},{"key":"e_1_2_1_40_1","unstructured":"Viraj Shah Svetlana Lazebnik and Julien Philip. 2023. JoIN: Joint GANs Inversion for Intrinsic Image Decomposition. arXiv preprint 2305.11321 [cs.CV] (2023)."},{"key":"e_1_2_1_41_1","volume-title":"Proc. CVPR.","author":"Shen Jianbing","year":"2011","unstructured":"Jianbing Shen, Xiaoshan Yang, Yunde Jia, and Xuelong Li. 2011. Intrinsic images using optimization. In Proc. CVPR."},{"key":"e_1_2_1_42_1","volume-title":"Proc. CVPR.","author":"Shen Li","year":"2008","unstructured":"Li Shen, Ping Tan, and Stephen Lin. 2008. Intrinsic image decomposition with non-local texture cues. In Proc. CVPR."},{"key":"e_1_2_1_43_1","volume-title":"Yu","author":"Shi Jian","year":"2017","unstructured":"Jian Shi, Yue Dong, Hao Su, and Stella X. Yu. 2017. Learning Non-Lambertian Object Intrinsics Across ShapeNet Categories. In Proc. CVPR."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2005.185"},{"key":"e_1_2_1_45_1","volume-title":"High quality rendered dataset and non-local graph convolutional network for intrinsic image decomposition. Journal of Image and Graphics","author":"Wang Yujie","year":"2022","unstructured":"Yujie Wang, Qingnan Fan, Kun Li, Dongdong Chen, Jingyu Yang, Jianzhi Lu, Dani Lischinski, and Baoquan Chen. 2022. High quality rendered dataset and non-local graph convolutional network for intrinsic image decomposition. Journal of Image and Graphics (2022)."},{"key":"e_1_2_1_46_1","volume-title":"Proc. ICCV.","author":"Wang Zian","year":"2021","unstructured":"Zian Wang, Jonah Philion, Sanja Fidler, and Jan Kautz. 2021. Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting. In Proc. ICCV."},{"key":"e_1_2_1_47_1","volume-title":"Proc. ICCP.","author":"Wu Jiaye","year":"2023","unstructured":"Jiaye Wu, Sanjoy Chowdhury, Hariharmano Shanmugaraja, David Jacobs, and Soumyadip Sengupta. 2023. Measured Albedo in the Wild: Filling the Gap in Intrinsics Evaluation. In Proc. ICCP."},{"key":"e_1_2_1_48_1","unstructured":"Yu-Ying Yeh Koki Nagano Sameh Khamis Jan Kautz Ming-Yu Liu and Ting-Chun Wang. 2022. Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptation. ACM Trans. Graph. (2022)."},{"key":"e_1_2_1_49_1","volume-title":"Proc. SIGGRAPH.","author":"Zeng Zheng","year":"2024","unstructured":"Zheng Zeng, Valentin Deschaintre, Iliyan Georgiev, Yiwei Hu Yannick Hold-Geoffroy, Fujun Luan, Ling-Qi Yan, and Milo\u0161 Ha\u0161an. 2024. RGB\u2194X: Image Decomposition and Synthesis Using Material- and Lighting-aware Diffusion Models. In Proc. SIGGRAPH."},{"key":"e_1_2_1_50_1","volume-title":"Learning Physically-based Material and Lighting Decompositions for Face Editing. Computational Visual Media","author":"Zhang Qian","year":"2022","unstructured":"Qian Zhang, Vikas Thamizharasan, and James Tompkin. 2022. Learning Physically-based Material and Lighting Decompositions for Face Editing. Computational Visual Media (2022)."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2012.77"},{"key":"e_1_2_1_52_1","volume-title":"Proc. ECCV.","author":"Zheng Jia","year":"2020","unstructured":"Jia Zheng, Junfei Zhang, Jing Li, Rui Tang, Shenghua Gao, and Zihan Zhou. 2020. Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling. In Proc. ECCV."},{"key":"e_1_2_1_53_1","volume-title":"Proc. SIGGRAPH Asia.","author":"Zhu Jingsen","year":"2022","unstructured":"Jingsen Zhu, Fujun Luan, Yuchi Huo, Zihao Lin, Zhihua Zhong, Dianbing Xi, Rui Wang, Hujun Bao, Jiaxiang Zheng, and Rui Tang. 2022b. Learning-Based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracing. In Proc. SIGGRAPH Asia."},{"key":"e_1_2_1_54_1","volume-title":"Proc. CVPR.","author":"Zhu Rui","year":"2022","unstructured":"Rui Zhu, Zhengqin Li, Janarbek Matai, Fatih Porikli, and Manmohan Chandraker. 2022a. IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenes. In Proc. CVPR."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/dl.acm.org\/doi\/10.1145\/3687984","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/dl.acm.org\/doi\/pdf\/10.1145\/3687984","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:09:58Z","timestamp":1750295398000},"score":1,"resource":{"primary":{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/dl.acm.org\/doi\/10.1145\/3687984"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,19]]},"references-count":54,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,12,19]]}},"alternative-id":["10.1145\/3687984"],"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.1145\/3687984","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,19]]},"assertion":[{"value":"2024-11-19","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}