{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T16:26:24Z","timestamp":1779294384513,"version":"3.51.4"},"reference-count":65,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2023,9,7]],"date-time":"2023-09-07T00:00:00Z","timestamp":1694044800000},"content-version":"vor","delay-in-days":366,"URL":"https:\/\/2.zoppoz.workers.dev:443\/http\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CNS-1815619, NGSDI-2105564"],"award-info":[{"award-number":["CNS-1815619, NGSDI-2105564"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100016682","name":"VMWare","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100016682","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2022,9,6]]},"abstract":"<jats:p>An accurate understanding of omnidirectional environment lighting is crucial for high-quality virtual object rendering in mobile augmented reality (AR). In particular, to support reflective rendering, existing methods have leveraged deep learning models to estimate or have used physical light probes to capture physical lighting, typically represented in the form of an environment map. However, these methods often fail to provide visually coherent details or require additional setups. For example, the commercial framework ARKit uses a convolutional neural network that can generate realistic environment maps; however the corresponding reflective rendering might not match the physical environments. In this work, we present the design and implementation of a lighting reconstruction framework called LITAR that enables realistic and visually-coherent rendering. LITAR addresses several challenges of supporting lighting information for mobile AR.<\/jats:p>\n          <jats:p>First, to address the spatial variance problem, LITAR uses two-field lighting reconstruction to divide the lighting reconstruction task into the spatial variance-aware near-field reconstruction and the directional-aware far-field reconstruction. The corresponding environment map allows reflective rendering with correct color tones. Second, LITAR uses two noise-tolerant data capturing policies to ensure data quality, namely guided bootstrapped movement and motion-based automatic capturing. Third, to handle the mismatch between the mobile computation capability and the high computation requirement of lighting reconstruction, LITAR employs two novel real-time environment map rendering techniques called multi-resolution projection and anchor extrapolation. These two techniques effectively remove the need of time-consuming mesh reconstruction while maintaining visual quality. Lastly, LITAR provides several knobs to facilitate mobile AR application developers making quality and performance trade-offs in lighting reconstruction. We evaluated the performance of LITAR using a small-scale testbed experiment and a controlled simulation. Our testbed-based evaluation shows that LITAR achieves more visually coherent rendering effects than ARKit. Our design of multi-resolution projection significantly reduces the time of point cloud projection from about 3 seconds to 14.6 milliseconds. Our simulation shows that LITAR, on average, achieves up to 44.1% higher PSNR value than a recent work Xihe on two complex objects with physically-based materials.<\/jats:p>","DOI":"10.1145\/3550291","type":"journal-article","created":{"date-parts":[[2022,9,7]],"date-time":"2022-09-07T14:54:27Z","timestamp":1662562467000},"page":"1-29","update-policy":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["LITAR"],"prefix":"10.1145","volume":"6","author":[{"given":"Yiqin","family":"Zhao","sequence":"first","affiliation":[{"name":"Worcester Polytechnic Institute, Worcester, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chongyang","family":"Ma","sequence":"additional","affiliation":[{"name":"Kuaishou Technology, Beijing, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haibin","family":"Huang","sequence":"additional","affiliation":[{"name":"Kuaishou Technology, Beijing, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tian","family":"Guo","sequence":"additional","affiliation":[{"name":"Worcester Polytechnic Institute, Worcester, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,9,7]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Article arXiv:1812.11941","author":"Alhashim Ibraheem","year":"2018","unstructured":"Ibraheem Alhashim and Peter Wonka. 2018. High Quality Monocular Depth Estimation via Transfer Learning. arXiv e-prints abs\/1812.11941, Article arXiv:1812.11941 (2018). arXiv:1812.11941 https:\/\/2.zoppoz.workers.dev:443\/https\/arxiv.org\/abs\/1812.11941"},{"key":"e_1_2_2_2_1","unstructured":"Amazon. 2020. Amazon AR View. https:\/\/2.zoppoz.workers.dev:443\/https\/www.amazon.com\/adlp\/arview. Accessed: 2020-7-2."},{"key":"e_1_2_2_3_1","first-page":"4","article-title":"Extended Reality in Medical","volume":"21","author":"Andrews Christopher","year":"2019","unstructured":"Christopher Andrews, Michael K Southworth, Jennifer N A Silva, and Jonathan R Silva. 2019. Extended Reality in Medical Practice. Curr. Treat. Options Cardiovasc. Med. 21, 4 (March 2019), 18.","journal-title":"Practice. Curr. Treat. Options Cardiovasc. Med."},{"key":"e_1_2_2_4_1","unstructured":"Apple. 2022. ARCoachingOverlayView. https:\/\/2.zoppoz.workers.dev:443\/https\/developer.apple.com\/documentation\/arkit\/arcoachingoverlayview."},{"key":"e_1_2_2_5_1","unstructured":"Apple. 2022. iPhone 13 Pro Tech Specs. https:\/\/2.zoppoz.workers.dev:443\/https\/www.apple.com\/iphone-13-pro\/specs\/."},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00255"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386901.3389033"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/2945.817351"},{"key":"e_1_2_2_9_1","volume-title":"Sensor fusion IV: control paradigms and data structures","author":"Besl Paul J","unstructured":"Paul J Besl and Neil D McKay. 1992. Method for registration of 3-D shapes. In Sensor fusion IV: control paradigms and data structures, Vol. 1611. Spie, 586--606."},{"key":"e_1_2_2_10_1","unstructured":"Dan Cernea. 2020. OpenMVS: Multi-View Stereo Reconstruction Library. (2020). https:\/\/2.zoppoz.workers.dev:443\/https\/cdcseacave.github.io\/openMVS"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2017.00081"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13561"},{"key":"e_1_2_2_13_1","unstructured":"RidgeRun Embedded Linux Developer Connection. 2022. NVIDIA CUDA Memory Management. https:\/\/2.zoppoz.workers.dev:443\/https\/developer.ridgerun.com\/wiki\/index.php?title=NVIDIA_CUDA_Memory_Management."},{"key":"e_1_2_2_14_1","volume-title":"Computer Graphics Forum","author":"Corsini Massimiliano","unstructured":"Massimiliano Corsini, Marco Callieri, and Paolo Cignoni. 2008. Stereo light probe. In Computer Graphics Forum, Vol. 27. Wiley Online Library, 291--300."},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1185657.1185686"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1401132.1401175"},{"key":"e_1_2_2_17_1","unstructured":"Devin Larson. 2014. Standard Proportions of the Human Body. https:\/\/2.zoppoz.workers.dev:443\/https\/www.makingcomics.com\/2014\/01\/19\/standard-proportions-human-body\/. Accessed: 2021-11-5."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6552\/aaadd9"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3379337.3415881"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14283"},{"key":"e_1_2_2_21_1","unstructured":"Epic Games. 2021. Unreal Engine - Real-Time 3D Creation Tool. https:\/\/2.zoppoz.workers.dev:443\/https\/www.unrealengine.com. Accessed: 2021-11-5."},{"key":"e_1_2_2_22_1","volume-title":"Learning to Predict Indoor Illumination from a Single Image. ACM Transactions on Graphics","author":"Gardner Marc-Andr\u00e9","year":"2017","unstructured":"Marc-Andr\u00e9 Gardner, Kalyan Sunkavalli, Ersin Yumer, Xiaohui Shen, Emiliano Gambaretto, Christian Gagn\u00e9, and Jean-Fran\u00e7ois Lalonde. 2017. Learning to Predict Indoor Illumination from a Single Image. ACM Transactions on Graphics (2017)."},{"key":"e_1_2_2_23_1","doi-asserted-by":"crossref","unstructured":"Marc-Andre Gardner Yannick Hold-Geoffroy Kalyan Sunkavalli Christian Gagne and Jean-Francois Lalonde. 2019. Deep Parametric Indoor Lighting Estimation.","DOI":"10.1109\/ICCV.2019.00727"},{"key":"e_1_2_2_24_1","volume-title":"Fast Spatially-Varying Indoor Lighting Estimation. CVPR","author":"Garon Mathieu","year":"2019","unstructured":"Mathieu Garon, Kalyan Sunkavalli, Sunil Hadap, Nathan Carr, and Jean-Fran\u00e7ois Lalonde. 2019. Fast Spatially-Varying Indoor Lighting Estimation. CVPR (2019)."},{"key":"e_1_2_2_25_1","unstructured":"Google. 2020. ARCore. https:\/\/2.zoppoz.workers.dev:443\/https\/developers.google.com\/ar."},{"key":"e_1_2_2_26_1","unstructured":"Google. 2022. Pixel 6 Tech Specs. https:\/\/2.zoppoz.workers.dev:443\/https\/store.google.com\/product\/pixel_6_specs?hl=en-US."},{"key":"e_1_2_2_27_1","unstructured":"Google for Education. 2022. Bringing virtual and augmented reality to school | Google for Education. https:\/\/2.zoppoz.workers.dev:443\/https\/edu.google.com\/products\/vrar\/?modal_active=none. Accessed: 2020-7-24."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1315184.1315207"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-020-2649-2"},{"key":"e_1_2_2_30_1","doi-asserted-by":"crossref","unstructured":"Vlastimil Havran Miloslaw Smyk Grzegorz Krawczyk Karol Myszkowski and Hans-Peter Seidel. 2005. Interactive System for Dynamic Scene Lighting using Captured Video Environment Maps.. In Rendering Techniques. 31--42.","DOI":"10.1145\/1187112.1187243"},{"key":"e_1_2_2_31_1","unstructured":"HUAWEI. 2021. HUAWEI Mate 30 Pro Specifications | HUAWEI Global. https:\/\/2.zoppoz.workers.dev:443\/https\/consumer.huawei.com\/en\/phones\/mate30-pro\/specs\/. Accessed: 2020-7-8."},{"key":"e_1_2_2_32_1","volume-title":"iPad Pro","author":"Apple Inc. 2020.","year":"2020","unstructured":"Apple Inc. 2020. iPad Pro 2020. https:\/\/2.zoppoz.workers.dev:443\/https\/www.apple.com\/ipad-pro\/specs\/."},{"key":"e_1_2_2_33_1","unstructured":"Apple Inc. 2022. Adding Realistic Reflections to an AR Experience. https:\/\/2.zoppoz.workers.dev:443\/https\/developer.apple.com\/documentation\/arkit\/camera_lighting_and_effects\/adding_realistic_reflections_to_an_ar_experience."},{"key":"e_1_2_2_34_1","unstructured":"Apple Inc. 2022. Introducing ARKit 5. https:\/\/2.zoppoz.workers.dev:443\/https\/developer.apple.com\/augmented-reality\/arkit\/."},{"key":"e_1_2_2_35_1","unstructured":"Inter IKEA Systems B. V. 2017. IKEA Place. https:\/\/2.zoppoz.workers.dev:443\/https\/apps.apple.com\/us\/app\/ikea-place\/id1279244498. Accessed: 2020-7-2."},{"key":"e_1_2_2_36_1","first-page":"3","volume-title":"Proc. Physically Based Shading Theory Practice 4","author":"Karis Brian","year":"2013","unstructured":"Brian Karis and Epic Games. 2013. Real shading in unreal engine 4. Proc. Physically Based Shading Theory Practice 4, 3 (2013)."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2833157.2833162"},{"key":"e_1_2_2_38_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10591--10600","author":"Li Junxuan","year":"2021","unstructured":"Junxuan Li, Hongdong Li, and Yasuyuki Matsushita. 2021. Lighting, Reflectance and Geometry Estimation From 360deg Panoramic Stereo. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10591--10600."},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00255"},{"key":"e_1_2_2_40_1","volume-title":"CollabAR: Edge-assisted Collaborative Image Recognition for Mobile Augmented Reality. In 2020 19th ACM\/IEEE International Conference on Information Processing in Sensor Networks (IPSN). 301--312","author":"Liu Z","year":"2020","unstructured":"Z Liu, G Lan, J Stojkovic, Y Zhang, C Joe-Wong, and M Gorlatova. 2020. CollabAR: Edge-assisted Collaborative Image Recognition for Mobile Augmented Reality. In 2020 19th ACM\/IEEE International Conference on Information Processing in Sensor Networks (IPSN). 301--312."},{"key":"e_1_2_2_41_1","volume-title":"Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting. (Aug","author":"Maier Robert","year":"2017","unstructured":"Robert Maier, Kihwan Kim, Daniel Cremers, Jan Kautz, and Matthias Nie\u00dfner. 2017. Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting. (Aug. 2017). arXiv:1708.01670 [cs.CV]"},{"key":"e_1_2_2_42_1","first-page":"73","article-title":"Efficient GPU screen-space ray tracing","volume":"3","author":"McGuire Morgan","year":"2014","unstructured":"Morgan McGuire and Michael Mara. 2014. Efficient GPU screen-space ray tracing. Journal of Computer Graphics Techniques (JCGT) 3, 4 (2014), 73--85.","journal-title":"Journal of Computer Graphics Techniques (JCGT)"},{"key":"e_1_2_2_43_1","volume-title":"International Workshop on Reproducible Research in Pattern Recognition. Springer, 60--74","author":"Moulon Pierre","year":"2016","unstructured":"Pierre Moulon, Pascal Monasse, Romuald Perrot, and Renaud Marlet. 2016. OpenMVG: Open multiple view geometry. In International Workshop on Reproducible Research in Pattern Recognition. Springer, 60--74."},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2017.2705103"},{"key":"e_1_2_2_45_1","unstructured":"Nvidia. 2022. Jetson AGX Xavier Developer Kit. https:\/\/2.zoppoz.workers.dev:443\/https\/developer.nvidia.com\/embedded\/jetson-agx-xavier-developer-kit."},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459872"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3307334.3326098"},{"key":"e_1_2_2_48_1","volume-title":"Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer","author":"Ranftl Ren\u00e9","year":"2020","unstructured":"Ren\u00e9 Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. 2020. Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) (2020)."},{"key":"e_1_2_2_49_1","volume-title":"Stepping Science: Estimating Someone's Height from Their Walk. https:\/\/2.zoppoz.workers.dev:443\/https\/www.scientificamerican.com\/article\/bring-science-home-estimating-height-walk\/. Accessed: 2021-11-5.","author":"Buddies Science","year":"2013","unstructured":"Science Buddies. 2013. Stepping Science: Estimating Someone's Height from Their Walk. https:\/\/2.zoppoz.workers.dev:443\/https\/www.scientificamerican.com\/article\/bring-science-home-estimating-height-walk\/. Accessed: 2021-11-5."},{"key":"e_1_2_2_50_1","unstructured":"Scott Stein. 2021. Lidar is one of the iPhone and iPad's coolest tricks. Here's what else it can do. https:\/\/2.zoppoz.workers.dev:443\/https\/www.cnet.com\/tech\/mobile\/lidar-is-one-of-the-iphone-ipad-coolest-tricks-its-only-getting-better\/. Accessed: 2021-11-5."},{"key":"e_1_2_2_51_1","volume-title":"HDR Environment Map Estimation for Real-Time Augmented Reality. CVPR","author":"Somanath Gowri","year":"2021","unstructured":"Gowri Somanath and Daniel Kurz. 2021. HDR Environment Map Estimation for Real-Time Augmented Reality. CVPR (2021)."},{"key":"e_1_2_2_52_1","volume-title":"Neural Illumination: Lighting Prediction for Indoor Environments. CVPR","author":"Song Shuran","year":"2019","unstructured":"Shuran Song and Thomas Funkhouser. 2019. Neural Illumination: Lighting Prediction for Indoor Environments. CVPR (2019)."},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00810"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323008"},{"key":"e_1_2_2_55_1","unstructured":"Three.js Organization. 2021. Three.js - JavaScript 3D library. https:\/\/2.zoppoz.workers.dev:443\/https\/threejs.org. Accessed: 2021-11-5."},{"key":"e_1_2_2_56_1","unstructured":"TornadoWeb. 2022. Tornado Web Server. https:\/\/2.zoppoz.workers.dev:443\/https\/www.tornadoweb.org\/en\/stable\/."},{"key":"e_1_2_2_57_1","volume-title":"2019 IEEE 2nd International Conference on Information and Computer Technologies (ICICT). IEEE, 170--174","author":"Mihran","unstructured":"Mihran Tuceryan et al. 2019. AR360: dynamic illumination for augmented reality with real-time interaction. In 2019 IEEE 2nd International Conference on Information and Computer Technologies (ICICT). IEEE, 170--174."},{"key":"e_1_2_2_58_1","volume-title":"21st European Signal Processing Conference (EUSIPCO","author":"Unger Jonas","year":"2013","unstructured":"Jonas Unger, Joel Kronander, Per Larsson, Stefan Gustavson, and Anders Ynnerman. 2013. Temporally and spatially varying image based lighting using HDR-video. In 21st European Signal Processing Conference (EUSIPCO 2013). IEEE, Marrakech, Morocco, 1--5."},{"key":"e_1_2_2_59_1","unstructured":"Unity. 2020. AR Foundation 4.2.0-preview.5. https:\/\/2.zoppoz.workers.dev:443\/https\/docs.unity3d.com\/Packages\/com.unity.xr.arfoundation@4.2\/manual\/index.html."},{"key":"e_1_2_2_60_1","unstructured":"Unity3D. 2022. Unity3D. https:\/\/2.zoppoz.workers.dev:443\/https\/docs.unity3d.com\/. Accessed: 2022-5-14."},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3458864.3467675"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3372224.3419192"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3517260"},{"key":"e_1_2_2_64_1","doi-asserted-by":"crossref","unstructured":"Yiqin Zhao and Tian Guo. 2020. PointAR: Efficient Lighting Estimation for Mobile Augmented Reality. In Computer Vision - ECCV 2020 Andrea Vedaldi Horst Bischof Thomas Brox and Jan-Michael Frahm (Eds.). Springer International Publishing Cham 678--693.","DOI":"10.1007\/978-3-030-58592-1_40"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3458864.3467886"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/dl.acm.org\/doi\/10.1145\/3550291","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/dl.acm.org\/doi\/pdf\/10.1145\/3550291","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/dl.acm.org\/doi\/pdf\/10.1145\/3550291","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T04:39:43Z","timestamp":1752467983000},"score":1,"resource":{"primary":{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/dl.acm.org\/doi\/10.1145\/3550291"}},"subtitle":["Visually Coherent Lighting for Mobile Augmented Reality"],"short-title":[],"issued":{"date-parts":[[2022,9,6]]},"references-count":65,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,9,6]]}},"alternative-id":["10.1145\/3550291"],"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.1145\/3550291","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,6]]},"assertion":[{"value":"2022-09-07","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}