{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T09:01:19Z","timestamp":1784019679932,"version":"3.55.0"},"reference-count":41,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2021,11,20]],"date-time":"2021-11-20T00:00:00Z","timestamp":1637366400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000185","name":"Defense Advanced Research Projects Agency","doi-asserted-by":"publisher","award":["FA8750-17-2-0146"],"award-info":[{"award-number":["FA8750-17-2-0146"]}],"id":[{"id":"10.13039\/100000185","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Deep Gaussian Processes (DGPs) were proposed as an expressive Bayesian model capable of a mathematically grounded estimation of uncertainty. The expressivity of DPGs results from not only the compositional character but the distribution propagation within the hierarchy. Recently, it was pointed out that the hierarchical structure of DGP well suited modeling the multi-fidelity regression, in which one is provided sparse observations with high precision and plenty of low fidelity observations. We propose the conditional DGP model in which the latent GPs are directly supported by the fixed lower fidelity data. Then the moment matching method is applied to approximate the marginal prior of conditional DGP with a GP. The obtained effective kernels are implicit functions of the lower-fidelity data, manifesting the expressivity contributed by distribution propagation within the hierarchy. The hyperparameters are learned via optimizing the approximate marginal likelihood. Experiments with synthetic and high dimensional data show comparable performance against other multi-fidelity regression methods, variational inference, and multi-output GP. We conclude that, with the low fidelity data and the hierarchical DGP structure, the effective kernel encodes the inductive bias for true function allowing the compositional freedom.<\/jats:p>","DOI":"10.3390\/e23111545","type":"journal-article","created":{"date-parts":[[2021,11,21]],"date-time":"2021-11-21T20:58:21Z","timestamp":1637528301000},"page":"1545","update-policy":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Conditional Deep Gaussian Processes: Multi-Fidelity Kernel Learning"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/2.zoppoz.workers.dev:443\/https\/orcid.org\/0000-0003-1329-6069","authenticated-orcid":false,"given":"Chi-Ken","family":"Lu","sequence":"first","affiliation":[{"name":"Mathematics and Computer Science, Rutgers University, Newark, NJ 07102, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Patrick","family":"Shafto","sequence":"additional","affiliation":[{"name":"Mathematics and Computer Science, Rutgers University, Newark, NJ 07102, USA"},{"name":"School of Mathematics, Institute for Advanced Study, Princeton, NJ 08540, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1093\/biomet\/87.1.1","article-title":"Predicting the output from a complex computer code when fast approximations are available","volume":"87","author":"Kennedy","year":"2000","journal-title":"Biometrika"},{"key":"ref_2","unstructured":"Wang, Z., Xing, W., Kirby, R., and Zhe, S. (2021, January 13). Multi-Fidelity High-Order Gaussian Processes for Physical Simulation. Proceedings of the International Conference on Artificial Intelligence and Statistics, San Diego, CA, USA."},{"key":"ref_3","unstructured":"Cutajar, K., Pullin, M., Damianou, A., Lawrence, N., and Gonz\u00e1lez, J. (2019). Deep Gaussian processes for multi-fidelity modeling. arXiv."},{"key":"ref_4","unstructured":"Salakhutdinov, R., Tenenbaum, J., and Torralba, A. (2012, January 2). One-shot learning with a hierarchical nonparametric bayesian model. Proceedings of the ICML Workshop on Unsupervised and Transfer Learning, Bellevue, WA, USA."},{"key":"ref_5","unstructured":"Finn, C., Abbeel, P., and Levine, S. (2017, January 6). Model-agnostic meta-learning for fast adaptation of deep networks. Proceedings of the International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_6","unstructured":"Titsias, M.K., Schwarz, J., Matthews, A.G.d.G., Pascanu, R., and Teh, Y.W. (2019, January 6\u20139). Functional Regularisation for Continual Learning with Gaussian Processes. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Rasmussen, C.E., and Williams, C.K.I. (2006). Gaussian Process for Machine Learning, MIT Press.","DOI":"10.7551\/mitpress\/3206.001.0001"},{"key":"ref_8","unstructured":"Duvenaud, D., Lloyd, J., Grosse, R., Tenenbaum, J., and Zoubin, G. (2013, January 17\u201319). Structure discovery in nonparametric regression through compositional kernel search. Proceedings of the International Conference on Machine Learning, Atlanta, GA, USA."},{"key":"ref_9","unstructured":"Sun, S., Zhang, G., Wang, C., Zeng, W., Li, J., and Grosse, R. (2018, January 10\u201315). Differentiable compositional kernel learning for Gaussian processes. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_10","unstructured":"Damianou, A., and Lawrence, N. (May, January 29). Deep Gaussian processes. Proceedings of the Artificial Intelligence and Statistics, Scottsdale, AZ, USA."},{"key":"ref_11","unstructured":"Snelson, E., Ghahramani, Z., and Rasmussen, C.E. (2004, January 13\u201318). Warped Gaussian processes. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_12","unstructured":"L\u00e1zaro-Gredilla, M. (2012, January 3\u20136). Bayesian warped Gaussian processes. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_13","unstructured":"Salimbeni, H., and Deisenroth, M. (2017, January 4\u20139). Doubly stochastic variational inference for deep Gaussian processes. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_14","unstructured":"Salimbeni, H., Dutordoir, V., Hensman, J., and Deisenroth, M.P. (2019). Deep Gaussian Processes with Importance-Weighted Variational Inference. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yu, H., Chen, Y., Low, B.K.H., Jaillet, P., and Dai, Z. (2019, January 8\u201314). Implicit Posterior Variational Inference for Deep Gaussian Processes. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.","DOI":"10.1109\/IJCNN.2019.8852481"},{"key":"ref_16","unstructured":"Lu, C.K., Yang, S.C.H., Hao, X., and Shafto, P. (2020, January 26\u201328). Interpretable deep Gaussian processes with moments. Proceedings of the International Conference on Artificial Intelligence and Statistics, Virtual."},{"key":"ref_17","unstructured":"Havasi, M., Hern\u00e1ndez-Lobato, J.M., and Murillo-Fuentes, J.J. (2018, January 3\u20138). Inference in deep Gaussian processes using stochastic gradient Hamiltonian Monte Carlo. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_18","unstructured":"Ustyuzhaninov, I., Kazlauskaite, I., Kaiser, M., Bodin, E., Campbell, N., and Ek, C.H. (2020, January 3). Compositional uncertainty in deep Gaussian processes. Proceedings of the Conference on Uncertainty in Artificial Intelligence, Toronto, ON, Canada."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1615\/Int.J.UncertaintyQuantification.2014006914","article-title":"Recursive co-kriging model for design of computer experiments with multiple levels of fidelity","volume":"4","author":"Garnier","year":"2014","journal-title":"Int. J. Uncertain. Quantif."},{"key":"ref_20","unstructured":"Raissi, M., and Karniadakis, G. (2016). Deep multi-fidelity Gaussian processes. arXiv."},{"key":"ref_21","first-page":"20160751","article-title":"Nonlinear information fusion algorithms for data-efficient multi-fidelity modelling","volume":"473","author":"Perdikaris","year":"2017","journal-title":"Proc. R. Soc. Math. Phys. Eng. Sci."},{"key":"ref_22","unstructured":"Requeima, J., Tebbutt, W., Bruinsma, W., and Turner, R.E. (2019, January 16\u201318). The Gaussian process autoregressive regression model (gpar). Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, Naha, Okinawa, Japan."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Alvarez, M.A., Rosasco, L., and Lawrence, N.D. (2011). Kernels for vector-valued functions: A review. arXiv.","DOI":"10.1561\/9781601985590"},{"key":"ref_24","unstructured":"Parra, G., and Tobar, F. (2017, January 4\u20139). Spectral mixture kernels for multi-output Gaussian processes. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_25","unstructured":"Bruinsma, W., Perim, E., Tebbutt, W., Hosking, S., Solin, A., and Turner, R. (2020, January 13\u201318). Scalable Exact Inference in Multi-Output Gaussian Processes. Proceedings of the International Conference on Machine Learning, Virtual."},{"key":"ref_26","unstructured":"Kaiser, M., Otte, C., Runkler, T., and Ek, C.H. (2018, January 3\u20138). Bayesian alignments of warped multi-output Gaussian processes. Proceedings of the Advances in Neural Information Processing Systems, Montreal, Canada."},{"key":"ref_27","unstructured":"Kazlauskaite, I., Ek, C.H., and Campbell, N. (2019, January 16\u201318). Gaussian Process Latent Variable Alignment Learning. Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, Naha, Okinawa, Japan."},{"key":"ref_28","unstructured":"Williams, C.K. (, January 2\u20134). Computing with infinite networks. Proceedings of the Advances in Neural Information Processing Systems, Denver, CO, USA."},{"key":"ref_29","unstructured":"Cho, Y., and Saul, L.K. (2009, January 6\u20138). Kernel methods for deep learning. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_30","unstructured":"Duvenaud, D., Rippel, O., Adams, R., and Ghahramani, Z. (2014, January 22\u201325). Avoiding pathologies in very deep networks. Proceedings of the Artificial Intelligence and Statistics, Reykjavik, Iceland."},{"key":"ref_31","first-page":"2100","article-title":"How deep are deep Gaussian processes?","volume":"19","author":"Dunlop","year":"2018","journal-title":"J. Mach. Learn. Res."},{"key":"ref_32","unstructured":"Shen, Z., Heinonen, M., and Kaski, S. (2020, January 26\u201328). Learning spectrograms with convolutional spectral kernels. Proceedings of the International Conference on Artificial Intelligence and Statistics, Virtual."},{"key":"ref_33","unstructured":"Wilson, A.G., Hu, Z., Salakhutdinov, R., and Xing, E.P. (2016, January 9\u201311). Deep kernel learning. Proceedings of the Artificial Intelligence and Statistics, Cadiz, Spain."},{"key":"ref_34","unstructured":"Daniely, A., Frostig, R., and Singer, Y. (2016, January 5\u201310). Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity. Proceedings of the NIPS: Centre Convencions Internacional Barcelona, Barcelona, Spain."},{"key":"ref_35","unstructured":"Mairal, J., Koniusz, P., Harchaoui, Z., and Schmid, C. (2014, January 8\u201313). Convolutional kernel networks. Proceedings of the Advances in Neural Information Processing Systems, Montr\u00e9al, QC, Canada."},{"key":"ref_36","unstructured":"Van der Wilk, M., Rasmussen, C.E., and Hensman, J. (2017, January 4\u20139). Convolutional Gaussian processes. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_37","unstructured":"GPy (2021, October 01). GPy: A Gaussian Process Framework in Python. Available online: https:\/\/2.zoppoz.workers.dev:443\/http\/github.com\/SheffieldML\/GPy."},{"key":"ref_38","unstructured":"Wilson, A., and Adams, R. (2013, January 16\u201321). Gaussian process kernels for pattern discovery and extrapolation. Proceedings of the International Conference on Machine Learning, Atlanta, GA, USA."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Lu, C.K., and Shafto, P. (2021). Conditional Deep Gaussian Processes: Empirical Bayes Hyperdata Learning. Entropy, 23.","DOI":"10.3390\/e23111387"},{"key":"ref_40","unstructured":"Paleyes, A., Pullin, M., Mahsereci, M., Lawrence, N., and Gonz\u00e1lez, J. (2019, January 8\u201314). Emulation of physical processes with Emukit. Proceedings of the Second Workshop on Machine Learning and the Physical Sciences, Vancouver, BC, Canada."},{"key":"ref_41","unstructured":"Shah, A., Wilson, A., and Ghahramani, Z. (2014, January 22\u201325). Student-t processes as alternatives to Gaussian processes. Proceedings of the Artificial Intelligence and Statistics, Reykjavic, Iceland."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/www.mdpi.com\/1099-4300\/23\/11\/1545\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:33:17Z","timestamp":1760167997000},"score":1,"resource":{"primary":{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/www.mdpi.com\/1099-4300\/23\/11\/1545"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,20]]},"references-count":41,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["e23111545"],"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.3390\/e23111545","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,20]]}}}