EfficientInferenceinFullyConnectedCRFswithGaussianEdgePotentials资源-CSDN下载

Computer

Vision

需积分: 50 92 浏览量 2017-01-02 17:11:28 上传评论收藏 3.86MB PDF 举报

资源推荐

资源详情

资源评论

Efﬁcient Inference in Fully Connected CRFs with

Gaussian Edge Potentials

Philipp Kr

ahenb

uhl

Computer Science Department

Stanford University

[email protected]

Vladlen Koltun

Computer Science Department

Stanford University

[email protected]

Abstract

Most state-of-the-art techniques for multi-class image segmentation and labeling

use conditional random ﬁelds deﬁned over pixels or image regions. While region-

level models often feature dense pairwise connectivity, pixel-level models are con-

siderably larger and have only permitted sparse graph structures. In this paper, we

consider fully connected CRF models deﬁned on the complete set of pixels in an

image. The resulting graphs have billions of edges, making traditional inference

algorithms impractical. Our main contribution is a highly efﬁcient approximate

inference algorithm for fully connected CRF models in which the pairwise edge

potentials are deﬁned by a linear combination of Gaussian kernels. Our experi-

ments demonstrate that dense connectivity at the pixel level substantially improves

segmentation and labeling accuracy.

1 Introduction

Multi-class image segmentation and labeling is one of the most challenging and actively studied

problems in computer vision. The goal is to label every pixel in the image with one of several prede-

termined object categories, thus concurrently performing recognition and segmentation of multiple

object classes. A common approach is to pose this problem as maximum a posteriori (MAP) infer-

ence in a conditional random ﬁeld (CRF) deﬁned over pixels or image patches [8, 12, 18, 19, 9].

The CRF potentials incorporate smoothness terms that maximize label agreement between similar

pixels, and can integrate more elaborate terms that model contextual relationships between object

classes.

Basic CRF models are composed of unary potentials on individual pixels or image patches and pair-

wise potentials on neighboring pixels or patches [19, 23, 7, 5]. The resulting adjacency CRF struc-

ture is limited in its ability to model long-range connections within the image and generally results

in excessive smoothing of object boundaries. In order to improve segmentation and labeling accu-

racy, researchers have expanded the basic CRF framework to incorporate hierarchical connectivity

and higher-order potentials deﬁned on image regions [8, 12, 9, 13]. However, the accuracy of these

approaches is necessarily restricted by the accuracy of unsupervised image segmentation, which is

used to compute the regions on which the model operates. This limits the ability of region-based

approaches to produce accurate label assignments around complex object boundaries, although sig-

niﬁcant progress has been made [9, 13, 14].

In this paper, we explore a different model structure for accurate semantic segmentation and labeling.

We use a fully connected CRF that establishes pairwise potentials on all pairs of pixels in the image.

Fully connected CRFs have been used for semantic image labeling in the past [18, 22, 6, 17], but the

complexity of inference in fully connected models has restricted their application to sets of hundreds

of image regions or fewer. The segmentation accuracy achieved by these approaches is again limited

by the unsupervised segmentation that produces the regions. In contrast, our model connects all

(a) Image

(b) Unary classiﬁers

CRF

(d) Fully connected CRF,

MCMC inference, 36 hrs

sky

tree

grass

bench

tree

road

grass

(e) Fully connected CRF,

our approach, 0.2 seconds

Figure 1: Pixel-level classiﬁcation with a fully connected CRF. (a) Input image from the MSRC-21 dataset. (b)

The response of unary classiﬁers used by our models. (c) Classiﬁcation produced by the Robust P

CRF [9].

(d) Classiﬁcation produced by MCMC inference [17] in a fully connected pixel-level CRF model; the algorithm

was run for 36 hours and only partially converged for the bottom image. (e) Classiﬁcation produced by our

inference algorithm in the fully connected model in 0.2 seconds.

pairs of individual pixels in the image, enabling greatly reﬁned segmentation and labeling. The

main challenge is the size of the model, which has tens of thousands of nodes and billions of edges

even on low-resolution images.

Our main contribution is a highly efﬁcient inference algorithm for fully connected CRF models in

which the pairwise edge potentials are deﬁned by a linear combination of Gaussian kernels in an ar-

bitrary feature space. The algorithm is based on a mean ﬁeld approximation to the CRF distribution.

This approximation is iteratively optimized through a series of message passing steps, each of which

updates a single variable by aggregating information from all other variables. We show that a mean

ﬁeld update of all variables in a fully connected CRF can be performed using Gaussian ﬁltering

in feature space. This allows us to reduce the computational complexity of message passing from

quadratic to linear in the number of variables by employing efﬁcient approximate high-dimensional

ﬁltering [16, 2, 1]. The resulting approximate inference algorithm is sublinear in the number of

edges in the model.

Figure 1 demonstrates the beneﬁts of the presented algorithm on two images from the MSRC-21

dataset for multi-class image segmentation and labeling. Figure 1(d) shows the results of approxi-

mate MCMC inference in fully connected CRFs on these images [17]. The MCMC procedure was

run for 36 hours and only partially converged for the bottom image. We have also experimented with

graph cut inference in the fully connected models [11], but it did not converge within 72 hours. In

contrast, a single-threaded implementation of our algorithm produces a detailed pixel-level labeling

in 0.2 seconds, as shown in Figure 1(e). A quantitative evaluation on the MSRC-21 and the PAS-

CAL VOC 2010 datasets is provided in Section 6. To the best of our knowledge, we are the ﬁrst to

demonstrate efﬁcient inference in fully connected CRF models at the pixel level.

2 The Fully Connected CRF Model

Consider a random ﬁeld X deﬁned over a set of variables {X

, . . . , X

}. The domain of each

variable is a set of labels L = {l

, l

, . . . , l

}. Consider also a random ﬁeld I deﬁned over variables

, . . . , I

}. In our setting, I ranges over possible input images of size N and X ranges over

possible pixel-level image labelings. I

is the color vector of pixel j and X

is the label assigned to

pixel j.

A conditional random ﬁeld (I, X) is characterized by a Gibbs distribution

P (X|I) =

Z(I)

exp(−

c∈C

|I)), where G = (V, E) is a graph on X and each clique c

剩余8页未读，继续阅读

评论收藏

内容反馈

1LOVESJohnny

粉丝: 275

Efficient Inference in Fully Connected CRFs with Gaussian Edge P...

最新资源

Efficient Inference in Fully Connected CRFs with Gaussian Edge P...

词向量-开山之作1-Efficient estimation of word representations in vector space.pdf

机器翻译PPT-nueral machine translation

matlab2016代码-densecrf:轻量级的MATLAB和Python包装器，用于PhilippKrähenbühl具有高斯边缘势能的

pydensecrf：PhilippKrähenbühl具有高斯边缘势能的密集（完全连接）CRF的Python包装器

MATLAB wrapper for Efficient Inference in Fully

Property Inference Attacks on Fully Connected Neural Networks

Causal Inference in Statistics.pdf

An introduction to Bayesian inference in econometrics

Collective Inference for Extraction MRFs Coupled with Symmetric Clique Potentials

Causal.Inference.in.Python.sanet.st.pdf

Inference in Hidden Markov Models

Causal Inference and Discovery in Python

Bayesian Inference in Statistical Analysis .zip

s7310-8-bit-inference-with-tensorrt.pdf

A First Course in Causal Inference

Tencent- CNN in MRF: Video Object Segmentation Spatio-Temporal MRF

因果推断书籍《causal inference in python》电子书，《使用Python进行因果推断：科技产业应用》

Robust data-driven inference in the.pdf

Models for Probability and Statistical Inference

Bayesian Network Inference with Java Objects (BANJO) v2.2.1

Sigma-Point Kalman Filters

Inference for a nonstationary process with linear GARCH errors

mlr.rar_Gaussian Mixture_MLR matlab_inference _matlab mlr_mixtur

Deterministic Variational Inference for Robust Bayesian Neural N

Inference in the Simple Regression Model

搭建https化的个人网站Nginx（一）

nodeschool:Nodeschool教程

最新资源