1 Introduction

Blind deconvolution is often used as a pre-processing step in many image processing-related applications. Blind deconvolution is often needed to improve the quality of images containing motion blur. A motion blur is a common problem that transforms an image to a blurry image to an inadequate level, and thus often incurs irreversible loss of information details. The formation of motion blur is attributed to the nature of image sensors that collect illuminations arriving for a certain time to capture an image. A motion blurred image is produced, if the position of the camera sensor changes during the exposure of the camera [1, 2].

The motion blur is considered as the convolution of a kernel of the blur operation with a latent version of the image observed (henceforth, called latent image), if this motion blur is shift-invariant. In this case, the effect of the sensor is defined by that kernel. Cancelation of the motion blur from the image can be performed by the deconvolution operation. If the corresponding blur kernel is given to retrieve the correct image from a blurred version, a non-blind deconvolution operation is performed. On the other hand, if the blur kernel is not given, blind deconvolution operation is performed. In general, blind deconvolution is significantly more challenging than non-blind deconvolution, while it is needed by a majority of the image sensors. This paper introduces a novel solution to the blind deconvolution of a single image, where both latent image and blur kernel can be estimated from an input blurred image.

Many blind deconvolution approaches employ an iterative method that gradually improves the latent image and the motion blur kernel. In the initial step, the iterative method retrieves an estimate of the latent image from the input blurred image and uses it to calculate the blur kernel. Then, the calculated kernel is utilized to compute an updated version of the latent image by exploiting non-blind deconvolution from the input blurred image. In the next iteration of the kernel computation process, the updated version of the latent image is employed. The intensive calculation overhead of this approach stems from the massive matrix operations needed for the kernel and latent image computation.

This work proposes an effective blind deconvolution approach that estimates a blur kernel accurately and recovers a deblurred image from a given blurred image. The key idea behind our approach lies in the homomorphic process that splits the observed image and in our method of utilizing the outliers in the observed images. The crucial concept of homomorphic image processing relies on representing the input image as a product of a fixed illumination and a changing reflectance. The product operation is converted to a summation using the logarithmic operation. Then, the reflectance part is extracted by a high-pass filter, while the illumination part is extracted by a low-pass filter. The most significant information of the image is contained in the reflectance part, while the illumination part contains only fixed and approximate information.

Our study investigates the blind deconvolution process in the homomorphic term on the reflectance part, which has the most of details of the image, instead of the input image. The illumination part, which has the redundancy component of the image, is ignored through kernel estimation. Kernel estimation methods and outlier-based image deblurring methods have received great attention recently. To extract blurred image using outliers, several approaches rely on computationally intensive methods such as Wiener filtering [3]. These approaches have less impact, if the light streaks are not separated and the outliers, such as non-Gaussian noise cannot be suppressed. We utilize the proposed homomorphic step to develop the outlier handling approach [4]. Our proposed approach does not need computationally extensive operations, such as light streak detection and outlier detection.

The main contributions of our work are as follows. The first is decomposition of a blurred image into two components and focus on one component that has most of the details. This process improves the output accuracy of blur kernel computation. The second is enhancement of the model data fidelity by removing redundant information so that the outliers can be reduced in the step of kernel computation. The third contribution is that the proposed homomorphic procedure can also be applied to other deblurring applications.

2 Problem formulation

Suppose that a motion blurred image y is created by convolving an unknown desired image x with an unknown motion blur kernel k. In addition, an additive white Gaussian noise n is commonly added to the operation, as a point spread function (PSF). We express the above model by Eq. (1) as follows:

$$ y = k * x + n $$
(1)

which is used in [5]. The operator * denotes convolution. The operation of blind deconvolution can be generally split into two main steps. The first step is to evaluate the unknown kernel k, while the second step is to conduct the deconvolution of the original image y with the estimated motion blur kernel k. One of popular blind deconvolution methods is optimizing x and k in an iterative manner. The latent image and blur kernel are estimated using Eqs. (2) and (3), respectively. These equations are similar to those used in [5] and are calculated as follows:

$$ x^{\prime } = \arg \min_{x} \left\{ {\left\| {y - k*x} \right\| + P_{x} \left( x \right)} \right\} $$
(2)
$$ k^{\prime } = \arg \min_{k} \left\{ {\left\| {y - k*x} \right\| + P_{k} \left( k \right)} \right\} $$
(3)

Here, the term \( \left\| {y - k*x} \right\| \) plays a role for data fitting, while Px and Pk are the regularization terms. Some regularization terms, such as a uniform prior and total variation, have been adopted in several methods.

The iterative refinement is proposed to improve the blur kernel k. The optimized deblurred image is obtained by the final step of non-blind deconvolution that is conducted using the optimized kernel k and the blurred input image y. The estimation of intermediate latent images does not have a direct impact on the deblurred output during the steps of iteration. It indirectly impacts the result by affecting the optimization process of the kernel k.

The iterative approach is inspired by two intuitive observations on its latent image computation: sharp edge retrieval and noise cancelation in smooth districts, which support perfect kernel approximation. A more optimal blur kernel is obtained when the image has a large number of sharp edges. This is the behavior of blur that usually exhibits a consistent property over a given blurred image. On the other hand, it is very difficult to estimate the blur kernel on an area with pixels of similar intensities. Typically, sharp edges are frequently found in a natural image. Therefore, a blur kernel is efficiently approximated from the edges restored in the intermediate latent images.

3 Related work

Blind deconvolution was extensively adopted by various applications in the last two decades [6, 7]. It is especially beneficial in the case when the number of given data points is less than the number of required data points. Early methods have put limitations on their kernel of motion blur and applied parameterized models to calculate this kernel [8,9,10,11]. Recent research made significant improvements in this regard. For example, Fergus et al. [12] successfully retrieved the correct image from its motion blurred observation. The readers can find more details related to different problems and comprehensive deblurring techniques [13, 14].

Several methods have introduced constrained hypothesis on motion blur, such as parameterized linear motion either in a single image or in multiple images. Two successively blurred images were considered by Chen et al. [8] to estimate the motion blur. Rav-Acha and Peleg [15] developed blurred images vertically and horizontally to recover the corrected image. Multiple images were employed by Cho et al. [16] with space-variant blurs to evaluate the kernels. In case of a single image, isotropy assumption was utilized by Yitzhaky et al. [10] to treat motion blur. In the work of Dai and Wu [17], the blur information, which is given by a method called Alpha Matting, was utilized to calculate per pixel blur kernels.

Recently, many studies have been published to address various types of motion blur problems. Jia [18] applied the Alpha Matting method, which defines transparency variations produced by motion blur to calculate the kernel. In Yuan et al. [19], two motion deblurred images have been employed: the first has noise but contains strong edges, while the second has motion blur. Shan et al. [20] used a concatenation of enhancement procedures. Although their proposed approach is an efficient deblurring technique, it requires a long computation time. Cho [21] evaluated common types of outliers, such as non-Gaussian noise and pixel saturation, which make deconvolution approaches unsuccessful. Jiangxin et al. [4] developed a method that describes the data fidelity term so that outliers can be less influential on the approximation of the blur kernel. Whyte et al. [22] developed a non-blind deconvolution technique that explicitly handles the saturated areas of pixels when retrieving a deblurred one.

4 Proposed framework

Our key idea of the proposed work is to develop a homomorphic image model with a data fidelity term to estimate blur kernel accurately for restoring the deblurred image from blurred one.

4.1 Homomorphic image representation concepts

An image can be represented by the product of two components. One part is the illumination and the other part is the reflectance [23]. The following equation shows the two components of an image.

$$ f(n_{1} \cdot n_{2} ) = i(n_{1} ,n_{2} ) \cdot r(n_{1} ,n_{2} ) $$
(4)

where f(n1, n2) is the value of the image pixel obtained, while r(n1, n2) and i(n1, n2) are the light reflectance and light illumination incident on the object, respectively.

It is known that the illumination is approximately constant over a part of the image, since the amount of light received by different objects is similar. The prominent variation between the different objects is mainly represented by the reflectance component.

If a logarithm operation is applied on Eq. (4), the multiplication operation can be transformed into an addition operation as follows:

$$ \log (f(n_{1} .n_{2} )) = \log (i(n_{1} ,n_{2} )) + \log (r(n_{1} ,n_{2} )) $$
(5)

While the first term in Eq. (5) has small variations, the second term exhibits large variations as it corresponds to the reflectance of the objects in an image. Figure 1 illustrates an example, where an image is decomposed to its reflectance and illumination components. By ignoring the illumination term and considering the reflectance term of Eq. (5), we can efficiently compute blur kernel.

Fig. 1
figure 1

Homomorphic image decomposition into illumination (upper) and reflectance components (lower)

Full size image

In addition, Fig. 2 shows an inverse process where the reflectance and illumination components are added again with exponential operation to reconstruct the original image.

Fig. 2
figure 2

Homomorphic reconstruction of the original image from the illumination and reflectance components

Full size image

4.2 Principles of kernel estimation

Based on [4], the latent image x and the blur kernel k are calculated from the input blurred image y. Then the deblurring equation is expressed by Eq. (6).

$$ \mathop {\hbox{min} }\limits_{x,k} \quad R(x*k - y) + \gamma P_{k} (k) + \lambda P_{x} (x) $$
(6)

In Eq. (6), R(·) is the Taylor polynomial approximation. γ and λ are weights to adjust Px(x) and Pk(k), which are the primary components.

Then we reformulate Eq. (6) to an optimization problem of obtaining the latent image version x and the blur kernel k from Eqs. (7) and (8), respectively.

$$ \mathop {\hbox{min} }\limits_{x} \quad R(x*k - y) + \lambda P_{x} (x) $$
(7)
$$ \mathop {\hbox{min} }\limits_{k} \quad R(x*k - y) + \gamma P_{k} (k) $$
(8)

Using kernel k from the prior iteration, the intermediate latent version x is computed by

$$ \mathop {\hbox{min} }\limits_{k} \quad R(x*k - y) + \lambda \left\| {\nabla x} \right\|_{0.8} $$
(9)

Our model employs the hyper-Laplacian prior distribution utilized by [24]. In Eq. (9), we set \( P_{x} \left( x \right) = \left\| {\nabla x} \right\|_{\rho } \) and ρ = 0.8.

We have difficulty to find a solution to the optimization problem expressed by Eq. (9), because R(·) is a nonlinear function. A previous method called iteratively reweighted least squares (IRLS) [24] is utilized to solve Eq. (9). The quadratic problem of Eq. (10) is represented iteratively as follows:

$$ x^{{\left[ {t + 1} \right]}} = \mathop {\arg \hbox{min} }\limits_{x} \sum\limits_{p} {\left\{ \begin{aligned} w^{x} \left| {\left( {x*k - y} \right)_{p} } \right|^{2} \hfill \\ + \lambda \left( \begin{aligned} w_{h}^{x} \left| {\left( {\partial_{h} x} \right)_{p} } \right|^{2} \hfill \\ + w_{v}^{x} \left| {\left( {\partial_{v} x} \right)_{p} } \right|^{2} \hfill \\ \end{aligned} \right) \hfill \\ \end{aligned} \right\}} $$
(10)

In Eq. (10), \( w^{x} = \frac{{R^{\prime } \left( {\left( {x^{\left[ t \right]} *k - y} \right)_{p} } \right)}}{{\left( {x^{\left[ t \right]} *k - y} \right)_{p} }} \), R′(·) is the derivative function of R(·). Similarly, \( w_{h}^{x} = \left| {\left( {\partial_{h} x^{\left[ t \right]} } \right)_{p} } \right|^{ - 1.2} \) and \( w_{v}^{x} = \left| {\left( {\partial_{v} x^{\left[ t \right]} } \right)_{p} } \right|^{ - 1.2} \) are also derivative functions. t indicates the iteration number, while p is the position of a pixel. Equation (10) can be considered as a least square problem, which is solved by calculating the conjugate gradient. Using Eqs. (9) and (10), we can calculate the intermediate latent version x. Then, we obtain the kernel k by solving Eq. (8).

The intensity of the image is substituted by the derivatives of image in the data fidelity term. Then we calculate kernel k using Eq. (11). As the image gradient has proved to determine more accurate blur kernel [5, 20], the image intensity is replaced with the image derivatives in the data fidelity term and we estimate the kernel k by Eq. (11).

$$ \mathop {\hbox{min} }\limits_{k} \quad R(\nabla x*k - \nabla y) + \gamma \left\| k \right\|_{1} $$
(11)

After estimating the intermediate latent image x, the blur kernel k is calculated by solving Eq. (8). Analogous to Eq. (9), the IRLS algorithm can be employed to solve Eq. (11) in the form of Eq. (12).

$$ k^{{\left[ {t + 1} \right]}} = \mathop {\arg \hbox{min} }\limits_{k} \sum\limits_{p} {\left\{ \begin{array}{l} w_{h}^{k} \left| {\left( {\partial_{h} x*k - \partial_{h} y} \right)_{p} } \right|^{2} \hfill \\ + w_{v}^{k} \left| {\left( {\partial_{v} x*k - \partial_{v} y} \right)_{p} } \right|^{2} \hfill \\ + \gamma w^{k} \left| {k_{p} } \right|^{2} \hfill \\ \end{array} \right\}} $$
(12)

Once the kernel k is obtained, the negative values in k are set to 0. Then, k is normalized such that the sum of its values is 1. We use the homomorphic decomposition to extract the reflectance component, while employing optimal blur kernel computation to accurately estimate the latent image x and the blur kernel k.

5 Experimental results

We evaluated the performance of the proposed deblurring method on various test images. We compare our simulation results with those of the existing methods [4, 21, 22]. In this experiment, we used three common image blurring algorithms to produce blurred target images: Gaussian blur, motion blur and out-of-focus blur. It is known that Gaussian blur is more challenging to deblur than motion blur and out-of-focus blur. Figure 3 shows eight test images and eight target blurred images produced by applying Gaussian blur, motion blur and out-of-focus blur. The images are named as follows: Bicycle, Lena, Boat, Motorbikes, Soldier, Text 1, RailCity and Text 2.

Fig. 3
figure 3

Original and blurred versions of Bicycle, Boat, and Lena, Motorbikes, Soldier, Text 1, RailCity, and Text 2 images: a original image, b blurred images produced by Gaussian blur, c blurred images produced by motion blur, and d blurred images produced by out-of-focus blur

Full size image

We implemented the proposed algorithm for estimating an optimal kernel in MATLAB and utilized peak signal-to-noise ratio (PSNR) as a metric for evaluating the performance of the deblurred results [25]. To show the quality of the estimated kernels, Fig. 4 presents the deblurred images of bicycle restored by the methods in [4, 21, 22], and the proposed approach.

Fig. 4
figure 4

Deblurred results of Gaussian blur for Bicycle image: a the result of [4]; b the result of [21]; c the result of [22]; and d the result of the proposed approach

Full size image

As Fig. 4d presents, the proposed approach outperforms the state-of-the-art methods in [4, 21, 22]. Due to space restrictions, we do not show the deblurred results of the seven remaining images in this article, while we include these results in the supplementary material. Supplementary figures 1 and 2 are two additional deblurred results of motion blur and out-of-focus for bicycle test image, respectively. Supplementary figures 3–23 represent the visual deblurred results of Gaussian blur, motion blur and out-of-focus for Lena, Boat, Motorbikes, Soldier, Text 1, RailCity and Text 2 images. Tables 1, 2 and 3 compare the deblurring performance of the proposed approach with three existing methods using PSNR (dB). We can observe that the proposed approach provides an improvement of 12% in PSNR compared with the methods of [4, 21, 22].

Table 1 Comparison of the deblurring performance of Gaussian blur of the methods in [4, 21, 22], and the proposed approach, which are measured in PSNR (dB) for eight images: Bicycle, Lena, Boat, Motorbikes, Soldier, Text 1, RailCity, and Text 2 images
Full size table
Table 2 Comparison of the deblurring performance of motion blur of [4, 21, 22], and the proposed approach, which are measured in PSNR (dB) for eight images: Bicycle, Lena, Boat, Motorbikes, Soldier, Text 1, RailCity, and Text 2 images
Full size table
Table 3 Comparison of the deblurring performance of out-of-focus blur of [4, 21, 22], and the proposed approach, which are measured in PSNR (dB) for eight images: Bicycle, Lena, Boat, Motorbikes, Soldier, Text 1, RailCity, and Text 2 images
Full size table

Tables 1, 2 and 3 illustrate the performance comparison of the proposed approach with existing methods. We observe from the tables that the proposed approach has higher PSNR values (up to 12% higher) than the methods reported in [4, 21, 22]. This is because the proposed approach has reflectance component effectively combined with outlier information to provide intermediate latent images for blur kernel estimation more efficiently.

6 Conclusion

A new homomorphic image deconvolution approach was presented to address the problem of image deblurring. This approach combines the homomorphic decomposition and optimal blur kernel computation. Homomorphic decomposition was used to extract the reflectance component from the given image as it contains most of the image information. We utilize only the reflectance component to estimate the kernel. This process removes redundant details in the given image and consequently plays a key role in efficiently extracting accurate motion blur kernels. We evaluated the proposed deblurring approach for various images in comparison with three previous methods. The extensive simulation results demonstrated an improvement of up to 12% of PSNR over the previous methods.