When the camera and the subject move about one another during the exposure, the result is a typical artifact known as motion blur. Computer vision tasks like autonomous driving, object segmentation, and scene analysis can negatively impact this effect, which blurs or stretches the image’s object contours, diminishing their clarity and detail. To create efficient methods for removing motion blur, it is essential to understand where it comes from.
There has been a meteoric rise in the use of deep learning in image processing in the past several years. The robust feature learning and mapping capabilities of deep learning-based approaches enable them to acquire intricate blur removal patterns from large datasets. As a result, picture deblurring has come a long way.
Over the past six years, deep learning has made great strides in blind motion deblurring. Deep learning systems can accomplish end-to-end picture deblurring by learning the blur features from the training data. Improving the effectiveness of image deblurring, they can directly produce clear photos from blurred ones. Deep learning approaches are more versatile and resilient in real-world circumstances than previous methods.
A new study by the Academy of Military Science, Xidian University, and Peking University explores everything from the causes of motion blur to blurred image datasets, evaluation measures for image quality, and methodologies developed. Existing methods for blind motion deblurring may be classified into four classes: CNN-based, RNN-based, GAN-based, and Transformer-based approaches. The researchers present a categorization system that uses backbone networks to organize these methods. Most picture deblurring methods use paired images to train their neural networks. Two main types of fuzzy image datasets are currently available: synthetic and genuine. The Köhler, Blur-DVS, GoPro, and HIDE datasets are only a few examples of synthetic datasets. Examples of real image databases are RealBlur, RsBlur, ReLoBlur, etc.
CNN-based Blind Motion Deblurring
CNN is extensively utilized in image processing to capture spatial information and local features. Deblurring algorithms based on convolutional neural networks (CNNs) have great efficiency and generalizability when trained with large-scale datasets. Denoising and deblurring images are good fits for CNN’s straightforward architecture. Image deblurring tasks involving global information or long-range dependencies may not be well-suited for CNN-based algorithms due to their potential limitations caused by a fixed-size receptive field. Dilated convolution is the most popular approach to dealing with a small receptive field.
By looking at the steps used to deblur the images, CNN-based blind deblurring techniques can be classified into two broad groups. The early two-stage networks and the modern end-to-end systems are two of the most effective ways for deblurring images.
The primary focus of early blind deblurring algorithms was on a single blur kernel image. Two steps comprised the process of deblurring images. The initial step is using a neural network to estimate the blur kernel. To accomplish deblurring, the blurred image is subjected to deconvolution or inverse filtering procedures using the estimated blur kernel. These two-stage approaches to picture deblurring put too much stock in the first stage’s blur kernel estimation, and the quality of that estimation directly correlates to the deblurring outcome. The blur is patchy, and it’s hard to tell how big or which way the image is getting distorted. Therefore, this approach does a poor job of removing complex genuine blur in real scenes.
The input blurred image is transformed into a clear one using the end-to-end image deblurring approach. It employs neural networks to understand intricate feature mapping interactions to improve picture restoration quality. There has been a lot of development in end-to-end algorithms for deblurring images. Convolutional neural networks (CNNs) were initially used for end-to-end restoration of motion blur images.
RNN-based Blind Motion Deblurring
The team investigated its connection to deconvolution to prove that spatially variable RNNs can mimic the deblurring process. They find that there is a noticeable improvement in model size and training speed when employing the proposed RNNs. In specific cases of picture sequence deblurring, RNN’s ability to grasp temporal or sequential dependencies, which applies to sequence data, could prove useful. When dealing with dependencies that span multiple periods, issues like gradient vanishing or explosion may arise. In addition, RNN struggles to grasp spatial information regarding image deblurring tasks. Consequently, RNNs are typically used in conjunction with other structures to achieve image deblurring tasks.
GAN-based Blind Motion Deblurring
Image deblurring is another area where GANs have shown success, following their success in computer vision tasks. With GAN and adversarial training, picture generation becomes more realistic, leading to better-deferred results. The generator and the discriminator receive input to fine-tune their training; the former learns to recover clear images from fuzzy ones, while the latter determines the integrity of the generated clear images.
However, the team states that the training could be shaky. Therefore, it’s important to strike a balance between training generators and discriminators. Pattern crashes or training patterns that do not converge are other possible outcomes.
Transformer-based Blind Motion Deblurring
Transformer offers processing benefits for some picture tasks that necessitate long-distance reliance and the ability to gather global information and handle the problem of remote spatial dependence. Nevertheless, the computational cost of the picture deblurring work is substantial because it requires processing a huge number of pixels.
The researchers highlight that big, high-quality datasets are required to train and optimize deep learning models because of how important data quality and label accuracy are in this process. There is hope that deep learning models can be fine-tuned in the future to make them faster and more efficient, opening up new possibilities for their use in areas like autonomous driving, video processing, and surveillance.
Check out the Paper and Github. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. Join our 36k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and LinkedIn Group.
If you like our work, you will love our newsletter..
Don’t Forget to join our Telegram Channel
Dhanshree Shenwai is a Computer Science Engineer and has a good experience in FinTech companies covering Financial, Cards & Payments and Banking domain with keen interest in applications of AI. She is enthusiastic about exploring new technologies and advancements in today’s evolving world making everyone’s life easy.
