深度学习驱动的超高清图修复技术——综述

article/2025/6/19 8:42:57

Deep Learning-Driven Ultra-High-Definition Image Restoration: A Survey

Liyan Wang, Weixiang Zhou, Cong Wang, Kin-Man Lam, Zhixun Su, Jinshan Pan

Abstract

Ultra-high-definition (UHD) image restoration​​ aims to specifically solve the problem of ​​quality degradation in ultra-high-resolution images​​. Recent advancements in this field are predominantly driven by ​​deep learning-based innovations​​, including enhancements in ​​dataset construction​​, ​​network architecture​​, ​​sampling strategies​​, ​​prior knowledge integration​​, and ​​loss functions​​. In this paper, we systematically review recent progress in ​​UHD image restoration​​, covering various aspects ranging from ​​dataset construction​​ to ​​algorithm design​​. This serves as a valuable resource for understanding ​​state-of-the-art developments​​ in the field. We begin by summarizing ​​degradation models​​ for various ​​image restoration subproblems​​, such as ​​super-resolution​​, ​​low-light enhancement​​, ​​deblurring​​, ​​dehazing​​, ​​deraining​​, and ​​desnowing​​, and emphasizing the ​​unique challenges​​ of their application to ​​UHD image restoration​​. We then highlight existing ​​UHD benchmark datasets​​ and organize the literature according to ​​degradation types​​ and ​​dataset construction methods​​. Following this, we showcase major milestones in ​​deep learning-driven UHD image restoration​​, reviewing the progression of ​​restoration tasks​​, ​​technological developments​​, and ​​evaluations of existing methods​​. We further propose a ​​classification framework​​ based on ​​network architectures​​ and ​​sampling strategies​​, helping to clearly organize existing methods. Finally, we share insights into the ​​current research landscape​​ and propose directions for ​​further advancements​​.

摘要​

​超高清(UHD)图像修复​​旨在针对性解决​​超高分辨率图像的质量退化问题​​。该领域的最新进展主要由​​基于深度学习的创新​​驱动,包括​​数据集构建​​、​​网络架构设计​​、​​采样策略优化​​、​​先验知识融合​​和​​损失函数改进​​等方面的增强。本文系统综述了UHD图像修复的最新进展,涵盖​​从数据集构建到算法设计的多个维度​​,为理解该领域前沿发展提供了重要参考。我们首先总结了​​多种图像修复子问题​​(如超分辨率、低光增强、去模糊、去雾、去雨、去雪等)的​​退化模型​​,并强调了其在UHD图像修复中面临的​​独特挑战​​;接着梳理了​​现有UHD基准数据集​​,并根据退化类型与数据集构建方法对文献进行分类;随后展现了​​深度学习驱动的UHD图像修复主要里程碑​​,回顾了修复任务、技术发展与现有方法的评估进展;进一步提出基于​​网络架构​​与​​采样策略的分类框架​​,助力系统性组织现有方法;最后,针对当前研究现状分享了见解并提出了未来发展方向的建议。


Introduction

Recently, with the rapid advancement of ​​imaging and acquisition equipment​​, ​​ultra-high-definition (UHD) images​​ featuring ​​high pixel density​​ and ​​resolutions​​ (e.g., 3,840 × 2,160 pixels or higher), become widely used in fields such as ​​video streaming [1], [2]​​, ​​virtual reality [3]​​, ​​medical imaging [4]​​, and ​​satellite remote sensing [5], [6]​​. This surge in applications has significantly heightened user demand for ​​enhanced image clarity​​ and ​​detail performance​​. However, ​​hardware limitations​​, ​​restricted transmission bandwidth​​, and ​​challenging acquisition environments​​ often impede the production of ​​high-quality UHD images​​. Common challenges include ​​insufficient resolution​​ and ​​degradation factors​​ such as ​​blur​​, ​​rain​​, ​​snow​​, ​​haze​​, ​​low light​​, etc. Fig. 1 illustrates examples of ​​image degradation​​ under different conditions, highlighting how these issues compromise ​​image resolution​​ and ​​overall quality​​. Consequently, ​​UHD image restoration​​ has emerged as a ​​critical area of research​​ within ​​computer vision​​ and ​​image processing​​.

近年来,随着​​成像与采集设备​​的快速发展,具有​​高像素密度​​和​​超高分辨率​​(如3,840 × 2,160像素及以上)的​​超高清(UHD)图像​​在视频流媒体、虚拟现实、医学影像、卫星遥感等领域广泛应用。这一应用热潮显著提升了用户对​​图像清晰度​​与​​细节表现​​的需求。然而,​​硬件限制​​、​​传输带宽约束​​以及​​复杂采集环境​​往往导致高质量UHD图像难以生成,常见挑战包括分辨率不足以及模糊、雨雪、雾霾、低光等退化因素。如图1所示,不同条件下的图像退化实例展示了这些问题如何损害图像分辨率与整体质量。因此,​​UHD图像修复​​已成为计算机视觉与图像处理领域的​​关键研究方向​​。

UHD image restoration​​ aims to recover ​​high-quality UHD images​​ from ​​degraded inputs​​, addressing various ​​sub-problems​​, such as ​​UHD image super-resolution​​, ​​deblurring​​, ​​dehazing​​, ​​low-light image enhancement​​, ​​deraining​​, and ​​desnowing​​. While ​​deep learning-based image restoration methods [7]–[17]​​ have achieved remarkable success on ​​lower-resolution images​​ (e.g., ​​low-light image enhancement [18]–[20]​​, ​​dehazing [21], [22]​​, ​​desnowing [23], [24]​​, ​​deraining [25]–[28]​​, and ​​deblurring [29], [30]​​) thanks to advances in techniques like ​​convolutional neural networks (CNNs) [31], [32]​​ and ​​Transformers [33], [34]​​, their effectiveness remains limited when applied to ​​UHD image restoration​​. This limitation stems from the fact that these models are often designed for ​​low-resolution inputs​​ and struggle to ​​scale efficiently to UHD images​​. In addition, ​​UHD images​​ inherently possess ​​more intricate details​​, a ​​wider color gamut​​, and a ​​significantly larger number of pixels​​, which pose ​​unique computational challenges​​. ​

UHD图像修复旨在从退化输入中恢复高质量的UHD图像,涵盖​​超分辨率​​、去模糊、去雾、低光增强、去雨、去雪等多种子问题。尽管基于深度学习的图像修复方法在低分辨率图像上(如低光增强、去雾、去雪、去雨、去模糊)凭借​​卷积神经网络(CNN)​​与​​Transformer等技术取得了显著成功,但其在UHD图像修复中的效果仍受限。这主要因为此类模型通常针对低分辨率输入设计,难以高效扩展到UHD图像。此外,UHD图像本身包含​​更精细的细节​​、​​更广的色域​​及​​显著增多的像素​​,这带来了​​独特的计算挑战​​。

Since the introduction of the ​​first large-scale dataset for UHD image super-resolution reconstruction tasks [35]​​ in 2021, a range of ​​UHD image restoration methods​​ and corresponding datasets have been developed. These include ​​multi-guided bilateral learning​​ for UHD image dehazing [36] with the ​​4KID dataset​​; ​​multi-scale separable-patch integration networks​​ for video deblurring [37] with the ​​4DRK dataset​​; the ​​Transformer-based LLFormer [38]​​ with the ​​UHD-LOL dataset​​ and the ​​Fourier embedding network UHDFour [39]​​ with the ​​UHD-LL dataset​​ for ​​UHD low-light image enhancement​​; and ​​UHDformer [40]​​, which explores ​​feature transformation between high- and low-resolution​​, with the ​​UHD-Haze/UHD-Blur datasets​​ for UHD image restoration. Additionally, the ​​dual interaction prior-driven network UHDDIP [41]​​, paired with the ​​UHDSnow/UHD-Rain datasets​​, addresses UHD image restoration.

These methods employ diverse strategies, including ​​downsampling-enhancement-upsampling structure [37], [39], [41]–[48]​​, ​​encoder-decoder structures with stepwise up-downsampling [38], [49]–[51]​​, and ​​resampling-enhancement structures [52], [53]​​. The evolution of network models has transitioned from ​​CNNs [37], [39], [42], [49]​​ (emphasizing ​​local feature extraction​​) to ​​Transformer-based models [38], [41], [43], [48]​​ (focusing on ​​global modeling​​), and most recently to ​​Multilayer Perceptrons (MLPs) [44]–[47]​​ and ​​Mamba [50]​​ (aiming to ​​reduce computational overhead​​). From 2021 to the present, approximately ​​20 studies​​ have explored ​​deep learning-based UHD image restoration methods​​. A concise summary of these developments is presented in ​​Fig. 2​​.

自2021年首个面向​​超高清(UHD)图像超分辨率重建任务​​的大规模数据集提出以来,一系列​​UHD图像修复方法​​及对应数据集相继涌现。例如:

  • 基于​​4KID数据集​​的​​多引导双边学习网络​​用于UHD图像去雾;
  • 基于​​4DRK数据集​​的​​多尺度可分离补丁集成网络​​用于视频去模糊;
  • 基于​​UHD-LOL数据集​​的​​Transformer架构LLFormer​​,以及基于​​UHD-LL数据集​​的​​傅里叶嵌入网络UHDFour​​用于UHD低光图像增强;
  • 基于​​UHD-Haze/UHD-Blur数据集​​的​​UHDformer​​,探索高低分辨率间的特征变换以实现UHD图像修复;
  • 结合​​UHDSnow/UHD-Rain数据集​​的​​双交互先验驱动网络UHDDIP​​处理UHD图像修复。

这些处理UHD图像的方法采用了​​多样化策略​​,包括:

  • ​下采样-增强-上采样结构
  • ​分步上下采样的编码器-解码器结构​​;
  • ​重采样-增强结构​​。

网络模型的演进从​​侧重局部特征提取的CNN​​,转向​​聚焦全局建模的Transformer​​,再到近年旨在​​降低计算开销的MLP​​与​​Mamba。2021年至今,已有约20项研究探索​​基于深度学习的UHD图像修复方法​​,其发展脉络可简要总结如图2所示。

Although ​​deep learning​​ has dominated research on UHD image restoration, there is a lack of ​​comprehensive and in-depth surveys​​ on deep learning-based solutions. Therefore, our work ​​systematically and comprehensively reviews​​ the research in the field to provide a useful starting point for understanding ​​major developments​​, ​​limitations of existing approaches​​, and ​​potential future research directions​​. The main contributions of this paper are threefold:

• We conduct a ​​systematic review​​ of research progress in ​​deep learning-based UHD image restoration​​, covering ​​problem definitions​​ for various UHD restoration tasks, ​​challenges​​, the development of ​​benchmark datasets​​, and the ​​improvements and limitations​​ of existing methods.
• We propose an ​​effective classification method​​ for existing deep learning-based UHD image restoration methods and analyze ​​representative benchmarks​​ under different subtasks.
• Finally, we discuss the ​​challenges faced by current deep learning methods​​ and outline ​​promising future directions​​.

尽管深度学习主导了UHD图像修复研究,但针对​​基于深度学习的解决方案​​仍缺乏全面且深入的综述。因此,本文通过​​系统性梳理与综合分析​​,为理解该领域的主要进展、现有方法的局限性及未来潜在研究方向提供参考。本文的主要贡献包括以下三点:

  • ​系统性综述​​:覆盖各类UHD修复任务的​​问题定义​​、​​挑战​​、​​基准数据集发展​​及现有方法的​​改进与局限性​​;
  • ​分类与基准分析​​:提出​​基于深度学习的UHD图像修复方法分类框架​​,并在不同子任务下分析代表性基准;
  • ​挑战与未来方向​​:探讨当前深度学习方法面临的挑战,并展望​​有潜力的研究方向​​。

Challenges 

UHD images​​, affected by the aforementioned degradation processes, pose ​​unique challenges​​ for restoration due to their ​​ultra-high resolution​​ and ​​dense pixel characteristics​​:
• ​High computational requirements​: Compared to ​​high-definition (HD) images​​, UHD images contain ​​significantly more pixels​​. Processing such ​​large-scale feature maps​​ demands ​​substantial computational resources and storage​​, necessitating ​​advanced GPUs​​ and ​​high-performance hardware​​.
• ​Difficulty in recovering details​: UHD images capture ​​intricate details​​, and degradation effects are ​​more pronounced at high resolutions​​. Although existing methods perform adequately with ​​low-resolution images​​, they often struggle to preserve ​​fine structures​​ in UHD images, resulting in ​​texture loss​​ or ​​artifacts​​ in the restoration output.
​Lack of training datasets​​: Methods trained on ​​low-resolution datasets​​ cannot directly process UHD images, as they require ​​large amounts of degraded-clear UHD image pairs​​ for fine-tuning. However, most publicly available datasets are dominated by ​​HD and lower-resolution images​​, with a notable lack of ​​datasets specialized for UHD scenarios​​. This limitation significantly hinders the ​​development and testing of algorithms​​.
• ​​Poor algorithm adaptability​: Many restoration algorithms are designed for ​​low-resolution images​​ and fail to ​​scale effectively to UHD images​​. Developing ​​new algorithms​​ that accommodate the ​​distinct characteristics of UHD images​​ is an urgent need.

受前述退化过程影响的​​超高清(UHD)图像​​,因其​​超高分辨率​​与​​密集像素特性​​,在修复过程中面临独特挑战:

  • ​高计算需求​​:相较于​​高清(HD)图像​​,UHD图像包含​​显著更多的像素​​。处理此类​​大规模特征图​​需消耗​​大量计算资源与存储空间​​,必须依赖​​高端GPU​​及​​高性能硬件​
  • ​细节恢复困难​​:UHD图像包含​​精细细节​​,且退化效应在高分辨率下更显著。现有方法虽能处理​​低分辨率图像​​,却常难以有效保留UHD图像的​​细微结构​​,导致修复结果出现​​纹理丢失​​或​​伪影​
  • ​训练数据匮乏​​:基于​​低分辨率数据集​​训练的方法无法直接处理UHD图像,需​​大量退化-清晰UHD图像对​​进行微调。然而,公开数据集以​​HD及以下分辨率图像为主​​,​​专为UHD场景设计的数据集严重不足​​,极大阻碍​​算法开发与测试​
  • ​算法适应性差​​:多数修复算法针对​​低分辨率图像​​设计,难以​​有效扩展至UHD图像​​。亟需开发适应​​UHD图像特性​​的新算法

Datasets

This section highlights the key benchmark datasets developed for UHD image restoration, categorized by the specific challenges they address.

Approaches

This section presents a comprehensive and systematic review of existing deep learning-based UHD image restoration methods. This review covers various tasks, including super-resolution reconstruction, low-light image enhancement, dehazing, deblurring, deraining, and desnowing.


Technical

In this section, we examine the evolution of deep learningbased UHD image restoration methods, focusing on three critical aspects:

  1. network architecture
  2. sampling strategy
  3. loss function

1. Network Architectures

​Existing UHD image restoration models​​ employ a variety of ​​advanced network architectures​​, including ​​convolutional neural networks (CNNs) [31]​​, ​​UNet [62]​​, ​​pyramid networks [63]​​, ​​multilayer perceptrons (MLPs) [64]​​, ​​Transformers [33]​​, and ​​Mamba [65]​​. These architectures provide ​​different processing methods​​ and ​​technical support​​ for UHD image restoration tasks. To systematically analyze their ​​characteristics​​ and ​​application scenarios​​, we categorize existing models into three forms based on ​​UHD image processing methods​​:

  1. ​Downsampling-enhancement-upsampling structure​​,
  2. ​Encoder-decoder structure with stepwise up-downsampling​​,
  3. ​Resampling-enhancement structure​​.

现有UHD图像修复模型​

现有模型采用多种​​先进网络架构​​,包括:

  • ​卷积神经网络(CNNs)​
  • ​UNet​
  • ​金字塔网络​
  • ​多层感知机(MLPs)
  • ​Transformer​
  • ​Mamba​

这些架构为UHD图像修复任务提供​​差异化处理方法​​与​​技术支持​​。为系统分析其​​特性​​与​​应用场景​​,依据​​UHD图像处理方法​​将现有模型分为三类:

  1. ​下采样-增强-上采样结构​
  2. ​分步上下采样的编码器-解码器结构​
  3. ​重采样-增强结构

1.1 Downsampling-Enhancement-Upsampling

Figure 4​​: Summary of the ​​downsampling-enhancement-upsampling structure​​ for UHD image restoration. (a) The ​​single-branch downsampling-enhancement-upsampling architecture​​ focuses on the design of ​​enhancement networks in the low-resolution space​​, utilizing some popular architectures such as ​​CNN, UNet, MLP, and Transformer​​. (b) The ​​dual-branch downsampling-enhancement-upsampling architecture​​ explores the ​​correlation between high- and low-resolution features​​ or incorporates ​​additional prior information​​ to guide the reconstruction process.

1.1 ​​下采样-增强-上采样结构​
​图4​​总结了用于​​超高清(UHD)图像恢复​​的​​下采样-增强-上采样结构​​。(a) ​​单分支下采样-增强-上采样架构​​侧重于在​​低分辨率空间​​中设计​​增强网络​​,采用流行架构如​​CNN、UNet、MLP和Transformer​​。(b) ​​双分支下采样-增强-上采样架构​​探索​​高分辨率与低分辨率特征之间的相关性​​,或融入​​额外先验信息​​以指导重建过程。

Figure 5​​: Overview of ​​dual branch frameworks​​ under the ​​downsampling-enhancement-upsampling structure​​. ​​DMixer [45]​​ ​​upsamples low-resolution features​​ and ​​merges them with high-resolution features​​ for reconstruction. ​​UDR-Mixer [47]​​ ​​feeds high-resolution features into the low-resolution branch​​ to facilitate reconstruction. ​​UHDformer [43]​​ ​​transforms features from high to low resolution​​ and ​​enhances high-resolution reconstruction through concatenation​​. ​​UHDDIP [41]​​ ​​extracts gradient and normal priors in the low-resolution space​​ to ​​interact with low-resolution features​​, ​​guiding high-resolution reconstruction​​.

图 5​​ 总结了​​下采样-增强-上采样结构​​下的​​双分支框架​​。​​DMixer 上采样低分辨率特征​​并将其​​与高分辨率特征融合​​以进行重建。​​UDR-Mixer​​ ​​将高分辨率特征输入低分辨率分支​​以促进重建。​​UHDformer 将特征从高分辨率转换为低分辨率​​,并通过​​拼接增强高分辨率重建​​。​​UHDDIP 在低分辨率空间中提取梯度和法线先验​​,以​​与低分辨率特征交互​​,​​指导高分辨率重建​​。

1.2 Encoder-Decoder with Stepwise Up-downsampling

Figure 6​​: Summary of the ​​Encoder-Decoder structure with stepwise up-downsampling​​ for UHD image restoration. ​​UHDVD [37]​​ and ​​LapDehazeNet [49]​​ progressively downsample the input image based on ​​separable-patch and Laplace pyramid architectures​​ to encode features respectively, while ​​LLFormer [38]​​ and ​​Wave-Mamba [50]​​ reduce the scale of potential features with ​​stepwise​​ and adopt the core components of the ​​Axis-based Transformer block​​ and the ​​Low-Frequency State Space Block (LFSS Block)​​ for restoration, respectively.

1.2 ​​带逐步上下采样的编码器-解码器结构​
​图 6​​ 总结了用于​​UHD图像恢复​​的​​带逐步上下采样的编码器-解码器结构​​。​​UHDVD 和 ​​LapDehazeNet 分别基于​​可分离块(separable-patch)和拉普拉斯金字塔架构​​逐步下采样输入图像以编码特征;而 ​​LLFormer 和 ​​Wave-Mamba 通过​​逐步方式​​降低潜在特征尺度,并分别采用​​基于轴向的Transformer块​​和​​低频状态空间块(LFSS Block)​​ 作为核心组件进行恢复。

1.3 Resampling-Enhancement

Figure 7​​: Summary of the ​​resampling-enhancement structure​​ for UHD image restoration. Unlike previous methods that rely on ​​uniform, content-agnostic downsampling​​ (represented by the gray arrows), (a) a ​​non-uniform sampling enhancement network (NSEN) [52]​​, which incorporates two core designs: 1) ​​content-guided downsampling​​ to generate ​​detail-preserving low-resolution images​​, and 2) ​​invertible pixel alignment​​ that computes inverse functions to remove distortions induced during the downsampling process; and (b) a ​​learning model-aware resampling (LMAR) [53]​​, which focuses on obtaining ​​compensated low-resolution features​​ from UHD input images guided by ​​model knowledge​​. These features are fed into the ​​enhancer​​, along with original low-resolution features, and are subsequently ​​upsampled to UHD results​​.

1.3 ​​重采样-增强结构​
​图 7​​ 总结了用于​​UHD图像恢复​​的​​重采样-增强结构​​。与依赖​​均匀、内容无关下采样​​(灰色箭头表示)的传统方法不同,(a) ​​非均匀采样增强网络(NSEN)包含两大核心设计:1) ​​内容引导下采样​​以生成​​保留细节的低分辨率图像​​;2) ​​可逆像素对齐​​,通过计算逆函数消除下采样过程中的失真。(b) ​​学习模型感知重采样(LMAR)专注于从UHD输入图像中获取​​模型知识引导的补偿低分辨率特征​​。这些特征与原始低分辨率特征一同输入​​增强器​​,最终​​上采样为UHD结果​​。

2 Sampling Strategy

在现有超高清(UHD)图像修复框架中,​​三种主要采样策略​​被广泛采用:

  1. ​高倍率上下采样​​(如4×、8×、16×、32×),常用方法包括​​双三次插值​​、​​双线性插值​​和​​PixelUnshuffle​​。这些方法虽能减少计算负担,但​​高倍率下导致显著信息丢失​​,影响修复质量(例如,PixelUnshuffle在PSNR/SSIM指标最优,但双线性插值在LPIPS更优)。
  2. ​逐步上下采样​​(如从2×降至16×),通常集成于​​编码器-解码器结构​​以提取多尺度特征。然而,​​下采样程度增加会不可逆损伤图像质量​​,后续上采样难以恢复。
  3. ​内容相关重采样​​,通过​​自适应调整采样率​​解决上述问题。例如:
    • ​小波变换​​避免关键信息丢失,
    • ​Min-p采样​​保留高置信度特征并丢弃次要特征,
    • ​非均匀下采样器​​根据图像细节丰富度动态采样,
    • ​模型感知重采样(LMAR)​​ 利用模型知识定制采样,兼容现有插值方法且无需重新训练增强网络。

​核心挑战​​在于​​平衡计算效率与信息保留​​,高倍率采样虽高效但牺牲细节,而内容相关策略通过智能优化(如动态采样和模型驱动)提升鲁棒性。

3 Loss Functions

1) Pixel-Level Loss

  • 𝐿1 loss (Mean Squared Error, MAE)
  • 𝐿2 loss (Mean Squared Error, MSE)
  • 𝑆𝑚𝑜𝑜𝑡ℎ𝐿1 loss [66]
  • 𝐶ℎ𝑎𝑟𝑏𝑜𝑛𝑛𝑖𝑒𝑟 loss
  • 𝑇𝑉 (Total Variation) loss

2)𝑆𝑆𝐼𝑀 loss [67] 

3)Frequency-Domain Loss [68]

4)Perceptual loss [69]

5)Adversarial loss [70]


Evaluation

1 Evaluation Metrics

1.1 Full-Reference Metrics

PSNR. Peak Signal-to-Noise Ratio (PSNR) [71] 

SSIM. Structural Similarity Index Measure (SSIM) [67] 

MAE/MSE. Mean Absolute Error (MAE) and Mean Squared Error (MSE) 

LPIPS. LPIPS [72] assesses perceptual similarity by calculating the distance between feature representations in a neural network. This deep learning-based metric approximates human judgment of image quality. Lower LPIPS values correspond to higher perceptual similarity.

全参考指标​
  • ​PSNR​​(峰值信噪比)[71]:衡量修复图像与真实图像之间的信噪比,值越高表示质量越接近真实图像。
  • ​SSIM​​(结构相似性指数)[67]:通过亮度、对比度和结构相似性综合评估图像质量,值越接近1表示修复效果越好。
  • ​MAE/MSE​​(平均绝对误差/均方误差):直接计算像素级差异,MAE值越小表示误差越低,MSE值越小表示修复精度越高。
  • ​LPIPS​​(基于学习的感知相似性指标)[72]:利用神经网络提取特征的距离评估​​感知相似性​​,更贴近人类对图像质量的判断。​​LPIPS值越低​​,表示感知质量越接近真实图像。

1.2 No-Reference Metrics

NIQE. The primary purpose of Natural Image Quality Evaluator (NIQE) [75] is to assess an image’s naturalness based on statistical models of natural scenes. It determines how visually natural and realistic an image appears without requiring reference images. Lower NIQE scores indicate more natural and realistic images.

MUSIQ. Multi-scale Image Quality (MUSIQ) [76] evaluates image quality by analyzing contrast preservation across multiple scales. It focuses on the preservation of fine details and textures in the image after processing. Higher MUSIQ values signify better image quality.

PI. Perceptual Index (PI) combines two metrics, MAE and NIQE, to evaluate image perceptual quality. It emphasizes both aesthetic appeal and naturalness. A lower PI score reflects better perceptual quality.

 无参考指标​
  • ​NIQE​​(自然图像质量评估器)[75]:基于自然场景的统计模型评估图像的​​自然性​​,无需参考图像。​​NIQE值越低​​,表示图像越接近真实自然场景。
  • ​MUSIQ​​(多尺度图像质量评估)[76]:通过​​多尺度对比度分析​​衡量细节和纹理的保留程度。​​MUSIQ值越高​​,表示图像质量越好。
  • ​PI​​(感知指数):结合​​MAE与NIQE​​,综合评价图像的​​美学吸引力​​和​​自然性​​。​​PI值越低​​,表示感知质量越优。

2 Comparision Results

--------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------

3 Computational Complexity


Future

1. ​​Effective Processing Paradigm​

As previously discussed, current ​​UHD image restoration methods​​ predominantly follow two main processing paradigms: ​​the downsampling-enhancement-upsampling​​ and ​​the encoder-decoder with stepwise up-downsampling​​. These paradigms aim to ​​reduce computational costs​​ by employing high magnification or stepwise downsampling; however, a significant drawback is ​​the information loss​​ which degrades overall restoration quality. Although some researchers have attempted to avoid information loss through ​​wavelet transforms​​ [50], ​​the connection with the augmentation network​​ has not been considered, and its effectiveness requires further improvement. Additionally, advancements in ​​learning model-aware resampling methods​​ [53] have shown potential for ​​preserving feature consistency​​ between UHD image inputs and their corresponding low-resolution inputs ​​without requiring retraining​​ of the enhancement network. Thus, exploring ​​the intrinsic connection between resampling and enhancement networks​​ is a potential research direction for developing ​​more effective processing paradigms​​.

高效处理范式​

如前所述,当前​​UHD图像修复方法​​主要遵循两种处理范式:​​下采样-增强-上采样​​和​​分阶段上下采样的编码器-解码器结构​​。这些范式通过采用高倍率或分步下采样来​​降低计算成本​​,但其显著缺点是​​信息丢失​​导致修复质量下降。尽管已有研究者尝试通过​​小波变换​​[50]避免信息丢失,但未考虑​​与增强网络的关联​​,且其效果仍需改进。此外,​​模型感知重采样方法​​[53]的最新进展展现了潜力,能够在​​无需重新训练增强网络​​的前提下,保持UHD图像输入与其对应低分辨率输入之间的​​特征一致性​​。因此,探索​​重采样操作与增强网络的内在关联​​是开发​​更高效处理范式​​的潜在研究方向。

2 ​​Lightweight Network Design​

Existing ​​deep learning models​​ face significant ​​computational challenges​​, which limit their application in UHD image restoration tasks. Current restoration methods, ranging from ​​traditional CNNs​​ to advanced ​​Transformer architectures​​, demand ​​substantial computational power​​, making them heavily reliant on ​​high-end GPUs​​. Although some networks have been proposed to reduce these complexities through ​​MLPs​​ and novel ​​Mamba frameworks​​, they fall short of enabling ​​direct implementation on mobile devices​​. Consequently, future research should prioritize the development of ​​lightweight network architectures​​ that balance ​​performance and efficiency​​, enabling broader applicability across ​​diverse devices​​.

轻量化网络设计​

现有​​深度学习模型​​面临显著的​​计算挑战​​,限制了其在UHD图像修复任务中的应用。当前修复方法(从​​传统CNN​​到先进的​​Transformer架构​​)需要​​大量计算资源​​,严重依赖​​高端GPU​​。尽管已有网络通过​​MLP​​和新型​​Mamba框架​​尝试降低复杂度,但仍无法实现​​在移动设备上直接部署​​。因此,未来研究需优先开发​​轻量化网络架构​​,平衡​​性能与效率​​,从而扩大其​​跨多样化设备的适用性​​。

3 ​​Developing Real-World Benchmark Datasets​

As shown in Table 1, except for the ​​UHD-LL dataset​​ [39], most existing benchmark datasets are ​​artificially synthesized​​. Although the data shortage in UHD image restoration has been temporarily alleviated, a significant gap remains between ​​synthetic images​​ and ​​real-world degraded images​​. Models trained on synthetic images often perform well on synthetic test samples but exhibit ​​poor performance on real-world images​​. For this reason, it is imperative to develop ​​large-scale paired training datasets​​ comprising ​​real-world UHD images​​.

开发真实世界基准数据集​

如表1所示,除​​UHD-LL数据集​​外,现有基准数据集多为​​人工合成​​。尽管UHD图像修复领域的数据短缺问题暂时缓解,但​​合成图像​​与​​真实世界退化图像​​之间仍存在显著差距。基于合成数据训练的模型在合成测试样本上表现良好,但对​​真实世界图像的修复效果较差​​。因此,亟需构建包含​​真实世界UHD图像​​的​​大规模成对训练数据集​​。

4. ​​Using Image Priors​

Current UHD image restoration networks predominantly focus on ​​learning intrinsic features​​; however, ​​the incorporation of image priors​​ plays a pivotal role in enhancing image quality. For instance, ​​UHDDIP​​ [41] incorporates ​​gradient and normal priors​​ into its model design, significantly improving ​​structural integrity​​ and ​​detail preservation​​ of restored images. This demonstrates the potential of leveraging image priors to guide the restoration process. Further exploration of ​​alternative image priors​​ opens promising avenues for advancing UHD image restoration methodologies.

利用图像先验​

当前的UHD图像修复网络主要关注​​学习图像固有特征​​,但​​图像先验的融合​​对提升修复质量至关重要。例如,​​UHDDIP​​在模型设计中融入了​​梯度和法向先验​​,显著提高了修复图像的​​结构完整性​​和​​细节保留能力​​。这证明了利用先验信息引导修复过程的潜力。进一步探索​​其他类型的图像先验​​,将为UHD图像修复方法的创新提供机遇。

5. ​​Specialized Evaluation Metrics​

Most image quality evaluation metrics, such as ​​PSNR, SSIM, and LPIPS​​, are designed for ​​standard-resolution images​​ and reflect image quality to a certain extent. However, the ​​high-resolution characteristics​​ of UHD images place greater demands on ​​detail restoration​​ and ​​subjective visual perception​​. For example, the perceived quality of an image may be affected by ​​local detail in different areas​​, which may not be evident in low-resolution images. Therefore, it is crucial to develop ​​evaluation indicators specifically tailored for UHD images​​. These metrics should accurately reflect the unique requirements of UHD images in terms of ​​detail representation​​ and ​​overall sensory quality​​.

专用评估指标​

大多数图像质量评估指标(如​​PSNR、SSIM和LPIPS​​)针对​​标准分辨率图像​​设计,仅能部分反映图像质量。然而,UHD图像的​​高分辨率特性​​对​​细节修复​​和​​主观视觉体验​​提出了更高要求。例如,图像感知质量可能受​​不同区域局部细节​​的影响,而此类现象在低分辨率图像中并不明显。因此,需开发​​专门针对UHD图像的评估指标​​,以准确反映其在​​细节还原​​与​​整体感官质量​​上的独特需求。

6. ​​UHD Images with Multiple Degradations​

Current UHD image restoration algorithms are usually designed to address ​​a single type of degradation​​. However, in practical applications, UHD images are often affected by ​​a combination of degradation factors​​, leaving the need for ​​mixed degradation processing​​ largely unresolved. It is worth noting that ​​SimpleIR​​ [48] is the first to propose ​​an all-in-one restoration method​​ for UHD images. While ​​all-in-one image restoration methods​​ have shown significant progress on low-resolution images using ​​prompt learning​​ and ​​dynamic network technologies​​, these techniques have not yet been applied to ​​UHD images​​.

多退化类型UHD图像修复​

现有UHD图像修复算法通常针对​​单一退化类型​​设计。然而,实际应用中UHD图像常受​​多种退化因素共同影响​​,导致​​混合退化处理需求​​尚未解决。值得注意的是,​​SimpleIR首次提出了​​面向UHD图像的一体化修复方法​​。尽管通过​​提示学习​​和​​动态网络技术​​,一体化修复方法已在低分辨率图像中取得显著进展,但这些技术尚未在​​UHD图像​​中应用。


http://www.hkcw.cn/article/psITTYNfjT.shtml

相关文章

【Docker系列】Docker 容器内安装`ps`命令

博客目录 一、为什么需要在 Docker 容器中安装ps命令二、不同 Linux 发行版的安装方法1. Alpine Linux 镜像的安装方法2. Debian/Ubuntu 镜像的安装方法3. CentOS/RHEL 镜像的安装方法 三、验证安装与基本使用四、永久解决方案:修改 Dockerfile1. Alpine 基础镜像的…

【KWDB 创作者计划】_再热垃圾发电汽轮机仿真与监控系统:KaiwuDB 批量插入10万条数据性能优化实践

再热垃圾发电汽轮机仿真与监控系统:KaiwuDB 批量插入10万条数据性能优化实践 我是一台N25-3.82/390型汽轮机,心脏在5500转/分的轰鸣中跳动。垃圾焚烧炉是我的胃,将人类遗弃的残渣转化为金色蒸汽,沿管道涌入我的胸腔。 清晨&#x…

对蚁群算法的理解和实例详解

目录 一、算法概述 二、实例详解 1)问题分析 2)初始化参数 2)设置蚂蚁初始位置 3)选择路径 4)记录本次最佳路径 5)更新信息素 6)清空禁忌表 三、计算结果 四、总结 一、算法概述 一群…

【PowerPoint专栏】PowerPoint的保存选项

在PowerPoint的保存选项中有非常多的可用选项,保存的类型也非常多。 在PowerPoint中的工具选项中同样有一些相关的菜单操作帮助用户完成一些特殊操作。 在

直击2025粤港澳大湾区车展 科技引领未来车展

5月31日,第二十九届粤港澳大湾区车展在深圳国际会展中心(宝安)拉开帷幕。本届车展延续“面向科技、面向未来、面向市场”的主题,以“科技Alpha车展”为核心方向,探索汽车前沿科技。展会规模超过26万平方米,有超8万平方米的户外活动体验区及试驾专区。车展期间,全球近百家…

新王加冕!巴黎5比0国米首夺欧冠 年轻风暴席卷欧洲

北京时间6月1日凌晨,2024至2025赛季欧冠联赛决赛在德国慕尼黑安联球场进行。经过90分钟的激战,法甲巴黎圣日耳曼队以5比0大胜意甲国际米兰队,夺得队史首座“大耳朵杯”。此前的淘汰赛中,“大巴黎”先后淘汰了利物浦队、阿斯顿维拉队和阿森纳队三支英超劲旅。而国米则在半决…

定制一款国密浏览器(13):预置国密根证书到浏览器

由于国密算法没有得到国外的认可,所以 Chromium、Firefox 等浏览器均不支持国密算法。即使我们修改了 Chromium 的源码,增加了国密算法的支持,但还不能在浏览器中正常使用。因为这涉及到证书的信任问题,国密证书都是国内厂商签发的,国密根证书并没有集成到系统和浏览器中。…

新闻数据加载(鸿蒙App开发实战)

本案例基于ArkTS的声明式开发范式,介绍了数据请求和onTouch事件的使用。包含以下功能: 数据请求。列表下拉刷新。列表上拉加载。 网络数据请求需要权限:ohos.permission.INTERNET 一、案例效果截图 操作说明: 点击应用进入主页…

Baklib企业CMS全流程管控与智能协作

企业CMS全流程管控方案解析 现代企业内容管理中,全流程管控的实现依赖于对生产、审核、发布及迭代环节的系统化整合。通过动态发布引擎与元数据智能标记技术,系统可自动匹配内容与目标场景,实现标准化模板驱动的快速部署。针对多分支机构的复…

当前用户的Git全局配置情况:git config --global --list

通过config命令可以查询当前用户的全局配置情况。这些配置项定义了 Git 在全局范围内的行为,包括如何处理大文件、SSL 证书验证以及提交时的用户信息。 git config --global --list http.sslVerifyfalse 这个配置项禁用了 SSL 证书验证。这在与自签名证书的 Git 服…

编辑器之神 Vim

终于到了主包最期待的环节了 简单介绍 这是linux中最能装逼的一个环节 服务器哪有界面版的,都是纯粹的命令行 这个时候操作就比较复杂,需要使用一些另类的编辑器 介绍几个编辑器吧 vi nano emacs vim,这些都是命令版的编辑器,基…

任务19:实现山东省气温预测

任务描述 知识点: 时间序列分析 重 点: 指数平滑法ARIMA模型Python连接数据库,查询数据 内 容: 读取并创建时序数据使用指数平滑法建立模型,并预测下一年山东省各月的平均气温使用ARIMA建立模型,并…

C++ - STL #什么是STL #STL的版本 #闭源开源 #STL的六大组件

文章目录 前言 一、什么是STL 二、STL的版本 1、原始版本 2、P.J.版本 3、RW版本 4、SGI版本 三、闭源、开源 四、STL的六大组件 总结 前言 路漫漫其修远兮,吾将上下而求索; 一、什么是STL STL(standard template libaray 标准模板库)&#…

学习STC51单片机22(芯片为STC89C52RCRC)

记住这个AT指令千万不要去脑子记,要用手册查 每日一言 努力不是为了感动谁,而是为了不辜负自己的野心。 硬件:ESP8266 wife模块 蓝牙,ESP-01s,Zigbee,NB-lot等通信模块都是基于AT指令的设计 老样子 我们用…

华为OD机试真题——统计匹配的二元组个数(2025A卷:100分)Java/python/JavaScript/C++/C语言/GO六种最佳实现

2025 A卷 100分 题型 本文涵盖详细的问题分析、解题思路、代码实现、代码详解、测试用例以及综合分析; 并提供Java、python、JavaScript、C++、C语言、GO六种语言的最佳实现方式! 2025华为OD真题目录+全流程解析/备考攻略/经验分享 华为OD机试真题《统计匹配的二元组个数》:…

LLMTIME: 不用微调!如何用大模型玩转时间序列预测?

今天是端午节,端午安康!值此传统佳节之际,我想和大家分享一篇关于基于大语言模型的时序预测算法——LLMTIME。随着人工智能技术的飞速发展,利用大型预训练语言模型(LLM)进行时间序列预测成为一个新兴且极具…

Unity3D仿星露谷物语开发56之保存角色位置到文件

1、目标 游戏中通过Save Game保存角色位置,当重启游戏后,通过Load Game可以恢复角色的位置。 2、Player对象操作 (1)组件添加 给Hierarchy下的Player组件添加Generate GUID组件。 (2)修改SceneSave.cs脚…

AI书签管理工具开发全记录(八):Ai创建书签功能实现

文章目录 AI书签管理工具开发全记录(八):AI智能创建书签功能深度解析前言 📝1. AI功能设计思路 🧠1.1 传统书签创建的痛点1.2 AI解决方案设计 2. 后端API实现 ⚙️2.1 新增url相关工具方法2.1 创建后端api2.2 创建crea…

【计算机网络】第3章:传输层—概述、多路复用与解复用、UDP

目录 一、概述和传输层服务 二、多路复用与解复用 三、无连接传输:UDP 四、总结 (一)多路复用与解复用 (二)UDP 一、概述和传输层服务 二、多路复用与解复用 三、无连接传输:UDP 四、总结 &#xff08…

leetcode hot100刷题日记——30.两数之和

解答: 方法一:迭代 迭代大致过程就是: 算两条链表的当前位的和,加上上一位留下来的进位,就是新链表的当前位的数字。计算当前的进位。 这样,我们迭代需要的东西是:链表1,链表2&…