Text Modality Assisted Guided Mural Inpainting Algorithm

CHEN Yong; DU Wanjun; ZHANG Shilong

doi:10.13203/j.whugis20240251

您当前的位置：

首页 >

文章列表页 >

Text Modality Assisted Guided Mural Inpainting Algorithm

Photogrammetry and Remote Sensing | 更新时间：2026-11-05

- Text Modality Assisted Guided Mural Inpainting Algorithm
- Geomatics and Information Science of Wuhan University Vol. 51, Issue 4, Pages: 741-750(2026)
- 作者机构：
  
  1.兰州交通大学电子与信息工程学院，甘肃兰州，730070
  2.甘肃省人工智能与图形图像处理工程研究中心，甘肃兰州，730070
- 作者简介：
- 基金信息：
- DOI：10.13203/j.whugis20240251
  CLC： P237
- Received：11 May 2025，
  
  Published：05 April 2026
- 稿件说明：
移动端阅览
陈永,杜婉君,张世龙.文本模态辅助引导的壁画修复算法[J].武汉大学学报（信息科学版）,2026,51(4):741-750.

CHEN Yong,DU Wanjun,ZHANG Shilong.Text Modality Assisted Guided Mural Inpainting Algorithm[J].Geomatics and Information Science of Wuhan University,2026,51(4):741-750.
陈永,杜婉君,张世龙.文本模态辅助引导的壁画修复算法[J].武汉大学学报（信息科学版）,2026,51(4):741-750. DOI：10.13203/j.whugis20240251

CHEN Yong,DU Wanjun,ZHANG Shilong.Text Modality Assisted Guided Mural Inpainting Algorithm[J].Geomatics and Information Science of Wuhan University,2026,51(4):741-750. DOI：10.13203/j.whugis20240251

摘要

现有深度学习算法在壁画修复时仅考虑壁画图像先验信息，缺乏文本信息引导壁画修复，导致修复结果出现语义不一致，细节缺失等问题，对此提出了一种文本模态辅助引导的壁画修复算法。首先，提出一个文本引导的壁画修复网络，利用文本信息作为壁画修复的辅助控制引导，为壁画图像提供上下文引导修复信息。其次，构建文本语义过滤模块，利用掩膜图像和互补图像过滤得到破损区域的文本特征，并设计了语义增强模块，对过滤后的文本特征进行增强，提高文本语义与图像语义的一致性。然后，设计上采样纹理细节修复网络，实现对壁画纹理细节的修复。最后，采用谱归一化判别器博弈对抗，完成修复。真实敦煌壁画的修复实验结果表明，所提方法能够有效完成破损壁画的修复，在主客观评价方面均优于比较算法。

Abstract

Objectives

Existing deep learning algorithms in the inpainting of murals only consider the prior information of mural images to guide the inpainting of damaged areas

and lack of text information to guide the inpainting of murals

resulting in semantic inconsistency and lack of details in the inpainting results. In order to solve this problem

a text modality-assisted guided mural inpainting algorithm is proposed.

Methods

First

a text modality-assisted guided mural inpainting network is proposed

which uses the text information as the control guidance for mural inpainting

and provides context-guided repair information for mural images. Second

a text filtering module is constructed to obtain the text features of the damaged area by filtering the mask image and the complementary image

and a cross-modal semantic enhancement module is designed to enhance the filtered text features to improve the consistency between text semantics and image semantics. Then

the upsampled texture detail inpainting network is designed to achieve bidirectional fusion of shallow and deep features to obtain mural images with fine-grained features. Finally

the normalized discriminator is used to match the recovered murals to the game

and the recovered murals are obtained.

Results

The experimental results of real mural restoration show that this method can effectively repair damaged murals

and is superior to comparative algorithms in subjective and objective evaluation.

Conclusions

The proposed method can effectively repair damaged murals

achieving better visual perception and coordination.

关键词

Keywords

references

李清泉，王欢，邹勤 . 一种基于稀疏表示模型的壁画修复算法［J］. 武汉大学学报（信息科学版）， 2018 ， 43 （ 12 ）： 1847 - 1853 .

LI Qingquan ， WANG Huan ， ZOU Qin . A Murals Inpainting Algorithm Based on Sparse Representation Model ［J］. Geomatics and Information Science of Wuhan University ， 2018 ， 43 （ 12 ）： 1847 - 1853 .

陈永，艾亚鹏，郭红光 . 改进曲率驱动模型的敦煌壁画修复算法［J］. 计算机辅助设计与图形学学报， 2020 ， 32 （ 5 ）： 787 - 796 .

CHEN Yong ， AI Yapeng ， GUO Hongguang . Inpainting Algorithm for Dunhuang Mural Based on Improved Curvature-Driven Diffusion Model ［J］. Journal of Computer Aided Design & Computer Graphics ， 2020 ， 32 （ 5 ）： 787 - 796 .

ZHANG X B ， ZHAI D H ， LI T R ， et al . Image Inpainting Based on Deep Learning： A Review ［J］. Information Fusion ， 2023 ， 90 ： 74 - 94 .

LI R ， ZHENG B . A Spatially Adaptive Hybrid Total Variation Model for Image Restoration Under Gaussian Plus Impulse Noise ［J］. Applied Mathematics and Computation ， 2022 ， 419 ： 126862 .

景慧莹，刘欣怡，张永军，等 . 低秩矩阵辅助的无人机影像建筑物立面纹理自动修复［J］. 武汉大学学报（信息科学版）， 2025 ， 50 （ 1 ）： 144 - 152 .

JING Huiying ， LIU Xinyi ， ZHANG Yongjun ， et al . Low-Rank Matrix Aided Automatic Texture Inpainting of Building Facades from UAV Images ［J］. Geomatics and Information Science of Wuhan University ， 2025 ， 50 （ 1 ）： 144 - 152 .

李晓霖，李刚，张恩琪，等 . 行列式点过程采样的文本生成图像方法［J］. 武汉大学学报（信息科学版）， 2024 ， 49 （ 2 ）： 246 - 255 .

LI Xiaolin ， LI Gang ， ZHANG Enqi ， et al . Determinant Point Process Sampling Method for Text-to-Image Generation ［J］. Geomatics and Information Science of Wuhan University ， 2024 ， 49 （ 2 ）： 246 - 255 .

HAN R Y ， LIU X ， LIAO S H ， et al . Adaptive Image Inpainting Algorithm Based on Sample Block by Kriging Pretreatment and Facet Model ［J］. Journal of Electronic Imaging ， 2021 ， 30 （ 4 ）： 043021 .

ZENG Y H ， FU J L ， CHAO H Y ， et al . Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting ［C］// IEEE/CVF Conference on Computer Vision and Pattern Recognition ， Long Beach， USA ， 2020 .

XIAO Q G ， LI G Y ， CHEN Q C . Image Inpainting Network for Filling Large Missing Regions Using Residual Gather ［J］. Expert Systems with Applications ， 2021 ， 183 ： 115381 .

LI J Y ， HE F X ， ZHANG L F ， et al . Progressive Reconstruction of Visual Structure for Image Inpain‑ting ［C］// IEEE/CVF International Conference on Computer Vision ， Seoul， Korea ， 2020 .

SUVOROV R ， LOGACHEVA E ， MASHIKHIN A ， et al . Resolution-Robust Large Mask Inpainting with Fourier Convolutions ［C］// IEEE/CVF Winter Conference on Applications of Computer Vision ， Waikoloa， USA ， 2022 .

ZHENG C X ， CHAM T J ， CAI J F ， et al . Bridging Global Context Interactions for High-Fidelity Image Completion ［C］// IEEE/CVF Conference on Computer Vision and Pattern Recognition ， New Orleans， USA ， 2022 .

NAZERI K ， NG E ， JOSEPH T ， et al . EdgeConnect： Generative Image Inpainting with Adversarial Edge Learning ［PP/OL］. arXiv （ 2019-01-01 ）［ 2024-01-11 ］. https：//arxiv.org/abs/1901.00212 https://arxiv.org/abs/1901.00212 .

ZHAO Y N ， PRICE B ， COHEN S ， et al . Guided Image Inpainting： Replacing an Image Region by Pulling Content from Another Image ［C］// IEEE Winter Conference on Applications of Computer Vision ， Waikoloa， USA ， 2019 .

GUO X F ， YANG H Y ， HUANG D . Image Inpainting via Conditional Texture and Structure Dual Generation ［C］// IEEE/CVF International Confe‑ rence on Computer Vision， Montreal， Canada ， 2022 .

CAO C J ， FU Y W . Learning a Sketch Tensor Space for Image Inpainting of Man-Made Scenes ［C］// IEEE/CVF International Conference on Computer Vision ， Montreal， Canada ， 2022 .

ZHANG L S ， CHEN Q C ， HU B T ， et al . Text-Guided Neural Image Inpainting ［C］// The 28th ACM International Conference on Multimedia ， Seattle， USA ， 2020 .

余凯，宾燚，郑自强，等 . 基于条件语义增强的文本到图像生成［J］. 软件学报， 2024 ， 35 （ 5 ）： 2150 - 2164 .

YU Kai ， BIN Yi ， ZHENG Ziqiang ， et al . Text-to-Image Generation with Conditional Semantic Augmentation ［J］. Journal of Software ， 2024 ， 35 （ 5 ）： 2150 - 2164 .

LIN Q ， YAN B ， LI J C ， et al . MMFL： Multimodal Fusion Learning for Text-Guided Image Inpainting ［C］// The 28th ACM International Conference on Multimedia ， Seattle， USA ， 2020 .

XU T ， ZHANG P C ， HUANG Q Y ， et al . AttnGAN： Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks ［C］// IEEE/CVF Conference on Computer Vision and Pattern Recognition ， Salt Lake City， USA ， 2018 .

LIU G L ， REDA F A ， SHIH K J ， et al . Image Inpainting for Irregular Holes Using Partial Convolutions ［C］// European Conference on Computer Vision ， Munich， Germany ， 2018 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Intelligent Change Detection Method and Application Based on Wuhan-1 Satellite Imagery

A Review of GNSS Intentional Interference Detection Technologies for Mass-Market Intelligent Terminals： Research Status and Prospects

Research Progress in Eco-Geological Environmental Monitoring and Construction of an Intelligent Monitoring System in the Qinling Mountains

Autonomous Robotic Mapping for Complex Spaces： Progress， Challenges and Future Perspectives

High-Resolution L-Band InSAR Study of Intense Near-Field Deformation Associated with Shallow Moderate Earthquakes

Related Author

No data

Related Institution

No data

AI问答

⁰