记录黑客技术中优秀的内容,传播黑客文化,分享黑客技术精华

图像拼接算法的综述

2016-08-05 06:05

一、简介

图像拼接在运动检测和跟踪、增强现实、分辨率增强、视频压缩和图像稳定等机器视觉领域有很大的应用。

如图所示,图像拼接分为四个步骤:图像匹配(registration)、重投影(reprojection)、缝合(stitching)和融合(blending)。

  • 图像匹配:是指一对描绘相同场景之间的几张图片的几何对应关系。一组照片可以是不同时间不同位置的拍摄,或者由多个传感器同时拍摄多张图像。
  • 重投影:通过图像的几何变换,把一系列图片转换成一个共同的坐标系
  • 缝合:通过合并重叠部分的像素值并保持没有重叠的像素值使之生成更大画布的图像
  • 融合:通过几何和光度偏移错误通常导致对象的不连续,并在两个图像之间的边界附近产生可见的接缝。因此,为了减小接缝的出现,需要在缝合时或缝合之后使用混合算法.

不同的技术已被用于不同的拼接算法来处理多个颜色波段。例如,在18192021中,综合所输入的RGB图像的颜色波段,获得的变换参数。在222324中,该RGB图像首先被转换成灰度,然后得到变换参数。在这两种情况下,找到最佳变换参数后,对所有的颜色波段进行处理,实施重投影(reprojection)步骤。

图一 图像拼接的步骤

二、图像拼接算法分类

“图像匹配”和“融合”是直接影响图像拼接性能两个显著的​​研究领域。作为图像拼接的第一个和最后一个步骤,如果没有正确的图像匹配和融合算法,几乎不可能进行成功的图像拼接。在本文中,我们侧重于根据其“图像匹配”和“融合”方法对现存的图像拼接算法进行分类。

如图二所示,根据“图像匹配方法”分类,图像拼接算法可分为基于“空间域”和“频域”。基于空间域的图像拼接可以进一步划分为基于区域的图像拼接和基于特征的图像拼接。基于特征的图像拼接可以再细分为低电平基于特征的图像拼接(low level feature-based image mosaicing)和基于轮廓的图像拼接(contour-based image mosaicing)。低级别的基于特征的拼接可以分为四类:基于Harris角点检测器的拼接、基于FAST角点检测器的拼接、基于SIFT特征检测器的拼接、以及基于SURF特征检测器的拼接。
如图三所示,根据“融合方法”,拼接算法可分为基于平滑过渡(transition smoothening-based)和基于最佳接缝(optimal seam-based)。基于平滑过渡拼接可以进一步被分成基于羽化(feathering-based)、基于金字塔(pyramid-based)、和基于梯度(gradient-based)的拼接。

图二

(图二 根据图像匹配方法的分类)

图三 根据融合方法的分类

(图三 根据融合方法的分类)

三、根据“图像匹配方法”对图像拼接的分类

图像匹配不仅是图像拼接的重要一步,也是它的基础。对于相同的目标,但来自不同的传感器、不同的角度和不同的时间产生的多源图像进行匹配,通过观察各对图像之间的对应关系来计算最佳几何变换。这一过程通过预估的几何变换把多源图像排列在一个共同的参考系中。如果多源图像对应点排列在一起,则图像匹配成功。上述的对应关系可以通过匹配图像之间的模板,或通过匹配从图像中提取的特征,或者通过利用在频域中的相位相关属性来建立。基于所述图像配准不同类别的图像拼接算法将在以下两个小节论述。

3.1 基于空间域图像拼接算法

这类算法使用像素的属性进行图像匹配,因此它们是最直接的图像拼接的方法。现有的图像拼接算法大部分都属于这一类。图像拼接算法大部分都属于这一类。“基于空间域图像拼接算法”可以是基于区域(area-based)或基于特征(feature-based)的。“基于区域“的图像拼接算法依赖于计算待拼接的两个图像的“窗口”像素值18 。基本方法是将图像有关联的“窗口”互相转移,看看有多少像素的匹配。随后,获得图像变换参数来弯曲和拼接图片。基于空间域的拼接算法通常被称为基于像素的拼接,因为它们使用的像素之间的匹配,而不是特征之间的匹配。的最常用的两个基于空间域的图像拼接算法 是基于“归一化互相关”(normalized cross correlation)的拼接和基于“互信息”(mutual information)的拼接。这两种方法都提供了图像相似性的量度,这些指标的较大值来自匹配区域或“窗口”大小。

3.1.1 基于归一化互相关(Normalized Cross Correlation, NCC)的拼接

此方法计算在两个图像中的每个位移(shifts)的“窗口”之间的相似性。它被定义为20

$$\overline{I_1} = {1\over N} \sum_i I_1(x_i)$$

$\overline{I_2}=\frac{1}{N}\sum_i I_2(x_i+u)$

$\overline{I_1}$ 和 $\overline{I_2}$ 是窗口的平均值图像。$I_1(x,y)$和$I_2(x,y)$分别是两张图片。$N$是“窗口”大小,$x_i=(x_i,y_i)$ 是窗口的像素坐标,$u=(u,v)$ 是通过NCC系数计算出的位移或偏移。NCC系数的范围为[-1,1]。 NCC峰值相对应的位移参数表示两个图像之间的几何变换。此方法的优点是计算简单,但是速度特别慢。此外,此类算法要求源图像之间必须有显著的重叠。

为了解决上述问题,[27] ,[44] ,[45] ,[46]几种技术已经被提出。Nasibov等人在图像匹配步骤之前使用的亮度校正矩阵,以便使算法对光照变化不敏感[27]。为了使计算速度更快,Berberidis等人提出的空间互相关来计算源的图像之间的位移的迭代算法[44]。Zhao等人提出了通过基于调整根据规模,从源图像中提取兴趣点的方向的相关窗口方法等[45] ,以增加计算速度。为了提高算法的非刚性变形的存在下的性能,Vercauteren等人[46]提出用黎曼统计(Riemannian statistics)与基于散乱数据拟合(scattered data fitting)的拼接。

3.1.2 基于互信息(Mutual Information, MI)的图像拼接

不同于基于图像强度值其计算相似性的NCC,互信息测量基于两个图像之间共享信息数量的相似性。两个图像$$I_1(X,Y)$$与$$I_2(X,Y)$$之间的MI以熵表示:

$$MI(I_1,I_2)=E(I_1)+E(I_2)-E(I_1,I_2)$$

$$E(I_1)$$和$$E(I_2)$$分别是$$I_1(x,y)$$和$$I_2(x,y)$$的熵。$$E(I_1,I_2)$$表示两个图像之间的联合熵。熵是一个随机变量的变异性指标。$$I_1(x,y)$$的变异性指标表示为:

$$E(I_1)=-\sumg p{I1}(g)log(p\{I_1}(g))$$

……

$$E(I_1,I2)=-\sum{g,h} p_{I_1,I2}(g,h)log(p{I1},p{I_2}(g,h)) $$

……

3.1.3 基于特征的低级拼接

这些拼接方法不需要大量重叠区域的图像,

3.1.3.1 基于Harris角点检测器的拼接

Harris角检测器检测角点从源图像强大的低级别的功能。最初被选择的图像中的局部检测窗口。接着在强度,通过在不同方向少量移动的窗口导致的变化被确定为以下[41] :

$$E(u,v)=\sum_i w(x_i,y_i)[I(x_i+u,y_i+v)-I(x_i,y_i)]^2$$
6

3.1.3.2 基于FAST角点检测器的拼接
3.1.3.2 基于SIFT特征检测器的拼接
3.1.3.2 基于SURF特征检测器的拼接

3.1.3 基于轮廓的拼接

3.2 基于频域图像拼接算法

四、根据“融合方法”对图像拼接的分类

4.1 基于平滑过渡融合的图像拼接算法

4.1.1 基于羽化融合的图像拼接算法

4.1.2 基于金字塔融合的图像拼接算法

4.1.2 基于梯度融合的图像拼接算法

4.2 基于最佳接缝混合的图像拼接算法

五、结论

参考文献

[18]: Sa. Ghannam, A.L. Abbott Cross correlation versus mutual information for image mosaicing Int. J. Adv. Comput. Sci. Appl. (IJACSA), 4 (2013)

[19]: A. Levin, A. Zomet, S. Peleg, Y. Weiss Seamless image stitching in the gradient domain Comput. Vis.-ECCV (2004), pp. 377–389

[20]: R. Szeliski Image alignment and stitching: a tutorial Found. Trends Comput. Graph. Vis., 2 (2006), pp. 1–104

[21]: A. Behrens, M. Guski, T. Stehle, S. Gross, T. Aach, Intensity based multi-scale blending for panoramic images in fluorescence endoscopy, in: IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2010, pp. 1305–1308.

[22]: G. Guandong, J. Kebin, A new image mosaics algorithm based on feature points matching, in: International Conference on Innovative Computing, Information and Control, 2007, pp. 471–471.

[23]: K.-I. Okumura, S. Raut, Q. Gu, T. Aoyama, T. Takaki, I. Ishii, Real-time feature-based video mosaicing at 500 fps, in: International Conference on Intelligent Robots and Systems (IROS), 2013, pp. 2665–2670.

[24]: S. Park, D. Ghosh, N. Kaabouch, R.A. Fevig, W. Semke, Hierarchical multi-level image mosaicing for autonomous navigation of UAV, in: IS&T/SPIE Electronic Imaging, 2012, pp. 830116–830116-9.

[25]L. Miao, Y. Yue, Automatic document image mosaicing algorithm with hand-held camera, in: International Conference on Intelligent Control and Information Processing (ICICIP), 2011, pp. 1094–1097.

[26]
H. Shejiao, H. Yaling, C. Zonghai, J. Ping, Feature-based image automatic mosaicing algorithm, in: in: World Congress on Intelligent Control and Automation, 2006, pp. 10361–10364.

[27]
A. Nasibov, H. Nasibov, F. Hacizade, Seamless image stitching algorithm using radiometric lens calibration for high resolution optical microscopy, in: International Conference on Soft Computing, Computing with Words and Perceptions in System Analysis, Decision and Control, 2009, pp. 1–4.

[28]
T. Vercauteren, A. Perchant, X. Pennec, N. Ayache, Mosaicing of confocal microscopic in vivo soft tissue video sequences, in: Medical Image Computing and Computer-Assisted Intervention–MICCAI, 2005, pp. 753–760.

[29]
R. Miranda-Luna, C. Daul, W.C. Blondel, Y. Hernandez-Mier, D. Wolf, F. Guillemin
Mosaicing of bladder endoscopic image sequences: distortion calibration and registration algorithm
IEEE Trans. Biomed. Eng., 55 (2008), pp. 541–553

[30]
G. Gao, K. Jia, A new image mosaics algorithm based on feature points matching, in: International Conference on Innovative Computing, Information and Control, 2007, pp. 471–471.

[31]
T. Botterill, S. Mills, R. Green, Real-time aerial image mosaicing, in: International Conference of Image and Vision Computing New Zealand (IVCNZ), 2010, pp. 1–8.

[32]
L. Yao, Image mosaic based on SIFT and deformation propagation, in: IEEE International Symposium on Knowledge Acquisition and Modeling Workshop, 2008, pp. 848–851.

[33]
G. Jun-Hui, Z. Jun-Hua, A. Zhen-Zhou, Z. Wei-Wei, L. Hui-Min, An approach for X-ray image mosaicing based on Speeded-up robust features, in: International Conference on Wavelet Active Media Technology and Information Processing (ICWAMTIP), 2012, pp. 432–435.

[34]
F. Yang, L. Wei, Z. Zhang, H. Tang
Image mosaic based on phase correlation and Harris operator
J. Comput. Inform. Syst., 8 (2012), pp. 2647–2655

[35]
M. Vivet, S. Peleg, X. Binefa, Real-time stereo mosaicing using feature tracking, in: IEEE International Symposium on Multimedia (ISM), 2011, pp. 577–582.

[36]
M. El-Saban, M. Izz, A. Kaheel, M. Refaat, Improved optimal seam selection blending for fast video stitching of videos captured from freely moving devices, in: IEEE International Conference on Image Processing (ICIP), 2011, pp. 1481–1484.

[37]
S.Z. Kovalsky, G. Cohen, J.M. Francos, Registration of joint geometric and radiometric image deformations in the presence of noise, in: IEEE/SP Workshop on Statistical Signal Processing, 2007, pp. 561–565.

[38]
D. Vaghela, K. Naina
A review of image mosaicing techniques
Int. J. Adv. Res. Comput. Sci. Manage. Stud., 2 (3) (2014)

[39]
P. Jain, V.K. Shandliya
A review paper on various approaches for image mosaicing
Int. J. Comput. Eng. Res., 3 (4) (2013)

[40]
R. Abraham, P. Simon, Review on mosaicing techniques in image processing, in: International Conference on Advanced Computing and Communication Technologies (ACCT), 2013, pp. 63–68.

[41]
H. Joshi, M.K. Sinha, A survey on image mosaicing techniques, in: International Journal of Advanced Research in Computer Engineering & Technology (IJARCET), vol. 2(2), 2013.

[42]
M.H.M. Patel, A.P.P.J. Patel, A.P.M.S.G. Patel
Comprehensive study and review of image mosaicing methods
Int. J. Eng. Res. Technol. (2012)

[43]
J.M. Fitzpatrick, D.L. Hill, C.R. Maurer Jr.
Image registration
Handbook Med. Imag., 2 (2000), pp. 447–513

[44]
K. Berberidis, I. Karybali, A new efficient cross-correlation based image registration technique with improved performance, in: Proceedings of the European Signal Processing Conference, 2002, pp. 3–6.

[45]
F. Zhao, Q. Huang, W. Gao, Image matching by normalized cross-correlation, in: IEEE International Conference on Acoustics, Speech and Signal Processing, 2006, pp. II–II.

[46]
T. Vercauteren, A. Meining, F. Lacombe, A. Perchant, Real time autonomous video image registration for endomicroscopy: fighting the compromises, in: Biomedical Optics (BiOS), 2008, pp. 68610C–68610C-8.

[47]
A. Dame, E. Marchand, Video mosaicing using a mutual information-based motion estimation process, in: IEEE International Conference on Image Processing (ICIP), 2011, pp. 1493–1496.

[48]
C. de Cesare, M.-J. Rendas, A.-G. Allais, M. Perrier, Low overlap image registration based on both entropy and mutual information measures, in: OCEANS, 2008, pp. 1–9.

[49]
M.B. Islam, M.M.J. Kabir
A new feature-based image registration algorithm
Comput. Technol. Appl., 4 (2013), pp. 79–84

[50]
V.S. Bind
Robust Techniques for Feature-Based Image Mosaicing
National Institute of Technology Rourkela (2013)

[51]
E. Zagrouba, W. Barhoumi, S. Amri
An efficient image-mosaicing method based on multifeature matching
Mach. Vis. Appl., 20 (2009), pp. 139–162

[52]
J. Jiao, B. Zhao, S. Wu, A speed-up and robust image registration algorithm based on fast, in: IEEE International Conference on Computer Science and Automation Engineering (CSAE), 2011, pp. 160–164.

[53]
X. Wang, J. Sun, H.-Y. Peng, Efficient panorama mosaicing based on enhanced-FAST and graph cuts, in: Recent Advances in Computer Science and Information Engineering, vol. 128, 2012, pp. 757–762.

[54]
D.G. Lowe
Distinctive image features from scale-invariant keypoints
Int. J. Comput. Vis., 60 (2004), pp. 91–110

[55]
D. Liqian, J. Yuehui, Moon landform images fusion and Mosaic based on SIFT method, in: International Conference on Computer and Information Application (ICCIA), 2010, pp. 29–32.

[56]
Y. Li, Y. Wang, W. Huang, Z. Zhang, Automatic image stitching using sift, in: International Conference on Audio, Language and Image Processing, 2008, pp. 568–571.

[57]
Y. Lei, W. Xiaoyu, Z. Jun, L. Hui, A research of feature-based image mosaic algorithm, in: International Congress on Image and Signal Processing (CISP), 2011, pp. 846–849.

[58]
H. Bay, T. Tuytelaars, L. Van Gool, Surf: speeded up robust features, in: Computer vision–ECCV, 2006, pp. 404–417.

[59]
N. Geng, D. He, Y. Song
Camera image mosaicing based on an optimized SURF algorithm
Indones. J. Electr. Eng., 10 (2012), pp. 2183–2193

[60]
R. Wen, C. Hui, L. Jiaju, X. Yanyan, R. Haeusler, Mosaicing of microscope images based on SURF, in: International Conference on Image and Vision Computing New Zealand, 2009, pp. 271–275.

[61]
H. Joshi, K. Sinha, Novel techniques image mosaicing based on image fusion using harris aand SURF, in: International Conference on Computer Science and Information Technology, 2013.

[62]
V.S. Bind, P.R. Muduli, U.C. Pati
A robust technique for feature-based image mosaicing using image fusion
Int. J. Adv. Comput. Res., 3 (2013), p. 263

[63]
K. Peng, M. Hongbing, An automatic airborne image mosaicing method based on the SIFT feature matching, in: International Conference on Multimedia Technology (ICMT), 2011, pp. 155–159.

[64]
J. Zhu, M. Ren
Image mosaic method based on SIFT features of line segment
Comput. Math. Methods Med., 2014 (2014)

[65]
J. Xiao, Y. Zhang, M. Shah, Adaptive region-based video registration, in: IEEE Workshops on Application of Computer Vision, 2005, pp. 215-220.

[66]
J. Prescott, M. Clary, G. Wiet, T. Pan, K. Huang, Automatic registration of large set of microscopic images using high-level features, in: IEEE International Symposium on Biomedical Imaging: Nano to Macro, 2006, pp. 1284–1287.

[67]
M. Deshmukh, U. Bhosle
A survey of image registration
Int. J. Image Process. (IJIP), 5 (2011), p. 245

[68]
H. Xie, N. Hicks, G.R. Keller, H. Huang, V. Kreinovich
An IDL/ENVI implementation of the FFT-based algorithm for automatic image registration
Comput. Geosci., 29 (8) (2003), pp. 1045–1055

[69]
C. Wang, Y. Cheng, C. Zhao, Robust subpixel registration for image mosaicing, in: Chinese Conference on Pattern Recognition, 2009, pp. 1–5.

[70]
R. Prados Gutiérrez
Image Blending Techniques and their Application in Underwater Mosaicing
Springer (2014)

[71]
D.K. Jain, G. Saxena, V.K. Singh, Image mosaicing using corner techniques, in: International Conference on Communication Systems and Network Technologies (CSNT), 2012, pp. 79–84.

[72]
Y. Xiong, K. Turkowski, Registration, calibration and blending in creating high quality panoramas, in: IEEE Workshop on Applications of Computer Vision Proceedings, 1998, pp. 69–74.

[73]
P. Liang, X. Zhiwei, D. Jiguang, Joint edge detector based on Laplacian pyramid, in: International Congress on Image and Signal Processing (CISP), 2010, pp. 978–982.

[74]
A. Pandey, U.C. Pati, A novel technique for non-overlapping image mosaicing based on pyramid method, in: IEEE India Conference (INDICON), 2013, pp. 1–6.

[75]
Y. Xiong, Eliminating ghosting artifacts for panoramic images, in: IEEE International Symposium on Multimedia, 2009, pp. 432–437.

[76]
R. Szeliski, M. Uyttendaele, D. Steedly, Fast Poisson blending using multi-splines, in: IEEE International Conference on Computational Photography (ICCP), 2011, pp. 1–8.

[77]
N. Gracias, M. Mahoor, S. Negahdaripour, A. Gleason
Fast image blending using watersheds and graph cuts
Image Vis. Comput. (2009), pp. 597–607

[78]
H. Wen, J. Zhou, An improved algorithm for image mosaic, in: International Symposium on Information Science and Engineering, 2008, pp. 497–500.


知识来源: s1nh.com/post/A-survey-on-image-mosaicing-techniques/

阅读:235158 | 评论:0 | 标签:无

想收藏或者和大家分享这篇好文章→复制链接地址

“图像拼接算法的综述”共有0条留言

发表评论

姓名:

邮箱:

网址:

验证码:

ADS

标签云

本页关键词