Mini-Unmanned Aerial Vehicle-Based Remote Sensing: Techniques, Applications, and Prospects (GRSM2019)

This repository contains a collection of research papers and resources for Mini-Unmanned Aerial Vehicle-Based Remote Sensing. The repository gives a survey of related mini-UAV-based remote sensing, aiming to boost the development of this field.

Table of Contents

• Background
• Survey
  • 1. Related Surveys
  • 2. Camera Calibration
  • 3. Combined Field of View
  • 4. UAV Image Matching
  • 5. Low-altitude AAT
  • 6. Dense Reconstruction
  • 7. Image Stitching
  • 8. Multi-sensor Data Registration
• Dataset & Tools
• Citation

Background

The past few decades have witnessed great progress of unmanned aircraft vehicles (UAVs) in civilian fields, especially in photogrammetry and remote sensing. In contrast with the platforms of manned aircraft and satellites, the UAV platform holds many promising characteristics: flexibility, efficiency, high-spatial/temporal resolution, low cost, easy operation, etc., which make it an effective complement to other remote sensing platforms and a cost-effective means for remote sensing. In light of the popularity and expansion of UAV-based remote sensing in recent years, this paper provides a systematic survey on the recent advances and future prospects of UAVs in the remote-sensing community.

Specifically, this paper is devoted to present:

• The main challenges and key technologies of remote-sensing data processing based on UAVs are discussed and summarized.
• A systematic survey of data processing technologies, categorized into eight different themes. In each section, we provide a critical overview of the state-of-the-art, illustrations, current challenges, and possible future works;
• A detailed overview of recent potential applications of UAVs in remote sensing;
• A discussion of the future directions and challenges of UAV-RS from the point of view of platform and technology.

We hope this work will provide remote-sensing researchers an overall picture of recent UAV-based remote sensing developments and help guide further research on this topic.

Survey

1. Related Surveys

Year	Pub.	Title	Links
2021	AIR	Applications, databases and open computer vision research from drone videos and images: a survey Younes Akbari, Noor Almaadeed, Somaya Al-maadeed & Omar Elharrouss	Paper
2019	GRSM	Mini-Unmanned Aerial Vehicle-Based Remote Sensing: Techniques, Applications, and Prospects Tian-Zhu Xiang, Gui-Song Xia, Liangpei Zhang	Paper
2017	RS	Hyperspectral Imaging: A Review on UAV-Based Sensors, Data Processing, and Applications for Agriculture and Forestry Adão, Telmo, Jonáš Hruška, Luís Pádua, José Bessa, Emanuel Peres, Raul Morais, and Joaquim João Sousa	Paper
2017	IJRS	UAS, Sensors, and Data Processing in Agroforestry: A Review Towards Practical Applications Luís Pádua, Jakub Vanko, Jonáš Hruška, Telmo Adão, Joaquim J. Sousa ORCID Icon, Emanuel Peres & Raul Morais	Paper
2017	IJRS	Forestry Applications of UAVs in Europe: A Review Chiara Torresan, Andrea Berton, Federico Carotenuto, Salvatore Filippo Di Gennaro, Beniamino Gioli, Alessandro Matese, Franco Miglietta, Carolina Vagnoli, Alessandro Zaldei & Luke Wallace	Paper
2017	GRSM	Unmanned Aerial Vehicles and Spatial Thinking: Boarding Education With Geotechnology and Drones Arnau Fombuena	Paper
2017	RS	Review of the Current State of UAV Regulations Stöcker, Claudia, Rohan Bennett, Francesco Nex, Markus Gerke, and Jaap Zevenbergen	Paper
2017	IJRS	UAVs: Regulations and Law Enforcement Arthur P. Cracknell	Paper
2016	RSE	UAVs as Remote Sensing Platform in Glaciology: Present Applications and Future Prospects Anshuman Bhardwaj, Lydia Sam, Akanksha, F. Javier Martín-Torres, Rajesh Kumar	Paper
2015	PERS	Overview and Current Status of Remote Sensing Applications Based on Unmanned Aerial Vehicles (UAVs) Gonzalo Pajares	Paper
2015	JCR	Coastal and Environmental Remote Sensing from Unmanned Aerial Vehicles: An Overview Victor V. Klemas	Paper
2014	ISPRS JPRS	Unmanned Aerial Systems for Photogrammetry and Remote Sensing: A Review I. Colomina, P. Molina	Paper
2014	GISRS	Recent Applications of Unmanned Aerial Imagery in Natural Resource Management Mozhdeh Shahbazi, Jérôme Théau & Patrick Ménard	Paper
2012	RS	Unmanned Aircraft Systems in Remote Sensing and Scientific Research: Classification and Considerations of Use Watts, Adam C., Vincent G. Ambrosia, and Everett A. Hinkley	Paper
2011	GISRS	Small-Scale Unmanned Aerial Vehicles in Environmental Remote Sensing: Challenges and Opportunities Perry J. Hardin, Ryan R. Jensen	Paper
🚩	Geometry
2022	GRSM	Unmanned Aerial Vehicle-Based Photogrammetric 3D Mapping: A survey of techniques, applications, and challenges San Jiang; Wanshou Jiang; Lizhe Wang	Paper
2016	ISPRS JPRS	Recent Developments in Large-Scale Tie-Point Matching Wilfried Hartmann, Michal Havlena, Konrad Schindler	Paper
2014	PHOR	State of the Art in High Density Image Matching Fabio Remondino, Maria Grazia Spera, Erica Nocerino, Fabio Menna, Francesco Nex	Paper
2012	PHOR	Development and Status of Image Matching in Photogrammetry Armin Gruen	Paper
🚩	Semantic
2022	GRSM	Deep Learning for Unmanned Aerial Vehicle-Based Object Detection and Tracking: A survey Xin Wu; Wei Li; Danfeng Hong; Ran Tao; Qian Du	Paper

^{_{*This table shows only surveys published in top remote sensing journals.
*PERS: Photogrammetric Engineering and Remote Sensing; ISPRS JPRS: International Society for Photogrammetry and Remote Sensing Journal of Photogrammetry and Remote Sensing; RS: Remote Sensing; IJRS: International Journal of Remote Sensing; RSE: Remote Sensing of Environment; GISRS: GIScience & Remote Sensing; PHOR: The Photogrammetric Record; GRSM: IEEE Geoscience and Remote Sensing Magazine.}}

2. Camera Calibration

• Awesome-Deep-Camera-Calibration
• Awesome-Image-Distortion-Correction
• Awesome-Image-Rectification

Year	Pub.	Title	Links
2023	arXiv	Deep Learning for Camera Calibration and Beyond: A Survey Kang Liao, Lang Nie, Shujuan Huang, Chunyu Lin, Jing Zhang, Yao Zhao, Moncef Gabbouj, Dacheng Tao	Paper/Proj
2023	ICCV	Innovating Real Fisheye Image Correction with Dual Diffusion Architecture Shangrong Yang, Chunyu Lin, Kang Liao, Yao Zhao	Paper
2023	ICCV	DarSwin: Distortion Aware Radial Swin Transformer Akshaya Athwale, Arman Afrasiyabi, Justin Lagüe, Ichrak Shili, Ola Ahmad, Jean-François Lalonde	Paper/Code
2023	ICCV	Deep Geometry-Aware Camera Self-Calibration from Video Annika Hagemann, Moritz Knorr, Christoph Stiller	Paper/Code
2023	CVPR	Wide-Angle Rectification via Content-Aware Conformal Mapping Qi Zhang, Hongdong Li, Qing Wang	Paper
2023	TIP	Model-aware Pre-training for Radial Distortion Rectification Wendi Wang; Hao Feng; Wengang Zhou; Zhaokang Liao; Houqiang Li	Paper
2023	TIP	SIR: Self-Supervised Image Rectification via Seeing the Same Scene From Multiple Different Lenses Jinlong Fan; Jing Zhang; Dacheng Tao	Paper/Code
2023	TCSVT	DaFIR: Distortion-aware Representation Learning for Fisheye Image Rectification Zhaokang Liao; Wengang Zhou; Houqiang Li	Paper/Code
2022	ECCV	Rethinking Generic Camera Models for Deep Single Image Camera Calibration to Recover Rotation and Fisheye Distortion Nobuhiko Wakai, Satoshi Sato, Yasunori Ishii & Takayoshi Yamashita	Paper
2022	IJCV	Wide-Angle Image Rectification: A Survey Jinlong Fan, Jing Zhang, Stephen J. Maybank & Dacheng Tao	Paper
2022	TCSVT	Revisiting Radial Distortion Rectification in Polar-Coordinates: A New and Efficient Learning Perspective Keyao Zhao; Chunyu Lin; Kang Liao; Shangrong Yang; Yao Zhao	Paper
2022	RAL	Learning-Based Distortion Correction and Feature Detection for High Precision and Robust Camera Calibration Yesheng Zhang; Xu Zhao; Dahong Qian	Paper
2021	ICCV	Towards Complete Scene and Regular Shape for Distortion Rectification by Curve-Aware Extrapolation Kang Liao, Chunyu Lin, Yunchao Wei, Feng Li, Shangrong Yang, Yao Zhao	Paper
2021	ICCV	Multi-Level Curriculum for Training a Distortion-Aware Barrel Distortion Rectification Model Kang Liao, Chunyu Lin, Lixin Liao, Yao Zhao, Weiyao Lin	Paper
2021	CVPR	Progressively Complementary Network for Fisheye Image Rectification Using Appearance Flow Shangrong Yang, Chunyu Lin, Kang Liao, Chunjie Zhang, Yao Zhao	Paper/Code
2021	TIP	A Deep Ordinal Distortion Estimation Approach for Distortion Rectification Kang Liao; Chunyu Lin; Yao Zhao	Paper
2020	ICASSP	Self-Supervised Deep Learning for Fisheye Image Rectification Chun-Hao Chao; Pin-Lun Hsu; Hung-Yi Lee; Yu-Chiang Frank Wang	Paper
2019	CVPR	Learning Structure-And-Motion-Aware Rolling Shutter Correction Bingbing Zhuang, Quoc-HuyTran, PanJi, Loong-Fah Cheong, Manmohan Chandraker	Paper
2019	CVPR	Blind Geometric Distortion Correction on Images Through Deep Learning Xiaoyu Li, Bo Zhang, Pedro V. Sander, Jing Liao	Paper/Code
2019	CVPR	Learning to calibrate straight lines for fisheye image rectification Xue, Zhucun and Xue, Nan and Xia, Gui-Song and Shen, Weiming	Paper
2018	ECCV	Fisheyerecnet: A multi-context collaborative deep network for fisheye image rectification Yin, Xiaoqing and Wang, Xinchao and Yu, Jun and Zhang, Maojun and Fua, Pascal and Tao, Dacheng	Paper
2016	ACCV	Radial lens distortion correction using convolutional neural networks trained with synthesized images Rong, Jiangpeng and Huang, Shiyao and Shang, Zeyu and Ying, Xianghua	Paper
2015	CVPR	Line-based multi-label energy optimization for fisheye image rectification and calibration Zhang, Mi and Yao, Jian and Xia, Menghan and Li, Kai and Zhang, Yi and Liu, Yaping	Paper/Code
2015	ISPRS JPRS	Generating 3D hyperspectral information with lightweight UAV snapshot cameras for vegetation monitoring: From camera calibration to quality assurance Aasen, Helge and Burkart, Andreas and Bolten, Andreas and Bareth, Georg	Paper
2014	IPOL	Automatic Lens Distortion Correction Using One-Parameter Division Models Miguel Alemán-Flores, Luis Alvarez, Luis Gomez, Daniel Santana-Cedrés	Paper/Code
2013	IROS	AprilCal: Assisted and repeatable camera calibration Richardson, Andrew and Strom, Johannes and Olson, Edwin	Paper
2010	PAMI	Equidistant fish-eye calibration and rectification by vanishing point extraction Hughes, Ciaran and Denny, Patrick and Glavin, Martin and Jones, Edward	Paper
2008	CVPR	On benchmarking camera calibration and multi-view stereo for high resolution imagery Strecha, Christoph and Von Hansen, Wolfgang and Van Gool, Luc and Fua, Pascal and Thoennessen, Ulrich	Paper
2006	PAMI	A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses Kannala, Juho and Brandt, Sami S	Paper
2000	PAMI	A flexible new technique for camera calibration Zhang, Zhengyou	Paper

3. Combined field of view

• Combined field of view to expand coverage
• Aerial path planning for optimal area coverage

Year	Pub.	Title	Links
2023	JSTARS	A Review on Viewpoints and Path Planning for UAV-Based 3-D Reconstruction Mehdi Maboudi; MohammadReza Homaei; Soohwan Song; Shirin Malihi, et al	Paper
2022	Auton Robot	Cooperative multi-UAV coverage mission planning platform for remote sensing applications Savvas D. Apostolidis, Pavlos Ch. Kapoutsis, Athanasios Ch. Kapoutsis & Elias B. Kosmatopoulos	Paper/Code
2021	ToG	Continuous aerial path planning for 3D urban scene reconstruction Han Zhang, Yucong Yao, Ke Xie, Chi-Wing Fu, Hao Zhang, Hui Huang	Paper
2021	TRO	View Path Planning via Online Multiview Stereo for 3-D Modeling of Large-Scale Structures Soohwan Song; Daekyum Kim; Sunghee Choi	Paper
2020	ToG	Offsite aerial path planning for efficient urban scene reconstruction Xiaohui Zhou, Ke Xie, Kai Huang, Yilin Liu, Yang Zhou, Minglun Gong, Hui Huang	Paper
2018	ToG	Aerial Path Planning for Urban Scene Reconstruction: A Continuous Optimization Method and Benchmark Neil Smith, Nils Moehrle, Michael Goesele, Wolfgang Heidrich	Paper
2018	Sensors	Multi-camera imaging system for UAV photogrammetry Wierzbicki, Damian	Paper
2018	TSG	Multi-UAV Pre-Positioning and Routing for Power Network Damage Assessment Gino J. Lim; Seonjin Kim; Jaeyoung Cho; Yibin Gong; Amin Khodaei	Paper
2015	Sensors	Multi-UAV Routing for Area Coverage and Remote Sensing with Minimum Time Gustavo S. C. Avellar, Guilherme A. S. Pereira, Luciano C. A. Pimenta and Paulo Iscold	Paper
2013	IJARS	A solution to cooperative area coverage surveillance for a swarm of MAVs Zheng-Jie, Wang and Wei, Li	Paper
2012	ISPRSC	UAV borne low altitude photogrammetry system Z. Lin, G. Su, and F. Xie	Paper

4. Low-altitude UAV image matching

🔸 Multi-view image matching

• Awesome-Image-Matching
• Awesome-Computational-Photography

Year	Pub.	Title	Links
2022	ISPRS JPRS	Leveraging vocabulary tree for simultaneous match pair selection and guided feature matching of UAV images San Jiang, Wanshou Jiang, Bingxuan Guo	Paper
2022	AAAI	Guide Local Feature Matching by Overlap Estimation Ying Chen, Dihe Huang, et al	Paper/Code
2021	IJCV	Image Matching from Handcrafted to Deep Features: A Survey Jiayi Ma, Xingyu Jiang, Aoxiang Fan, Junjun Jiang & Junchi Yan	Paper
2020	IJCV	Image Matching across Wide Baselines: From Paper to Practice Yuhe Jin and Dmytro Mishkin and Anastasiia Mishchuk and Jiri Matas and Pascal Fua and Kwang Moo Yi and Eduard Trulls	Paper/Code
2019	ISPRS JPRS	LAM: Locality affine-invariant feature matching Jiayuan Li, Qingwu Hu, Mingyao Ai	Paper
2018	ECCV	Geodesc: Learning local descriptors by integrating geometry constraints Z. Luo, T. Shen, L. Zhou, S. Zhu, R. Zhang, Y. Yao, T. Fang, and L. Quan	Paper/Code
2018	JARS	Unmanned aerial vehicle oblique image registration using an ASIFT-based matching method Wang, Chengyi and Chen, Jingbo and Chen, Jiansheng and Yue, Anzhi and He, Dongxu and Huang, Qingqing and Zhang, Yi	Paper
2018	IJRS	Registrating oblique images by integrating affine and scale-invariant features Yu, Mei and Yang, Huachao and Deng, Kazhong and Yuan, Kai	Paper
2018	ISPRS JPRS	Hierarchical motion consistency constraint for efficient geometrical verification in UAV stereo image matching Jiang, San and Jiang, Wanshou	Paper
2017	ICCV	Progressive Large Scale-Invariant Image Matching in Scale Space Lei Zhou, Siyu Zhu, Tianwei Shen, Jinglu Wang, Tian Fang, Long Quan	Paper
2017	ICCV	Learning discriminative and transformation covariant local feature detectors X. Zhang, F. X. Yu, S. Karaman, and S.-F. Chang	Paper/Code
2017	ICCV	Learning spread-out local feature descriptors X. Zhang, F. X. Yu, S. Kumar, and S.-F. Chang	Paper/Code
2017	CVPR	HPatches: A benchmark and evaluation of handcrafted and learned local descriptors Balntas, Vassileios and Lenc, Karel and Vedaldi, Andrea and Mikolajczyk, Krystian	Paper/Code
2017	RS	On-board GNSS/IMU assisted feature extraction and matching for oblique UAV images S. Jiang and W. Jiang	Paper
2016	CVPR	Learning to match aerial images with deep attentive architectures H. Altwaijry, E. Trulls, J. Hays, P. Fua, and S. Belongie	Paper
2015	CVPR	TILDE: A Temporally Invariant Learned DEtector Yannick Verdie, Kwang Yi, Pascal Fua, Vincent Lepetit	Paper/Code
2015	PERS	Reliable spatial relationship constrained feature point matching of oblique aerial images Hu, Han and Zhu, Qing and Du, Zhiqiang and Zhang, Yeting and Ding, Yulin	Paper
2015	RS	A robust photogrammetric processing method of low-altitude UAV images Ai, Mingyao and Hu, Qingwu and Li, Jiayuan and Wang, Ming and Yuan, Hui and Wang, Shaohua	Paper
2014	ISPRS JPRS	L2-SIFT: SIFT feature extraction and matching for large images in large-scale aerial photogrammetry Sun, Yanbiao and Zhao, Liang and Huang, Shoudong and Yan, Lei and Dissanayake, Gamini	Paper
2012	PhotoRec	Development and status of image matching in photogrammetry Armin Gruen	Paper

🔸 Matching with non-UAV images

Year	Pub.	Title	Links
2022	RS	Multiview Image Matching of Optical Satellite and UAV Based on a Joint Description Neural Network Chuan Xu, Chang Liu, Hongli Li, Zhiwei Ye, Haigang Sui and Wei Yang	Paper
2021	RS	A Practical Cross-View Image Matching Method between UAV and Satellite for UAV-Based Geo-Localization Lirong Ding, Ji Zhou, Lingxuan Meng and Zhiyong Long	Paper
2017	RS	Automatic UAV image geo-registration by matching UAV images to georeferenced image data Xiangyu Zhuo, Tobias Koch, Franz Kurz, Friedrich Fraundorfer and Peter Reinartz	Paper
2016	ISPRS JPRS	Illumination-invariant image matching for autonomous UAV localisation based on optical sensing Xue Wan, Jianguo Liu, Hongshi Yan, Gareth L.K. Morgan	Paper
🚩	G2A
2023	AAAI	Cross-View Geo-Localization via Learning Disentangled Geometric Layout Correspondence Xiaohan Zhang, Xingyu Li, Waqas Sultani, Yi Zhou, Safwan Wshah	Paper
2019	ICCV	Bridging the Domain Gap for Ground-to-Aerial Image Matching Krishna Regmi, Mubarak Shah	Paper/Code
2019	ICCV	Ground-to-Aerial Image Geo-Localization With a Hard Exemplar Reweighting Triplet Loss Sudong Cai, Yulan Guo, Salman Khan, Jiwei Hu, Gongjian Wen	Paper
2018	CVPR	CVM-net: Cross-view matching network for image-based ground-to-aerial geo-localization Sixing Hu, Mengdan Feng, Rang MH Nguyen, and Gim Hee Lee	Paper/Code
2016	CVPR	Regularity-driven building facade matching between aerial and street views M. Wolff, R. T. Collins, and Y. Liu	Paper
2014	3DV	Accurate geo-registration by ground-to-aerial image matching S. Qi, C. Wu, B. Curless, Y. Furukawa, C. Hernandez, and S. M. Seitz	Paper

5. Low-altitude automatic aerial triangulation

🔸 Structure from motion

Year	Pub.	Title	Links
2020	ECCV	DeepSFM: Structure From Motion Via Deep Bundle Adjustment Xingkui Wei, Yinda Zhang, Zhuwen Li, Yanwei Fu, Xiangyang Xue	Paper/Code
2020	ISPRS JPRS	Efficient structure from motion for large-scale UAV images: A review and a comparison of SfM tools San Jiang, Cheng Jiang, Wanshou Jiang	Paper
2019	CFR	Structure from Motion Photogrammetry in Forestry: a Review Jakob Iglhaut, Carlos Cabo, Stefano Puliti, Livia Piermattei, James O’Connor & Jacqueline Rosette	Paper
2017	ICCV	Parallel Structure from Motion from Local Increment to Global Averaging Siyu Zhu, Tianwei Shen, Lei Zhou, Runze Zhang, Jinglu Wang, Tian Fang, Long Quan	Paper
2017	ICCV	Distributed Very Large Scale Bundle Adjustment by Global Camera Consensus Runze Zhang, Siyu Zhu, Tian Fang, Long Quan	Paper
2017	CVPR	HSfM: Hybrid Structure-from-Motion Hainan Cui, Xiang Gao, Shuhan Shen, Zhanyi Hu	Paper/Code
2017	CVPR	Unsupervised learning of depth and ego-motion from video Tinghui Zhou, Matthew Brown, Noah Snavely, David G. Lowe	Paper/Code
2017	ActaNum	A survey of structure from motion Onur Özyeşil, Vladislav Voroninski, Ronen Basri and Amit Singer	Paper
2016	CVPR	Structure-from-Motion Revisited Schonberger, Johannes Lutz and Frahm, Jan-Michael	Paper/Code
2015	ICCV	PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization Alex Kendall, Matthew Grimes, Roberto Cipolla	Paper/Code
2014	ECCV	Robust global translations with 1DSFM Kyle Wilson & Noah Snavely	Paper/Code
2012	CVPR	Semantic structure from motion with points, regions, and objects Sid Yingze Bao; Mohit Bagra; Yu-Wei Chao; Silvio Savarese	Paper
2012	Geom	‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications M.J. Westoby, J. Brasington, N.F. Glasser, M.J. Hambrey, J.M. Reynolds	Paper
2011	PhotoRec	Photogrammetric processing of low-altitude images acquired by unpiloted aerial vehicles Yongjun Zhang, Jinxin Xiong, Lijuan Hao	Paper

🔸 Rolling-shutter SfM

Year	Pub.	Title	Links
2023	CVPR	Revisiting Rolling Shutter Bundle Adjustment: Toward Accurate and Fast Solution Bangyan Liao, Delin Qu, Yifei Xue, Huiqing Zhang, Yizhen Lao	Paper/Code
2023	RS	Experimental Tests and Simulations on Correction Models for the Rolling Shutter Effect in UAV Photogrammetry Nazarena Bruno and Gianfranco Forlani	Paper
2021	IJCV	Solving Rolling Shutter 3D Vision Problems using Analogies with Non-rigidity Yizhen Lao, Omar Ait-Aider & Adrien Bartoli	Paper
2019	PAMI	Accurate 3D Reconstruction from Small Motion Clip for Rolling Shutter Cameras Im, Sunghoon and Ha, Hyowon and Choe, Gyeongmin and Jeon, Hae-Gon and Joo, Kyungdon and Kweon, In So	Paper
2019	CVPR	Learning Structure-And-Motion-Aware Rolling Shutter Correction Bingbing Zhuang, Quoc-Huy Tran, Pan Ji, Loong-Fah Cheong, Manmohan Chandraker	Paper
2018	PRL	Robustified Structure from Motion with rolling-shutter camera using straightness constraint Yizhen Lao, Omar Ait-Aider, Helder Araujo	Paper
2017	CVPR	Self-Calibration-Based Approach to Critical Motion Sequences of Rolling-Shutter Structure From Motion Eisuke Ito, Takayuki Okatani	Paper
2017	CVPR	Unrolling the shutter: CNN to correct motion distortions Vijay Rengarajan, Yogesh Balaji, A. N. Rajagopalan	Paper
2016	CVPR	From bows to arrows: Rolling shutter rectification of urban scenes Vijay Rengarajan, Ambasamudram N. Rajagopalan, Rangarajan Aravind	Paper
2015	ICCV	High quality structure from small motion for rolling shutter cameras Sunghoon Im, Hyowon Ha, Gyeongmin Choe, Hae-Gon Jeon, Kyungdon Joo, In So Kweon	Paper
2013	ICCV	Street view motion-from structure-from-motion Bryan Klingner, David Martin, James Roseborough	Paper
2013	CVPR	Rolling Shutter Camera Calibration Luc Oth, Paul Furgale, Laurent Kneip, Roland Siegwart	Paper
2012	CVPR	Rolling shutter bundle adjustment Johan Hedborg; Per-Erik Forssén; Michael Felsberg; Erik Ringaby	Paper

🔸 SfM Toolkit

Name	Year	Pub
OpenSfM		Homepage
ColMap	2016	CVPR
OpenMVG	2016
TheiaSfM	2015
MVE	2014	Proceedings
VisualSFM	2011
Bundler	2006
Ceres Solver	2023	Doc
Regard3D	2019

6. Dense reconstruction

• Awesome MVS

Year	Pub.	Title	Links
2023	ISPRS JPRS	Deep learning based multi-view stereo matching and 3D scene reconstruction from oblique aerial images Jin Liu, Jian Gao, Shunping Ji, Chang Zeng, Shaoyi Zhang, JianYa Gong	Paper/Code
2023	ISPRS JPRS	Multiple View Stereo with quadtree-guided priors Elisavet Konstantina Stathopoulou, Roberto Battisti, Dan Cernea, Andreas Georgopoulos, Fabio Remondino	Paper/Code
2023	TGRS	Select-and-Combine (SAC): A Novel Multi-stereo Depth Fusion Algorithm for Point Cloud Generation via Efficient Local Markov Netlets Mostafa Elhashash; Rongjun Qin	Paper
2023	TGRS	A Hierarchical Deformable Deep Neural Network and an Aerial Image Benchmark Dataset for Surface Multiview Stereo Reconstruction Jiayi Li; Xin Huang; Yujin Feng; Zhen Ji; Shulei Zhang; Dawei Wen	Paper
2022	ECCV	Multiview Stereo with Cascaded Epipolar RAFT Ma, Zeyu and Teed, Zachary and Deng, Jia	Paper/Code
2022	ECCV	MVSTER: Epipolar Transformer for Efficient Multi-View Stereo Xiaofeng Wang, Zheng Zhu, Fangbo Qin, Yun Ye, Guan Huang, Xu Chi, Yijia He, Xingang Wang	Paper/Code
2022	PAMI	A Survey on Deep Learning Techniques for Stereo-Based Depth Estimation H Laga, LV Jospin, F Boussaid, M Bennamoun	Paper
2021	CVPR	PatchmatchNet: Learned Multi-View Patchmatch Stereo Fangjinhua Wang and Silvano Galliani and Christoph Vogel and Pablo Speciale and Marc Pollefeys	Paper/Code
2021	PAMI	Visibility-Aware Point-Based Multi-View Stereo Network Chen, Rui and Han, Songfang and Xu, Jing and Su, Hao	Paper/Code
2021	ISPRS JPRS	Attention aware cost volume pyramid based multi-view stereo network for 3D reconstruction Anzhu Yu, Wenyue Guo, Bing Liu, Xin Chen, Xin Wang, Xuefeng Cao, Bingchuan Jiang	Paper/Code
2021	ISPRS JPRS	Automatic 3D building reconstruction from multi-view aerial images with deep learning Dawen Yu, Shunping Ji, Jin Liu, Shiqing Wei	Paper/Code
2021	ISPRS JPRS	IMGTR: Image-triangle based multi-view 3D reconstruction for urban scenes Zhihua Hu, Yaolin Hou, Pengjie Tao, Jie Shan	Paper
2021	Displays	Multi-view stereo in the Deep Learning Era: A comprehensive review Xiang Wang, Chen Wang, Bing Liu, Xiaoqing Zhou, Liang Zhang, Jin Zheng, Xiao Bai	Paper
2020	CVPR	BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks Yao, Yao and Luo, Zixin and Li, Shiwei and Zhang, Jingyang and Ren, Yufan and Zhou, Lei and Fang, Tian and Quan, Long	Paper/Code
2020	CVPR	Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching Xiaodong Gu, Zhiwen Fan, Zuozhuo Dai, Siyu Zhu, Feitong Tan, Ping Tan	Paper/Code
2020	CVPR	Cost Volume Pyramid Based Depth Inference for Multi-View Stereo Yang, Jiayu and Mao, Wei and Alvarez, Jose M. and Liu, Miaomiao	Paper/Code
2020	ISPRS JPRS	Pruning multi-view stereo net for efficient 3D reconstruction Xiang Xiang, Zhiyuan Wang, Shanshan Lao, Baochang Zhang	Paper
2019	ICCV	Point-based Multi-view Stereo Network Chen, Rui and Han, Songfang and Xu, Jing and Su, Hao	Paper/Code
2019	CVPR	GA-Net: Guided Aggregation Net for End-To-End Stereo Matching Feihu Zhang, Victor Prisacariu, Ruigang Yang, Philip H.S. Torr	Paper/Code
2019	CVPR	Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference Yao, Yao and Luo, Zixin and Li, Shiwei and Shen, Tianwei and Fang, Tian and Quan, Long	Paper/Code
2018	ECCV	Specular-to-diffuse translation for multi-view reconstruction Shihao Wu, Hui Huang, Tiziano Portenier, Matan Sela, Daniel Cohen-Or, Ron Kimmel, Matthias Zwicker	Paper
2018	ECCV	MVSNet: Depth inference for unstructured multi-view stereo Yao Yao, Zixin Luo, Shiwei Li, Tian Fang, Long Quan	Paper/Code
2018	CVPR	DeepMVS: Learning Multi-View Stereopsis Po-Han Huang, Kevin Matzen, Johannes Kopf, Narendra Ahuja, Jia-Bin Huang	Paper/Code
2017	ICCV	Monocular Dense 3D Reconstruction of a Complex Dynamic Scene From Two Perspective Frames Suryansh Kumar, Yuchao Dai, Hongdong Li	Paper
2017	ICCV	Surfacenet: An end-to-end 3d neural network for multiview stereopsis Mengqi Ji, Juergen Gall, Haitian Zheng, Yebin Liu, Lu Fang	Paper/Code
2017	ICCV	End-To-End Learning of Geometry and Context for Deep Stereo Regression Alex Kendall, Hayk Martirosyan, Saumitro Dasgupta, Peter Henry, Ryan Kennedy, Abraham Bachrach, Adam Bry	Paper/Code
2017	CVPR	SGM-Nets: Semi-Global Matching With Neural Networks Akihito Seki, Marc Pollefeys	Paper
2017	NeurIPS	Learning a Multi-View Stereo Machine Abhishek Kar, Christian Häne, Jitendra Malik	Paper/Code
2016	JMLR	Stereo matching by training a convolutional neural network to compare image patches Zbontar, Jure and LeCun, Yann	Paper/Code
2016	TGRS	Hierarchical and adaptive phase correlation for precise disparity estimation of UAV images Jie Li; Yiguang Liu; Shuangli Du; Pengfei Wu; Zhenyu Xu	Paper
2016	RS	Multi-view stereo matching based on self-adaptive patch and image grouping for multiple unmanned aerial vehicle imagery Xiao, Xiongwu and Guo, Bingxuan and Li, Deren and Li, Linhui and Yang, Nan and Liu, Jianchen and Zhang, Peng and Peng, Zhe	Paper
2015	CGV	Multi-view stereo: A tutorial Yasutaka Furukawa and Carlos Hernández	Paper
2015	ICCV	Massively Parallel Multiview Stereopsis by Surface Normal Diffusion Silvano Galliani, Katrin Lasinger, Konrad Schindler	Paper
2014	PhotoRec	State of the art in high density image matching Fabio Remondino, Maria Grazia Spera, Erica Nocerino, Fabio Menna, Francesco Nex	Paper
2014	AppGeo	UAV for 3D mapping applications: a review Francesco Nex & Fabio Remondino	Paper
2012	RS	Assessing the accuracy of georeferenced point clouds produced via multi-view stereopsis from unmanned aerial vehicle (UAV) imagery Steve Harwin and Arko Lucieer	Paper

🔸 MVS Toolkit

Name	Year	Pub
Meshroom	2021	MMSys
Colmap	2016	CVPR
MicMac	--
MVE	2014	Paper
MVS-Texturing	2014	ECCV
CMVS-PMVS	2010	PAMI/Homepage
S2P - Satellite Stereo Pipeline	2014
SAT-PP
Danesfield	2019	CVPRW

7. Image stitching

• Awesome-Computational-Photography

Year	Pub.	Title	Links
2022	GRSL	Efficient Seamline Network Generation for Large-Scale Orthoimage Mosaicking Junxing Yang; Lulu Liu; Qingsong Yan; Fei Deng	Paper
2021	TGRS	Superpixel-Based Seamless Image Stitching for UAV Images Yiting Yuan; Faming Fang; Guixu Zhang	Paper
2021	JSTARS	An Efficient Method for Generating UAV-Based Hyperspectral Mosaics Using Push-Broom Sensors Juan M. Jurado; Luís Pádua; Jonas Hruška; Francisco R. Feito; et al	Paper
2020	ISPRS JPRS	Jointly optimizing global and local color consistency for multiple image mosaicking Li Li, Menghan Xi, Chi Liu, Liang Li, Hanyun Wang, Jian Yao	Paper
2020	JSTARS	UAV Image Mosaicking Based on Multiregion Guided Local Projection Deformation Quan Xu; Jun Chen; Linbo Luo; Wenping Gong; Yong Wang	Paper
2019	GRSM	Remote Sensing Image Mosaicking: Achievements and Challenges Xinghua Li; Ruitao Feng; Xiaobin Guan; Huanfeng Shen; Liangpei Zhang	Paper
2018	ECCV	Object-centered image stitching Charles Herrmann, Ramin Zabih, et al	Paper/Code
2018	ECCV	Robust image stitching with multiple registrations Herrmann, Charles, Zabih, Ramin, et al	Paper/Code
2018	TIP	Dynamic Video Stitching via Shakiness Removing Yongwei Nie, et al	Paper/Code
2018	RAL	Unsupervised Deep Homography: A Fast and Robust Homography Estimation Model Ty Nguyen; Steven W. Chen; Shreyas S. Shivakumar; Camillo Jose Taylor; Vijay Kumar	Paper/Code
2018	PR	Image stitching by line-guided local warping with global similarity constraint Tian-Zhu Xiang, Gui-Song Xia, Xiang Bai, Liangpei Zhang	Paper
2018	TMM	Parallax-Tolerant Image Stitching Based on Robust Elastic Warping Jing Li; Zhengming Wang; Shiming Lai; Yongping Zhai; Maojun Zhang	Paper/Code
2018	TMM	Quasi-Homography Warps in Image Stitching Nan Li, Yifang Xu, Chao Wang	Paper/Code
2018	IJRS	Mosaicking UAV orthoimages using bounded Voronoi diagrams and watersheds Mengxiao Song, Zheng Ji, Shan Huang &Jing Fu	Paper
2018	IJRS	Seam-line determination via minimal connected area searching and minimum spanning tree for UAV image mosaicking Yumin Tian,Aifeng Sun &Di Wang	Paper
2017	CVPR	Direct Photometric Alignment by Mesh Deformation Kaimo Lin, Shuaicheng Liu, et al	Paper/Code
2017	PR	Globally consistent alignment for planar mosaicking via topology analysis Menghan Xia, Jian Yao, Renping Xie, Li Li, Wei Zhang	Paper
2017	AutCon	UAV-based automatic generation of high-resolution panorama at a construction site with a focus on preprocessing for image stitching Seongdeok Bang, Hongjo Kim, Hyoungkwan Kim	Paper
2016	ECCV	Natural Image Stitching with the Global Similarity Prior Yu-Sheng Chen & Yung-Yu Chuang	Paper/Code
2016	ECCV	SEAGULL: Seam-Guided Local Alignment for Parallax-Tolerant Image Stitching Kaimo Lin, Nianjuan Jiang, Loong-Fah Cheong, Minh Do & Jiangbo Lu	Paper
2016	TIP	Multi-viewpoint panorama construction with wide-baseline images Guofeng Zhang; Yi He; Weifeng Chen; Jiaya Jia; Hujun Bao	Paper
2016	RS	Mosaicking of Unmanned Aerial Vehicle Imagery in the Absence of Camera Poses Yuhua Xu, Jianliang Ou, Hu He, Xiaohu Zhang and Jon Mills	Paper/Code
2016	RSS-W	Deep Image Homography Estimation Daniel DeTone, et al	Paper/Code
2016	JARS	Computer vision-based orthorectification and georeferencing of aerial image sets Mohammad Reza Faraji, Xiaojun Qi, Austin Jensen	Paper
2016	Sensors	An Efficient Seam Elimination Method for UAV Images Based on Wallis Dodging and Gaussian Distance Weight Enhancement Jinyan Tian, Xiaojuan Li, Fuzhou Duan, Junqian Wang and Yang Ou	Paper
2016	ICPR	Locally warping-based image stitching by imposing line constraints T.-Z. Xiang, G.-S. Xia, L. Zhang, and N. Huang	Paper
2015	ICCV	Dual-Feature Warping-Based Motion Model Estimation Shiwei Li, Lu Yuan, Jian Sun, Long Quan	Paper/Code
2015	CVPR	Adaptive As-Natural-As-Possible Image Stitching Chung-Ching Lin, S. U. Pankanti, K. N. Ramamurthy, and Aleksandr Y. Aravkin	Paper/Code
2015	ISPRS JPRS	A robust mosaicking procedure for high spatial resolution remote sensing images Xinghua Li, Nian Hui, Huanfeng Shen, Yunjie Fu, Liangpei Zhang	Paper
2015	CluCom	A review of parallel computing for large-scale remote sensing image mosaicking Lajiao Chen, Yan Ma, Peng Liu, Jingbo Wei, Wei Jie & Jijun He	Paper
2014	CVPR	Shape-preserving half-projective warps for image stitching Che-Han Chang, Yoichi Sato, Yung-Yu Chuang	Paper/Code
2014	CVPR	Parallax-tolerant Image Stitching Fan Zhang and Feng Liu	Paper
2014	PAMI	As-Projective-As-Possible Image Stitching with Moving DLT Julio Zaragoza; Tat-Jun Chin; Quoc-Huy Tran; Michael S. Brown; David Suter	Paper/Code/ Code
2013	EG	Seam-Driven Image Stitching Junhong Gao, Yu Li, Tat-Jun Chin, Michael S. Brown	Paper/Code
2011	CVPR	Smoothly varying affine stitching Wen-Yan Lin, Siying Liu, Y Matsushita, Tian-Tsong Ng, Loong-Fah Cheong	Paper/Code
2011	CVPR	Constructing image panoramas using dual-homography warping Junhong Gao, Seon Joo Kim, Michael S. Brown	Paper/Code
2007	IJCV	Automatic panoramic image stitching using invariant features Matthew Brown and David G. Lowe	Paper/Demo
2007	CGV	Image Alignment and Stitching: A Tutorial Rick Szeliski	Paper

🔸 Stitching Toolkit

• Image composition editor (ICE)
• Autostitch

8. Multi-sensor data registration

🔸 Multi-band image registration

Year	Pub.	Title	Links
2024	InfFus	POS-GIFT: A Geometric and Intensity-Invariant Feature Transformation for Multimodal Images Zhuolu Hou, Yuxuan Liu, Li Zhang	Paper/Code
2023	ISPRS JPRS	Multimodal image matching: A scale-invariant algorithm and an open dataset Jiayuan Li, Qingwu Hu, Yongjun Zhang	Paper/Code
2023	InfFus	Shape-Former: Bridging CNN and Transformer via ShapeConv for multimodal image matching Jiaxuan Chen, Xiaoxian Chen, Shuang Chen, Yuyan Liu, et al	Paper
2023	JMASS	Advances and Challenges in Multimodal Remote Sensing Image Registration Bai Zhu; Liang Zhou; Simiao Pu; Jianwei Fan; Yuanxin Ye	Paper
2022	ISPRS JPRS	A robust multimodal remote sensing image registration method and system using steerable filters with first- and second-order gradients Yuanxin Ye, Bai Zhu, Tengfeng Tang, Chao Yang, Qizhi Xu, Guo Zhang	Paper/Code
2022	ISPRS JPRS	A robust registration method for UAV thermal infrared and visible images taken by dual-cameras Lingxuan Meng, Ji Zhou, Shaomin Liu, Ziwei Wang, et al	Paper/Code
2022	TGRS	A Multiscale Framework With Unsupervised Learning for Remote Sensing Image Registration Yuanxin Ye; Tengfeng Tang; Bai Zhu; Chao Yang; Bo Li; et al	Paper/Code
2022	TGRS	LNIFT: Locally Normalized Image for Rotation Invariant Multimodal Feature Matching Jiayuan Li; Wangyi Xu; Pengcheng Shi; Yongjun Zhang; Qingwu Hu	Paper/Code
2021	JSTARS	Self-Similarity Features for Multimodal Remote Sensing Image Matching Xin Xiong; Guowang Jin; Qing Xu; Hongmin Zhang	Paper
2021	GRSM	Advances and Opportunities in Remote Sensing Image Geometric Registration: A systematic review of state-of-the-art approaches and future research directions Ruitao Feng; Huanfeng Shen; Jianjun Bai; Xinghua Li	Paper
2021	RS	Multimodal Remote Sensing Image Registration Methods and Advancements: A Survey Xinyue Zhang, Chengcai Leng, Yameng Hong, Zhao Pei, Irene Cheng and Anup Basu	Paper
2021	IJRS	A comprehensive review on remote sensing image registration Sourabh Paul &Umesh C. Pati	Paper
2021	InfFus	A review of multimodal image matching: Methods and applications Xingyu Jiang, Jiayi Ma, Guobao Xiao, Zhenfeng Shao, Xiaojie Guo	Paper
2019	TIP	RIFT: Multi-Modal Image Matching Based on Radiation-Variation Insensitive Feature Transform Jiayuan Li; Qingwu Hu; Mingyao Ai	Paper/Code
2019	ISPRS JPRS	Robust registration for remote sensing images by combining and localizing feature- and area-based methods Ruitao Feng, Qingyun Du, Xinghua Li, Huanfeng Shen	Paper
2019	TGRS	Fast and Robust Matching for Multimodal Remote Sensing Image Registration Yuanxin Ye; Lorenzo Bruzzone; Jie Shan; Francesca Bovolo, Qing Zhu	Paper/Code
2019	JSTARS	Registration of Multimodal Remote Sensing Image Based on Deep Fully Convolutional Neural Network Han Zhang; Weiping Ni; Weidong Yan; Deliang Xiang, et al	Paper
2019	PRL	Robust visible infrared image matching by exploiting dominant edge orientations H. Chen, N. Xue, Y. Zhang, Q. Lu, and G.-S. Xia	Paper
2018	TGRS	Robust registration of multimodal remote sensing images based on structural similarity Yuanxin Ye; Jie Shan; Lorenzo Bruzzone; Li Shen	Paper/Code
2015	ISPRS JPRS	A fast and mobile system for registration of low-altitude visual and thermal aerial images using multiple small-scale UAVs S. Yahyanejad and B. Rinner	Paper
2013	PRL	Visible and infrared image registration in man-made environments employing hybrid visual features Jungong Han, Eric J. Pauwels, Paul de Zeeuw	Paper
2008	PR	Multi-sensor image registration based on intensity and edge orientation information Yong Sun Kim, Jae Hak Lee, Jong Beom Ra	Paper

🔸 Registration of LiDAR and optical images

Year	Pub.	Title	Links
2021	ISPRS JPRS	Robust registration of aerial images and LiDAR data using spatial constraints and Gabor structural features Bai Zhu, Yuanxin Ye, Liang Zhou, Zhilin Li, Gaofei Yin	Paper
2019	ISPRS JPRS	NRLI-UAV: Non-rigid registration of sequential raw laser scans and images for low-cost UAV LiDAR point cloud quality improvement Jianping Li, Bisheng Yang, Chi Chen, Ayman Habib	Paper
2019	ISPRS JPRS	Marker-free coregistration of UAV and backpack LiDAR point clouds in forested areas Przemyslaw Polewski a, Wei Yao a, Lin Cao b, Sha Gao	Paper
2016	RS	A linear feature-based approach for the registration of unmanned aerial vehicle remotely-sensed images and airborne LiDAR data S. Liu, X. Tong, J. Chen, X. Liu, W. Sun, H. Xie, P. Chen, Y. Jin, and Z. Ye	Paper
2015	ISPRS JPRS	Automatic registration of UAV-borne sequent images and LiDAR data Bisheng Yang and Chi Chen	Paper
2015	ISPRS JPRS	Automatic registration of optical aerial imagery to a LiDAR point cloud for generation of city models Bernard O. Abayowa, Alper Yilmaz, Russell C. Hardie	Paper

Dataset and Tools

🔸 Tools

• awesome-photogrammetry-projects

Name	Year	Links	Comments
OpenDroneMap	--	Github	An open source command line toolkit for processing aerial drone imagery to generate maps, point clouds, 3D models and DEMs
VisWebDrone	2020	IJGI	A web application for photogrammetric processing
TeleSculptor	--	--	A cross-platform desktop application for aerial photogrammetry
CloudCompare	--	--	3D point cloud and mesh processing software

🔸 Dataset

Name	Year	Comments
SUIRD	2019	Small UAV Image Registration Dataset (SUIRD)
Urban Drone Dataset (UDD)	2018	Structure from Motion of Aerial Images (PRCV)
UAV image mosaicking dataset	2016	Dataset for Unmanned Aerial Vehicle (UAV) Image Mosaicking (RS)
ISPRS/EuroSDR Benchmark for Multi-Platform Photogrammetry	2015	For dense Image Matching and Image Orientation
Benchmark on High Density Aerial Image Matching	2015	For dense image matching
🚩 Semantic
MAVREC	2024	Ground and aerial-view dataset for multiview aerial visual recognition (object detection) (Paper)
InsPLAD	2023	Inspection of Power Line Assets: the Dataset (InsPLAD) (Paper)
HIT-UAV	2023	A high-altitude UAV-based infrared thermal dataset for object detection (Paper/Homepage)
UAV-ROD	2022	Car detection (rotating object detection) dataset for UAV imagery (NeuroCom)
VisDrone	2021	Object detection and tracking from UAV data (PAMI)
UAV-Human	2021	Human Behavior Understanding with UAVs (CVPR)
DAC-SDC	2021	Lower power object detection challenge (PAMI)
UAVid	2020	Semantic segmentation dataset for UAV imagery
UAVDT	2018	Object detection, single object tracking, and multiple object tracking (ECCV)
Okutama-Action	2017	Concurrent Human Action Detection (CVPRW)
UAV123	2016	Object tracking dataset (ECCV)

Citation

Please cite our paper if you find the work useful:

@article{xiang2019mini,
  title={Mini-Unmanned Aerial Vehicle-Based Remote Sensing: Techniques, Applications, and Prospects},
  author={Tian-Zhu Xiang, Gui-Song Xia, Liangpei Zhang},
  journal={IEEE Geoscience and Remote Sensing Magazine},
  pages={29 - 63},
  volume={7},
  number={3},
  doi={10.1109/MGRS.2019.2918840},  
  year={2019}
}