Hello everyone,
In this final article, I’d like to introduce the role of deep learning in remote sensing.
In the previous articles, we explored the foundations of remote sensing — a technology grounded in electromagnetic (EM) measurements. By analyzing reflection spectra, we can obtain valuable information about the Earth’s surface, such as the type of objects present and their condition. Remote sensing is essentially large-scale EM measurement, extending laboratory-scale physics to a geographic scale.
We also discussed the basics of machine learning and deep learning. Convolutional Neural Network (CNN), for instance, is designed to obtain highly abstract features in images and has already been applied in various domains, including autonomous driving, medical imaging, and surveillance systems, showing a broad range of impact.
However, each application domain comes with its own challenges, and remote sensing is no exception. In this article, I will introduce some representative applications of deep learning in remote sensing and discuss the unique difficulties and open challenges that arise in this field.
Application of deep learning in remote sensing
As I mentioned in the previous article, convolutional neural networks (CNNs) are inspired by the way humans process visual information. By stacking multiple layers, CNNs can automatically extract features from images step by step, eventually identifying what objects are present in the scene.
In remote sensing, this becomes particularly powerful. Satellites and aerial platforms capture large-scale images of the Earth’s surface from high altitudes. Deep learning models can then classify the categories of land cover, detect specific objects, or estimate their number and spatial extent. In other words, AI plays a crucial role in the image analysis stage, turning raw satellite images into meaningful information.
What makes this even more remarkable is the processing speed. With GPU acceleration and modern computational infrastructure, AI systems can analyze massive datasets much faster than humans ever could.
With this background in mind, let’s now explore some concrete applications of deep learning in remote sensing.
1) Land Cover Classification
One of the most representative applications of deep learning in remote sensing is land cover classification.
A widely used approach here is transfer learning. In this method, networks that have been pre-trained on general-purpose datasets such as ImageNet (e.g., ResNet or Inception-v3) are fine-tuned on remote sensing datasets, enabling classification into about 20–30 land cover categories.
A well-known example is the UC Merced Land Use dataset, where deep learning models have achieved over 90% accuracy in distinguishing categories such as residential areas, forests, agricultural land, and industrial zones.
This technology is expected to play an important role in urban planning, environmental monitoring, and resource management, as it enables large-scale and automated land use classification.
In more advanced studies, researchers go beyond RGB imagery by utilizing multispectral data from Landsat or leveraging time-series data from the Sentinel satellites. The latter allows the incorporation of temporal continuity, making it possible to detect land cover changes through change detection techniques.
2) Ship and Aircraft Detection
Another important application of deep learning in remote sensing is object detection, particularly for ships and aircraft.
In maritime surveillance, for example, deep learning models are used to automatically locate ships within vast ocean images captured by satellites. This task, however, comes with several challenges; 1) False detections: non-ship objects such as cloud shadows, waves, or coastal structures can sometimes be misclassified as ships, 2)Orientation variability: ships can appear at arbitrary angles, and simple bounding-box annotations may not capture their shapes well, leading to inaccurate counts or localization errors.
Recent advances, especially in the YOLO family of models (e.g., YOLOv8), have shown remarkable performance. For instance, using the Airbus Ship Detection dataset, YOLOv8 has achieved both high speed and high accuracy, making it highly promising for real-world applications. Moreover, research has extended toward detecting diverse ship types in both coastal and offshore regions, with applications expected across civilian and military domains.
These object detection techniques are also applied to aircraft monitoring. Studies targeting airports and airfields have demonstrated that deep learning models can successfully detect aircraft from high-resolution satellite images. Comparative research shows that algorithms such as YOLO, Faster R-CNN, and DETR each offer different trade-offs between precision, recall, and inference speed, highlighting active progress in this field.
3) Forest Monitoring
High-resolution imagery from satellites and aircraft can be combined with deep learning to monitor vegetation conditions. For example, it is possible to study how forest cover changes across seasons or years, or to measure how much forest area has been lost over time.
A common method for this is segmentation, which classifies images at the pixel level and enables the automatic identification of classes such as forests, farmland, or wetlands. In my own research, I focused on mangrove forests, using very high-resolution WorldView satellite imagery to build a model that distinguishes mangrove areas from surrounding vegetation. This makes it possible to track the condition of mangroves, detect annual changes, and capture variations in the surrounding environment — information that can be valuable for ecosystem conservation and for supporting public agencies.
That said, this field still faces a number of challenges, such as multi-scale analysis, refinement of boundary regions, and dealing with imperfect annotations. Continued advances in these areas are expected to further improve the technology.
Challenges of Applying Deep Learning
So far, we have looked at several applications of deep learning in remote sensing. At first glance, deep learning appears highly useful: it enables automation, fast image analysis, and — by combining high-dimensional spatial and spectral information — it can achieve accuracy levels approaching human visual interpretation.
However, there are still many challenges when applying deep learning to remote sensing:
- Black-box nature
Deep learning models have millions of parameters, and it is often unclear what criteria they use to make decisions. This makes interpretation difficult, and it is not always obvious when users should rely on AI versus when expert judgment is necessary. - Mismatch with human intuition
Since deep learning relies on numerical optimization, its results can sometimes diverge significantly from what humans would intuitively expect. - Data dependence and domain shift
The performance of AI depends heavily on the data it is trained on. A model may achieve high accuracy on training and evaluation datasets, but perform poorly on new data collected under different seasons, regions, or sensors — a phenomenon known as domain shift. - Difficulty of improvement
When errors occur, tracing the cause and improving the model is not straightforward. The complexity of the optimization process makes it difficult to identify exactly why the model produced a certain output.
In short, while deep learning offers great potential for remote sensing, interpretability, reliability, and generalization remain open challenges. Addressing these issues will be crucial for making the technology more widely applicable in scientific and practical contexts.
The Future of Deep Learning in Remote Sensing
So how should we address these challenges?
At the heart of the issue lies a cultural mismatch between remote sensing and deep learning.
Remote sensing is a science-based technology, grounded in electromagnetic analysis and underpinned by physical principles. Reflectance properties and other engineering characteristics allow us to interpret real-world phenomena in a way that is firmly tied to scientific reasoning.
Deep learning, on the other hand, has followed a very different path. Its development was not about “understanding” in the human sense, but about optimizing for performance — even without full interpretability. By embracing enormous parameter spaces and refining optimization techniques, researchers created powerful models that excel at tasks, even if the exact reasoning behind their decisions remains unclear. This is where the fundamental gap emerges: the scientific foundations assumed in remote sensing are not inherently built into AI.
How, then, can we bridge this gap? One approach is through Explainable AI (XAI), where models are restricted to outputs that can be traced and explained. But this risks going against the very history of deep learning, which gained its power by accepting black-box complexity in exchange for performance. Another perspective, common in the remote sensing community, is that human knowledge should be combined directly with algorithms. Yet, as history shows, AI has advanced precisely by reducing human intervention. Ultimately, we must accept the fact that we can only approach AI through algorithms themselves.
What is required, then, is a deeper understanding of algorithms and their behavior. We need to specialize and refine them, while ensuring scientific validity within remote sensing applications. And this is not unique to remote sensing — the same challenge extends across all domains where AI is applied.
We have entered a new era. Instead of attempting to trace every computational step at the microscopic level, we must take a macroscopic perspective, understanding AI’s behavior through algorithmic design and theoretical frameworks. In physics, this shift is reminiscent of the move from particle-level mechanics to thermodynamics and statistical mechanics: from trying to explain every particle to describing the system as a whole. Likewise, we cannot hope to perfectly interpret every parameter in a neural network. Instead, we must study the nature of networks, their processing of information, and the robustness of features they learn — whether spatial, abstract, or context-based. And importantly, we must debate whether sacrificing robustness for performance under narrow conditions is truly justified.
Conclusion
In this article, we have explored how remote sensing and deep learning intersect, and what challenges arise from their collaboration. This is not only my area of research but also my conviction: progress in this field will take long-term, sustained effort. While the core technologies of AI evolve rapidly, integrating them with established scientific practices like remote sensing will require patience, reflection, and persistence.
I believe that the sooner we confront these challenges head-on — without turning away from the cultural and methodological differences — the more effectively we can shape the future of AI-driven remote sensing for science and society.
