Detecting solar panels from satellite images
Abstract
This study leverages deep learning for segmenting solar panels in satellite imagery to enable tracking of installation process on large solar farms. A custom segmentation ResNet-based model was developed in PyTorch, to achieved 71% Jaccard index (aka IoU) on a validation dataset. ColorJitter was shown to be a good augmentation to combat influence of clouds and shadows in the images. GitHub repo.
Data
Raw images of solar farms were provided using Airbus Defence and Space satellites, followed by manual labeling using LabelMe software. The large (~1GB) satellite images were split into smaller ones (256x256 pixels size), and total of 449 labels of 3 classes: racks, common panels, and dense panels were obtained.
Augmentation
It was insightful to plot mean and standard deviation of labeled and unlabeled images to identified range for ColorJitter as augmentation: 
Labeled and unlabeled data have decent overlap, with non-covered datapoints typically being clouds (high mean, low stdev) or with partial out-of-bounds, i.e. black, zones (low mean). See main_experimentation.py for examples.
Training
I used Adam optimizer and weighed CrossEntropyLoss. Segmentation was based on UNet-like architecture and this paper (see models.py):
Training for 120 epoch took about 6 minutes (RTX5000). Achieved 71% Jaccard index (aka IoU) on validation dataset.
Inference
Finally, I evaluated and stitched together predicted images back into large satellite images (size ~1Gb, showing here only smaller section):
(Note: some missing patches in the image were used for training).
Shadows and clouds are probably the biggest obstacle for precise counting. While augmentations can help to some extent, ideally multiple images of the same solar farm should be obtained and combined for thorough coverage.
The following are some notable examples.
Handling multiple classes:
Handling different backgrounds:
And here are the common failures, mostly shades and clouds:
Conclusion
Object segmentation has shown to perform well for solar panel detection. Augmentation using brightness and contrast improved detection significantly.
Number of solar panels is calculated knowing pixel resolution is 50cm/pixel and panels width of about 100 cm (i.e. 2 pixels).
Small edge improvements can be done by allowing for some overlap when cropping large images and combining masks subsequently.
References: Boiler-plate code based on a book “PyTorch: Step by Step” by Daniel Voigt Godoy.