The purpose of this project is to perform a statewide large scale solar radiation analysis on South Carolina using ESRI’s Area Solar Radiation tool and a high resolution DEM from The United States Geological Survey (USGS). This analysis was performed in order to determine the solar suitability of various locations in the state. The DEM we used has a resolution of 1/3 arc-seconds (approximately 9.8 meters). The biggest issue with this undertaking is that this kind of big data calculation is very computationally expensive. On our workstations, this calculation was taking upwards of a week to process and could potentially crash the computer.

​

To tackle this problem, we wrote python scripts to split, distribute, and mosaic our data so the solar radiation analysis could be performed quickly and effectively. These scripts can be accessed from our GitHub page. Distribution was performed using an “embarrassingly parallel” model of computing to run our split data on our local distributed computing cluster, GalaxyGIS.

 

GalaxyGIS is our High Throughput Computing pool at Clemson University which includes over 700 computational nodes and a High Throughput scheduler called HTCondor. HTCondor is a specialized workload management system for compute-intensive jobs that provides a job queuing mechanism, scheduling policy, priority scheme, resource monitoring, and resource management. 

 

By using our parallel computing method on GalaxyGIS, we were able to reduce processing time from around a week down to about 3 hours! For more information on our methodology, check out the Methods page.