Collaborative deep learning is an inspiring vision, but what makes these projects succeed in practice? We launched the Drought Watch benchmark for drought detection from satellite in August 2019 to encourage the broader machine learning community to collaborate on a problem that is challenging and meaningful—in terms of planetary-scale environmental sustainability. This week we presented the work—and the accompanying dataset of over 100,000 satellite images of Northern Kenya expertly labeled with four levels of drought—at the Computer Vision for Agriculture workshop at ICLR 2020 . What was the collective progress on the benchmark in the interim?
Besides the original authors, 9 participants have logged over 2,500 experiments to W&B, increasing the model’s validation accuracy by 2%. At least one paper, to be presented at the CVPR 2020 Vision for Agriculture workshop in June—Climate Adaptation: Reliably Predicting from Imbalanced Satellite Data by Ruchit Rawal and Prabhu Pradhan—cites our dataset and platform as helpful in their work. This is exactly what we were hoping to see: folks building on the existing benchmark dataset, code, and models to support and accelerate their research. We’re very excited about these developments, and we’re just getting started.
In this report, I review the community submissions to Drought Watch so far, highlighting some of the architecture choices and modifications that seem to improve the model’s validation accuracy. I describe my process for developing the baseline model for the benchmark, in case the approach or insights are useful to other folks working on this or similar problems. Finally, I provide an example of a W&B Sweep as an easy entry point to exploring your own model variants—as well as some next steps for folks to try.
 In case you missed it, you can watch the recording here—Andrew Hobbs and Stacey present the benchmark around 10:57:00.