Kubeflow for Satellite Machine Learning at Scale

Abstract:

Kubeflow and Kubernetes have become commonly used toolkits in industry that allow data scientists to package, deploy and train machine learning models in a portable, scalable and efficient way. We think these open source model training and deployment tools can be useful for the Earth Science Community especially for the training, testing, post-processing, accuracy assessment portions of Earth Science workflows. These tools are powerful and flexible enough to accommodate the complexities of Earth system science data machine learning workflows that can be deployed to any cloud computing environments.

Development Seed is eager to share how we have used kubeflow pipelines to scale our object detection workflows of Earth Science problems such as detecting wildlife or point of interests, e.g. schools and health centers. For classification machine learning problems in Earth Science such as rare point of interest building detection, at a global scale, with a data-set spanning three continents. We have also taken advantage of the Kubeflow tool Katib, to facilitate deep learning model hyper-parameter searches. Katib also helps facilitate reproducible workflows, and the creation of a record of reproducible experiments with different hyperparameter configurations. In our wildlife detection workflow, having Kubeflow pipeline allow us to deploy object detection model to dominant cloud providers’ platform e.g. Google Cloud Platform, AWS, or Microsoft Azure. Kubeflow allows for quick iterations of model training and model evaluation. The ability to iterate over model training and evaluation cycles efficiently is important for all Earth Science workflows. Specifically for our work with wild-life detection in Africa, these tools allow us to share model progress with our volunteer training data annotators, who can improve label quality (ie adjust bounding boxes that are too wide), and/or provide additional training data samples of certain classes to further fine-tune the model.

In the talk, we will discuss how we set up Kubeflow with Kubernetes, and hyperparameters tunning tools like Katib for satellite machine learning workflow, and share steps on how Earth Scientists can get started incorporating Kubeflow into their workflows.