Amazon SageMaker Object Detection for Protecting the Great Barrier Reef
Every now and then, I receive updates from Kaggle about new competitions. I rarely do participate and if I do, I never won anything. I was never a data scientist, but I believe participating in Kaggle competitions is one of the ways data scientists can build project portfolios. — However, the main reason I stay connected with the Kaggle community is to find datasets I can use for personal projects.
Recently, TensorFlow in collaboration with The Great Barrier Reef Foundation launched a competition on Kaggle with the goal to “ accurately identify starfish in real-time by building an object detection model trained on underwater videos of coral reefs.” This is inline with The Great Barrier Reef Foundation’s innovation program to develop new survey and intervention methods to provide a step change in COTS Control. You can find more details on the Kaggle competition page:
Why Amazon SageMaker Built-in Object Detection Algo?
The fact that the problem scope “accurately identify starfish in real-time by building an object detection model trained on underwater videos of coral reefs” was clearly defined caught my attention. More so, the use of object detection. Definitely, one can train a TensorFlow model on Amazon SageMaker but I asked myself: “Why not give the Amazon SageMaker Built-in Object Detection Algorithm a go at this?” The answers were quite straightforward:
- I did not have to write any neural network architectures from scratch.
- It is very easy to use.
- It reduces my time to solution. (I completed data preparation and training in 24 hours of work)
- Amazon SageMaker Object Detection Algo has an excellent documentation.
With all checks marked, it was time to get my hands dirty with some code.
PS: This was my first time building any object detection use case on Amazon SageMaker. The reason is that I always felt object detection was in the realm of computer vision ninjas.
The Work
If you are interested in reproducing this work, you can download the Jupyter Notebook from my AWS Samples Github repository following the link below:
The Dataset
As with all Kaggle competitions, the dataset is accessible from the Kaggle environment. You can also access Kaggle datasets outside the Kaggle environment using the Kaggle Python API and your Kaggle API keys. Once setup, you can download the dataset (~15GB) using the command below:
kaggle competitions download -c tensorflow-great-barrier-reefPrepare Image Dataset and Upload to S3
The team behind the competition had an amazing work collection images from video frames and annotating the images, as seen in the image below in the annotations column. Since the goal was to train an object detection model, this means that I only needed images with coral-eating crown-of-thorns starfish annotations.
df['num_bbox'] = df['annotations'].progress_apply(lambda x: len(x))
data = (df.num_bbox>0).value_counts(normalize=True)*100
print(f"No BBox: {data[0]:0.2f}% | With BBox: {data[1]:0.2f}%")Approximately 30% of the entire dataset contained annotations, which is what we need to train our object detection model.
The next step here is then to structure the annotations in the valid format for the Amazon SageMaker Object Detection built-in algorithm. There are 3 formats, namely: RecordIO Format, Image Format and Augmented Manifest Image Format. In this case, I used the Image Format which meant I needed to write the annotations to a json file in a specific format. Example below for the validation image. Read more about training data formats compatible with the Amazon SageMaker built-in Object Detection Algorithm
{
"file": "video_0_1000.jpg",
"image_size": [
{
"width": 50,
"height": 32,
"depth": 3
}
],
"annotations": [
{
"class_id": 0,
"left": 559,
"top": 213,
"width": 50,
"height": 32
}
],
"categories": [
{
"class_id": 0,
"name": "starfish"
}
]
}
Finally, the annotations and image are uploaded to s3 and ready to train our model
Train and deploy Image Classification Model
Our image classification model is trained on Amazon SageMaker GPU instance ml.p2.xlarge. To save up to 70% on training costs, I used the managed spot settings on Amazon SageMaker. You can read here for more information on Amazon SageMaker managed spot training. See image below:
With all training infrastructure configurations in place, we then set our hyperparameters. There is an excellent documentation as well as an example on how to use Amazon SageMaker Object Detection for Bird Species where the necessary parameters are covered.
Then we call the fit method on our object detection estimator and wait for the machine to learn:
object_detector_model.fit(inputs=data_channels, logs=True)If you follow the example notebook, I will recommend that you do not wait. This is because the num_epochs to 100, which took about 13 hours to complete training.
Training Results
The accuracy metric for this algorithm is validation:mAP and as seen in the image above, it begins to plateau at ~43%. This means that our first model is 43% accurate. Is this good enough for a first iteration? Most likely. However, to make this production ready, we use techniques such as image data augmentation, hyperparameter tuning, or trying out new models like YoloV[2 to 5] or Facebook Detectron2.
Production Deployment: Protecting Coral Reefs
Amazon Web Services provides multiple options to deploy Amazon SageMaker models. Because, this will be used at the edge in real time, we may consider the following:
- Amazon SageMaker Edge: Amazon SageMaker Edge enables machine learning on edge devices by optimizing, securing, and deploying models to the edge, and then monitoring these models on your fleet of devices, such as smart cameras, robots, and other smart-electronics, to reduce ongoing operational costs.
- AWS Panorama: The AWS Panorama Appliance is a compact edge appliance that uses a powerful system-on-module (SOM) that is optimized for machine learning workloads. The appliance can run multiple computer vision models against multiple video streams in parallel and output the results in real time. It is designed for use in commercial and industrial settings and is rated for dust and liquid protection (IP-62).
The options above are not listed in any preference order, however, my first choice would be the AWS Panorama device or SDK. This is because it is already optimized similar use cases and little to do to make it operational.
Conclusion
Even though I had previous experience with Amazon SageMaker, this was my first object detection use case that is not an example from the AWS SageMaker samples repository. Hopefully, we are able to leverage machine learning capabilities to better protect our environment, in this case the Great Barrier Reef.
I hope you found this a good read. If you would like to discuss your machine learning use cases, you can reach me via email, follow me on Twitter or connect with me on LinkedIn. Can’t wait to hear from you!!
Jupyter Notebook available below:
