Articles
Tutorials
Interactive Guides
Introduction to ImageLabeller
ImageLabeller is a simple demo application that uses machine learning to apply labels to images. It uses AWS Lambda, AWS DynamoDB, AWS S3, AWS SageMaker, AWS API Gateway, AWS CloudWatch, and AWS CloudFormation, and is written in Go, and Python. This guide covers how to use Atlassian Open DevOps tools to deploy, and monitor a subset of these components.
ImageLabeller Architecture
How it works
A user invokes AWS Lambda SubmitImage, passing a valid image URL as a parameter. SubmitImage downloads the image, uploads it to an AWS S3 bucket open-devops-images-
The AWS Lambda InvokeLabeller has an AWS DynamoDB event trigger. InvokeLabeller downloads the corresponding image from open-devops-images-
The user invokes AWS Lambda GetImageLabel, passing the image ID token as a parameter. GetImageLabel looks up the labels associated with the image ID token in ImageLabels, and returns them to the user. The user gets a NOT_CLASSIFIED label if the AWS SageMaker model hasn’t finished processing an image.
ImageLabeller is deployed to five environments. A Test environment in us-west-1, a Staging environment in us-east-2, and three Production environments in us-west-2, us-east-1, and ca-central-1.
Components
The AWS S3 buckets and AWS DynamoDB tables that are part of ImageLabeller are deployed using AWS CloudFormation templates. This guide provides CI/CD configuration information for how to deploy the AWS S3 buckets. The process for deploying the AWS DynamoDB tables is almost identical and is left as an exercise for you to complete on your own.
There are three AWS Lambda in ImageLabeller: SubmitImage, InvokeLabeller, and GetImageLabel. This guide provides CI/CD configuration information for how to deploy SubmitImage. The process for deploying GetImageLabel is almost identical and is also left for you to complete on your own.
InvokeLabeller is a python-based AWS Lambda and requires minor updating before it can be published. Specifically, InvokeLabeller is dependent on AWS SageMaker configuration which is done through the AWS console.
The system tests included in the ImageLabeller project test ImageLabeller as a whole. The tests will fail if any component of ImageLabeller is not present or working as expected. SubmitImage, GetImageLabel, and InvokeLabeller deployments all execute the system tests as part of their respective CI/CD pipelines.
For the initial deployment of SubmitImage, GetImageLabel, and InvokeLabeller you need to comment out system test execution steps. Once the deployments have successfully run once, you can uncomment the system test execution steps, and redeploy.
Order to deploy components
Deploy AWS SageMaker pre-trained model manually through the AWS console
Deploy AWS infrastructure repositories
- s3_infra
- dynamodb_infra
Deploy AWS Lambdas
- SubmitImage
- GetImageLabel
- InvokeLabeller
Deploy System Tests
Deploy CloudWatchAlarms
.png?cdnVersion=1568)
Share this article
Next topic
Recommended reading
Bookmark these resources to learn about types of DevOps teams, or for ongoing updates about DevOps at Atlassian.
DevOps community
DevOps learning path
