This is a basic tutorial that explains the importance of tracking results and how to collect data using Custom Metrics using AWS CloudWatch Metrics.
The ultimate purpose of this story is to allow the reader to understand some of the terminologies of CloudWatch and to always push for challenging goal that leads to higher performance teams/individual.
Not interested in the business impact, jump directly to the technical side!
Business Impact (KPI)
A Key Performance Indicator is a measurable value that demonstrates how effectively a company or an individual is achieving key objectives.
If you want to be successful in achieving your goal, you need to show to yourself and the rest of the business progressive results in a specific time-frame, specifying why those objectives are relevant to yourself or the company and how realistic is the completion.
We live in a world driven by data, and the easy way to track progress is to use that data in our favour! With real-time/daily/weekly update, you will be able to:
- Track your progress and share it to the rest of the business.
- Knowing if you are on track for the deadline.
- Being able to control the outcome.
I love a high-performance mindset, where THE INDIVIDUAL:
- Know what you want to achieve, “must be measurable and tangible!”
- Set the goal! “Realistic but hard to achieve”
- Timeframe it! “Your time is precious, manage it!”
- Deadline; “Failure is a chance to improve, get better on estimation and managing expectation.”
- Track Progress “Re-iterate, re-prioritise, keep up morale.”
Custom Metrics (Mesure and Achieve Success)
In this part, I’ll show how to use AWS Custom Metrics to be able to measure any part of your application, moreover, making sure you are on-track on your KPI and be able to re-iterate and re-prioritise when you are trending bearish.
AWS CloudWatch Metrics is a service that allows you to publish data-point in a time-ordered way and quickly visualise it using a statistical graph.
A metric is a set of data-point that will always have: Name, Namespace, Value and timestamp.
At the moment that your metric is defined, is up to you to send as many data-point as you need! However, remember that Metrics are costly; therefore, the more you have, the more you pay. Metrics are not stored forever; you only have 15 months retention, after that the metrics will be deleted automatically. Metrics cannot be deleted, therefore if you are experimenting, remember to use a test-namespace or experiment in an unused region as the metrics are stored just in the region where they are created.
Metric Timestamp: usage and retention period
Metrics are time-series that are stored for 15 months; nevertheless, you are allowed to publish a metric of up to 2 weeks old or 2 hours in the future!
Metrics can have different retention based on the granularity of the publisher, for example, if we publish seven metrics per seconds, those will be available only for 3 hours! Behind the scene, CloudWatch allows you to publish metrics in a short period and visualise them for a minimum period of 1 second!
After a particular time, he will automatically aggregate the data, and you lose granular visibility! This means you need to use a more considerable Period to visualise the data on the graph.
The timestamp has the following convention:
YYYY-MM-DDTHH:MM: SS (for example, 2020–11–18T23:59:59Z).
The namespace will help you to define your domain and group metrics that are part of that domain-space.
You need always a namespace to publish a metric!
A dimension is a set of name/value that is associated with your metric and enable you to add information related to the metric you are publishing. You can add up to 10 dimensions per metric.
The more dimensions you have, the more comfortable it is to create filters based on the different key on the metric and the dimensions contained in it!
Let’s get real!
Let’s imagine to create a namespace where we are putting a few metrics without dimension such: NumberOfRequests, NumberOfClicks etc. etc.
aws cloudwatch put-metric-data --namespace NAMESPACE-NoDimension --metric-name NumberOfRequests --value 1 --unit Countaws cloudwatch put-metric-data --namespace NAMESPACE-NoDimension --metric-name NumberOfClicks --value 1 --unit Countaws cloudwatch put-metric-data --namespace NAMESPACE-NoDimension --metric-name NumberOfFailure --value 1 --unit Countaws cloudwatch put-metric-data --namespace NAMESPACE-NoDimension --metric-name NumberOfConversion --value 1 --unit Coun
As said before dimensions are optional, and you can always integrate more dimensions during the development lifecycle of your application.
If you want to expand your data, you can always add dimensions to your dataset, however for the sake of this example we will create an entirely new domain, and we start to insert metric with dimensions.
We will use the same example of before. However, we would like to know who is making all the request, which user?
aws cloudwatch put-metric-data --namespace NAMESPACE-with-dimension --metric-name NumberOfRequests --dimensions User=Alberto --value 1 --unit Countaws cloudwatch put-metric-data --namespace NAMESPACE-with-dimension --metric-name NumberOfRequests --dimensions User=Adriana --value 1 --unit Countaws cloudwatch put-metric-data --namespace NAMESPACE-with-dimension --metric-name NumberOfRequests --dimensions User=Joseph --value 1 --unit Countaws cloudwatch put-metric-data --namespace NAMESPACE-with-dimension --metric-name NumberOfRequests --dimensions User=Daniel --value 1 --unit Countaws cloudwatch put-metric-data --namespace NAMESPACE-with-dimension --metric-name NumberOfRequests --dimensions User=Patrick --value 1 --unit Count
As you can see from the example above, using dimension, you are able to define any kind of metrics, and this will help you track any of your tangible objective e.g.
- Number of clicks in a certain part of your website
- Reaching $X amount of sells in a timeframe
- Reaching $X amount of sell of product Y (dimensions) in a timeframe
- Cross product selling when a user purchase product X and then Y.
This is a very basic tutorial; nevertheless, don’t underestimate the metrics!
They are a vital part of your business, helping to transform a goal into a subset of numbers that can map the progress of your objectives accurately.