INTRODUCTION
Today the ability to save and store options within the Amazon Marketing Cloud UI does not exist
This case study presents an overview of a project developed during my summer internship at Amazon. The project aimed to implement a new feature for saving and storing SQL statements and executions within the AMC UI. The project's goal is to provide AMC customers with a way to prevent work loss, ensure efficient process tracking, and offer a convenient storage solution for quickly retrieving saved information. It also aims to facilitate the reuse of queries across instances within the same account.
WHAT IS AMAZON MARKETING CLOUD
Amazon Marketing Cloud can help customers discover advertising insights
Amazon Marketing Cloud (AMC) empowers businesses with data-driven insights, targeted advertising capabilities, cross-channel integration, advanced analytics, and efficient budget allocation, all of which contribute to improved marketing strategies, customer engagement, and overall business growth.
Main points of improvement:
1. SQL statements cannot be saved on AMC
2. Users have to open new tabs and rely on the browser's auto-storage function
3. Queries cannot be named, increasing the challenge of locating a specific SQL statement
4. Focused on individual users with limited functionality for streamlining collaboration
All the query executions have the same name and it is difficult for customers to
differentiate between one query and the other
HYPOTHESIS
How did I approach the problem?
By implementing a system for saving and storing queries, mistakes can be decreased, workflows can be improved, productivity can be increased, and collaboration can be enabled.
Amazon Marketing Cloud (AMC) original console design and distribution
THE PROBLEM
AMC lacks essential features to prevent information loss
AMC customers are consistently at risk of losing important information. They are forced to use external tools to save their queries, as they lack trust in the limited options available within the AMC UI for storing information.
RESEARCH
1. AMAZON MARKETING CLOUD STRUCTURE
The structure of AMC is different from any other Amazon advertising product
In AMC, an account can consist of a variable number of instances, which can be thought of as distinct workspaces. For example, a marketing agency could have an AMC account and manage multiple instances, with each one representing a different brand/client they work with.
2. VOC - VOICE OF THE CUSTOMER
AMC's customers are forced to use external tools to save their queries
Queries are currently restricted to individual users due to the lack of collaboration options. Queries remain visible exclusively to their respective owners and cannot be shared with other users, regardless of whether they both belong to the same account. Users must turn to external tools to be able to share the SQL statements. Although this is essential to promote collaboration among team members and facilitate the reuse of queries across diverse workspaces, it often leads to a disorganized and less efficient process that consumes more time than necessary.
“Right now there are about four people on the Media Monks team that specialize in AMC. To find a query again, we have to remember the query name, the type of query, the status of the query, who created that. That’s how we can actually work as a team with the (Google) Data Hub. For AMC, I think it’s mostly individual work. Any query I run, the other person in the team wouldn’t be able to see it. So that’s one of the reason why we work with an Amazon sales maker notebook. It’s the only way we can see what other people are working on”
“My clients (LG / Publicis / BidX etc) have asked for an AMC feature allowing them to store queries / share these with different teams internally as well as collaborate on queries. Clients who would run sometimes hundred queries with different advertisers find it difficult to search or look for specific queries (they use external tools to save queries since these allow them to put them into folders, sort by date, etc.). Sharing queries and especially knowing who added and amended a query within the query editor will be helpful since agencies like Publisics would have one or two SQL experts who could share queries with all teams within AMC etc.”
3. COMPETITIVE ANALYSIS
Other competitors have already solved this problem
As AMC primarily functions as a database engine, its main competitors are products such as BigQuery by Google, Snowflake, and Azure Synapse by Microsoft, which already provide saving and storing query features. These features enable users to save their ongoing work and final versions to prevent data loss while also facilitating collaboration with other users within the same account. Furthermore, competitors offer additional functionalities like query naming, adding details, and results sharing, which are currently absent in AMC. Based on this analysis, it became evident that AMC could benefit from incorporating similar functionalities to enhance the product's value and contribute to enhanced collaboration, error prevention, streamlined workflows, and increased speed.
DEFINE
1. TARGET CUSTOMER
Who was the target customer for this project?
AMC customers fall into three categories: Business, Strategic, and Technical. Among these, the Technical users are the primary target for this new feature. Technical users heavily utilize the query editor, leveraging their SQL skills to retrieve AMC data. Their Python understanding might be solid, but their business knowledge might range from medium to low. The skill level of technical users varies depending on their roles and job descriptions.
Benefits for technical users:
Save their work-in-progress
Prevent work lost
Minimize potential mistakes.
Main benefits for Business and Strategic users:
Although their SQL skills are moderate to low, public saved queries could still be valuable for them. They might use them as templates to build new queries or to reuse the same query to gather information regarding other instances.
2. SCOPE OF THE PROJECT
What are the technical customer’s needs?
The project is divided into three components: Save and store, collaboration, and potential scalability. The primary focus lies in the first one: saving and storing, which are the main priorities (P0) of the project. After completing the development of P0, the priorities will shift to P1, which includes collaboration functionalities related to editor access and sharing results. These functionalities are considered "nice to have" elements that can be incorporated after finishing the development of P0. Finally, P2 encompasses ideas on how this project could scale in the future, demonstrating its potential for rapid growth and providing multiple benefits for AMC customers.
3. BRD - BUSINESS REQUIREMENTS DOCUMENT
I defined potential use cases based on the problem
During the elaboration of the BRD document for the project, I explored potential use cases based on the problem and followed the priorities established in the step above. The creation of this document was vital since it allowed me to understand the customer journey and all the elements I needed to take into consideration to make this project a reality.
As an AMC UI user, I want to …
Save my queries to review it later
Be able to share saved queries with other users in my account
Edit, update, and delete my queries
Be able to identify my queries quickly
Have a place to store all my saved queries
Be able to review my work in progress
Extracts of the BRD I wrote for my intern project
CREATE
1. WIREFRAMES
The project introduced a new feature that allows customers to save and access their work in progress, prevent mistakes and reuse their queries in multiple instances by enabling collaboration among teams. During the first 3 weeks of my internship, while I was learning about AMC and its customers, I sketched some ideas to start thinking of potential solutions to the problem I was working on.
Some sketches I created during the week 2 and 3 of my summer internship
2. DESIGN EVOLUTION
How did the design solution change over time?
One of the most challenging parts of this project, particularly concerning the design aspect, was determining the optimal placement for the saved queries. In the images displayed below, I aim to present a timeline depicting the evolution of the design. Each week, I collected valuable feedback from other designers and stakeholders, which helped me enhance the design solution. Eventually, I reached a point where the design was ready for review and customer viewing.
Week 3 - Starting the design process
Not the best place to store queries since this column is more related to the available content within an instance.
Week 6 - Mind point review of my internship
It might cause problems if the customers open the ‘library’ and the Schema Explorer at the same time.
Week 9 - Before the user testing
I changed the name ‘account library’ to ‘saved queries’ because it was difficult for others to understand this was the place where the queries were stored.
DESIGN PROPOSAL
What are the customer’s needs?
After analyzing the information shown above, I broke down the design solution into three main parts. The first part is the ability to save queries, the second part is the option to store them, and the third part is the ability to enable collaboration through one of the saving methods.
Saved queries placement
The saved queries are under the Explore panel, next to the ‘Schema Tables’
Categories
The queries are stored in two categories that are related to the visibility option chosen during the saved process.
Saved queries options
For each saved query, you will have four options: open it in the query editor, change its visibility, delete it or view its details.
STEP 3: Saved query options
For each saved query, you will have four options: open it in the query editor, change its visibility, delete it, or view its details.
DELETE A QUERY
The option to delete a saved query, gives you 30 days to recover the information, as all deleted query remains saved in the 'Deleted queries' section for that period of time. Only the owner has the ability to edit saved queries, regardless of whether they are stored as public or private.
Users who are not the owner cannot update, change, or delete a saved query.
QUERY DETAIL PAGE
Each saved query has a specific detail section attached to it. By opening the side panel containing the search details, you can change the name and description of the search. This panel also offers important information such as the search's creator, when it was created, the latest date range used, the time zone and time it takes for the search to complete, and the most recent update made. If the query has been run previously, the query details will provide an additional page showing information related to the results history. However, it's crucial to note that the full query detail page is accessible only to users within the same 'instance'. This measure is in place to mitigate potential privacy concerns linked to the query result history.
Query detail preview
Full query detail page
STEP 1: SAVE A QUERY AND CHOOSE THE VISIBILITY
During the saving process, customers can name their query, provide an optional description, and choose who has permission to view the query by choosing between the two visibility options: private or public.
Private queries are exclusively accessible to the owner, while public queries are visible to all users within the same AMC account.
Additionally, customers can update saved queries, create as many copies as they wish, and delete queries that belong to them.
STEP 2: LOOK FOR A SAVED QUERY
All the queries that you save are kept in the "Saved queries" section, which you can find in the Explorer panel. Here, you can choose to view either public or private queries, depending on their visibility settings. When you want to edit a saved query, you can simply click on the query's name, which will act as a hyperlink and automatically open the query in the query editor.
Delete a saved query
Process of saving a query as private
USER TESTING
1. THE PARTICIPANTS
Who were the participants of the Cognitive Walkthrough?
I tested the design with internal and external customers to evaluate the solution, the prototype’s functionality, and the overall customer experience. All the customers possess SQL abilities. Among them, four were very familiar with AMC, while the remaining two were not. However, the latter two do have experience using other clean rooms such as Big Query and Azure.
2. The user
I asked the participants to portray the following user, Danielle
Danielle works at a full-service media agency. Her team works on the programmatic advertising side of the business where she works to find insights into her client’s customer base to inform their advertising campaigns.
The task Danielle has to complete in her role
Write queries to retriever data from AMC
Analyze the results
Re-use queries to repeat the same strategy with different clients in the same account
3. THE SCENARIOS
What task did the participants perform?
The purpose of testing 3 different scenarios was to examine the main issues of the project. For example, I wanted to find out if customers could tell the variations between saving a query as public or private. I also wanted to see if they could find where the saved queries were stored. Lastly, I aimed to measure how easy it was for them to navigate and identify areas that could be improved.
The three scenarios tested during the cognitive walkthrough
4. THE POSITIVE FEEDBACK
The participants found the design solution interesting and exciting
The participants stayed very interested and involved throughout the whole process. In general, they strongly believed that the project was a fantastic idea, especially because many AMC customers have been asking for similar features for a long time. They also highlighted how simple it is to save and find stored queries, showing that the user experience is very positive and a major improvement for the product.
Some positive feedbacks coming from the internal customers, Amazon’s employees
5. IMPROVEMENTS
The majority of the confusion was related to UX Strings
During the testing with users, I realized that most of the difficulties came from the words used in the user interface, rather than the way things were done or the design I was working on. The users also gave me valuable information about AMC, which helped me improve certain parts of how people navigate through the console. For example, they pointed out that to change the visibility of a query, the user had to take too many steps. Thanks to their feedback, I was able to make the experience smoother by reducing the number of steps needed.
Improvements I needed to make:
They didn’t understand what account means
They didn’t understand why public queries were stored at the account level
To change the visibility of a query, the user has to take too many steps
Difficulties understanding the differences between the two ‘save’ button
They want to see when a query is private or public
Examples of the part that were confusing for the participants
PROTOTYPE
Interact with the high-fidelity prototype here
I created the prototype of the main user experience of the project, which included saving and searching for a query, using Figma. I made sure to keep the design consistent with the overall look and feel of Amazon Marketing Cloud, therefore, I used the same font, colors, and components that are used in all Amazon advertising products.
FAQ - FREQUENTLY ASK QUESTIONS
-
Today, the ability of saving and storing options within the AMC UI does not exist. To store queries, users must run them, and AWS-generated files compile executed queries in the 'Submitted Queries' section.
Alternatively, queries can be saved by keeping them in an open tab in the query editor, but the tab cannot be closed. Both methods make it challenging to save work in progress, quickly find specific query results due to the lack of labeling, and increase the potential ricks of loss vital information.
In addition, as AMC continues to evolve with new functionalities and features such as Template Analytics, Paid features and more options available in the instructional query library that provides more material to the customers, the saving feature is essential to ensures that users can preserve their work in progress and have an option to properly organize the information. This is particularly crucial for complex marketing campaigns that involve multiple iterations of a single query or the creation of several of them, enabling users to maintain continuity and avoid repetitive work.
-
Since this project is part of a three-month internship, the short-term success measures involve aligning all parties involved— product managers, designers, and engineers—with the proposed solution. Another measure of success would be the potential consideration of this problem for further exploration or implementation, which signifies long-term success.
Regarding the success of the feature, We will measure success of this initiative through the percentage of active users who utilize the saved feature each month, breaking it down in the following points:
Customer behavior
Number of queries saved per user as private
Number of queries saved per user as public
Percentage of customers that access public queries
Percentage of customers that reuse public queries
Percentage of customers that recover delete queries
Number of time per day a user accesses their private library
Number of time per day a user accesses their public library
Performance
Range between the number of private and saved queries
Percentage of queries that are change from private to public queries
From the total of public queries, what is the percentage of the queries that are being used
Percentage of public queries accessed by users other than the query owner
This will help us to understand how many people find the product valuable and keep using it.
Additionally, tracking the percentage of users over time serves as a crucial metric for assessing whether the performance of the feature is improving or declining. Finally, by comparing the percentages of users who choose to save their features privately versus publicly, we can assess whether users find value in both options.
-
Storing queries containing sensitive information could create security risks if the information is not properly protected. Unauthorized access to saved queries might lead to the exposure of confidential data, affecting the privacy and security of
campaigns and strategies. Therefore, the distinction of private and public must be very clear for the customers before using this feature, and the developers must plan, develop and make the maintenance of this feature carefully to prevent exporse and display saved queries in the wrong places. Additionally, users that are used to their current workflow might resist adopting the new feature, leading to slower adoption rates and limited utilization during a period of time.
-
Since this project currently only offers the ability to save queries, we aim to provide an option that allows customers to save information beyond just queries. In the future, we plan to create an Account library, serving as a repository of information exclusive to each particular account. This library will function similarly to the instructional query library but will only store information that is part of a specific account. Consequently, the Account Library will house all saved public queries, template analytics, audiences, instructional queries, and more.
Project details
Please have a look at the entire FAQ document project here
I have made a document that summarizes the whole project and addresses the most frequently asked questions. To see it, click the button below.
Would you like to view additional projects?
Netflix - Feature for content recommendations
Gmail - AI chat to improve content discoverability