Integrating AWS Secrets Manager with Parameter Store

Problem

AWS Secrets Manager helps customers easily rotate, manage, and retrieve secrets throughout their lifecycle. The Parameter Store (PS) feature of AWS Systems Manager has significant overlapping use cases with AWS Secrets Manager (ASM), leaving customers to wander which is the best service in which to store their secure information.

Result

An integration that eliminates overlap and provides a more fluid UX for customers to centrally manage their resources.

Industry

Cloud security, Cryptography

Duration

Sep 2022 — June 2023

Product

Web service integration

My Roles

User research

UX design

UI & Interaction design

Development support

(6+ person team)

Objectives

Conduct a multi-step research process into both services.

Support feature/product design with mockups.

Fine-tune to user needs and API constraints.

User research

Several outreach sessions were organized with mid-large clients. Topics covered:

• Resource storage practices & organization
• Information search patterns
• Pain points
• Unique service usage patterns
• Feature usage.

Critical to understand were current resource organization practices, as well as secret type distribution.

A sizable number of customers use Parameter Store as part of the same workflow as Secrets Manager. 12.5% of next service page visits after Secrests Manager were found to be to Systems Manager. The parameters list of Parameter Store represents 29% of landing page traffic. In addition, 44%+ of exit pages in Systems Manager are from the parameters list.

With qualitative and quantitative insights, we felt prepared to answer the customer need for this integration as comprehensively as possible. Findings helped decide new resource creation flow, import flow, and presentation of the new resource type in console. 

Defining users

Our users included 3 main groups:

• DevOps — Efficiently delegating resource use
• Cloud Architect — Making broad usage decisions
• IT Admin — Producing and managing resources

Several high- and mid-level user flows were diagrammed based on customer workflows, seeking to define how customers would decide on a service to use for their data storage, how they might manage resources across both services, and the prospected steps they would take to upgrade, create, import, and manage a new resource type: the static secret.

Early mockups

Drafts were created to explore fully vs. partially integrated list and create views, type selection flows, upgrade & import flows, as well as possible changes to the Paramater Store console.

Designs were shared with UX designer and PM stakeholders from the Parameter Store team to refine the idea.

Design decisions

Modifications to the existing Secrets Manager service included the addition of a new resource type, as well as the ability for customers to effectively import existing Parameter Store resources. Designs were aligned with the Cloudscape design system current release and planned beta.

Creating new resources

 Create resource flows were combined for several reasons:

• More cleanly centralize and integrate new resource type instead of confining it to a discrete section of the service.
  
• Aid customers in feature discovery and exploring secret tiers. Facilitate customer feature discovery with effective descriptions and links to learn more.

We focused on absolving any mis-alignment between how each service orders/presents the purpose of each field. It took many iterations to balance tier display, flow integration, form field design, and messaging.

Resource list management

Contrary to the create flow, a tough decision was made to separate the static secret list for several reasons:

• A conceptual split between configuration-style secrets and functional secrets, as well as a known division of responsibility within customer teams with respect to these secrets (secret-producing roles vs. secret-consuming roles).
• API changes meant different data would be returned for each resource type, resulting in an inefficient use of table columns.
• Afforded actions only pertained to one resource type.
• We could start introducing the side panel navigation to user workflows, to leverage more extensively in future projects.

Resource type labels

There were several issues related to how best to integrate established parameter types/tiers with established secret types. ASM refers to "type" as a secret's purpose, while PS refers to it as the type of string the parameter holds (only 1 of which would be integrated, hence selection of static secret type would be unnecessary). 2 solutions were considered to address this issue that were ultimately rejected:

• Flattening resource selection (Standard, Advanced, Fully-Managed), renaming this selection to "Secret type", and renaming or removing label from the previous "Secret type" selection (AWS service, DB, Other).
  
• Adding a "Category" selection & resource key/value pair to differentiate fully-managed and static secrets, and have "type" only apply to fully-managed secrets.

Both ideas were discarded due to potential confusion of adding additional generalist terms and conflicts with research findings and API changes.

Solution: Integrate! "Tier" was included as an input in the create static secret flow to align with the original PS console, and the "Secret type" selection was left unchanged for creating fully-managed secrets. The "type" and "tier" are then combined within the "Secret type" label for display purposes, using a sub-type in parentheses. This results in secret types being formatted as "Fully-managed (<Service or DB or Other>)" or "Static (<Tier>)". This solution aligned best with our understood persona goals/tasks, and made for better consistency with original create flows as well as effective and space-efficient information design when displaying in console.

Importing or "referencing" resources

Importing resources from Parameter Store was a requirement from a product perspective to ensure that the integration avoided characteristics of a one-way door for customers. In addition, some customers expressed the need to keep vital resources in the current service, as as replicating the resource would mean extensive rework.

In collaboration with product management and content writing stakeholders,  a "parameter reference" was conceptualized as a new AWS resource type. It's not quite a copy or image, and not strictly an imported resource because the original resource would continue to exist in the original service. There was some precedent in a little-known Parameter Store service feature.

A user can view and select any or all resources in Parameter Store, then create a reference in Secrets Manager. The feature had to be discoverable, but not spotlighted, as we projected that only a subset of migrating customers would use it. As a result, it was made an action button on top of the "Static secrets" list.

Upgrading resources

In collaboration with the Parameter Store team, we discussed integration from the other console's side. Customers would be redirected to create "SecureString"-type parameters as secrets in Secrets Manager. Ultimately, this change was evaluated to be too disruptive and abandoned.

Customers would also have the oprtion to upgrade a regular or advanced parameter into a fully-managed secret. This feature was categorized as a P1 feature for implementation after general availability.

Opportunity: Updating encryption key select

Secrets Manager

Parameter Store

New Solution

I consulted with the lead front-end engineer to make a decision to update the encryption key selection component: the PS one had more versatile functionally, the ASM one was more efficient, we synthesized a solution that was quick to implement, introduced overlooked functionality, and was more efficient UX for a large subgroup of users.

The solution was also modernized to design system pattern conventions.

API alignment

1st draft - Create flow & Resource details

Solution - Create flow

Solution - Resource details

Problem: The API design was such that we wouldn’t be able to replicate a major feature in the integration that was a key differentiator between resource tiers. Customers would expect this feature to be available when trying to create/view a resource from our console, and the lack of it could undermine our goal to bring new users in.

Rejected idea: Use an “empty table” pattern to communicate that the feature would be available soon.
Minimize customer dissatisfaction by presenting the feature as being introduced at some point in the future, even if unspecified. However, this was rejected by the design review team as not a good enough reason to break the established pattern in the create flow context, but not in the description page context.

Solution: Include additional messaging to an existing inline info alert, centralizing comprehensive information about limitations and differences of the integration, complete with links to Parameter Store console and documentation. Retain empty table pattern on the resource details page. New action to “Edit policies in Parameter Store”.

This was identified as the highest potential area for dropoff, to be validated with console metrics, support tickets, and direct feedback post-launch.

Measuring results

Goals/measures were developed in alignment with established UX goals for Secrets Manager. With one of our internal scripts, we examined the Parameter Store console to define console element tags for the feature in collaboration with front-end engineers.

• New Goal: Make users of Parameter Store aware of Secrets Manager as a secure “upgrade”.
  
• Measure(s):
  • Number of customers entering ASM from PS documentation or console.
• Existing Goal: Minimize drop-off in create secret flow.
• Measure(s):
  • Completion rate by type.
    Completion rate by account size/newness.
• Existing Goal: Minimize customer confusion locating secrets , versions, or resource information.
• Measure(s):
  
  • Session duration.
    
  • Number of instances where multiple clicks back and forth between “fully-managed” and “static”. Compare to number of page views.

Conclusion

This project required involved collaboration not only to design a customer-centric UX, but also to develop the actual feature design in collaboration with product. Early mockups supported socialization and iteration on the idea.

Later on, collaboration with the lead API engineer and console engineer ensured that the UX design aligned with technical scope.

As a result, customers will have more centralized, customizable options for storing their encrypted information, widening the scope of what they’ll want to store in Secrets Manager.