BACKGROUND & OBJECTIVES

What are Salesforce Product & Service emails, and what do they do?

The Technology Communications & Readiness (TCR) team, made up of Technical Writers and Project Managers, is responsible for crafting and sending out communications to ensure that customers are prepared for planned technology changes. These communications can be categorized as Product & Service Notifications ↗ — and common examples of this include feature retirement emails, incident emails, infrastructure change emails, and more.

These emails are critical for Salesforce Admins to avoid disruption and successfully do their jobs!

Who are these “Salesforce customers” most impacted by these emails? They're Salesforce Administrators (aka, Admins).

Salesforce Administrators manage and customize the Salesforce platform for their company or organization, allowing other employees, who might be anything from Marketers, to Sales Reps, to Engineers, use Salesforce in their day-to-day role. These users are called "end-users", while you can think of Admins as the Salesforce “guru” at their specific company.

As Admins are the point-of-contact for all Salesforce platform needs at their company, they need to be updated whenever there’s a change, update, release, etc. to Salesforce so they can stay up to date and successfully manage the platform for their end-users.

PROBLEM STATEMENT

The problem is...these emails are super complex! 

In reality, these were lengthy and technical emails, ranging in complexity. This is problematic when looking at data on the Salesforce Administrator user base — 68% of Admins are on the most basic level of Salesforce support and from small-to-midsize companies. This means that...

Most of these Admins are on the lower end of the proper Salesforce training and education needed digest the complexity of Salesforce, and lack the proper access to Salesforce customer service when wanting to have their questions answered.

So the burning question is...

💡 “How effective and understandable are Salesforce product service email communications for small-to-mid-size business Salesforce Administrators on the lowest level of Salesforce support?”

NARROWING IN ON RESEARCH QUESTIONS

The Research Goal

Understand the efficacy of Salesforce Product & Service email communications* for Admins who lack access to Premier support* 

*defined by the various types of Salesforce Product & Service emails
*defined by Basic support level, <100k AOV admins (SMB)

Research Questions

DIVING INTO EXISTING INSIGHTS

Methods: Stakeholder Interviews, Literature Review

Filling in Knowledge Gaps

After spending a week talking to various stakeholders to gather context on what I needed to know, I started off by conducting a Literature Review on existing insights at Salesforce seeking to learn more about the following relevant spaces:

What did I learn?

Here's a brief look into some critical learnings that helped guide the rest of my process:

Insight #1: There's existing principles used to define what makes Salesforce content effective to users. While these have just been started to be defined, technical writers for Trailhead (Salesforce education) use these to guide their work. There are 6 principles:

Insight #2: Content and Experience are two separate, but interrelated, concepts. There’s content (like the words, layout, visuals) and the experience (search, page layout, etc) that we need to account for.  

Insight #3: SMB Admins on Basic-level support juggle many challenges. An example for a typical SMB Admin on Basic support: ‍

DEFINING CONTENT EFFECTIVENESS
‍
Methods: Stakeholder Workshop, Jobs-to-be-Done

Revising Content Effectiveness Principles Using Jobs-to-be-Done

While I had found existing principles around content effectiveness when assessing Salesforce-branded content, I had to ask —

“Are these the same principles we should use to define what makes Product & Service email content effective?”.

How do we define what makes an effective email? In this case, it depends on what the Salesforce Administrator, the our target user, defines an “effective” and “successful” email by. To start understanding this, I conducted a workshop with the TCR team utilizing a Jobs-to-be-Done framework to understand the goals and motivations Salesforce Administrators have when encountering these product service email communications. Here’s an example of a “job” created from this workshop:

But why a JTBD stakeholder workshop?
This was a great way to quickly draw from the knowledge of email creators on the team (Technical Writers, Project Managers, etc.) to think more about the mental models they use when writing these emails, as well as gather quick insight about what they know about Admins. Jobs-to-be-Done was a great way at getting into the user goals for these email types, and overall, this workshop helped introduce this framework to a non-researcher crowd.

From this set of JTBDs created, I revised the content effectiveness principles to better fit the context of Product & Service emails.

TALKING TO ADMINISTRATORS
‍
Methods: Survey, Interviews, Heuristic Evaluation

Survey with 100+ Salesforce Administrators

What did I do?
Acting as a part-survey and part-screener, I built a survey gathering details on participant criteria (What size is their company/organization? Are they an Admin? What support level are they on?) as well as asking initial insights about their behavior and past experience when interacting with Salesforce Product & Service emails.

Why incorporate a part-survey part screener format?
As this was my first time talking to Admins, I wanted to make sure I was properly equipped to handle these interviews. By incorporating a screener that gathered additional information about the Admin experience and their perceptions of these emails, I was able to go into interviews with a better background knowledge on this space. This was also a great way to gather some supplemental data to help in forming interview questions for follow-up.

1:1 Deep-Dive User Interviews

What did I do?
After screening through 100+ screener participants, I selected 10 Salesforce Administrators to talk to through 60-minute 1:1 remote interviews. Towards the latter half of each session, I conducted a Heuristic Evaluation activity to further understand and observe their perceptions and understandability of these emails over a series of metrics.

1. Over screen-share, each participant read two Salesforce product service emails that differed in style/content.
2. Following each email, participants rated a series of statements created from the revised content effectiveness principles and “Jobs-to-be-Done” list. This aimed to quantitatively measure the effectiveness of the email content.

Why design a 1:1 interview with a Heuristic Evaluation activity?
In order to dig in deeper on the Admin's experiences with these emails, a 1:1 interview allowed me to probe in further. To answer the research question, I also needed to measure how "effective" these emails might be: leveraging the revised content effectiveness principles to build a Heuristic Evaluation activity allowed me to properly gather this data in a more structured way.

TAKEAWAYS

Some Learnings...

While I was at Salesforce, I actually kept a doc of learnings I had throughout the internship — every time something stood out or was meaningful, I jotted it down. Here's a few things I had written down at the time.

Conducting research can be ambiguous; it's important to learn how to embrace ambiguity.

Embracing ambiguity is important when conducting research. This project was nuanced and vague from the start, and having an open mind (instead of stubbornly going down one path) was critical, as the more we uncovered about the problem, the more we could figure out where to go next.

Narrow your research scope by cutting down on what you don't need to know.

And along with that — knowing which rabbit hole you shouldn't dive into is a skill to learn! Salesforce is complex. I was absorbing tons of new information every day, but with a limited amount of time, I had to learn to figure out quickly what was noise in order to effectively answer this problem statement within the 12-week time frame. (ex: Do I really need to know everything about Astro ↗ ?!)

MUCH THANKS

Many thanks to Robin, the best manager, mentor, and someone I could just talk life with, and Arun, a wonderful project sponsor and collaborator! Grateful to the Content Experience, Technology Readiness & Communications, and Research & Insights teams for a wonderful and growing summer.