Putting Voice behavior into context

Timeline: Fall 2023

Team: 1 product manager plus Zone Product lead

Question: How do providers in Orthopedic practices use Dictation and Ambient tools to create their clinical documentation?

When I first started working with the team, they were focused on the Ortho specialty, which were heavy users of both the mobile app and Voice services such as Dictation. I started with three site visits to clients in Washington State to observe mobile and dictation use in context.

Three site visits let us see how things differed — and stayed the same — across multiple clients.

The site visits helped contextualize problems the team had only seen in Voice of the Customer feedback by mapping them to the user’s daily experience. They also helped the team understand exactly how dictated content in the note combines with content from accelerators and content put in by medical assistants.

I use workflow/process maps like this one to help teams understand how activities and content map against the user’s actual workflow.

Evaluating the Ambient Notes experience in context

Timeline: Summer 2024

Team: 2 product managers, 1 designer, 1 product architect

Question: How do Ambient Listening apps work for the providers using them today? What will differentiate our offering from the others?

The following summer, the team was hard at work integrating Ambient listening platforms into the Mobile app as part of the upcoming Ambient Notes launch. They had done usability tests on Figma prototypes to validate the interaction model. However, they hadn’t been able to see how the actual models held up against real-world use. I was called in to help a few months before our launch at Thrive, our annual customer conference.

We started with a set of site visits to clients in Texas and North Carolina, where a visiting product manager and I used a demo version of the app on our phones while shadowing providers. After getting consent from provider and patient, we recorded the visit using the Ambient feature, and asked the provider to give feedback on the generated summary.

A demo version of the app allowed us to test the Ambient feature against real visits directly on a test patient. It also gave us the ability to see what the model created against a transcript of the visit.

After each visit, I wrote up a complete report in Confluence to send to the product team in Pune. By the next morning, they were eagerly consuming the findings and logging JIRA tickets for bugs we had found in testing.

The team was surprised and delighted by how quickly I could turn around site visit findings.

A critical finding

The most important thing we saw in site visits wasn’t about the quality of the Ambient summary. It was about how providers understood this new technology, and the potential disaster waiting if the technology failed them. Providers were excited about the potential for an Ambient solution that delivered the summary directly into their clinical note. However, multiple issues with the UI and the limitations of cell networks in doctor’s offices led to repeated failures. This led to visible panic in the providers testing the system.

The team was surprised and delighted by how quickly I could turn around site visit findings.

This finding led the team to focus design and dev effort on UX copy, error messaging, and data loss prevention. We also made sure that training and marketing materials for the feature recommended using Wi-Fi for the app instead of the user’s cell network.

Testing the model’s output across specialties

After the site visits, we needed to pressure test the model’s output beyond Orthopedics. I worked with ChatGPT to brainstorm scripts for common clinic visit conversations, then tapped our Patient Safety clinicians to record the visits, using my improv skills to riff on the script. Finally, I ran the recordings through the Ambient functionality to get the resulting clinical note.

These were two of what ended up as ten mock visits we were able to generate notes for. Some I recorded, others were donated from salespeople who had done demos already.

Once we had the content, a designer and I mocked up each visit in a comprehensive Figma prototype, using variables for each data element to keep things easy for participants to switch between visits as needed for testing.

A complex Figma prototype with variables helped us iterate after each week’s testing, so we got a validated solution to engineering teams more efficiently. Recreated to preserve confidentiality.

The full test protocol led participants through the process of recording the visit all the way to reviewing the output. They were then asked what they’d need to do to finish the note, and gave feedback on some enhancements we were exploring.

The lead designer and I tested the finished prototype with over 20 participants of varying specialties over 4 weeks. After each week’s calls, we would meet to review the findings, and decide what to change based on what we saw. The lead designer would make the changes, and testing would begin again. The designer worked closely with Product during this time to prioritize fixes we were confident in along the way.

By the end of the four weeks, we had a clear design direction for launch.

Validating at scale during our customer conference

In fall of 2024, Ambient notes was launched at Thrive, athenahealth’s annual user conference. I was there with several members of the product team, who flew in from Pune. They worked the Ambient booth while I stayed in the Innovation Lab, testing our prototype at scale with a constant stream of attendees.

The Ambient Notes team was the belle of the ball at Thrive; we stayed busy answering questions and further validating the solution with customers.

The conference also gave us an opportunity to validate fast-follow hypotheses on the solution, including user management, ambient-inferred diagnoses, and preference settings.

Outcome

The Ambient Notes launch was objectively successful, and the functionality is popular among our customers. As of April 2026, athenahealth's own Ambient Notes page reports the following metrics:

OrthoLoneStar, an early adopter, saw the following outcomes: