How ClassHero reversed a 48% drop in conversions

Meet John

John Gilmore is the Sales Operations Manager at the startup ClassHero. John’s been at the company for around two years, helping the team grow.

As with most startups, there’s a lot on John’s plate, but one of his key tasks is finding opportunities for product improvement. 

John shared that he’s “in charge of growth and tasked with bringing growth priorities to the product team for fitting into their roadmap.” 

He keeps a close eye on the site metrics too, making sure conversions are happening as expected.

Meet John’s worst nightmare 

Every Monday, John looks at the onboarding rates on the site (that’s the % of new users who complete setup).

Onboarding is a multistep process, and usually, about 75% of new users complete it. Those who don't complete onboarding in their first session rarely come back to the app.

When John sat down for work one Monday morning in October, he couldn’t quite believe the stats. 

The onboarding rate had dropped from 75% to 39%. That’s a 48% drop. 

“It was brutal. Users who DON'T finish onboarding in the first session almost never come back to our app. And worst of all, we'd made a bunch of product changes, so we didn't know if it was a technology problem or if users just hated the changes.”

Fortunately, John had installed Hotjar a few months earlier. He went straight to the session recordings to try and spot the problem. 

“On that morning I started watching a crazy amount of videos. I needed to find what was causing the problem, otherwise, we’d be bleeding sales.”

Identifying the issue

John watched session recordings of pretty much every user on the site that morning. He saw folks failing to finish onboarding and quickly identified a few bugs. 

Overwhelming autocomplete 

During the onboarding flow, users needed to search for their school. 

The team had built an auto-complete feature to help users search through the thousands of schools on the ClassHero database. Sounds great, right? 

Well, John spotted a problem. 

“Users would enter the first two numbers of their school ZIP code and suddenly over 4,000 results would appear. I expected users to ignore this, but many didn’t. They simply clicked through hundreds of schools to try and find their own before giving up.”

Fortunately, this was an easy fix. But John wasn’t done there. After watching a few more Hotjar recordings he spotted another issue. 

Dropdown difficulties 

"I was watching users select from a dropdown form and noticed a serious issue."

Like most onboarding forms, ClassHero’s contained some dropdowns that would allow you to select an answer to complete the step. 

But after a recent update, the dropdown wouldn’t minimize after the user entered their answer, hiding the next question from the user. 

“Users would get a pop-up saying ‘Please fill out every field’. But they couldn’t see some of the questions because it was hidden by the drop-down.”

According to John, spotting these issues was “very time-intensive, but very valuable.”

Missing grade 

“Many teachers sign up using the single sign-on service.” 

SSO (single sign-on) is a way for users to sign-up to a tool using an existing account, like Google. It saves heaps of time and makes onboarding a breeze—or so you’d think. 

"During onboarding, teachers would try to change their grade but the system wouldn't let them. Then they couldn't assign work for the proper grade. Assigning is the last step of onboarding. We had 6th-grade teachers being asked to assign to the 1st-grade."

John had spotted a number of bugs, and the next task was to share them with his team. 

For a lot of issues, this was simple. 

“For the obvious bugs I can just send my developers and my executive team a list of recordings. As soon as they see it they understand the issue.” 

But John wanted to make sure the team was fixing the right issues first. He needed a way to group insights and quantify them. That’s where Hotjar’s latest feature, Highlights, came in.

Highlighting the solution 

With a 48% drop in conversions, John needed to move fast. Fortunately, just a couple of weeks earlier Hotjar Highlights had launched, so John had a way to group the issues, prioritize them, and share with his team. 

“Hotjar Highlights enables me to tee up my ask: ‘Here's a 30-second Highlight of the issue. Here's a Collection of the same issue across multiple users for you to view at your leisure.’ That drives near-immediate understanding and clarity around what I'm asking for and why.” 

John built highlights for all of the core issues he’d spotted. He tagged recordings with the problem he identified and sent them over. 

John shares how this approach led to an almost instantaneous solution: 

“Once I showed the Chief Product Officer he got it. He said ‘Yeah, this is the top priority. Fix this before we carry on with the rest of our backlog.” 

Hotjar Highlights helped John push his fixes right to the top of the priority list.

But a word of caution to others who follow John’s approach: it’s so impactful it can feel like cheating. 

"I knew my role was limited to one area, and seeing that Hotjar allowed me to convince the entire company to my priorities made me feel almost like I was cheating! I mean, surely others had to pass on other high-priority issues because they couldn't explain their own problems as well. They weren't yet using Hotjar."

John has some tips for other Hotjar users starting to use Highlights:

“Personally I love using Highlights and Collections together. I see each highlight as a self-contained explanation for the case I’m making to the product team. I’ve got the video. I’ve got the tags if I need them. I’ll add comments and a thread for a response. 

There’s no risk of distraction and minimal lift to get my stakeholders directly to the central issue. 

And then the Collections view basically creates a visual report of all the evidence I have for a given case. Here's how much this hurts (one Highlight); click on the rest of these if you can stomach it (a Collection of Highlights).” 

Finding tomorrow’s bugs today 

John doesn’t rest on his laurels. Even though the 48% onboarding drop was reverted in just a week due to his changes, he’s well aware that a similar issue might occur again. After all, the team at ClassHero is constantly launching new features and building new functionality. 

To keep on top of any new issues or opportunities that pop up, John uses Slack. 

“It’s important for me to see all the priority recordings in Hotjar. Typically that’s everyone who signs up.” 

To make sure he’s on top of all priority videos, John has set up a Slack channel that shows recordings of every user who has dropped out during sign-up. 

“Using Hotjar filters, I was able to home in on every recording of users who fail to onboard and use the automated integration to send them straight to Slack. That way, as soon as something goes wrong, I’m able to see the issue in Slack and coordinate with the team.” 

But it’s not just bugs John has discovered—it’s opportunities to improve, as well. 

Lightbulb moments

Through Slack channels and Highlights, John has got a firm hold on the user experience. He’s able to understand what users go through at each stage of the funnel. 

This helps him spot little things that could dramatically improve the user experience. 

"After watching hundreds of recordings I started to spot a pattern.

See, teachers assign math homework in our app, which students log in and complete, and teachers then review. It's our most central feature. 

But first-time teachers didn't want to send students into practice until they knew exactly what it was going to look like for students. 

And most first-time users could not find anywhere to see this preview.”

Hotjar recordings showed that users would click on a bunch of different links related to homework, but increasingly rage clicked or grew frustrated.

"I could tell they weren't finding what they wanted. But the lightbulb moment was when I saw the very few who DID find a way to preview homework. Suddenly they would move quickly toward inviting students into the app. Which is the next critical milestone for post-onboarded users: students practicing."

Fueled with this latest insight, John shared his findings with the product team who promptly updated the layout to make the homework preview section more visible so users would take the next critical step: inviting their students in for practice.

"Since the change, the percentage of new teachers who hit the critical next milestone after onboarding has more than doubled. It was 30% and jumped to 65%. 

Many teachers simply need to see this preview before they are comfortable sending students in. We learned that from Hotjar."

–  

Inspired by John’s findings? Install Hotjar yourself and try it out. John started on the free plan and installed the tracking code on his own. “I didn’t want to waste developers’ time, so I installed it and went from there.” It’s that simple.