In the world of SaaS, understanding why our customers leave us is just as important as knowing why they chose us in the first place.
That’s why cancellation surveys are such a great opportunity to learn more about your customers and what was really going on when they decide to cancel.
Writing questions that gather this valuable insight isn’t easy, however, but if you can get it right, the data collected is one of the most straight-forward opportunities your business can have.
So today, I’m going to walk through an exit survey that was sent to me recently and discuss how the wording and content of the questions can be adjusted to achieve maximum impact.
But first — a quick disclaimer before we dig in. This analysis is not in any way meant to disparage the very hardworking folks who created this survey. My goal is to offer ways to enhance your own exit surveys based on the real-life ones that exist in the wild (and who knows — maybe give FreshBooks some completely free and equally unsolicited advice 😂).
I am also SO HAPPY that they are sending one in the first place since so many teams skip this step. Most SaaS companies feel that collecting a cancellation reason at the moment of churn is enough, but my work in retention has proven that this just doesn’t provide enough context for all the real reasons behind the churn (but this is a whole other post I need to write at some point). Regardless, still awesome FreshBooks is even collecting this data in the first place.
Alright let’s dig in with the first question.
1. “Which of the following factors influenced your decision to cancel your subscription with FreshBooks? (Select up to your top three answers that apply)”
The survey starts off strong. FreshBooks gets it — this first question tells me they understand there are many reasons why someone decides to cancel their subscription.
From my work in JTBD and retention, I can both appreciate and confirm the way this question is structured because customers rarely ever cancel for just one reason. Usually it’s the culmination of several reasons over time until one extra thing topples the whole thing over.
Think of the game Jenga where one weakened area with one extra block on top makes the whole tower come down.
By asking for the top three reasons, FreshBooks can dive deep and try to understand the various reasons that led to the final breakup.
While I don’t have any major complaints about this question, there is one underlying angle I believe is missing: the bigger picture.
In my own case, I left the FreshBooks platform because I was tired of managing both Xero and FreshBooks. There wasn’t any one singular thing FreshBooks did wrong. This question forces users like me to fit into the box of users who use the platform for one or two things and leave it for an alternative that does the same thing.
This sentiment reappears in other questions down the line, and I think it’s a missed opportunity to gather more context about the cancellation in the first place.
Recommendation: Add an open-ended question asking “What led you to cancel FreshBooks?” so I can give more context about what was going on here.
Understanding this backstory is critical because it gives product owners and GTM leaders a sense for what is controllable along with how qualified the churned customer was in the first place.
2. “After leaving FreshBooks, what platform did you switch to?”
This question makes sense to ask, but I find it interesting because it assumes that I’m leaving FreshBooks for something else instead of potentially already having another solution in place (which admittedly might not be that common, unless of course it was and you’d really want to know that).
Given my use case for the platform, I think it would be beneficial for FreshBooks to question that customers may not be replacing but reallocating their JTBD to other tools.
But I select “Xero” since this is kind of the case even if technically not true since I was always a customer of Xero.
Recommendation: Add a question asking what customers were using or doing before FreshBooks so it’s clear if there are more users like me who used FB in addition to other solutions.
3. “What does Xero have that FreshBooks does not?”
Like the second question, this question assumes that someone like me is leaving the platform for its faults rather than a larger, overarching context switch. For example: I’m tired of managing two platforms.
This direct comparison question seeks to identify specific features or services that may lead customers to prefer Xero over FreshBooks.
While it's valuable to understand where FreshBooks might be lacking from a product standpoint, it’s also important to consider that a cancellation might not always be due to product features alone but could involve broader operational or strategic needs.
For example, perhaps my business has doubled in size in the last six months or maybe I was going through a really slow growth period. Again, this would be important for product and GTM leaders to know because the implications change based on what’s within the control of the FreshBooks AND also what can they anticipate as a potential cause of churn (and therefore have a planned reaction for churn mitigation).
4. “How satisfied were you with the clarity of our subscription plan options?”
At first blush, this question seems fine and I totally get what this question is trying to accomplish. I think it wants me give them a sense of understanding: “Do I understand the plans?”
But I’m stuck on the word “clarity.” Am I being asked about the clarity of the content of the plans themselves or the way they're presented on the website?
Recommendation: Get specific about potentially vague adjectives and nouns used in questions like this. Otherwise, it can be misinterpreted and give you false positives (or negatives).
Because I’m not exactly sure what the survey creator is looking for here, I'm most likely to respond somewhere in the middle ground — aka the central tendency bias.
The central tendency bias suggests that survey takers are more likely to choose the middle of a scale, especially if they’re unsure of what the question is asking or there’s a heavy cognitive load in understanding the question. This means there may be a lot of respondents selecting “neutral” for this question, which is unlikely to accurately reflect their experience.
So we want to try to avoid the possibility of confusion or thinking too hard while answering the survey.
6. “Tell us about your utilization of the features included in your plan.”
Jumping ahead to question number 6, we run into more ambiguity and cognitive overload. FreshBooks is known for its invoices, but it does a bunch of other stuff too.
This question is asking about how much I use the features, but I’m left to wonder what the question means by “features.”
How do I, the survey taker, define “feature”?
But also, shouldn't FreshBooks already have a pretty solid understanding of utilization?
Assuming they’re using something like Amplitude or Mixpanel, FreshBooks should have a strong and clear understanding of how many of its users are using feature X, feature Y, both, or neither.
That leaves me to think that the survey creator wants to understand my understanding of what I think a feature is and what my concept is of how much of the product I use.
Very meta! But that’s my best guess. 😂
Recommendation: Rewrite this question to say: “Select the features you used most when you were a paying customer of FreshBooks.” With this adjustment, your survey taker doesn’t have to try to define “feature” because you have already provided the options for them to select from.
You can also create more of a matrix where you ask what features the user used, but also if they’ve never heard of it or if it just didn’t apply to their business. That way, you don’t leave the word “feature” up to interpretation of the survey taker.
The bottom line — if you ask survey takers to perform mental gymnastics, you’re just going to end up with survey churn or inaccurate results.
Final Thoughts
Overall, FreshBooks’ cancellation survey is a good one. It's clear they've got a good understanding of their customer base and the reasons behind churn. However, there are a few tweaks that could polish up the survey experience and enrich the insights they gather:
Be careful of cognitive overload in survey questions. If respondents have to think too hard about what you’re asking, they may just select the middle option (central tendency bias) or make stuff up to make the question end quickly.
Specificity is your best friend. Get as specific as necessary in your questions so you can gather stronger, more accurate answers that lead to better insights. If there’s too much room for interpretation, you may also get a bunch of non-answers or false positives/negatives.
And finally, make sure to gather context before asking more pointed questions. This will help ensure you’re not making too many assumptions about a respondent’s behavior and therefore drawing incorrect or half-assumptions. You’d be surprised at how little things like this can snowball later into an entire insight that has a massive impact on GTM or the product (but actually is only partially true). In this case, they were the assumptions that I was switching to a different platform rather than already having used a competing product in the first-place and there was no underlying story to why I chose the cancellation reasons that I chose.
All in all, still amazing they send this in the first place. What was your biggest takeaway? Does this make you look at your exit survey a little differently now? ✨
Survey Teardown: FreshBooks' Cancellation Survey