Your Users are Telling You What's Missing from Your Docs - Here's How to Listen

Users are one of the most important feedback loops that you can have with your documentation - taking advantage of this feedback loop to improve your documentation, and in turn, your product, can provide real business value.

Kapa.ai offers various metrics and analytics that you can take advantage of to improve both your documentation and your product. Remember, improving your documentation only makes Kapa better at answering your users questions - further enabling Kapa to answer questions from your users.

What Users are Asking

When a user has a question for your team, there's usually a few reasons why they've asked:

1. They haven't read the documentation

2. They haven't found the answer

3. They didn't understand the answer

All three scenarios lead to a customer contacting your support team, or using your Ask Ai bot, to get more information or to clarify. While we can't help it when customers do not read the documentation, we can improve their experience of finding the answer or understanding the answer.

Top Questions analyses your conversations in order to surface themes from the conversations on topics that your users are asking questions about - these topics are great for understanding what is important to your users.

From a product point of view, this data can highlight features within the product that are being heavily used, or have room for improvement. For example, if a customer is asking for clarification on a setting of a feature within your product, it may be worth investigating if this is something that may be worth making clear within the product itself.

What Your Docs is Missing

Creating documentation takes a lot of iteration - there will always be gaps, or questions that your docs can no answer for your customers. Identifying these gaps can be difficult and often requires analysing the support tickets that your team are receiving.

One of the advantages of using Kapa to help answer your customer questions is it's ability to say "I don't know" when asked a question that it is unable to get an answer from your documentation on - it won't try to make up an answer when it's not sure, this leads to inaccurate answers. These conversations are marked as 'uncertain' which then feeds into another one of our analytics features - Coverage Gaps.

Coverage Gaps is designed to identify gaps in your documentation where you may need to investigate if there is room for improvement. This gives you the opportunity to improve your documentation which will in turn improve the quality of answers that Kapa can give next time your users ask similar questions.

You can see in the example below that our customers ask about customisation features and a new data source - in these examples, we would go ahead and update our documentation to either add information about the features, or we may add a custom answer that tells users that this is not available, if appropriate.

What Users are Telling You

Conversations themselves can be a great source of feedback - especially if a user has provided feedback. The Conversations view within the dashboard allows you to filter conversations, helpful if you have a large volume of conversations. Some helpful filters for reviewing feedback are 'has comments' and 'has downvotes'.

If a user has left constructive feedback as a comment to a response that they were given from Kapa.ai, this can be used to inform product or documentation decisions - for example, they may leave a comment seeking more information. While this may also be picked up by coverage gaps, it's still worth reviewing to check if there are other areas of improvement.

If the user has left a downvote, this may involve additional investigatory work. Users may leave downvotes for a couple of reasons, either they don't like the answer, despite it being correct, or the answer is correct. You may need to read more of the conversation to understand the users dissatisfaction here. Tip, concentrate on the conversations that are also tagged as 'uncertain', these are more likely to be gaps in your knowledge base.

Using These Analytics

Now that you have the analytics to hand, it's time to do something with them to provide business value.

For Technical Writers

In the hands of a technical writer, this analytics data may be used to improve or add new documentation sources to your documentation to help with customer questions. This may range from a simple one-liner that explains that a feature does not exist, a re-wording of existing data, or the creation of a new knowledge base article.

For Product Managers

Understanding what your users are asking about shows what's important to them. When combined with product analytics, this can give product managers more insight on how real users are using the features that they are launching and how they can iterate on them to deliver improved features.

Conclusion

Understanding your users is core to improving your documentation and your products. Using Kapa's suite of analytics features lets you understand what your users are asking about and help identify gaps in your documentation that may be plugged to help improve the user experience. As an added benefit, improving your documentation will improve the answers that Kapa can give back to your users and lead to a further reduction in support tickets that your support team need to worry about.