How Raspberry Pi tested kapa and Inkeep head to head
"I'd built a retrieval engine myself, so I knew what running one properly would take, and that it was not worth for us to maintain. When we decided to look for an external solution, we chose Kapa over Inkeep because they performed better on accuracy, and supported all of our sources"
Gordon Hollingworth
Chief Technology Officer
Challenge
Documentation and other information for more than 75 million devices is spread across HTML docs, PDF datasheets and product briefs, a monthly magazine, and GitHub repos. Keyword search only covered the HTML, and usually the answer to your question was not given on just one page, but instead spread across domains.
Solution
Perform a public A/B test of Kapa against Inkeep on the Raspberry Pi blog, to determine the best solution for knowledge retrieval. Now docs, PIP briefs, magazines, PDFs, and GitHub repos are crawled daily, so every source stays up to date and Kapa helps thousands of users self-serve on the documentation.
Results
18,000+ documentation questions answered since launch
% decrease of uncertain answers from 21% doing public trial, to 10% in production
24% of questions answered in a non-english language
88% self-serve rate among users
68% of answers included non-documentation knowledge as a source
THE SITUATION
Documentation spread across HTML pages and hundreds of PDFs
Evaluating vendors
A live A/B test on the Raspberry Pi blog
On 13 May 2026, Raspberry Pi published "Search our documentation by meaning, not keywords".
The post explained how RAG works, then invited readers to try it for themselves. Half of visitors got Kapa embedded in the article and the other half got Inkeep, both connected to the same sources: the documentation site, PIP whitepapers and books, and selected GitHub repositories. Readers were asked to rate each answer with a thumbs up or down and to say more in the comments.
The trial produced real questions from real users at volume, roughly 1,000 in the first few weeks, and it showed the team, in public, where the documentation had gaps.
Why Raspberry Pi chose Kapa over alternatives
The deciding factor was the depth and breadth of sources that Raspberry Pi could ingest into their knowledge base. A large share of what engineers need sits outside the actual documentation webpages. PIP product briefs, datasheets, and magazine issues hundreds of pages long. Kapa ingested that content alongside the web docs and GitHub repos and answered from all of it with source links.
The trial also confirmed the accuracy behaviour Gordon cared about. Answers cited the exact source, and when the documentation does not cover something, the assistant says so.
THE SOLUTION
From A/B test to live in six weeks
Raspberry Pi went live with Kapa on the documentation site in mid-June 2026. Readers type a question into the docs header and get an answer built from the documentation, with links back to the source. The sources keep themselves current through Kapa's built-in connectors: the docs site is crawled directly, the Product Information Portal is crawled from its sitemap with a part-number filter so only public briefs and whitepapers come in, and Official Magazine PDFs are picked up weekly with answers linking to permanent URLs. GitHub repos such as rpi-image-gen round it out, so code and documentation stay together.
Play video
THE BUSINESS IMPACT
From A/B test to live in six weeks
Since going live on the documentation site, Kapa has answered more than 18,000 questions across roughly 10,000 conversations, and the volume has settled at more than 4,500 questions a month.
Questions arrive in more than 20 languages, after English, the most common ones are Japanese, Spanish, Chinese, French, and German, all answered from the same English documentation in the actual language the user is asking in.
88% self-serve rate. Kapa's self-serve rate estimates the share of conversations that users resolve on their own, without needing help from the Raspberry Pi team.
Uncertain answers halved. During the public trial, about 21% of answers were flagged as uncertain, meaning the documentation did not clearly cover the question. That figure is now around 10% after re-iterating on the sources and closing out the knowledge gaps flagged by kapa.
Insights for the documentation team. Every conversation is stored with an uncertainty flag and user feedback. Raspberry Pi's documentation team uses that record to see what people are trying to do, in their own words, and where the written docs fall short.
The broader lesson from Raspberry Pi's approach is the evaluation itself. Rather than choosing a vendor from a feature list, the team put two products in front of real users, measured the results, and published the whole thing. It is a model any documentation team can copy, run the tools on your real content, with your real users, and let the answers decide.
