AI Knowledge Base Experiment
Running AI-powered experiments to get more admins and agents to adopt Jira's Knowledge Base.
What is it?
This experiment targets to encourage admins and agents to create more articles and publish in Knowledge Base. Right now, they often avoid writing and updating it as it takes too much time.
My contribution
Led discovery research to uncover pain points
Facilitated ideation workshop with the triad
Translating ideas into design concepts for experiments
Company
Atlassian
Role
Product design lead
Time frame
Jan - Apr 2025
Why the experiment?
Currently, admins and agents weren't publishing and updating articles regularly, which means there'a not enough good quality content getting fed to the Knowledge Base. This results in AI Virtual Agent not generating good and accurate answers.
I initiated to start understanding what was stopping admins and agents from writing, and as a business focus we wanted to see if AI could make the process easier.
SEQ testing insights
To hear from our users directly, I ran user interviews and SEQ testing. I organised the research, wrote the discussion guide, ran every session and pulled all the insights to share with the wider team of 30 people.
What we learned was very simple — writing an article from scratch is overwhelming and takes too long. People either save notes elsewhere or avoid writing altogether.
Forming the hypotheses
Fail fast, learn fast
From that, we broke it into smaller hypotheses for each experiment. I designed all the prototypes, explained the flows to engineers, and made sure everything fit technically.
Our hypotheses include:
By using AI generated articles, we will reduce effort so they create more articles.
By using AI prompts, we will give control so they publish more confidently.
By nudging from tickets, we will remind users so they create articles in context.
By inserting snippets, we will save time so they create more articles efficiently
04 Snippet
Close collaboration
Once the designs were ready, I worked with the engineers to launch the experiments to 10% of users first and move to 50%. The goal wasn’t to make them perfect — just real enough to learn what worked and what didn’t.
I also worked with data analyst to define what we were measuring and track: publishing rate, AI draft usage, when users edited the AI-generated articles etc.
Experiment & results
The experiment gave us clear result of what actually helped them and what didn’t.
This includes:
AI-generated articles was a winner, users published 25% more articles
AI prompts was slowing down agents' work, so we changed the logic so they appear at the right time.
Prompts and snippets overlapped with Rovo's functionality, so we merged them to avoid duplication of tools.
My learnings 💙
Small, simple experiments can answer questions faster than long projects.
Timing matters — showing AI at the wrong moment can hurt the experience.
Collaborating with engineers early makes launching experiments smoother.
Testimonials
I would like to thank Lorraine for being very quick in implementing additional designs related to "KB feedback from Admins" project, related to various scenarios. This would surely speed up the delivery. Thanks again!!
———
Senior Engineer, Knowledge Base
I've had the pleasure of working closely with Lorraine, and she has been an incredible design partner. She consistently brings strong design instincts and a collaborative mindset to every project. One of the things I really appreciate is her ability to quickly spin up mockups for sparring — this has been incredibly helpful in enabling the team to explore multiple approaches and land on the best solution for our users before moving into high-fidelity design.
Lorraine ramped up impressively fast, especially given the complexities of the knowledge base space. Her ability to grasp nuanced problems and translate them into thoughtful design solutions has made a big impact.
On top of all that, her communication is clear and effective, and she’s been a joy to work with. I'm genuinely grateful for all the hard work she's put in — it’s made a real difference.
———
Senior Product Manager, Knowledge Base