User blog:Pikushi/Updates on Quick Answers and Adding a Vetting Period
Template:Featured blog
We introduced Quick Answers back in September when Brandon shared a comprehensive blog post about the tool's development up to that point. If you haven't read it yet, take a moment to check it out!
Let’s start with a quick summary of what we’re talking about: Quick Answers.
Quick Answers is a product that reformats long-form information from wiki pages into a Q&A (questions & answers) format that would appear as a module on the wiki page it’s based on. These modules directly address and answer questions about the subject of the page.
There are a large group of people who come to your community in search of the answer to a question or two. Many times, those people will stay to learn more information, but getting them through that first door into your wiki is a big step, and it is important to help them get that answer.
That is what led us to Quick Answers and using AI to create questions that reflect what people are searching for. The advantage of this approach is that the format allows for easy search indexing from Google, in a structure that Google finds really informative and likely to rank well. Better rankings from Google means more readers visiting and engaging with your community. Your content will be in front of more fans like yourself looking for answers and wanting to know more.
In our last blog, Brandon talked about how we would be introducing a way for admins to curate and control Quick Answers questions that will display on their communities. Over the past few months, we have been working on an Editor Dashboard, where admins can edit questions that have been loaded in. We are also adding a vetting period to give time for admins to review questions before they go live.
As a result of an extended and successful beta for the Dashboard, and introducing a vetting period for our admins to review new AI-generated question-answer pairs, we are now moving forward with rolling this feature out to more communities.
Experimentation & Learnings
The dashboard we showed you in that September blog has been tested for a few months, and the full feature (dashboard and question modules) has been up on 12 different wikis. We’ve been directly engaged with those communities to see their feedback, benefits, and suggestions for improvement. The participating wikis provided valuable insights, and we've learned a lot!
The vast majority of what we heard from these community members has been positive - the most common thing we heard was “Can we have more questions loaded in?”.
We also saw a SEO boost to wiki pages that had Quick Answers, so more people were visiting the wiki, as well as an increase in engagement, so more people were staying on the wiki once they were there. Specifically, we saw a 1.5% rise in views on pages featuring Quick Answers after their addition, translating to thousands of fresh visitors and new editors. We learned that Quick Answers were helping fans discover the content communities had worked so hard on, in ways that we had not been tested before. We understand that Google sees pages that are gathering more views as beneficial content and will more likely continue to scale outward and boost the page in search rankings, leading more fans to the wikis.
During this beta test, we also tested different methods for using AI to generate questions and answers. There were a few trip ups, but after testing, testing, and testing again, we are in a much better place than we were before with AI generation.
We heard critical feedback from users, too, focusing on minor design & functionality improvements we can make to the tool. We heard concerns that there was no way for admins to check the Quick Answer questions before they went live.
A Community-Based Pivot
We took that concern and made a big change - we introduced a vetting period.
Now, when a question gets preloaded into the dashboard, the admins and other trusted editors will have time to review a question before it goes live. That way, the community has a clear amount of time to be able to correct any issues with a question before it goes live, rather than having to correct any issues on the fly. This specifically addresses concerns regarding AI not providing a reliable question or getting the answer wrong. Ultimately, we want to make sure you have control over what content is on your community, including what questions and answers the Quick Answers tool provides. This vetting period has been implemented on four communities out of the beta test wiki group, and we are working with them to find a golden spot for how long this vetting period should be as well as how many questions are loaded in at a time.
If no editor makes any edits to the Quick Answer, it just goes live once the countdown has run down. If an editor saves a change to a new to-review QA, it goes live right away, showing a "last edited" date instead of the publication countdown.
Here are two examples of editing the Quick Answers questions:
(Both videos show editing without the vetting period added.)
Thank you RinasaurusRex from the The Last of Us wiki for showing how to use the dashboard!
Next Steps
As I mentioned above, we are working with the beta test wikis to test this vetting period model. After we finetune th details, we will begin to gradually role out the Quick Answers tool to more wikis. I want to emphasize the word “gradually” - we want to make sure that wikis that receive the QA tool are aware of what is happening and understand how to use the dashboard. The tool will also not go live everywhere all at once. We will be turning on the tool to select wikis when their preloaded questions are ready for review.
Our team, including myself, will reach out to admins on wikis where the tool will be turned on to keep them informed. We want to ensure you are well-informed and provide a channel for any feedback you may have after using the tool.
We will be holding an AMA for questions about Quick Answers in the Fandom Discord on February 8. Check the announcements channel in the server to see the time in your time zone.
If you have any questions, please feel free to send them our way via your Wiki Representative (if you have one) or a Support ticket. Check our the Quick Answers Help page, too!
User:Pikushi/Profile
Template:Blog footer