Taxonomy plays a vital role in most information architecture or user experience initiatives but does not lend itself well to visual representation. This can lead to taxonomy being overlooked during the design process. How do you convince decision makers of the importance of taxonomy when one of the main deliverables is a spreadsheet? Recently, while working on a project with Fell Swoop, I was introduced to a usability methodology called tree testing that allows you to test the effectiveness of a taxonomy and visualize the test results.
I had the opportunity to work with Matt Turpin at Fell Swoop and help redesign the FAQ page taxonomy for the website of CLEAR.com, a wireless communications service provider. First, we took a close look at their current taxonomy to understand how they were currently being grouped. I read many of the individual FAQs and sorted them into new categories to make the first draft of the new taxonomy. We then began the first round of user testing using OptimalSort, an online card sorting service from Optimal Workshop. A card sort is a usability testing method where users are given topics and asked to put them into groups. While the results that OptimalSort gave us were helpful, they were challenging to decipher and most of them would be confusing for a client.
Figure 1: A screenshot from Optimal Sort showing the affinity diagrams that users made between various subjects.
Figure 2: A screenshot of a dendogram from OptimalSort showing a large scale view of the clusters of similar terms made by the users. While this is very useful for a taxonomist, it is challenging for a client (or anyone) to decipher at a glance.
We incorporated some of the user testing data we obtained from the card sort and created another draft of the new taxonomy. For the next phase of user testing, we conducted the first of two tree tests using Optimal Workshop’s TreeJack program. A tree test is a reverse card sort where the participants are asked to find information about particular subjects by navigating through the taxonomy to the category where they would most likely find the FAQ that would answer the question. Tree tests are a very effective method for seeing how usable the taxonomy will be for users to navigate.
The information about the tree test that TreeJack provides is exactly the type of actionable user testing feedback that a taxonomist can use and a client can intuitively understand. Everyone can clearly see how usable the taxonomy is on the whole, and drill down to see what parts are working (or not), and where the users think that the information is.
One of the questions we asked during the tree test was to have users try to find information about how much data they have used:
Figure 3: Screenshot from TreeJack showing a summary of users’ answers to one of the tree test questions.
For each question, TreeJack provides a pie graph showing the success rate of each user, a cumulative score of how successful they were, if they were able to navigate directly to the FAQ and how long it took them to make their choice.
You also get a more detailed breakdown of the navigation that the users took to get to the node they chose as the answer:
Figure 4: Screenshot from TreeJack showing the two correct paths to the answer by navigating through the taxonomy, the other places users explored, and pie charts showing the choices of the users.
When we examined the paths that the users were taking, we saw that most of them expected the information about data usage to be under “My Account Management” whereas in this draft of the taxonomy, “Data Usage/ Allowances” had been placed under “Billing and Payments” and “Common Questions For New Customers.”
We incorporated this and a lot of the other information we received from the tree testing into the next version of the taxonomy. When we tested again, the results were greatly improved:
Figure 5: Screenshot from TreeJack showing a summary of users’ answers to the same question during second round of tree testing.
Upon further examination of the taxonomy, we realized that the My Account node was confusing to users on many levels so we eliminated it and promoted the Data Usage/ Allowances to become a top-level term:
Figure 6: Screenshot from TreeJack showing the users’ navigation through the taxonomy from the second round of tree testing.
As a result of making Data Usage/Allowances a top-level term, users were now quickly able to find the answer to this very common question rather than having to search through many levels of the taxonomy to find it. Given the fact that the majority of users had been able to find the answer rather quickly and effectively, we were able to discern the fact that most of the other navigation choices were likely due to one user.
The most important infographic TreeJack provides as far as presenting to the client is the bottom line: how effective is the taxonomy?
The first round of testing showed that the taxonomy was that users were able to find the right FAQ 57% of the time:
Figure 7: Summary from TreeJack of the overall success of the first round of tree testing.
Using the feedback from the first round of tree testing, we were able to get the Overall Success rate up 20%:
Figure 8: Summary from TreeJack of the overall success of the second round of tree testing.
The overview infographics cut right to the chase and provide the client with a way to measure the overall effectiveness of the taxonomy while validating the expense of the user testing. The only thing that was missing from the project was a tree test before work began, which would have provided a snapshot of the current state of the taxonomy to contrast against the improved effectiveness of the one that was created. I will definitely be doing that next time!
A taxonomy is the “skeleton” of any information architecture project, it creates the navigational structure for an organization’s information and makes it findable. You can see a wireframe and visualize how the site is going to look, but all too often the taxonomy deliverable is a spreadsheet. It may be the best site navigation ever devised, but the last thing anyone wants to look at during a meeting is another spreadsheet.
Tree testing, and the visualization provided by TreeJack solves the issue of how to visually present a taxonomy to clients. Armed with this valuable new tool, taxonomists can finally show, not just tell clients how vital an effective taxonomy is to the success of their information architecture initiative.
Contact Brian LeBlanc at firstname.lastname@example.org or see him on the web at www.infogrationconsulting.com