I’ve split the card sorting articles in two. The previous one covered what card sorting is, how it’s used and how you shouldn’t use it. If you haven’t read it and are unfamiliar with card sorting, please read that one first.
In this article I’m talking about the problems with card sorting.
There are lots of problems with every kind of testing methodology. So card sorting is not an isolated use case where it gets it wrong.
Accessibility
This was the point that started the thread I created on Saturday. The original tweet I’d seen was asking about the accessibility of card sorting for people who are blind or have a visual impairment.
I assume (and I might be wrong) the original poster was wondering about the accessibility of online card sorting tools.
That wasn’t where my mind went, however.
For me, there are two things that worry me with the accessibility of card sorting.
Ableist by design
Card sorting works on the premise that someone can see all the cards.
This is because there is usually a lot of information on a screen. Being able to see the information means we don’t need to hold it in our memories.
There is a hotly debated mathematical law, called Millers Law. It states we can only hold 7 +/-2 things in our working memory at any one time.
Arguably, with card sorting we are trying to reduce multiples of 7 down to that magic number (or less).
If someone cannot visually see those cards we have made the whole thing almost impossible.
I’m aware that people who don’t have one or more senses find the others make up for the missing ones. It would be wrong to assume that all people who are blind can hold more than 7 items in their working memory.
The cognitive load of trying to retain all the labels in order to sort them would be close to impossible.
Would would mean they would constantly need the cards to be read back to them. In the case of a screen reader, it would be reading what’s on the screen, repeatedly.
If it were in person, the risk of bias occurring would be higher, because the tester may want to make the task easier.
Either way, it renders the process of card sorting a wholly impractical method of testing with people who are blind or visually impaired.
ADHD
If you hang around on twitter as much as I do. You’ll be aware that people with ADHD are struggling with platforms like Mural or Miro.
I believe this is to do with the amount of information presented on a screen, and probably cursors moving around rapidly.
Card sorting is essentially like having a bit canvas board with lots of cards on them.
If you are using this method of testing with someone who has ADHD then this approach is quite harmful.
Participant recruitment
You’re probably wondering why I see this as problematic, let me explain.
Card sorting as a loose concept is pretty easy to get your head around. It’s a bit like sorting your spice rack, you put things in the order that’s logical to you.
When it comes to simple structuring, recruiting participants and defining the task is straight forward.
The problem comes when you try to test something complex or are trying to use it as the only way to create labels.
Running an in person test means you have the time to explain to people what they’re doing. But it doesn’t offer you the volume at scale needed for an authoritative outcome.
Running at scale, for example using Optimal Workshop gives you sample volume authority.
However, they’re notoriously hard to recruit for (unless you pay Optimal Workshop). You also need to consider that people struggle with the task if you’ve recruited your own participants.
Validation
Card sorting does not replace the need for information architecture or taxonomy design.
Please re-read that last sentence again.
If that seems like a strange thing to read, I’ve seen card sorting used to replace that skill.
Getting eyes on labelling or structure early on in the process is like a building block.
It should be used as a gateway checkpoint. Complete with the understanding that it tested well in the context of the structure. It can (and does) fall apart when you put it into the visual hierarchy.
If card sorting is being used, it absolutely must be used as one part of the testing journey. Not as the final answer.
Please read my article The Cult of Validation, to understand why I find this concept so frustrating.
Metadata and polyhierarchies
If card sorting is the only method of creation used, it poses other challenges with navigability.
A lot of large websites, especially ecommerce or news sites, rely on metadata to create polyhierarchical structures.
Card sorting can be used to generate metadata labelling or structures, but it then has to be augmented with the structure. That cannot be done with card sorting.
A polyhierarchical structure is a fluid structure. One that allows you to put one piece of content in multiple locations. Card sorting doesn’t allow for something to be categorised in two or more locations.
