In a previous job, we spent a lot of time traveling to be on-site with customers. This was an essential (and expensive) part of our business. We took the “go to where the users are” mantra to a maximal degree, and partnering closely with end users in commitment engineering loops was core to how we built product.
It wasn’t until a couple of years in, however, that I learned my most valuable lesson.
My team and I had just come back from a grueling on-site to a distant land. We went there to roll out a new product in our platform, intended to replace a legacy tool the customer had developed in-house. This was a high-stakes, high-profile project, and the Big Boss wanted to do a debrief on how it went.
I gave a glowing report: my team ran a group training Monday, and then spent the rest of the week doing desk sides with users to get them using it “in anger”. By the end of the week, I reported, we had a burgeoning group of power users who had completely converted over from their old workflow.
“Ok – tell me about one of these users. Walk me through what she’s doing”, the Boss asked.
I told him her name (let’s say Susie) and described how she was using our product over a couple of desk-side sessions I had done personally.
“Ok”, he asked. “Tell me more. What does she have on her desk? What other browser tabs does she have open?”
I was a bit taken aback. Why does he care what else she’s doing on her computer? I’m not a spy – why should I be snooping on her other tabs? I was there to focus on our product, not talk about whatever else she’s doing.
He was clearly disappointed I didn’t have an answer, and although I still felt the meeting went well, this kept bothering me until our next on-site a few weeks later.
I sat down with Susie as soon as I got there, eager to see how it had gone with our new tool since I was there. And sure enough, she had our product open, gamely clicked through a few workflows to show me she knew how to use it, and claimed she’d been using it regularly. Success!
But when I looked up, my heart sank – to the left of our tab, there were a dozen favicons of the old, in-house product she was supposed to have moved off of. She clearly had just opened our product when she knew I was coming by.
“Susie”, I asked with the energy of a parent observing their child pushing vegetables around on their plate. “I noticed you have a bunch of tabs of the old tool open. I’m not disappointed, just curious – would you mind telling me about that?”
And thus begun the best user interview of my career. She walked me through the old tool in new depth, explaining exactly why some of the workflows were better than our new product. We opened them side-by-side, and she showed me how the loading lag on our UI – beautiful though it was – made it hard to do her job, and why the “p2” chart option we had put down as “nice to have” was the reason she kept finding herself back in her old workflow. And – absolved of the need to pretend to be a power user – she told me she wasn’t alone, and that the whole floor was grumbling about having to switch workflows.
The moral of the story, for me, is that it’s very easy to want to talk about your product with users. You want to show them your features, and see how they’re adapting their workflows to your new way. You want to believe, and can easily adopt happy ears and eyes.
And users don’t want to disappoint you, either! You worked so hard, so they’re more likely to tell you what you want to hear.
So, next time you’re doing a user interview, look up, and pay attention to their other tabs. You might learn something.