Blogs

5 min read

The Case for User Acceptance Testing

Oct 22, 2019, 12:00:00 PM

New phone, who dis?

On a random Sunday, my husband surprised me by forcing me into the AT&T Store. You see, I had recently been struggling with my phone recently and although it was still a great phone, it just wasn't as fast as I needed it to be. My Bluetooth was sketchy at best, I had to use headphones or the speaker for phone calls, and my battery life meant I carried a backup battery everywhere. At almost four years old, I was carrying a dinosaur. I picked out my new iPhone Xs Max and started the data transfer process.

"Would you like a case?" 

Of course, I want a case. My shiny new toy needs to be protected from my own idiocy of sleeping with it, shoving it in my back pocket, throwing it on my car seat, stuffing it in my purse, and all the other things I do with my phone (read: play games while in the bathroom. You do it too. Don't judge). 

I like clear cases. After all, Apple didn't spend its resources thinking about the aesthetics of these devices for nothing. And while I want my precious..er..phone to be protected, I also want to showcase its beauty. I pulled two clear cases off the wall, debated for all of 10 seconds and made my choice: a Pelican Marine case that promises "total protection from the elements". Fancy. 

User Acceptance Testing

It took two days for me to put the case on my phone. Why? User Acceptance Testing (UAT). 

Yup. Those are the instructions for installing my case.

"Attention: this is the best way to do this and, by the way, we need you to test this before installing your phone."

Wait...what?

"Put a piece of tissue in the case and submerge it in water for 30 minutes."

Um...okay. 

In case you're wondering, my case passed the test. I could move forward with the rest of the installation steps. But for me, this triggered a thought about User Acceptance Testing. 

Whenever we work with clients, we expect them to engage in User Acceptance Testing. We've spent a lot of time together implementing a solution that fits your needs and requirements. While we provide role-based test cases, it's up to you and your testing team to make sure the tools and solution match. Without this, the go-live for the solution will fail. According to The Standish Group's CHAOS report, user involvement is either number one or number two factor in successful, challenged, or failed projects. Considering we spend approximately $250 Billion on IT application development projects in the United States, wouldn't we want to engage our users early and often? 

Agile as a Feedback Mechanism

The CHAOS report includes comparisons between Agile and Waterfall frameworks and methodologies. However, the critical thing to remember is it's not HOW you implement a project, but whether or not you involve your users. Scott Ambler and Associates makes this comparison using the Cost of Change curve. While the Cost of Change curve is used to advocate using Agile, the message is clear regardless of framework. Early and active stakeholders lead to fewer defect costs because of the shortened feedback cycle. Users gotta use so we can make adjustments. 

This is what we forget about Agile. After all in the Agile Manifesto, two of the four points emphasize the people side of how we develop technology. We are supposed to value people over process and customer collaboration over contract negotiation. I have to ask, then, when was the last time you and your team held a stakeholder demo? Or created a focus group? Or at least bounced an idea off a mentor? This is where many organizations fail at agile: we're afraid to engage our users early for feedback. As a result, 50% of IT projects fail. And while the failure rate is dropping, I have to challenge whether it's because we're getting better at feedback or whether we're getting better at manipulating data. 

Test Plan: User, Customer, Stakeholder

Your audience is critical as are their roles. To solicit feedback early and often, you must place yourself in the role of the user, customer, or stakeholder. We've moved away from calling them User Stories to just Stories. However, the user profiles of these roles are critical to the creation of a good story. Who's the audience? What do they really want?

I like to think of the use case of my Pelican phone case. As a frequent traveler, I want the maximum protection for my phone for whatever I'm doing. I should be confident that it provides me impact, water, dust, and "being stupid" protection. I should also expect it to be lightweight, but aesthetically pleasing. In order to fulfill my requirements and acceptance criteria, what should my involvement be when I'm finally ready to use it? Is asking me to do a single, 30-minute test too much? At the price point of both the phone and the case, should I not be eager to be part of the testing process to make sure I have the highest quality product? I think this is a great way to validate whether or not the user gets what they wanted. And while my test was successful, there were clear instructions as to what to do if the test failed. 

Defining not only what you're intending to deliver, but to whom makes us better able to deliver the right thing with the right quality. Users gotta use. 

Lessons Learned: Users gotta use

No matter how cool or innovative or disruptive, if users don't use, we've failed. Complicated technology implementations or clunky user experiences or missing a key demographic are all failures. For as much as you expect to "fail fast", do you actually solicit the feedback needed early and often to adjust? I, for one, would love to see the failure statistics for my new phone case. My brain is putting together a Pareto chart of root causes as I type this. However, I also would like to see the number of users that didn't engage in the testing steps. After all, functional testing and user involvement is only as effective as the users that use. 

What do you think you can do better to engage the various roles of your users? And remember, it's not just tools and technology, it's the processes that support these as well. Want to learn how Praecipio Consulting outlines the importance of UAT in a migration? Watch our on demand webinar, Plan Your Jira Server to Cloud Migration

Written by Amanda Babb

Upcoming Webinars

Past Webinars

Case Studies

Blog