QA and "The Testers"

January 6, 2024

Tom McLaughlin

What It's Like Now

Around the office we've been bemoaning the state of QA in modern software development. At our clients, consultants reported seeing these things

  • QA used to be a gatekeeper of quality, now it's an afterthought
  • Advances in technology have allowed developers to develop more faster, but testing technology hasn't advanced at the same rate
  • QA Teams have been disbanded entirely
  • QA is about testing, not about quality
  • QA engineers are sprinkled across scrum teams and outnumbered by developers 4:1
  • We don't value QA the same way we value product and engineering
  • QA gets squeezed into the end of the sprint
  • Unit testing is mistaken for QA
  • PMs are rewarded for shipping features, not shipping stable product

One QA professional at a big technology company told us he is expected to test software as it goes out the door. Regardless of what he finds, the software gets shipped. Worse, his performance is based on how many bugs the developers fix.

What It Was Like

My early development career was in government, defense, and insurance. At best, these are industries that hold on to development practices of the past, with ample gatekeepers to make sure things go smoothly. At worst, they move at a snails pace and reject change as violently as my body rejects mayonnaise.

To their credit, though, these industries held on to a gatekeeping model for QA. The software is not ready when the developers say it is - it's when QA approves it. We do not want the Department of Revenue to "move fast and break things."

Given this power (and responsibility), QA Engineers take it personally when bugs are found by users. They challenge developers, and develop an uncanny knowledge of edge cases. While a scrum team's tacit knowledge my lie in particular applications or APIs, the QA team's knowledge encompasses the entire system, including the gray areas just outside any scrum team's knowledge. Their system-wide knowledge often surpasses that of the architects and senior programmers.

Got a bug report for some weird edge case? QA will be able to craft the data to reproduce faster than the developers will.

Alas, even in these industries, QA is less the steward of overall system quality than it was. They are, increasingly, just "the testers."

What Happened

I don't know if the disbandment of powerful QA organizations was inevitable or not. The direction I've observed is:

  • Smaller, more frequent releases
  • Scrum teams with one or two QA engineers operating in isolation without a QA practice org
  • Increased pressure to ship features

Notably, even in the early days of Agile, the focus was more on shipping systems than it was on features. Painting very (overly?) broad strokes, we are trending toward a world where systems are the mere agglomeration of features (except in the few organizations with exemplary Product teams).

Maybe the stakes aren't high enough for any of this to matter. But I know at my clients, it's generally the case that their data must be accurate, secure, and timely. Defects that impact any of these factors have major ramifications on their business (and therefore my business).

Renaissance of Quality

I'd love to see a Renaissance of Quality. Where we re-empower QA to set the bar that must be cleared. Where they aren't merely scouring for bugs, but bring to bear their systems-level knowledge to help us make more robust and more usable products. Maybe then I'd get to see teams spend less time on fix releases an more time focused on product. I understand, however, that this is unlikely to happen. Currently, "Product" gets all the hype, followed by "Development."

We too often see QA struggling prior to each release to get things in order, ship the product, breathe a sigh of relief, and then brace for the next round. Still, none of this will change until the pain becomes too great to bear. In the meantime, be kind to "the testers," they're out there swimming against the tide trying to save us from ourselves.

About Me

Tom McLaughlin is a principal consultant at devobsessed. You can find me on linkedin and github.