Posts

Showing posts with the label QA

Replicating bugs is the *pits*

Image
/via https://www.monkeyuser.com/2017/steps-to-reproduce/ This, tragically, is entirely true. It takes you weeks of fiddling around with increasingly arcane maneuvers to replicate the precise set of circumstances in which that extremely weird edge-case that you didn’t account for shows up. But, and this is key, your end-user replicated it just a few femto-seconds after you deployed the release. Sigh…

The Truth About Testing

Image
/via https://www.monkeyuser.com/2016/testing-hammering-nails/ This totally nails it when it comes to Manual Testing, doesn’t it? Oh, I agree, there is a place for manual testing — smoke tests to build confidence, complex tests that simply can’t be automated, and so forth. However, what I  do  see all too often is Manual Testing being used as the default approach to existence. This, typically in organizations where QA consists of “ those folks over there ”, and developers consider testing a four-letter word. My advice — if you’re working at one of those places, and if you have agency, leave. It’s just not worth the candle…

Beware the Happy Flow

Image
/via https://www.monkeyuser.com/2018/happy-flow/ We all have some variation of the  Happy Flow , (the  Golden Path , the  Happy Day… ). You know the drill — it’s the optimal situation, the one that we’re all aiming for, where there are no exceptions, error conditions, or BadThings™ happening. It’s actually a good thing, in that it allows us to focus, and make sure that we get the important parts Just Right. Heck, it’s actually baked philosophically into erlang, in the form of  Let It Crash   (•). And then again, it can be a Very Bad Thing, if you get tunnel vision, and start believing that your version of the  Happy Flow  is the One True Way. I mean, have you thought about the RealUsers™ out there, who have their own version of reality? After all, your tunnel vision is only valid if you also represent the vast majority of the RealUsers™. (and yes, that’s why you need metrics!). And then of course there is the oh-so-common situation where yo...

It passed the tests, it’s good to go…

Image
It’s amazing how frequently I see this, y’know? Where “ passing tests ” is considered   proof , as if it has validated the bug-free nature of your code, when all it means is that   your code passed the regression tests . Thats it. No more (and no less, mind you). Are there still bugs in there? —  Yeah! Will your users find them for you? —  Hell yeah! Will they be found 2am Saturday? —  Certamente! Mind you, if your organization has leveled up, and you’ve built out your deployment pipelines, and you   understand   that your regression tests are your last barrier, the “ it’s the best we can do at this point in time ” validation of your system, then, well, you still have bugs, but at the very least you   understand that you still have bugs. Tragically, in this world of   #CowboyDevelopers , that is not the case. I mean, how do you argue with someone who genuinely believes the following? “I don’t see why I need to write tests for it — *I* wrote this...

Realistic Tests, but not in PROD

Image
So yeah, I’ve never done this. Nope. Not me. Nuh-uh.

What does “Testing” *actually* mean to you?

Image
“QA testers do not provide assurance; they help provide analysis and feedback.”  (•) Ask yourself the following —  • Does your team believe this? • Internalize this? • Do they all implicitly have tests with each of their commits? • Do they care about code-coverage? • Are the tests explicating quality? Testing, and QA, are not just about checking functionality — they are about embedding quality in your system, in your processes, and in your maintainability. To summarize, you should • Test throughout the process  not  only at the end  : Test early, test often, and start testing as early in the development process as you can • Build maintainability (and quality!) into the code  : Looking for places you didn’t meet the specs is a never-ending task. Instead, write for fault-tolerance/reliability, and then find bugs where you  weren’t  reliable • Understandability of your code-base is vital — test for it  : Oh,  you  may know ...