Build Quality In!

Here are some insightful quotes come from one of the 20th century's leading experts on management and quality, William Edwards Deming.

 

"Build quality in."

 

"Quality can not be inspected into a product or service; it must be built into it."

 

"Inspection does not improve the quality nor guarantee quality. Inspection is too late. The quality, good or bad, is already in the product."

 

This blog will focus on coding and testing, but these quotes apply more universally. For example: Maintaining yourself – your body, your mental state, your skills – is one way to build quality in. Please don’t code drunk.

 

“Building quality in” is the philosophy I think we need to have about software testing. If you’ve heard the term “shift left,” it’s the same idea (test earlier).

If we let testing be a phase that happens after coding, here are some of the problems we encounter:

• Rework: Bugs found in testing result in test-and-fix cycles.
• More defects missed or found after initial testing: Testing at the end can't keep up. As the code grows, running regression tests takes longer and longer, and there are always new features to be tested. Either the test cycle keeps growing or some tests get skipped, causing defect leakage.
• Context switching and information loss: Often, by the time a bug feeds back to the devs, they are working on something else. They might have forgotten the details and are more likely to make mistakes. Merge issues more likely.
• High work-in-process (WIP): As a result of the above two. Many things will be "dev complete" but stuck in the test-and-fix cycle -- never truly done.
• Integration and stabilization sprints: If the devs keep writing untested code, at some point they have to stop writing code so we can stabilize things to rerun all the tests.
• Overreliance on black-box testing, which is usually slower and harder to run in a continuous-integration (CI) environment.
• Large batch size (large stories, large releases): When tests take too long and require manual effort, feedback cycles are too long and developers learn to cram as much stuff as possible into a change.
• Long cycle time: Influenced by all of the above. This includes long time to recover from production issues.
• Unpredictability: Large batch sizes, long cycle times, and long feedback loops combine to produce an opaque process with unpredictable timelines.
• Missed deadlines: When you do the testing at the end and find a serious problem, it might be too late to fix (and retest) in time.
• Partial testing: Rerunning all the tests becomes long and expensive, so we take shortcuts and say the code didn't change enough to have to rerun tests. And that's when it breaks.
• Developer and QA -- separate roles, possibly even separate teams. "Us vs. them." Low trust. Low cohesion. Disparate goals.
  • While I believe there is value in having test specialists and in doing exploratory and usability testing, high performers in the 21st century (not just 2021 -- I mean like 2001) do not outsource quality. A development task is not complete until the developer can prove that it's working in an automated way, not "it works on my machine."
• Unrefactorability (I just made that word up): When testing takes too long, developers will be too scared to change the code.
  • Future changes will be hacked and patched in instead of designed in.
• Overwork and extra hours: If you have all of the above, what else would you expect? The only way anything is going to get done is by people working their tails off. Often there is a tendency to reward those heroes, which helps perpetuate the behavior.
• No time to learn or collaborate: You think the business is going to allow you slack time to learn and experiment, or pair/mob program, when you've missed all your deadlines and are drowning in bugs because we're not building quality in?
• More micromanaging; Similar to above, if you're missing all your dates because of testing and quality issues, be prepared for more control measures: multiple status updates per day, more estimates, more requirements that dictate implementation details.
• All of the above can lead to low morale, low retention, and difficulties in recruitment.
• Leakage of customer data, poor products, bad customer experience, loss of reputation, downsizing, bankruptcy: If all of the above were happening, doesn't this seem inevitable?

 

You may think I'm fearmongering. Keep in mind that most developers and teams nowadays are doing at least some test automation, in which case the above problems might not be as pronounced.

 

Image credit: https://www.slideshare.net/lazygolfer/testing-does-not-equal-quality

 

Here are some ideas on how to improve:

• The team, especially leadership, needs to vigorously resist schedule pressure and overcommitment. Refuse to cut corners on quality, and do not offer it as a tradeoff. Speak up about the risks caused by technical debt and testing debt.
• For testing to have true value, it should influence, and ideally be an indivisible part of, the development process. Revisit your "definition of done" with this in mind.
• Developers need to write unit tests together with their code -- in the same commit. If you want tests that inform the design and lead to better, more testable code, consider writing unit tests before the code.
• All unit tests should be run in an automated environment (CI) on every commit. All tests must pass (without cheating and rerunning failures) before code is accepted.
• Measure and improve your code coverage.
  • Warning! While code coverage is a useful metric to indicate risk -- low coverage indicates lack of testing and high risk -- do not rely on it blindly. High coverage does NOT indicate good testing or low risk! You need to write tests that exercise your code with appropriate sets of inputs (including boundary conditions and error cases), and your tests need to perform appropriate assertions.
• You can set a code-coverage gate to make your build fail if coverage drops below a specified percent. Start with the current level and increase it over time as you improve your testing.
• To set up a coverage gate in Gradle you can do something like the following:

gradle.properties:

minimumLineCoverage=0.90

build.gradle:

jacocoTestCoverageVerification {

violationRules {

rule {

limit {

counter = 'LINE' // If not specified then default is branch. Options are: INSTRUCTION, LINE, BRANCH, COMPLEXITY, METHOD, CLASS

  minimum = minimumLineCoverage as BigDecimal

}

}

}

}

check.dependsOn jacocoTestCoverageVerification

• Other types of tests, not just unit, can be planned, and ideally automated, at the beginning of development. Consider a "Three Amigos" approach to BDD and executable specifications.
• Rome wasn't built in a day. For a code base with no automated unit tests, try starting the practice with new code. And as you revisit and change old code, start writing unit tests for the changed code. You may also find the code is untestable, in which case I recommend putting in place some higher-level integration tests so you at least get some coverage; and once those are in place you will have more confidence to refactor and make your code unit-testable.
• Developers and testers can work together -- pair on test cases, pair on coding, and both will learn. Here's a fun video related to that: Sleeping with the enemy
• Pairing or mobbing can serve as a live code review, and are ways to “build quality in” instead of inspecting code at the end with merge requests. It may not work in every situation, but at least do some collaboration early in development.

 

Image credit: https://www.methodsandtools.com/archive/archive.php?id=94

 

Thanks for reading!