This may sound strange, but I will prove it: no matter how big or stable a piece of software is, it has an unlimited number of bugs not yet found. No matter how many of them we have already managed to find and fix, there are still too many left to count.
Letâs take this simple Java method that calculates a sum of two integers as an example:
int sum(int a, int b) {
return a + b;
}
This simple program has an unlimited number of bugs.
To prove this claim we just need to put two thoughts together:
First, a bug is something that compromises the quality of software, which, according to IEEEÂ 610.12-1990, is âthe degree to which a system meets specified requirements or user expectations.â
Second, requirements and expectations may be functional and non-functional. The latter include performance, resilience, robustness, maintainability, and a few dozen other NFRs.
It is obvious that there are at least two variables in this equation that are ambiguous: user expectations and maintainability. We canât be precise about them and thatâs why the number of bugs they will produce has no limit.
Of course, only a very limited subset of the entire set of bugs has any real business impact. Most of the bugs that exist in a program may stay there even after it is shipped to its usersânobody will ever find them or else the damage they cause to the user experience will be insignificant.
Finally, take a look at the method sum() one more time. How about these bugs:
- It doesnât handle overflows
- It doesnât have any user documentation
- Its design is not object-oriented
- It doesnât sum three or more numbers
- It doesnât sum
doublenumbers - It doesnât cast
longtointautomatically - It doesnât skip execution if one argument is zero
- It doesnât cache results of previous calculations
- There is no logging
- Checkstyle would complain since arguments are not
final
Iâm sure you can find many more.
BTW, Glenford J. Myers said something very similar in his book âThe Art of Software Testing,â which I reviewed earlier.
Bill Hetzel, The Complete Guide to Software Testing (1993): âSome Theoretical Limits to Testing: âWe can never be sure the specifications are correct,â âNo testing system can identify every correct program,â âWe can never be certain that a testing system is correct.â These theoretical limits tell us that there will never be a way to be sure we have a perfect understanding of what a program is supposed to do (the expected or required results) and that any testing system we might construct will always have some possibility of failing. In short, we cannot achieve 100 percent confidence no matter how much time and energy we put into it!â
