Friday, August 8, 2008

Great advise and a relevant take on software QA testing.

I have to say that I agree with the following information. This is taken directly from the source and has not been modified.


* Expose all issues. As a tester, you must report all software problems that you encounter. The ones in your own product take first priority, but defects found in other products along the way should not go unreported. A Tester has the moral obligation to make all problems visible to the appropriate people. It doesn't matter if the bug is in someone else's product. The user will see the issue in your product and they will blame you if they don't know what to do. At the very least, you should document issues like this in the Readme or Troubleshooting docs and provide warnings or workarounds.

* Every minute wasted is a bug lost. Good Testers are constantly active constantly looking at the product and constantly looking for ways to save time. There is never enough time to test the product completely and so every minute wasted is another bug that you're letting slip by. Cannot afford to take the stance of being "Perfectly Content to do absolutely nothing until someone tells me twice." Never stop searching.

* Every minute of someone else's time that is wasted is 2 lost bugs. The bug that you could have been finding and the bug that the other person could have been finding. DO NOT WASTE OTHER PEOPLES TIME. If you have a question, Google it, read the doc, stop and think about your question. There are not very many questions in this Universe whose answers exist only in the mind of one person. Dropping in on a developer or another tester who is busy working should be your last resort.

* Regressing defects. Testing that a defect is fixed, that the issue no longer happens, is only the beginning of regressing a defect. The most valuable part is also the hardest part, it comes in testing around the fixed area to verify that nothing else was broken. Knowing what is "around the fixed area", guessing what could have been affected by a fix (or not affected when it should have been), attaining a knowledge of the architecture of the product.

* Defects must be fixable. There is no point in entering a report for a defect that cannot be fixed. An issue cannot be fixed if it cannot be duplicated and there are not sufficient trace information to determine the cause. There cannot be a fix if the subject of a defect is not an identifiable, quantifiable issue and if a case cannot be made that the issue will cause damage or inconvenience to a user. Writing observations without qualifying them as Defects is an unacceptable waste of other peoples time. Part of writing fixable defects is to investigate the issue so that you can provide enough information to determine (1) Where the problem is happening (at a minimum, who's component it is happening in, i.e. who should be responsible for fixing) and (2) How to reproduce the problem consistently. When the resulting damage or inconvenience of a defect is not obvious, you may need to make a case for why it really is a defect and why it must be fixed.

* Developer helpers. Our job has nothing to do with helping the developers to do things that they are able to do but would rather not do. We are not machine jockeys. We are not customer support. Our #1 priority is the quality of the product that we have been assigned. Anything else is a distraction and should be avoided. Our job is to examine all aspects of a product to evaluate the products quality and it's fitness to meet the expectations of end users. Playing the part of developer-helper, or unit-test-automater should be reserved for Developer Interns. It wastes of the time of a QA Engineer and ultimately stunts the quality of the product. Occasionally we will help other groups when they're under the gun, because we're geared for it. However, we can not afford to let those groups believe that helping them is our only calling in life. The most useless waste of a Quality Assurance persons time is engaging in a task that a developer could do themselves. Sometimes for the overall good we must allow ourselves to be distracted from testing, but don't let distractions run your day. We are not Developer Helpers or lackys.

* Study voraciously. Google is your best friend. Always search for an answer to a question before asking it. Always research the technology of the product that you will be testing. In many ways the tester has a responsibility to know, even more intimately than the developer, the architecture of the product, it's environment, and the technologies that it is built upon and that it interacts with.

* Functional testing is only the first step.
In order to be truly useful in this job, a tester must get beyond functionality testing. Functionality testing is preliminary to any other testing. If the product doesn't do what it's supposed to do, then any other issues are not terribly relevant. Developers could and should be doing functionality testing themselves in the form of Unit tests and other pre-check-in checks. If this is not being done, then that issue in the software development process of the team must be tackled before testing can archive full productivity. Where a tester adds value to the process is when he is able to go beyond those things that should be readily apparent to developers and casual users.

* Credibility is everything. There is no room for lies in this profession. Conjecture about something you don't really understand is the same as lying. Agreeing with something that someone else says that you don't know to be true, is the same as lying. Since our primary objective is to show the true state of the product, if what we say is not trusted by others then, even when we have done the work and the information is right, we have failed the primary objective. "Honra y dinero se ganan despacio y se pierden ligero" ~ "Reputation and money are earned slowly and lost quickly".

* Be a Devil for Details. Writing a bug report that just says "IT DON'T WORK", is NOT good enough. You must investigate the problem enough to provide the "Why it doesn't work", or the "When it doesn't work".

* If you're not part of the solution... Good testers don't just point out problems, they also suggest solutions. Examining the problem to the level that you can suggest solutions helps to verify that you understand the problem.

* Stand for quality. The #1 job of QA is to hold software development to a high standard of quality. This entails both the product and the process, in order to enforce both, you need to strive to have a maven understanding of both.


1 comment:

Anonymous said...

Thank you for sharing, Tim. Great points that all testers should follow!