Testing bugs not in the specification

Tags:
Software Quality
Software Quality Assurance
Software testing
How do you test bugs that are not covered in the specification?

Answer Wiki

Thanks. We'll let you know when a new response is added.

All bugs should be tested and reported, and enough time must be allowed to do it. In fact, a software testing/quality assurance engineer should always be trying to find new bugs no matter what stage of the quality assurance process he is in, but how you should manage these new bugs depends on your organization’s procedures.

I think you should investigate if your organization have some procedure/policy in place which contemplate this aspect of the testing process. Maybe a established procedure says that you should not spend too much time on these just-discovered bugs, but report them to your boss, so he/she can assign them to other engineer.

Regards,

****************************************
The program <a href=”http://www.google.com/search?sourceid=navclient&ie=UTF-8&rls=RNWE,RNWE:2005-51,RNWE:en&q=Bugzilla”>Bugzilla</a> is a nice general purpose bug management program. If you project is of any size, a program like this makes it easier to hunt and squash those pesky bugs. The program is both free and open source.
Good Luck and happy bug hunting!
-Flame

******************************************
If you have to ask, “How do you test bugs that are not covered in the specification”, then you have no business testing. First, Testing, be it Software or Hardware, is an art, a talent, a mindset – you don’t get it through college degrees or books or side courses – you either have it or you don’t !

Second, “Bugs” are not defined in design specifications; bugs are mistakes, bugs happen. “Bugs” are the result of developers thinking of function and not misfunction. Think about it, Product Requirement Specifications (PRS) are design guidelines, built from customer requirements that specify function – overall function. The PRS can not specify each and every detail of the design and that leaves a tremendous number of nooks and crannies for bugs to hide in. You assign five people a programatic problem and you’ll have five different solutions. The good Software Test Engineer thinks of everything that the developer though of and everything the developer did not.

Discuss This Question: 2  Replies

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when members answer or reply to this question.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
  • RobinGoldsmith
    These answers don't seem responsive to the question as I interpret it. I think the question in fact is asking how testing can reveal specification defects. Most testing won't, because it is reactive, simply confirming that the what the developer created conforms to the design/specification. Specification defects, whether incorrect or overlooked in the specification, are a major source of errors in the developed system, since neither the developer nor tester is likely to detect the specification defect. Effective requirements and design reviews can reduce the number of defects in the design/specification. In addition, my Proactive Testing(tm) methodology uses a variety of powerful special test planning and design risk analysis techniques which indeed help spot many otherwise overlooked defects, most of which are overlooked because they are defects in the design/specification.
    135 pointsBadges:
    report
  • Saimadhu
    Thanks Mr. Robin for clear & concise info. I agree with you. Here I am adding elaborated info on pro-active testing. Pro-active testing will help prevent the specification defects and helps to be in sync with all the stakeholders regarding ‘what the final product will be & do?’. How to do this Pro-active testing? The tester should be pro-active to understand the product, product scope, testing scope, on-going changes in the product etc. i.e. 1. What the product is? 2. Why this product is needed? (What are the business drivers for this product?) 3. Who are the customers? How they will deploy in their environments and use? 4. When (which circumstances) this product will to be used? How many concurrent users are expected etc? 5. How the product is Architected? How this Architecture fits for our purpose? How different components in the product communicate? Example: if the product is built as a strong security server then How this Architecture is providing the required security ? etc 6. What is the scope of the testing i.e. Check whether the product requirements are Clear, Correct & Completely specified? What makes the customer to refuse the acceptance of the product? Understand what balance of quality characteristics need to be tested in this product? Quality characteristics can be: Accuracy (Correctness, Completeness), Reliability, Security, Robustness (fault-tolerant), Performance (resource utilization, timeliness), Usability, and Adaptable in multiple environments etc. Why so many Quality characteristics? Which characteristics must to have and which are nice to have? Accuracy (Correctness, Completeness), Reliability Gives value to the product, so must to have. And other characteristics Add Value to the Product so nice to have. But providing only must to have characteristics will not win the market. Remember most of the products are rebuilt NOT because they are functionally deficient but because they are vulnerable (security is compromised), difficult to maintain/ port/ scale etc. So we need to also provide the characteristics which add value. But in competitive environment, providing all the characteristics completely in a single product is NOT feasible since it takes lot of time. So, to win the market, we need to see what balance of quality characteristics our product is required? So discuss with your key-stake holders to identify this balance. 7. Tester should be pro-active in verification of the balance of these Quality characteristics. What does u mean by balance of Quality characteristics? Example: Performance Vs Reliability. Do we require Performance or Reliability? If we want to improve the performance of the product in I/O activities then we may prefer to have Cache in between. The cache will helps in reducing the IO activities since the data will be transferred to the database once in a while. But what happens if we lose the cached data (due to power failure etc) just before transferring to the database? the product reliablity is lost right. So one characteristic is affecting the other characteristic. i.e, providing performance is affecting the reliability of the product. And you know if reliability is lost then the market for the product is lost. So, to increase the Performance and at the same time maintaining Reliability requires the balance of these 2 characteristics in the product design. 7. What are the 3rd party tools used? What are their limitations? Are they affecting the balance of Quality etc? 8. It is Tester’s responsibility to make sure all the stakeholders are in sync with ‘What the final product will be & do?’ i.e., the required requirements, plan, design, implementation etc. For this participate pro-actively in reviews, writing test cases, expected output etc at each phase of the product. What do you think? Any thing to add?
    455 pointsBadges:
    report

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

To follow this tag...

There was an error processing your information. Please try again later.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Thanks! We'll email you when relevant content is added and updated.

Following