SMOKE Vs SANITY

I have gathered a few points about the difference between smoke and sanity testing from the responses of two software testing groups. I have added the points below. However, my experience of executing the Smoke and Sanity testing has been the following: Smoke Test: When a build is received, a smoke test is run to ascertain if the build is stable and it can be considered for further testing. Smoke testing can be done for testing the stability of any interim build. Smoke testing can be executed for platform qualification tests. Sanity testing: Once a new build is obtained with minor revisions, instead of doing a through regression, a sanity is performed so as to ascertain the build has indeed rectified the issues and no further issue has been introduced by the fixes. Its generally a subset of regression testing and a group of test cases are executed that are related with the changes made to the app. Generally, when multiple cycles of testing are executed, sanity testing may be done during the later cycles after through regression cycles.

2 3

A Sanity test is used to determine a small section of the application is still working after a minor change. 4 Sanity testing is a cursory testing; it is performed whenever a cursory testing is sufficient to prove the application is functioning according to specifications. This level of testing is a subset of regression testing. 5 Smoke testing is normal health check up to a sanity testing is to verify whether build of an application before taking it to requirements are met or not, testing in depth. checking all features breadth-first.

Smoke Smoke testing originated in the hardware testing practice of turning on a new piece of hardware for the first time and considering it a success if it does not catch fire and smoke. In software industry, smoke testing is a shallow and wide approach whereby all areas of the application without getting into too deep, is tested. A smoke test is scripted--either using a written set of tests or an automated test A Smoke test is designed to touch every part of the application in a cursory way. It's is shallow and wide. Smoke testing will be conducted to ensure whether the most crucial functions of a program work, but not bothering with finer details. (Such as build verification).

Sanity A sanity test is a narrow regression test that focuses on one or a few areas of functionality. Sanity testing is usually narrow and deep.

A sanity test is usually unscripted.

What is test case? A test case is a set of conditions or variables under which a tester will determine whether an application or software system meets specifications. The mechanism for determining whether a software program or system has passed or failed such a test is known as a test oracle Test cases is a sequence of steps to test the correct behavior of a functionality/feature of an application What is a good test case? Find bugs that will be considered relevant Get these bugs fixed More powerful More likely to yield significant (more motivating, more persuasive) results More credible Representative of events more likely to be encountered by the user Easier to evaluate. More useful for troubleshooting More informative More Appropriately complex More likely to help the tester or the programmer develop insight into some aspect of the product, the customer, or the environment

Test case template What is test log? Test Log is a chronological record of the all the Information about the test case execution Or Test log is a collection of raw output captured during a unique execution of one or more tests, usually representing the output resulting from the execution of a test suite for a single test cycle plan. Test Log Table Name: Test Item

Expected

Actual

Comments

result

Result

Author: Version: What is test script?

Page Date:

of

A test script is a short program written in a programming language used to test part of the functionality of a software system. A written set of steps that should be performed automatically can also be called a test script What is test suite? In software development, a test suite, less commonly known as a validation suite, is a collection of test cases that are intended to be used to test a software program to show that it has some specified set of behaviors.

IEEE 829-1998 Test Plan Structure

Test plan identifier Introduction Test items Features to be tested

Features not to be tested Approach Item pass/fail criteria Suspension criteria and resumption requirements Test deliverables Testing tasks Environmental needs Responsibilities Staffing and training needs Schedule Risks and contingencies Approvals

IEEE STANDARDS FOR SOFTWARE

Software Engineering - Descriptions

1. ANSI/IEEE 1002-1987 IEEE Standard Taxonomy for Software Engineering Standards 2. ANSI/IEEE 1008-1987 IEEE Standard for Software Unit Testing 3. IEEE Std 1012-1986 IEEE Standard for Software Verification and Validation Plans 4. IEEE Std 1012-1998 IEEE Standard for Software Verification and Validation 5. IEEE Std 1016-1998 IEEE Recommended Practice for Software Design Descriptions 6. IEEE Std 1016.1-1993 IEEE Guide to Software Design Descriptions 7. IEEE Standard for Software Reviews and Audits 8. IEEE Std 1028-1997 IEEE Standard for Software Reviews 9. ANSI/IEEE 1042-1987 IEEE Guide to Software Configuration Management 10. IEEE Std 1044-1993 IEEE Standard Classification for Software Anomalies 11. IEEE Std 1044.1-1995 IEEE Guide to Classification for Software Anomalies 12. IEEE Std 1045-1992 IEEE Standard for Software Productivity Metrics 13. IEEE Std 1058-1998 IEEE Standard for Software Project Management Plans 14. IEEE Std 1059-1993 IEEE Guide for Software Verification and Validation Plans

15. IEEE Std 1061-1992 IEEE Standard for a Software Quality Metrics Methodology 16. IEEE Std 1061-1998 IEEE Standard for a Software Quality Metrics Methodology 17. IEEE Std 1062-1993 IEEE Recommended Practice for Software Acquisition 18. IEEE Std 1062, 1998 and 1062-1993 and 1062-1998 aIEEE Recommended Practice for Software Acquisition 19. IEEE Std 1063-1987 IEEE Standard for Software User Documentation 20. IEEE Std 1063-2001 IEEE Standard for Software User Documentation 21. IEEE Std 1074-1997 IEEE Standard for Developing Software Life Cycle Processes 22. IEEE Std 1074.1-1995 IEEE Guide for Developing Software Life Cycle Processes 23. IEEE Std 1175-1992 IEEE Trial-Use Standard Reference Model for Computing System Tool Interconnections 24. IEEE Std 1209-1992 IEEE Recommended Practice for the Evaluation and Selection of CASE Tools 25. IEEE Std 1219-1998 IEEE Standard for Software Maintenance 26. IEEE Std 1220-1994 IEEE Trial-Use Standard for Application and Management of the Systems Engineering Process 27. IEEE Std 1220-1998 IEEE Standard for Application and Management of the Systems Engineering Process 28. IEEE/EIA 12207.0-1996 IEEE/EIA Standard Industry Implementation of International Standard ISO/IEC12207:1995 and (ISO/IEC 12207) Standard for Information Technology Software life cycle processes 29. IEEE Std 12207.1-1997 IEEEsol;EIA Guide for Information Technology Software life cycle processesLife cycle data 30. IEEE/EIA12207.2-1997 IEEE/EIA Guide Software life cycle processesImplementation considerations 31. IEEE Std 1228-1994 IEEE Standard for Software Safety Plans 32. IEEE Std 1233-1996 IEEE Guide for Developing System Requirements Specifications

33. IEEE Std 1233, 1998 and 1233-1996 aIEEE Guide for Developing System Requirements Specifications IEEE Std 1298-1992 Software Quality Management System Part 1: Requirements 34. IEEE Std 1320.1-1998 IEEE Standard for Functional Modeling LanguageSyntax and Semantics for IDEF0 35. IEEE Std 1320.2-1998 IEEE Standard for Conceptual Modeling Language Syntax and Semantics for IDEF1X97 (IDEFobject) 36. IEEE Std 1348-1995 IEEE Recommended Practice for the Adoption of Computer-Aided Software Engineering (CASE) Tools 37. IEEE Std 1362-1998 IEEE Guide for Information TechnologySystem Definition Concept of Operations (ConOps) Document 38. IEEE Std 1420.1-1995 IEEE Standard for Information TechnologySoftware Reuse Data Model for Reuse Library Interoperability: Basic Interoperability Data Model (BIDM) 39. IEEE Std 1420.1b-1999 IEEE Trial-Use Supplement to IEEE Standard for Information TechnologySoftware ReuseData Model for Reuse Library Interoperability:Intellectual Property Rights Framework 40. IEEE Std 1430-1996 IEEE Guide for Information TechnologySoftware ReuseConcept of Operations for Interoperating Reuse Libraries 41. IEEE Std 1471-2000 IEEE Recommended Practice for Architectural Description of Software-Intensive Systems 42. IEEE Std 1490-2003 Adoption of PMI StandardA Guide to the Project Management Body of Knowledge 43. IEEE Std 1517-1999 IEEE Standard for Information TechnologySoftware Life Cycle ProcessesReuse Processes 44. IEEE Std 1540-2001 IEEE Standard for Software Life Cycle ProcessesRisk Management 45. IEEE Std 610.12-1990 IEEE Standard Glossary of Software Engineering Terminology 46. IEEE Std 730-1989 IEEE Standard for Software Quality Assurance Plans 47. IEEE Std 730-1998 IEEE Standard for Software Quality Assurance Plans 48. IEEE Std 730-2002 IEEE Standard for Software Quality Assurance Plans

49. IEEE Std 730.1-1995 IEEE Guide for Software Quality Assurance Planning 50. IEEE Std 828-1998 IEEE Standard for Software Configuration Management Plans 51. ANSI/IEEE 829-1983 IEEE Standard for Software Test Documentation 52. IEEE Std 829-1998 IEEE Standard for Software Test Documentation 53. IEEE Std 830-1993 IEEE Recommended Practice for Software Requirements Specifications 54. IEEE Std 830-1998 IEEE Recommended Practice for Software Requirements Specifications 55. IEEE Std 982.1-1988 IEEE Standard Dictionary of Measures to Produce Reliable Software 56. IEEE Std 982.2-1988 IEEE Guide for the Use of IEEE Standard Dictionary of Measures to Produce Reliable Software 57. IEEE Std 990-1987 IEEE Recommended Practice for Ada As a Program Design Language