Software Testing

Software Testing ________________________________________________________________________

Softwar e Testi ng
- made easy

Arcus Infotech Pvt Ltd -1-

Software Testing ________________________________________________________________________

33

This book is dedicated to

Arcus Infotech Pvt Ltd -2-

Software Testing ________________________________________________________________________

Lord Vignesh

TABLE CONTENTS

1. Testing …………………………………………………………….11 1.1 Definition: …………………………………………………………………..11 1.2 Objective: ………………………………………………………………......11 1.3 Benefits of testing: ………………………………………………………..12

Fundamentals:

Arcus Infotech Pvt Ltd -3-

Software Testing ________________________________________________________________________ 1.4 Defects: …………………………………………………………………......12 1.5 Bugs: ………………………………………………………………………...12 2. Quality Assurance, Quality Control, Verification & Validation: …………….13 2.1 Quality ………………………………………………………..13 2.2 Quality ……………………………………………………………13 2.3 ………………………………………………………………...13 2.4 …………………………………………………………………..13 3. SDLC ………………………………………………………………………14 3.1 Requirement ……………………………………………………14 3.1.1 …………………………………………………………….....15 3.1.2 ………………………………………………………….16 3.1.3 ……………………………………………………………….16 3.2 Software ………………………………………………………….18 3.2.1 ……………………………………………………………….18 i. Purpose of HLD ……………………………….18 Assurance: Control: Verification: Validation: Process: analysis: BRS: Analysis: SRS: Design: HLD: document:

Arcus Infotech Pvt Ltd -4-

Software Testing ________________________________________________________________________ 3.2.2 ……………………………………………………………….19 3.3 ……………………………………………………………………..19 LLD: Coding:

3.4 ……………………………………………………………………..20 3.4.1 Levels ………………………………………………20 3.4.1.1 Unit …………………………………………..20 a. Benefits of …………………………...20 b. Component ………………………...20 of

Testing: Testing: Testing:

unit test

testing: process:

3.4.1.2 Integration testing: …………………………………..22 3.4.1.2.1 Incremental Integration testing: …………23 a. Top down integration: ……………………..24 b. Bottom up integration: …………………….25 i. Stub & Drivers: ………………………27 c. Sandwich testing: …………………………..27 3.4.1.3 ………………………………………..28 3.4.1.4 …………………………………………28 3.4.1.5 ………………………………………..29 Smoke Sanity System testing: testing: testing:

Arcus Infotech Pvt Ltd -5-

Software Testing ________________________________________________________________________ 3.4.1.5.1 ………………….30 A. …………………………30 a. Functionality ………………30 i. GUI ………………..30 ii. Input domain coverage: …...31 iii. coverage: ...31 iv. coverage: …31 V. ………..31 a. Data validation b. data integrity vi. Order of functionality: …….32 b. …………………32 B. …………………...32 i. …………………….32 ii. …………………...33 iii. ……………………34 iv. ……………..35 v. ……………..35 vi. ……………...36 vii. ……………..36 a. Load ………………..37 testing: Performance testing: Data-Volume testing: Configuration testing: Compatibility testing: Security testing: Recovery testing: Usability testing: Non-Functional testing: Sanitation testing: database coverage: Error handling Output domain testing: coverage: Functional testing: Types of system testing:

Arcus Infotech Pvt Ltd -6-

Software Testing ________________________________________________________________________ b. Stress ……………...37 3.4.1.6 User-Acceptance …………………………...37 a. Alpha ……………………………….38 b. Beta …………………………………38 testing: testing: testing: testing:

3.5 Implementation: …………………………………………………………………...39 3.6 Maintenance: ……………………………………………………………………….39 3.6.1 Bug Fixing: ……………………………………………………………….39 3.6.2 Enhancement: …………………………………………………………...39 3.6.3 Upgradation: ……………………………………………………………..39 4. SDLC ……………………………………………………………………….39 4.1 Waterfall …………………………………………………………...40 4.2 …………………………………………………………………….42 4.3 Spiral ……………………………………………………………….46 4.4 Prototype ………………………………………………………….49 5. Testing ……………………………………………………………….50 Models: Model: V-Model: Model: Model: Techniques:

5.1 Black-Box Testing: ……………………………………………………….50 a. Tools used for Black-Box testing: ………………………………50 b. Advantage: …………………………………………………………..51 c. Disadvantage: ……………………………………………………….51

Arcus Infotech Pvt Ltd -7-

Software Testing ________________________________________________________________________ 5.1.1 Types of black-box testing: …………………………………...51 i. Boundary Value Analysis: …………………………………51 a. BVA Techniques: …………………………………...51 b. Limitations: ………………………………………….52 ii. Equivalent Class Partioning: ……………………………..52 iii. Error Guessing: …………………………………………….52 5.2 White Box ………………………………………………………..52 Testing:

5.2.1 Types of White-Box testing: …………………………………..53 i. Basic Path Testing: …………………………………………53 a. Cyclomatic Complexity: …………………………...54 ii. Control Structure Testing: ………………………………..54 a. Branch testing: …………………………………….54 b. Condition testing: ………………………………….56 c. Dataflow testing: …………………………………..56 d. Loop testing: ……………………………………….57 iii. Program Technique testing: …………………………….59 iv. Mutation testing: …………………………………………...59 5.2.2. Why we do white ………………………………..59 5.2.3. Need of white ……………………………………60 5.2.4. ……………………………………………………….60 box box testing: testing: Limitation:

Arcus Infotech Pvt Ltd -8-

Software Testing ________________________________________________________________________ 5.3 Grey-Box ………………………………………………………….60 testing:

6. STLC Process: ……………………………………………………………………… 61 6.1 Test Initiation Phase: ……………………………………………………..61 a. Test Strategy document: ………………………………………….62 i. Scope & Objective: ………………………………………….62 ii. Budget (or) Business Issues: …………………………….63 iii. Roles & Responsibilities: ………………………………...63 iv. Communication & Status Report: ………………………63 v. Test Automation & Tools: …………………………………63 Vi. Change & Configuration management: ………………..63 vii. Training Plan: ………………………………………………64 viii. Risk & Assumptions: …………………………………….64 6.2. Test plan ………………………………………………………….64 6.2.1. Master Test ………………………………………………64 a. testing Team …………………………………..64 b. Identifying Tactical ………………………………...64 6.2.2 Detailed Test ……………………………………………...65 a. Test Plan ………………………………………...65 b. …………………………………………………65 c. Test …………………………………………………….65 Phase: Plan: Formation: Risks: Plan: Identifier: Introduction: item:

Arcus Infotech Pvt Ltd -9-

Software Testing ________________________________________________________________________ d. Features to be tested: ……………………………………..66 e. Features not to be tested: …………………………………66 f. Approach: …………………………………………………….66 g. Item Pass/ Fail Criteria: ……………………………………66 h. Test deliverables: …………………………………………..67 i. Testing tasks: ………………………………………………..67 j. Environmental needs: ………………………………………67 k. Responsibilities: ……………………………………………67 l. Staff & Training Needs: …………………………………….67 m. Schedule: …………………………………………………… 68 n. Risks & Contingencies: ……………………………………68 o. Approvals: …………………………………………………...68 6.3. Test design ……………………………………………………….68 6.3.1. Test Scenario …………………………………….68 6.3.2. Test case ………………………………………….68 6.4. Execute Test ………………………………………………69 Case Phase: Document: Document: Phase: Phase:

6.5. Test Report ……………………………………………………….69

6.5.1. Defect Report Document: …………………………………….69 a. Defect Id (or) Name: ………………………………………..69 b. Defect Description (or) introduction: …………………...69

Arcus Infotech Pvt Ltd - 10 -

Software Testing ________________________________________________________________________ c. Severity: ………………………………………………………69 i. High severity: ………………………………………...69 ii. Medium severity: ……………………………………70 iii. Low severity: ………………………………………..70 d. Priority: ……………………………………………………….70 e. Reprodusable: ………………………………………………70 f. Status: ………………………………………………………… 70 g. Tested By: ……………………………………………………70 h. fixed by: ……………………………………………………..70 i. Reported on: …………………………………………………70 6.5.2. Tools Used: ……………………………………………………..70 a. Clear Quest: ………………………………………………… 70 b. Test Director: ………………………………………………..71 c. Defect tracker: ………………………………………………71 6.6. Test Closure Phase: ……………………………………………………..72 a. Sign off: ……………………………………………………… 72 b. Authorities: ………………………………………………….72 c. Deliverables: ………………………………………………..72 d. Metrics: ……………………………………………………...72 i. Defect metrics: ……………………………………….72 ii. Defect age: …………………………………………...72 iii. Defect analysis: …………………………………….73

Arcus Infotech Pvt Ltd - 11 -

Software Testing ________________________________________________________________________ 7. Types of testing: ……………………………………………………………………73 7.1. Compliance testing: ……………………………………………………..73 7.2. Intersystem testing/ Interface testing: ……………………………….73 7.3 parallel testing: ……………………………………………………………73 7.4 database testing: ………………………………………………………….74 7.5. Manual support testing: …………………………………………………74 7.6. Ad-hoc testing: ……………………………………………………………74 7.7. Configuration testing: …………………………………………………...75 7.8. Pilot testing: ……………………………………………………………….75 7.9. Automated testing: ………………………………………………………75 7.10. Load Testing: ……………………………………………………………75 7.11. Stress & Volume Testing: ……………………………………….. ……76 7.12. Usability Testing: …………………………………………………. ……76 7.13. Environmental testing: ………………………………………... ………76 7.14. Active testing: …………………………………………………... ………77 7.15. Passive testing: ………………………………………………...…. ……77 7.16. Client / Server testing: …………………………………………....……77 7.17. Web testing: …………………………………………………….…. ……77 8. Review: …………………………………………………………………….………… 78 8.1. Definition: ………………………………………………………………….78 8.2. Types of review: ……………………………………………………. ……78

Arcus Infotech Pvt Ltd - 12 -

Software Testing ________________________________________________________________________ 8.2.1. Walkthrough: ……………………………………………………78 8.2.2. Inspection: ………………………………………………………79 8.2.3. Informal review: ……………………………………….. ………79 8.2.4. Technical review: ………………………………………………80 8.3. Comparison of review types: ……………………………….…. ………80 8.4. Activities performed during review: ……………………….. …………80 8.5. review of the specification/ Planning & preparing system test : .. 82 8.6. Roles & Responsibilities: …………………………………………….…83 8.6.1. Moderator: …………………………………………………. ……83 8.6.2. Recorder: …………………………………………….. …………83 8.6.3. Presenter: …………………………………………. ……………83 8.6.4. Reviewers: ………………………………………………………84 . 9. Difference ………………84 tables: ………………………………………………….

9.1. Quality Vs Testing: …………………………………….. ………………84 9.2. Testing Vs Debugging: ………………………………….. ……………84 9.3. Quality Assurance Vs Quality Control: …………………. …………84 9.4. Verification Vs Validation: ……………………………………………86 9.5. Black-Box Testing Vs White-Box Testing: …………………………86 9.6. IST Vs UAT: ……………………………………………………………… 87 9.7. SIT Vs IST: ……………………………………………………….…….… 87 9.8. Alpha testing Vs Beta testing: ………………………………. …….…87 Arcus Infotech Pvt Ltd - 13 -

Software Testing ________________________________________________________________________ 9.9. Test Bed ………………………………………88 9.10. Re-Testing …………………………………88 Vs Vs Test Regression Environment: testing:

10. Regression testing & re-testing: …………………………………….. ………88 10.1. Factors favour automation of regression testing: ………………89 10.2. Tools used in regression testing: …………………………………90 11. Roles & Responsibilities: ……………………………………………….. ……90 11.1. Test Associate: …………………………………………………….… 90 11.2. Test engineer: ………………………………………………………… 90 11.3. Senior test engineer: ………………………………………….. ……91 11.4. test lead: ……………………………………………………………… 92 11.5. test manager: ………………………………………………………… 92 12. Types of defect: ………………………………………………………………… 93 12.1. Test procedure related defects: ……………………………………94 12.2. Test data (or) test input related defects: …………………………94 12.3. Coding related defects: ………………………………………………94 12.4. Hardware (or) Infrastructure related defects: ……………………94 13. Types of risks & its Solutions: ……………………………………………….94 13.1. Buddy testing (Due to lack of time): ………………………………94 13.2. Exploratory testing (Due to lack of Documentation): ……….…95

Arcus Infotech Pvt Ltd - 14 -

Software Testing ________________________________________________________________________ 13.3. Pair testing (Due to lack of domain knowledge): ………………95 14. Bug life cycle (or) Defect life cycle: ………………………………….. ……96 15. Test strategy: ………………………………………………………………..… 98 15.1. Static testing: ………………………………………………..……… 99 15.2. Dynamic testing: ……………………………………………….…… 99 16. Testing Process / Methodology: ………………………………………... …99 16.1. Test initiation phase: ……………………………………………… 100 16.2. Test planning phase: ……………………………………………… 101 16.3. Test design phase: ………………………………………………… 103 16.4. Test execution process: ……………………………………..…… 104 16.5. Test management process: ………………………………………105 16.6. Test closure phase: …………………………………………...…… 106 17. Test deliverable template: ……………………………………………….. …108 17.1. Project details form: ………………………………………….…… 108 17.2. Minutes of meeting: …………………………………….…….…… 109 17.3. Top level project checklist: ……………………………………. …109 17.4. Test environment request: ………………………………….. ……111 17.5. Clarification document: ……………………………………………112 17.6. Test condition / test case document: ………………………...…112 Arcus Infotech Pvt Ltd - 15 -

Software Testing ________________________________________________________________________ 17.7. Test script document: ………………………………………………113 17.8. Traceability document: …………………………………………. …113 17.9. Daily status report: ………………………………………………… 113 17.9.1. Design phase: ……………………………………….…… 114 17.9.2. Execution phase: ………………………………………… 114 17.9.3. Defect distribution: ……………………………………… 114 17.9.4. Environment non-Availability: …………………………115 17.9.5. Other issues / Concerns: …………………………… 115 17.9.6. General remarks: ……………………..……………… 115 17.10. Weekly status report: ………………………………………… 115 17.10.1. Life cycle / Process: …………………..…………… 116 17.10.2. Highlights of the week: ………………….………… 116 17.10.2.1. Design phase: …………………..………… 116 17.10.2.2. Execution Phase: ………………………… 116 17.10.3. Defect distribution: ………………………………… 116 17.10.4. Environment non-Availability: …………………… 117 17.10.5. Other issues/ concerns: …………………………… 117 17.10.6. General remarks: …………………………….……… 117 17.11. Defect report: …………………………………………………… 117 17.12. Final test checklist: …………………………………….……… 117 17.13. Project de-briefs form: ………………………………………… 118 18. CMM Levels: ……………………………………………………..…………119 Arcus Infotech Pvt Ltd - 16 -

Software Testing ________________________________________________________________________

19. 10 Tips you should read before automating your testing work: …120 20. Question & Answer: ……………………………………………………… 123 21. Glossary: ……………………………………………………………………157

1. Testing Fundamentals
1.1. Definition “The process of exercising software to verify that it satisfies specified requirements and to detect errors “ (Or) “Testing is the process of executing a program with the intent of finding errors” (Or) Testing identifies faults, whose removal increases the software quality by increasing the software’s potential reliability. Testing is the

Arcus Infotech Pvt Ltd - 17 -

Software Testing ________________________________________________________________________ measurement of software quality. We measure how closely we have achieved quality by testing the relevant factors such as correctness, reliability, usability, maintainability, reusability and testability. (Or) Software testing is a process to check whether our application or product meets the client or customer requirements or not. 1.2. Objective • • • • • Testing is a process of executing a program with intent of finding an error. A good test is one that has a high probability of finding an as-yetundiscovered error. A successful test is one that uncovers an as-yet-undiscovered error. Testing should also aim at suggesting changes or modifications if required, thus adding value to the entire process. The objective is to design tests that systematically uncover different classes of errors and do so with a minimum amount of time and effort. • • • Demonstrating that the software application appears to be working as required by the specification. Meeting performance requirements. Software reliability and software quality based on the data collected during testing. 1.3. Benefits of Testing • • • Increase accountability and Control. Cost reduction. Time reduction. Arcus Infotech Pvt Ltd - 18 -

Software Testing ________________________________________________________________________ • • • Defect reduction. Increase productivity of the Software developers. Quantitative Management of Software delivery.

1.4. Defect: If software misses some feature or function from what is there in requirement it is called as defect. (Or) It is nothing but mismatch between the developer developed requirement and the customer specific requirement, then the mismatch is called as defect. Missing of the customer requirement during development

1.5. Bug: A fault in a program which causes the program to perform in an unintended or unanticipated manner. (Or) Bug means deviation from the expectation. (or) Actual result not equal to expected result.

2. Quality Assurance, Quality Control, Verification & Validation 2.1. Quality Assurance “A planned and systematic pattern for all actions necessary to provide adequate confidence that the item or product conforms to established technical requirements”

Arcus Infotech Pvt Ltd - 19 -

Software Testing ________________________________________________________________________ 2.2. Quality Control “QC is a process by which product quality is compared with applicable standards, and the action taken when nonconformance is detected.” “Quality Control is defined as a set of activities or techniques whose purpose is to ensure that all quality requirements are being met. In order to achieve this purpose, processes are monitored and performance problems are solved.” 2.3. Verification “The process of evaluating a system or component to determine whether the products of the given development phase satisfy the conditions imposed at the start of that phase.” (Or) “Are we building the Product Right?” 2.4. Validation Determination of the correctness of the products of software development with respect to the user needs and requirements. (Or) “Are we building the Right Product”?

Difference Table: Quality Analysis
Study on Process followed in Project development

Quality Control
Study on Project for its Function and Specification

Verification
Process of determining

Validation
whether Process of determining whether a

Arcus Infotech Pvt Ltd - 20 -

Software Testing ________________________________________________________________________
output of one phase of development fully conforms to its previous phase developed system conforms to its SRS document Verification is concerned with phase containment of errors Validation is concerned about the final product to be error free

3. SDLC Process (Software Development Life Cycle)
3.1. Requirement Analysis: Roles: Business Analyst (B.A), Engagement Manager (E.M) Process: First of all the Business analyst will take an appointment from the customer, collects the templates from the company, meets the customer on appointed day, gathers the requirements with the help of template and comes back to the company with the requirements documents. Once the requirement document has come to the company the engagement manager will check whether the customer gives any extra requirements or confused requirements. In case of extra requirements he deals the excess cost of the project. In case of confused requirements he is the responsible for prototype demonstration and gathering the clear requirements. Proof: The proof document of this phase is Requirement Document. This is called with different names in different companies.

Arcus Infotech Pvt Ltd - 21 -

Software Testing ________________________________________________________________________ 1. FRS (Functional Requirements Specification) 2. CRS (Customer Requirement Specification) 3. URS (User Requirement Specification) 4. BDD (Business Design Document) 5. BD (Business Document) 6. BRS (Business Requirement Specification) Some companies may maintain the over all business flow information in one document and the detailed functional requirement information in the other document.

3.1.1. BRS (Business Requirement Specification)
Roles: Business Analyst (B.A), Project Manager (P.M) Process: BRS is developed by Business Analyst. BRS is a Business Requirement Specification Initially client will give the req's in their own format then it will be converted in to Standard format by which s/w people can understand. In BRS the requirements are defined in general format, where as in SRS the requirements will be divided in to modules and each module contains How many interfaces and screens.

3.1.2. Analysis:
Arcus Infotech Pvt Ltd - 22 -

Software Testing ________________________________________________________________________ (a) Tasks: 1. Feasibility Study. 2. Tentative planning. 3. Technology Selection. 4. Requirement Analysis. (b)Roles: System Analyst (S.A), Project Manager (P.M), and Team Manager (T.M) Process: 1. Feasibility Study 2. Tentative Planning : It is detailed study of the requirements in order to check Weather the requirements are possible or not. : In this section the resource planning and the time planning (Scheduling) is done temporarily. 3. Technology Selection : The list of all the technologies that is required to accomplish this project. Successfully will be analyzed and listed out in this section. 4. Requirement Analysis : The list of all the requirements that is required to accomplish this project.

3.1.3. SRS (Software Requirement Specification):
Roles: Project Manager (P.M) Process: SRS Document is one of the Client Requirement Document. The SRS is often referred to as the "parent" document because all subsequent project management documents, such as design specifications, statements of work, software architecture specifications, testing and validation plans, and documentation plans, are related to it. An SRS is basically an organization's understanding (in writing) of a customer or potential client's system requirements and dependencies at a particular point in Arcus Infotech Pvt Ltd - 23 -

Software Testing ________________________________________________________________________ time (usually) prior to any actual design or development work. It's a two-way insurance policy that assures that both the client and the organization understand the other's requirements from that perspective at a given point in time. It's important to note that an SRS contains functional and nonfunctional requirements only; it doesn't offer design suggestions, possible solutions to technology or business issues, or any other information other than what the development team understands the customer's system requirements to be. A well-designed, well-written SRS accomplishes four major goals: * It provides feedback to the customer. An SRS is the customer's assurance that the development organization understands the issues or problems to be solved and the software behavior necessary to address those problems. Therefore, the SRS should be written in natural language in an unambiguous manner that may also include charts, tables, data flow diagrams, decision tables, and so on. * It decomposes the problem into component parts. The simple act of writing down software requirements in a well-designed format organizes information, places borders around the problem, solidifies ideas, and helps break down the problem into its component parts in an orderly fashion. * It serves as an input to the design specification. As mentioned previously, the SRS serves as the parent document to subsequent documents, such as the software design specification and statement of work. Therefore, the SRS must contain sufficient detail in the functional system requirements so that a design solution can be devised. * It serves as a product validation check. The SRS also serves as the parent document for testing and validation strategies that will be applied to the requirements for verification.

Arcus Infotech Pvt Ltd - 24 -

Software Testing ________________________________________________________________________

3.2. Software Design:
The development process is the process by which the user requirement are elicited and software satisfying this requirement is designed, Build, tested and delivered to the customer.

3.2.1. HLD (High Level Designed Document): a. Purpose of this Document This HLD is also called as Architectural Designed Document (or) System Design Document (Or) Macro Level Design. It is the phases of the life cycle when a Logical view of the computer implementation of the solution to the customer requirement is developed. It gives the solution at high level of abstraction. The solution contains two main components. • • The functional Architecture of the application and the Database design. (Or) This High-Level Design (HLD) document specifies the implementation, including intercomponent dependencies, and provides sufficient design detail that any product based on this HLD will satisfy the product requirements. (Or) This is the first place other developers/maintainers will look to learn how your project works. It must provide a comprehensive outline of the entire design. The design lead is responsible for writing the overview. Write this section as HTML and place it in the javadoc overview file.

3.2.2. LLD (Low Level Designed Document): Arcus Infotech Pvt Ltd - 25 -

Software Testing ________________________________________________________________________ This LLD Design is also called as detailed design (Or) Micro level Design. During the detailed design phase, the view of the application developed during the high-level design is broken down into modules and programs. Logic design is done for every program and then documented as program specifications. For every program a unit test is created. Important activities in the detailed design stage include the identification of common routines and programs development of skeleton programs, and development of utilities and tool for productivity improvement. (Or) This document describes each and every module in an elaborate manner so that the programmer can directly code the program based on this. There will be at least 1 document for each module and there may be more for a module. The LLD will contain: - detailed functional logic of the module in pseudo code - database tables with all elements including their type and size - all interface details with complete API references (both requests and responses) - all dependency issues -error message listings - complete input and outputs for a module.

3.3 Coding:
This coding part will be done by only Programmers nothing but Developers. First the Developer write the coding for the Application (or) Product based on the Low Level Designed (LLD) Document, because this document is the detailed document based on the SRS Document. After writing the coding for the LLD design the Programmer will write the coding for the HLD design.

Arcus Infotech Pvt Ltd - 26 -

Software Testing ________________________________________________________________________

3.4. Testing: 3.4.1. Levels of Testing: 3.4.1.1. Unit Testing:
Unit testing is also called as Component Testing. Unit testing is defines as Testing the individual module. The testing is done to a unit or to a smallest piece of software. Unit testing are done to verify if it satisfies its functional specification or its intended design structure. (Or) This is the first and the most important level of testing. As soon as the programmer develops a unit of code the unit is tested for various scenarios. As the application is built it is much more economical to find and eliminate the bugs early on. Hence Unit Testing is the most important of all the testing levels. As the software project progresses ahead it becomes more and more costly to find and fix the bugs. In most cases it is the developer’s responsibility to deliver Unit Tested Code.

a. Benefits of Unit Testing:
• • • • Assurance of working components before integration Tests are repeatable - Every time you change something you can rerun your suite of tests to verify that the unit still works. Tests can be designed to ensure that the code fulfills the requirements. All debugging is separated from the code.

b. Component Test Process: a) Component Test Planning;

Arcus Infotech Pvt Ltd - 27 -

Software Testing ________________________________________________________________________ b) Component Test Specification; c) Component Test Execution; d) Component Test Recording; e) Checking for Component Test Completion.

This Figure illustrates the generic test process described in Component Test Planning shall begin the test process and Checking for Component Test Completion shall end it; these activities are carried out for the whole component. Component Test Specification, Component Test Execution, and Component Test Recording may however, on any one iteration, be

Arcus Infotech Pvt Ltd - 28 -

Software Testing ________________________________________________________________________ carried out for a subset of the test cases associated with a component. Later activities for one test case may occur before earlier activities for another. Whenever an error is corrected by Making a change or changes to test materials or the component under test, the affected activities shall be repeated.

3.4.1.2. Integration Testing:
“Testing performed to expose faults in the interfaces and in the interaction between integrated components” Testing of combined parts of an application to determine they function together correctly. The 'parts' can be code modules, individual applications, client and server applications on a network etc. This type of testing is especially relevant to client/server and distributed systems. Objective: The typical objectives of software integration testing are to: • • • • Cause failures involving the interactions of the integrated software components when running on a single platform. Report these failures to the software development team so that the underlying defects can be identified and fixed. Help the software development team to stabilize the software so that it can be successfully distributed prior to system testing. Minimize the number of low-level defects that will prevent effective system and launch testing. Entry criteria: • • • The integration team is adequately staffed and trained in software integration testing. The integration environment is ready. The first two software components have: Arcus Infotech Pvt Ltd - 29 -

Software Testing ________________________________________________________________________  Passed unit testing.  Been ported to the integration environment.  Been integrated. • • • Documented Evidence that component has successfully completed unit test. Adequate program or component documentation is available Verification that the correct version of the unit has been turned over for integration. Exit criteria: • • A test suite of test cases exists for each interface between software components. All software integration test suites successfully execute (i.e., the tests completely execute and the actual test results match the expected test results). • • • • Successful execution of the integration test plan No open severity 1 or 2 defects Component stability The iterative and incremental development cycle implies that software integration testing is regularly performed in an iterative and incremental manner. • • Software integration testing must be automated if adequate regression testing is to occur. Software integration testing can elicit failures produced by defects that are difficult to detect during system or launch testing once the system has been completely integrated.

Guidelines:

3.4.1.2.1. Incremental Integration Testing
“Integration testing where system components are integrated into the system one at a time until the entire system is integrated”

Arcus Infotech Pvt Ltd - 30 -

Software Testing ________________________________________________________________________ Continuous testing of an application as new functionality is added; requires that various aspects of an application's functionality be independent enough to work separately before all parts of the program are completed, or that test drivers be developed as needed; done by programmers or by testers. Integration testing where system components are integrated into the system one at a time until the entire system is integrated. The Incremental Integration Testing will divided into 3 types, a.) Top down Integration. b.) Bottom up Integration. c.) Sandwich Integration.

a.) Top down Integration
“An approach to integration testing where the component at the top of the component hierarchy is tested first, with lower level components being simulated by stubs. Tested components are then used to test lower level components. The process is repeated until the lowest level component has been tested.” • • Modules integrated by moving down the program design hierarchy. Can use depth first or breadth first top down integration.

Arcus Infotech Pvt Ltd - 31 -

Software Testing ________________________________________________________________________ Steps: • • • • • • Main control module used as the test driver, with stubs for all subordinate modules. Replace stubs either depth first or breadth first Replace stubs one at a time. Test after each module integrated Use regression testing (conducting all or some of the previous tests) to ensure new errors are not introduced. Verifies major control and decision points early in design process.

b.) Bottom up Integration:
“An approach to integration testing where the lowest level components are tested first then used to facilitate the testing of higher level components. The process is repeated until the component at the top of the hierarchy is tested.” • • Begin construction and testing with atomic modules (lowest level modules). Use driver program to test.

Steps: Arcus Infotech Pvt Ltd - 32 -

Software Testing ________________________________________________________________________ • • • Low level modules combined in clusters (builds) that perform specific software sub functions. Driver program developed to test. Cluster is tested. Driver programs removed and clusters combined, moving upwards in program structure.

Bottom-Up
Major Features aimed to proving of particular modules. b.) Modules can be integrated in various Clusters as desired. c.) Major emphasis is on module functionality and Advantages performance. a.) No test stubs are needed manpower needs c.) Errors in critical modules are found early

Top-Down
is tested first integrated one at a time c.) Major emphasis is on interface testing

a.) Allows early testing a.) The control program Feasibility and practicality b.) Modules are

a.) No test drivers are needed plus a few modules forms a basic early prototype c.) Interface errors are discovered early d.) Modular features aid debugging a.) Test stubs are needed b.) The extended early

b.)It is easier to adjust b.) The control program

Disadvantages

a.) Test drivers are needed

Arcus Infotech Pvt Ltd - 33 -

Software Testing ________________________________________________________________________ b.)Many modules must phases dictate a slow be integrated before a manpower build-up working program is available c.) Errors in critical modules at low levels are

c.) Interface errors are found late Comments discovered late a.) At any given point, a.) more code has been with top down testing. bottom-up is a more Intuitive test philosophy. An early working

program raises morale management progress to maintain a pure topdown strategy in practice.

Written and tested that and helps convince Some people feel that Is being made. It is hard

i. Stub and Drivers:
Stubs: Stubs are program units that are stands for the other (more complex) program units that are directly referenced by the unit being tested. Stubs are usually expected to provide the following: An interface that is identical to the interface that will be provided by the actual program unit, and the minimum acceptable behaviour expected of the actual program unit. (This can be as simple as a return statement) Drivers: Drivers are programs or tools that allow a tester to exercise/examine in a controlling manner the unit of software being tested. A driver is usually expected to provide the following: A means of defining, declaring, or otherwise creating, any variables, constants, or other items needed in the testing of the unit, and a means of monitoring the states of these items, any input and output mechanisms needed in the testing of the unit Arcus Infotech Pvt Ltd - 34 -

Software Testing ________________________________________________________________________

c.) Sandwich Testing:
This testing is also called as Bidirectional Testing (Or) Hybrid Testing. It is a Combination of both bottom-up and top-down testing using testing layer.

3.4.1.3. Smoke Testing:
• • • • • • This Smoke Testing is done by Developers. This testing is one of the last tests for testers. The smoke testing is also called as Build Verification Testing. Build is nothing but .exe (execution) file. The execution file contains the Source code and Executable code. When a build is received, a smoke test is run to ascertain if the build is stable and it can be considered for further testing. Smoke testing can be done for testing the stability of any interim build. Smoke testing can be executed for platform qualification tests.

3.4.1.4. Sanity Testing:
Sanity Testing is one of the basic tests for testers. Once a new build is obtained with minor revisions, instead of doing a through regression, sanity is performed so as to ascertain the build has indeed rectified the issues and no further issue has been introduced by the fixes. Its generally a subset of regression testing and a group of test cases are executed that are related with the changes made to the app. Generally, when multiple cycles of testing are executed, sanity testing may be done during the later cycles after through regression cycles.

Smoke

Sanity Arcus Infotech Pvt Ltd - 35 -

Software Testing ________________________________________________________________________
Smoke testing originated in the hardware 1 testing practice of turning on a new piece of hardware for the first time and considering it a success if it does not catch fire and smoke. In software industry, smoke testing is a shallow and wide approach whereby all areas of the application without getting into too deep, is tested. 2 3 A smoke test is scripted--either using a written set of tests or an automated test A Smoke test is designed to touch every part of the application in a cursory way. It's is shallow and wide. 4 Smoke testing will be conducted to ensure whether the most crucial functions of a program work, but not bothering with finer details. (Such as build verification). A Sanity test is used to determine a small section of the application is still working after a minor change. Sanity testing is a cursory testing; it is performed whenever a cursory testing is sufficient to prove the application is functioning according to specifications. This level of testing is a subset of regression testing. 5 Smoke testing is normal health check up to a build of an application before taking it to testing in depth. sanity testing is to verify whether requirements are met or not, Checking all features breadth-first. A sanity test is usually unscripted. A sanity test is a narrow regression test that focuses on one or a few areas of functionality. Sanity testing is usually narrow and deep.

3.4.1.5. System Testing:
“System testing is the process of testing an integrated system to verify that it meets specified requirements". System test Entrance Criteria: • • • • Successful execution of the Integration test cases No open severity 1 or 2 defects 75-80% of total system functionality and 90% of major functionality delivered System stability for 48-72 hours to start test

Arcus Infotech Pvt Ltd - 36 -

Software Testing ________________________________________________________________________ System Test Exit Criteria: • • • • Successful execution of the system test cases ,and documentation that shows coverage of requirements and high-risk system components System meets pre-defined quality goals 100% of total system functionality delivered

3 .4.1.5.1. Types of system testing: a.) Functional Testing: b.) Non Functional Testing: A.) Functional Testing: Functional testing can be subdivided into two types:  Functionality Testing.  Sanitation Testing.

a. Functionality Testing:
Functionality testing can be done by using some coverage’s. i. ii. iii. iv. v. vi. GUI Coverage. Input Domain Coverage. Output Domain Coverage. Error Handling Coverage. Database Coverage. Order of Functionality.

i.) GUI Coverage: Under GUI Coverage the testers are checking the properties for the particular application. Properties are Arcus Infotech Pvt Ltd - 37 -

Software Testing ________________________________________________________________________ • • • • • • Height. Width. Text. Length. Start Position(X). End Position(Y).

ii.) Input Domain Coverage: Under Input Domain Coverage the Testers are checking the input for the particular application based on the customer or client requirements. Checking whether the input will be accepted by our application or not. iii.) Output Domain Coverage: Under output domain coverage the testers are checking the output for the particular application based on the given input in the input domain coverage. Output is nothing but customer expectation. iv.) Error handling Coverage: Under error Handling Coverage the testers are checking that our application have the capability to move from Abnormal State to Normal State or not. Abnormal State is nothing but Error Handling State. v.) Database Coverage: Database Coverage is also called as Backend Coverage. Database coverage is nothing but retrieving the value from the database. Database Coverage can be Sub Divided into • • Data Validation. Data Integrity.

Data Validation: It is nothing but to check whether the new values entered in the front end, that same value will be entered in the database or not. Data Integrity: Arcus Infotech Pvt Ltd - 38 -

Software Testing ________________________________________________________________________ It is nothing but to check whether the modified values will be stored in the database or not.

vi.) Order of Functionality: To check the functionality of the particular application in the order wise, that is nothing but to check the input for application in the procedure wise. If you not check the input in procedure wise it will show the error message. b. Sanitation Testing: Sanitation testing is nothing but to check the extra functionalities in the particular application. Because if you not check the extra functionalities it will affect the main functionality also. That’s why the tester first check all the main functionalities which is available in the customer requirement document, then only he/she checks the extra functionalities which is not mentioned in the customer requirement document. B.) Non Functional Testing: Non functional testing is nothing but just checking the performance of the application. Non functional testing can be sub divided into i.) Usability testing. ii.) Recovery testing. iii.) Security testing. iv.) Compatibility testing. v.) Configuration testing. vi.) Data-Volume Testing. vii.) Performance testing.

i.) Usability Testing:
Usability Testing is also called as User-Interface Testing. Arcus Infotech Pvt Ltd - 39 -

Software Testing ________________________________________________________________________ Usability Testing is nothing but User-friendliness check. Application Flow is tested, Can new user understand the application easily, Proper help documented whenever user stuck at any point. Basically system Navigation is checked in this testing. User friendliness means • • • • Easy to use. Easy to follow. Easy to understand. Look & Attractive. (Or) “Testing the ease with which users can learn and use a product.” All aspects of user interfaces are tested: • • • • Display screens. Messages. Report formats. Navigation and selection problems.

ii.) Recover testing: “Testing aimed at verifying the system's ability to recover from varying degrees of failure.” Recovery is the ability to restart operations after the integrity of the application has been lost. The process normally involves reverting to a point where the integrity of the system is known, and then reprocessing transactions up until the point of failure. The importance of recovery will vary from application to application. Objectives: Recovery testing is used to ensure that operations can be continued after a disaster, recovery testing not only verifies the recovery process, but also the effectiveness of the component parts of that process. Specific objectives of recovery testing include: Arcus Infotech Pvt Ltd - 40 -

Software Testing ________________________________________________________________________ • • • • • Adequate backup data is preserved Backup data is stored in a secure location Recovery procedure are documented Recovery personnel have been assigned and trained Recovery tools have been developed and are available

When to use Recovery Testing: Recovery testing should be performed whenever the user of the application states that the continuity of operation of the application is essential to the proper functioning of the user area. The user should estimate the potential loss associated with inability to recover operations over various time spans. The amount of the potential loss should both determine the amount of resource to be put into disaster planning as well as recovery testing. iii.) Security testing: “Testing whether the system meets its specified security objectives.” Security is a protection system that is needed for both secure confidential information and for competitive purposes to assure third parties their data will be protected. Protecting the confidentiality of the information is designed to protect the resources of the organization. Security testing is designed to evaluate the adequacy of the protective procedures and countermeasures. Objectives: Security defects do not become as obvious as other types of defects. Therefore, the objectives of security testing are to identify defects that are very difficult to identify. Even failures in the security system operation may not be detected, resulting in a loss or compromise of information without the knowledge of that loss. The security testing objectives include: • Determine that adequate attention has been devoted to identifying security risks

Arcus Infotech Pvt Ltd - 41 -

Software Testing ________________________________________________________________________ • • • Determining that a realistic definition and enforcement of access to the system has been implemented Determining that sufficient expertise exists to perform adequate security testing Conducting reasonable tests to ensure that the implemented security measures function properly When to Use security Testing: Security testing should be used when the information and/or assets protected by the application system are of significant value to the organization. The testing should be performed both prior to the system going into an operational status and after the system is placed into an operational status. The extent of testing should depend on the security risks, and the individual assigned to conduct the test should be selected based on the estimated sophistication that might be used to penetrate security.

iv.) Compatibility testing: Compatibility testing, part of software non-functional tests, is testing conducted on the application to evaluate the application's compatibility with the computing environment. Computing environment may contain some or all of the below mentioned elements:
• • •

Operating systems (MVS, UNIX, Windows, etc.) Other System Software (Web server, networking/ messaging tool, etc.) Browser compatibility (Fire fox, Netscape, Internet Explorer, Safari, etc.)

v.) Configuration testing: This testing is also called as Hardware Compatibility Testing. During this testing tester validate how well our current project is able to supports on different types of hardware technologies like as different types of printers, n/w Arcus Infotech Pvt Ltd - 42 -

Software Testing ________________________________________________________________________ interface cord(NIC),topology etc. this testing is also called as hardware testing or portable testing

vi.) Data-Volume testing: Data Volume testing is also called as Storage testing (Or) Memory Testing. Volume Testing belongs to the group of non-functional tests, which are often misunderstood and/or used interchangeably. Volume testing refers to testing a software application with a certain amount of data. This amount can, in generic terms, be the database size or it could also be the size of an interface file that is the subject of volume testing. For example, if you want to volume test your application with a specific database size, you will expand your database to that size and then test the application's performance on it. Another example could be when there is a requirement for your application to interact with an interface file (could be any file such as .dat, .xml); this interaction could be reading and/or writing on to/from the file. You will create a sample file of the size you want and then test the application's functionality with that file in order to test the performance. Volume Testing means testing the software with large volume of data in the database. vii.) Performance testing: It is nothing but to check the time response for the particular application. Time response is nothing but to check how much time it takes to perform a particular task. (Or) “Testing conducted to evaluate the compliance of a system or component with specified performance requirements.”

Arcus Infotech Pvt Ltd - 43 -

Software Testing ________________________________________________________________________ Performance testing is designed to test run time performance of software within the context of an integrated system. It is not until all systems elements are fully integrated and certified as free of defects the true performance of a system can be ascertained. Performance tests are often coupled with stress testing and often require both hardware and software infrastructure. That is, it is necessary to measure resource utilization in an exacting fashion. External instrumentation can monitor intervals, log events. By instrument the system, the tester can uncover situations that lead to degradation and possible system failure. Performance testing can be sub divided into • • Load Testing. Stress Testing.

Load testing: Load Testing involves stress testing applications under real-world conditions to predict system behaviour and performance and to identify and isolate problems. Load testing applications can emulate the workload of hundreds or even thousands of users, so that you can predict how an application will work under different user loads and determine the maximum number of concurrent users accessing the site at the same time. Stress Testing: Testing conducted to evaluate a system or component at or beyond the limits of Its specified requirements. 3.4.1.6. User Acceptance Testing:

Arcus Infotech Pvt Ltd - 44 -

Software Testing ________________________________________________________________________ Formal testing conducted to enable a user, customer, or other authorized entity to determine whether to accept a system or component” User Acceptance Testing (UAT) is performed by Users or on behalf of the users to ensure that the Software functions in accordance with the Business Requirement Document. UAT focuses on the following aspects: • • • • • • • • All functional requirements are satisfied All performance requirements are achieved Other requirements like transportability, compatibility, error recovery etc. are satisfied. Acceptance criteria specified by the user is met. SIT must be completed. Availability of stable Test Environment with the latest version of the Application. Test Cases prepared by the testing team to be reviewed and signed-off by the Project coordinator (AGM-Male). All User IDs requested by the testing team to be created and made available to the testing team one week prior to start of testing. Exit Criteria • All Test Scenarios/conditions would be executed and reasons will be provided for untested conditions arising out of the following situations • • • Non -Availability of the Functionality Deferred to the Future Release All Defects Reported are in the ‘Closed’ or ‘Deferred’ status. The client team should sign off the ‘Deferred’ defects. Entry Criteria

Arcus Infotech Pvt Ltd - 45 -

Software Testing ________________________________________________________________________ User Acceptance can be sub divided into  Alpha Testing.  Beta Testing. a. Alpha Testing: Alpha testing is conducted at the developer's site by a customer. The customer uses the software with the developer 'looking over the shoulder' and recording errors and usage problems. Alpha testing conducted in a controlled environment. b. Beta Testing: Beta testing is conducted at one or more customer sites by end users. It is 'live' testing in an environment not controlled by the developer. The customer records and reports difficulties and errors at regular intervals.

3.5. Implementation:
Just installing the developed application in the client or customer side and working with the application in the customer side.

3.6. Maintanance:
Just we main maintaining the application by using some methods.  Bug Fixing.  Enhancement.  Upgradation. 3.6.1. Bug Fixing: After Implementation we customer or client found the defect, then we need to clear the bug that is nothing but bug fixing. 3.6.2. Enhancement:

Arcus Infotech Pvt Ltd - 46 -

Software Testing ________________________________________________________________________ Adding extra features or modifying the already developed application is called enhancement. 3.6.3. Upgradation: After adding extra features just modifying the version name for the particular application.

4. SDLC Models (Software Development Life Cycle):
SDLC models are 4.1. 4.2. 4.3. 4.4. Waterfall Model. V-Model (Verification & Validation). Spiral Model (Incremental Model) Prototype Model.

4.1.) Waterfall Model: Waterfall model is also called as Sequential Model (or) Linear Model. One such approach/process used in Software Development is "The Waterfall Model". Waterfall approach was first Process Model to be introduced and followed widely in Software Engineering to ensure success of the project. In "The Waterfall" approach, the whole process of software development is divided into separate process phases. The phases in Waterfall model are: Requirement Specifications phase, Software Design, Implementation and Testing & Maintenance. All these phases are cascaded to each other so that second phase is started as and when defined set of goals are achieved for first phase and it is signed off, so the name "Waterfall Model". All the methods and processes undertaken in Waterfall Model are more visible.

Arcus Infotech Pvt Ltd - 47 -

Software Testing ________________________________________________________________________

The stages of "The Waterfall Model" are: Requirement Analysis & Definition: All possible requirements of the system to be developed are captured in this phase. Requirements are set of functionalities and constraints that the end-user (who will be using the system) expects from the system. The requirements are gathered from the end-user by consultation, these requirements are analyzed for their validity and the possibility of incorporating the requirements in the system to be development is also studied. Finally, a Requirement Specification document is created which serves the purpose of guideline for the next phase of the model. System & Software Design: Before a starting for actual coding, it is highly important to understand what we are going to create and what it should look like? The requirement specifications from first phase are studied in this phase and system design is prepared. System Design helps in specifying hardware and system requirements and also helps in defining overall system architecture. The system design specifications serve as input for the next phase of the model. Arcus Infotech Pvt Ltd - 48 -

Software Testing ________________________________________________________________________ Implementation & Unit Testing: On receiving system design documents, the work is divided in modules/units and actual coding is started. The system is first developed in small programs called units, which are integrated in the next phase. Each unit is developed and tested for its functionality; this is referred to as Unit Testing. Unit testing mainly verifies if the modules/units meet their specifications. Integration & System Testing: As specified above, the system is first divided in units which are developed and tested for their functionalities. These units are integrated into a complete system during Integration phase and tested to check if all modules/units coordinate between each other and the system as a whole behaves as per the specifications. After successfully testing the software, it is delivered to the customer. Operations & Maintenance: This phase of "The Waterfall Model" is virtually never ending phase (Very long). Generally, problems with the system developed (which are not found during the development life cycle) come up after its practical use starts, so the issues related to the system are solved after deployment of the system. Not all the problems come in picture directly but they arise time to time and needs to be solved; hence this process is referred as Maintenance. There are some disadvantages of the Waterfall Model. 1) As it is very important to gather all possible requirements during the Requirement Gathering and Analysis phase in order to properly design the system, not all requirements are received at once, the requirements from customer goes on getting added to the list even after the end of "Requirement Gathering and Analysis" phase, this affects the system development process and its success in negative aspects. 2) The problems with one phase are never solved completely during that phase and in fact many problems regarding a particular phase arise after the phase is Arcus Infotech Pvt Ltd - 49 -

Software Testing ________________________________________________________________________ signed off, this results in badly structured system as not all the problems (related to a phase) are solved during the same phase. 3) The project is not partitioned in phases in flexible way. 4) As the requirements of the customer goes on getting added to the list, not all the requirements are fulfilled, this results in development of almost unusable system. These requirements are then met in newer version of the system; this increases the cost of system development. 4.2. V-Model (Verification & Validation): The V-model is a software development process which can be presumed to be the extension of the waterfall model. Instead of moving down in a linear way, the process steps are bent upwards after the coding phase, to form the typical V shape. The V-Model demonstrates the relationships between each phase of the development life cycle and its associated phase of testing. The V-model deploys a well-structured method in which each phase can be implemented by the detailed documentation of the previous phase. Testing activities like test designing start at the beginning of the project well before coding and therefore saves a huge amount of the project time.

Arcus Infotech Pvt Ltd - 50 -

Software Testing ________________________________________________________________________

The Phases of the V-model
The V-model consists of a number of phases. The Verification Phases are on the left hand side of the V, the Coding Phase is at the bottom of the V and the Validation Phases are on the right hand side of the V.

Verification Phases
Requirements analysis
In the Requirements analysis phase, the requirements of the proposed system are collected by analyzing the needs of the user(s). This phase is concerned about establishing what the ideal system has to perform. However it does not determine how the software will be designed or built. Usually, the users are interviewed and a document called the user requirements document is generated. The user requirements document will typically describe the system’s functional, physical, interface, performance, data, security requirements etc as expected by the user. It is one which the business analysts use to communicate their understanding of the system back to the users. The users carefully review this document as this document would serve as the guideline for the system designers

Arcus Infotech Pvt Ltd - 51 -

Software Testing ________________________________________________________________________ in the system design phase. The user acceptance tests are designed in this phase. See also Functional requirements, and Non-functional requirements

System Design
Systems design is the phase where system engineers analyze and understand the business of the proposed system by studying the user requirements document. They figure out possibilities and techniques by which the user requirements can be implemented. If any of the requirements are not feasible, the user is informed of the issue. A resolution is found and the user requirement document is edited accordingly. The software specification document which serves as a blueprint for the development phase is generated. This document contains the general system organization, menu structures, data structures etc. It may also hold example business scenarios, sample windows, reports for the better understanding. Other technical documentation like entity diagrams, data dictionary will also be produced in this phase. The documents for system testing is prepared in this phase.

Architecture Design
The phase of the design of computer architecture and software architecture can also be referred to as high-level design. The baseline in selecting the architecture is that it should realize all which typically consists of the list of modules, brief functionality of each module, their interface relationships, dependencies, database tables, architecture diagrams, technology details etc. The integration testing design is carried out in this phase.

Module Design
The module design phase can also be referred to as low-level design. The designed system is broken up into smaller units or modules and each of them is explained so that the programmer can start coding directly. The low level design document or program specifications will contain a detailed functional logic of the module, in pseudocode:

Arcus Infotech Pvt Ltd - 52 -

Software Testing ________________________________________________________________________
• • • • •

database tables, with all elements, including their type and size all interface details with complete API references all dependency issues error message listings complete input and outputs for a module.

The unit test design is developed in this stage.

Validation Phases
Unit Testing
In the V-model of software development, unit testing implies the first stage of dynamic testing process. According to software development expert Barry Boehm, a fault discovered and corrected in the unit testing phase is more than a hundred times cheaper than if it is done after delivery to the customer. It involves analysis of the written code with the intention of eliminating errors. It also verifies that the codes are efficient and adheres to the adopted coding standards. Testing is usually white box. It is done using the Unit test design prepared during the module design phase. This may be carried out by software developers.

Integration Testing
In integration testing the separate modules will be tested together to expose faults in the interfaces and in the interaction between integrated components. Testing is usually black box as the code is not directly checked for errors.

System Testing
System testing will compare the system specifications against the actual system. The system test design is derived from the system design documents and is used in this phase. Sometimes system testing is automated using testing tools. Once all the modules are integrated several errors may arise. Testing done at this stage is called system testing. Arcus Infotech Pvt Ltd - 53 -

Software Testing ________________________________________________________________________

User Acceptance Testing
Acceptance testing is the phase of testing used to determine whether a system satisfies the requirements specified in the requirements analysis phase. The acceptance test design is derived from the requirements document. The acceptance test phase is the phase used by the customer to determine whether to accept the system or not.

c.) Spiral Model (Incremental Model): The spiral model is a software development process combining elements of both design and prototyping-in-stages, in an effort to combine advantages of top-down and bottom-up concepts. Also known as the spiral lifecycle model (or spiral development), it is a systems development method (SDM) used in information technology (IT). This model of development combines the features of the prototyping model and the waterfall model. The spiral model is intended for large, expensive and complicated projects.

Arcus Infotech Pvt Ltd - 54 -

Software Testing ________________________________________________________________________

History
The spiral model was defined by Barry Boehm in his 1988 article "A Spiral Model of Software Development and Enhancement". This model was not the first model to discuss iterative development, but it was the first model to explain why the iteration matters. As originally envisioned, the iterations were typically 6 months to 2 years long. Each phase starts with a design goal and ends with the client (who may be internal) reviewing the progress thus far. Analysis and engineering efforts are applied at each phase of the project, with an eye toward the end goal of the project.

Steps
The steps in the spiral model can be generalized as follows: 1. The new system requirements are defined in as much detail as possible. This usually involves interviewing a number of users representing all the external or internal users and other aspects of the existing system. Arcus Infotech Pvt Ltd - 55 -

Software Testing ________________________________________________________________________ 2. A preliminary design is created for the new system. This phase is the most important part of "Spiral Model". In this phase all possible (and available) alternatives, which can help in developing a cost effective project are analyzed and strategies are decided to use them. This phase has been added specially in order to identify and resolve all the possible risks in the project development. If risks indicate any kind of uncertainty in requirements, prototyping may be used to proceed with the available data and find out possible solution in order to deal with the potential changes in the requirements. 3. A first prototype of the new system is constructed from the preliminary design. This is usually a scaled-down system, and represents an approximation of the characteristics of the final product. 4. A second prototype is evolved by a fourfold procedure: 1. evaluating the first prototype in terms of its strengths, weaknesses, and risks; 2. defining the requirements of the second prototype; 3. planning and designing the second prototype; 4. constructing and testing the second prototype.

Applications
The spiral model is used most often in large projects. For smaller projects, the concept of agile software development is becoming a viable alternative. The US military has adopted the spiral model for its Future Combat Systems program.

Advantages
1. The spiral model promotes quality assurance through prototyping at each stage in systems development. 2. The Software is produced early in the software life cycle. 3. It facilitates high amount of risk analysis.

Arcus Infotech Pvt Ltd - 56 -

Software Testing ________________________________________________________________________ 4.4. Prototype Model: Rapid
prototyping

models are essential for identifying design flaws and gaining
design process.

valuable feedback during the

Functional rapid prototyping models

allow product designers and engineers to see how their designs look and function in real world situations. And using rapid prototyping models allows product companies to win new business, develop better products and improve production planning. But in order to be truly effective for functional testing and other applications, engineers need to create multiple iterations of rapid prototyping models, and they need to be able to create them quickly and inexpensively. (Or) It is a extension to waterfall model. The prototype model in the design phase, the designers after developing design, he sends these designed document to the client or customer who specified the requirements. He will go through the design and if the designs are not acceptable, satisfy able he will simply reject the design. The designer make modifications to the required design and again sends the modified designs to the client or customer for getting feedback on these design. If the client or customer rejects the design again, the modifications are done & they are sent to client or customer until the designers receives positive feedback on the developed designs. Requirement gathering

Quick design

Customer Evaluation of the prototype

Design

Arcus Infotech Pvt Ltd - 57 -

Software Testing ________________________________________________________________________ Coding

Testing

Maintenance

5. Testing Techniques:
Testing Techniques will be 5.1. Black-Box Testing. 5.2. White-Box Testing. 5.3. Grey Box Testing. 5.1. Black-Box Testing: Black box testing treats the system as a “black-box”, so it doesn’t explicitly use Knowledge of the internal structure or code. Or in other words the Test engineer need not know the internal working of the “Black box” or application. Main focus in black box testing is on functionality of the system as a whole. Each testing method has its own advantages and disadvantages. There are some bugs that cannot be found using only black box or only white box. Majority of the application are tested by black box testing method. We need to cover majority of test cases so that most of the bugs will get discovered by blackbox testing. a. Tools used for Black Box testing:

Black box testing tools are mainly record and playback tools. These tools are used for regression testing that to check whether new build has created any bug in previous working application functionality. These record and playback tools records test cases in the form of some scripts like TSL, VB script, Java script, Perl.

Arcus Infotech Pvt Ltd - 58 -

Software Testing ________________________________________________________________________

b. Advantages of Black Box Testing • •
•

Tester can be non-technical. Used to verify contradictions in actual system and the specifications. Test cases can be designed as soon as the functional specifications are complete.

c. Disadvantages of Black Box Testing: • • The test inputs needs to be from large sample space. It is difficult to identify all possible inputs in limited testing time. So writing test cases is slow and difficult • Chances of having unidentified paths during this testing

5.1.1. Types of Black-box Testing: i. Boundary Value Analysis: Many systems have tendency to fail on boundary. So testing boundry values of application is important. Boundary Value Analysis (BVA) is a test Functional Testing technique where the extreme boundary values are chosen. Boundary values include maximum, minimum, just inside/outside boundaries, typical values, and error values. Extends equivalence partitioning Test both sides of each boundary Look at output boundaries for test cases too Test min, min-1, max, max+1, typical values a. BVA techniques:
1. 2.

Number of variables For n variables: BVA yields 4n + 1 test cases. Kinds of ranges Generalizing ranges depends on the nature or type of variables

Advantages of Boundary Value Analysis
1.

Robustness Testing - Boundary Value Analysis plus values that go beyond the limits Arcus Infotech Pvt Ltd - 59 -

Software Testing ________________________________________________________________________ 2. Min - 1, Min, Min +1, Nom, Max -1, Max, Max +1 3. Forces attention to exception handling b. Limitations of Boundary Value Analysis

Boundary value testing is efficient only for variables of fixed values i.e boundary. ii. Equivalence Class Partitioning: Equivalence partitioning is a black box testing method that divides the input domain of a program into classes of data from which test cases can be derived. How this partitioning is performed while testing: 1. If an input condition specifies a range, one valid and one two invalid classes are defined. 2. If an input condition requires a specific value, one valid and two invalid equivalence classes are defined. 3. If an input condition specifies a member of a set, one valid and one invalid equivalence class is defined. 4. If an input condition is Boolean, one valid and one invalid class is defined. iii. Error Guessing: This is purely based on previous experience and judgment of tester. Error Guessing is the art of guessing where errors can be hidden. For this technique there are no specific tools, writing the test cases that cover all the application paths. 5.2. White Box Testing: White box testing (WBT) is also called Structural or Glass box testing. White box testing involves looking at the structure of the code. When you know the internal structure of a product, tests can be conducted to ensure that the internal operations performed according to the specification. And all internal components have been adequately exercised. 5.2.1. Types of white-box testing: Arcus Infotech Pvt Ltd - 60 -

Software Testing ________________________________________________________________________ • • • • Basic Path Testing: Control Structure testing: Program technique testing: Mutation Testing:

i. Basic Path Testing: The white box testers are using this technique to estimate the execution of a program, without any disturbance such that the program should cover all the independent paths defined in it. That is the program has to be executed depending upon how many no of independent paths defined in it. To implement this technique , the programmers are follow the approaches. Step1: preparing a program with respect to design logic. Step2: prepare flowchart for that program. Step3: calculating cyclomatic complexity. Step 4: run the program more than 1 time to cover all the independent paths. Eg: 1 2

3

6

4

7

8

5

Arcus Infotech Pvt Ltd - 61 -

Software Testing ________________________________________________________________________

The no of independent paths are Path1: 1-2-3-6-7 Path2: 1-2-3-6-8 Path3: 1-2-3-4-5 a. Cyclomatic Complexity: The cyclomatic complexity is a measurement for finding the no of independent paths in a flow graph. ii. Control Structure Testing: Validating every input statement and output statement correctness for a control structure. a. Branch testing:
• •

also called Decision Testing Definition: "For every decision, each branch needs to be executed at least once." Shortcoming - ignores implicit paths that result from compound conditionals. Treats a compound conditional as a single statement. (We count each branch taken out of the decision, regardless which condition lead to the branch.)

• •

•

This example has two branches to be executed:

IF ( a equals b) THEN statement 1 ELSE statement 2 END IF

Arcus Infotech Pvt Ltd - 62 -

Software Testing ________________________________________________________________________
•

This examples also has just two branches to be executed, despite the compound conditional:

IF ( a equals b AND c less than d ) THEN statement 1 ELSE statement 2 END IF

•

This example has three branches to be executed:

IF ( a equals b) THEN statement 1 ELSE IF ( c equals d) THEN statement 2 ELSE statement 3 END IF END IF
• •

Obvious decision statements are if, for, while, switch. Subtle decisions are return (Boolean expression), ternary expressions, and try-catch. For this course you don't need to write test cases for IOException and OutOfMemory exception.

•

Arcus Infotech Pvt Ltd - 63 -

Software Testing ________________________________________________________________________

b. Condition testing: Validating a simple Boolean “if” condition with respect to its input statements and output statements correctness. (Or) Condition testing is a test construction method that focuses on exercising the logical conditions in a program module. Errors in conditions can be due to:
• • • • •

Boolean operator error Boolean variable error Boolean parenthesis error Relational operator error Arithmetic expression error

Definition: "For a compound condition C, the true and false branches of C and every simple condition in C need to be executed at least once."

Multiple-condition testing requires that all true-false combinations of simple conditions be exercised at least once. Therefore, all statements, branches, and conditions are necessarily covered. c. Dataflow testing: Validating the flow of data with respect to a control statement. (Or) Selects test paths according to the location of definitions and use of variables. This is a somewhat sophisticated technique and is not practical for extensive use. Its use should be targeted to modules with nested if and loop statements.

Arcus Infotech Pvt Ltd - 64 -

Software Testing ________________________________________________________________________

d. Loop testing: Validating the looping control structures for its defined no of iterations. (Or) Loops are fundamental to many algorithms and need thorough testing. There are four different classes of loops: simple, concatenated, nested, and unstructured. Examples:

Create a set of tests that force the following situations:
•

Simple Loops, where n is the maximum number of allowable passes through the loop.
o o o

Skip loop entirely Only one pass through loop Two passes through loop

Arcus Infotech Pvt Ltd - 65 -

Software Testing ________________________________________________________________________
o o •

m passes through loop where m<n. (n-1), n, and (n+1) passes through the loop. Start with inner loop. Set all other loops to minimum values. Conduct simple loop testing on inner loop. Work outwards Continue until all loops tested. If independent loops, use simple loop testing. If dependent, treat as nested loops. Don't test - redesign.

Nested Loops
o o o o

•

Concatenated Loops
o o

•

Unstructured loops
o

public class loopdemo { private int[] numbers = {5,-3,8,-12,4,1,-20,6,2,10}; /** Compute total of numItems positive numbers in the array * @param numItems how many items to total, maximum of 10. */ public int findTotal(int numItems) { int total = 0; if (numItems > 0 && numItems <= 10) { for (int count=0; count < numItems; count = count + 1) { if (numbers[count] > 0) { total = total + numbers[count]; } } } return total; } } public void testOne() Arcus Infotech Pvt Ltd - 66 -

Software Testing ________________________________________________________________________ { loopdemo app = new loopdemo(); assertEquals(0, app.findTotal(0)); assertEquals(5, app.findTotal(1)); assertEquals(5, app.findTotal(2)); assertEquals(17, app.findTotal(5)); assertEquals(26, app.findTotal(9)); assertEquals(36, app.findTotal(10)); assertEquals(0, app.findTotal(11)); } iii. Program technique Testing: During this testing the programmers are calculating the execution time of a program using monitors. If the program execution is not acceptable then the programmers are performing changes in the structures of a program without disturbing the functionality. I.e. if a program takes more time to complete its execution the programmers are reducing the internal steps of the program without disturbing the external functionality of the program. iv. Mutation testing: It means a change in a program. Mutation testing means the programmers are performing changes in a testing program to estimate the correctness & completeness of program testing .i.e. they will make modifications within a program and validates that modified program to check whether it is working properly or not with respect to the modifications.

5.2.2. Why we do White Box Testing? To ensure:
•

That all independent paths within a module have been exercised at least once. All logical decisions verified on their true and false values. All loops executed at their boundaries and within their operational bounds internal data structures validity.

• •

Arcus Infotech Pvt Ltd - 67 -

Software Testing ________________________________________________________________________ 5.2.3. Need of White Box Testing? To discover the following types of bugs:
•

Logical error tend to creep into our work when we design and implement functions, conditions or controls that are out of the program The design errors due to difference between logical flow of the program and the actual implementation Typographical errors and syntax checking

•

•

Skills Required: We need to write test cases that ensure the complete coverage of the program logic. For this we need to know the program well i.e. we should know the specification and the code to be tested. Knowledge of programming languages and logic. 5.2.4. Limitations of WBT: Not possible for testing each and every path of the loops in program. This means exhaustive testing is impossible for large systems. This does not mean that WBT is not effective. By selecting important logical paths and data structure for testing is practically possible and effective. 5.3. Grey Box Testing: • Grey box Testing is the new term, which evolved due to the different architectural usage of the system. This is just a combination of both Black box & White box testing. Tester should have the knowledge of both the internals and externals of the function. • • Tester should have good knowledge of White Box Testing and complete knowledge of Black Box Testing. Grey box testing is especially important with Web and Internet applications, because the Internet is built around loosely

Arcus Infotech Pvt Ltd - 68 -

Software Testing ________________________________________________________________________ integrated components that connect via relatively well defined interfaces

6. STLC Process: (Software Test Life Cycle):
STLC Process is one of the guideline for testing the particular application. The STLC is including in the system testing process. In the system testing the test engineers have to test the developed software by following this STLC phases Test Strategy Document

Test Initiation

Test Plan

Test Plan Document

Test Design

Test Scenario, Test Case

Execute test Case

Test Report

Defect Report Document

Test Closure

Arcus Infotech Pvt Ltd - 69 -

Software Testing ________________________________________________________________________ 6.1. Test Initiation Phase: In this phase the project manager category people will be involved. They will receive the reviewed BRS & SRS documents & prepares a testing document.

a. Test Strategy Document: This is a company level document which is prepared by the project manager category people, this document defines the testing approach. The first stage is the formulation of a test strategy. A test strategy is a statement of the overall approach to testing, identifying what levels of testing are to be applied and the methods, techniques and tools to be used. A test strategy should ideally be organization wide, being applicable to all of an organizations software developments. Developing a test strategy which efficiently meets the needs of an organization is critical to the success of software development within the organization. plan. IEEE for Test Strategy Document: i. Scope And Objective: Scope means the purpose or need for testing the developed project (i.e). What is the need of testing? (Or) why we require testing for developed project. The process or need of testing in this project is to validate the developed software with respect to customer specification. (i.e.) to make the developed software to meet the customer expectations. Arcus Infotech Pvt Ltd - 70 The application of a test strategy to a software development project should be detailed in the projects software quality

Software Testing ________________________________________________________________________ The objective of testing in this project is to find the defects as many as possible while testing the developed software.

ii. Budget (or) Business issues: This component defines how much amount of the budget is allocated for testing in this project. Eg: 100% Project Cost

60% For development process & Maintenance iii. Roles and responsibilities:

40% for testing process

The names of jobs of test engineer in a testing team and their responsibilities. The name of the jobs is the various levels of test engineer in a testing team such as senior test engineer & junior test engineer. iv. Communication & status reports: Communication defines the way of communication between roles in a testing team & the way of communication of testing team with others who worked in this project. Status reporting means reporting the daily statuses to the test lead by the test engineers. v. Test Automation & Tools:

Arcus Infotech Pvt Ltd - 71 -

Software Testing ________________________________________________________________________ It defines the need of automation testing in this project and if automation testing is required then whether that particular automation tool is available in our organization or not for this project. vi. Change & Configuration Management: Change means changes or modifications done to the test deliverables. Configuration management means maintaining all the test deliverables , modifications done to test deliverable have to be maintained in the database of organization for future reference. Change & Configuration management means the project manager gives the information regarding the changes in the test deliverables & Maintaining these test deliverables for future reference. vii. Risk & Assumptions: The list of analyzed risks & solutions to overcome these risks by the testing team while testing the developed software in future. The risks & solutions for the risks are prepared by the project manager by analyzing the risks. viii. Training plan: The required no of training sessions that the testing team requires to understand the requirements developed in the project properly and perfectly. s 6.2. Test Plan Phase: A test plan states what the item to be tested are, at what level they will be tested, what sequence they are to be tested in , how the test strategy will be applied to the testing of each item, and describes the test Environment. Test Plan Document will be divided into  Master Test Plan.  Detailed Test Plan.

Arcus Infotech Pvt Ltd - 72 -

Software Testing ________________________________________________________________________ 6.2.1. Master Test Plan: Master Test Plan is the high level view of the testing approach, a. Testing Team Formation. Test Manager concentrates on below factor to form Testing Team for corresponding project. Test Manager will check for Availability of test engineers (Selection will be made in 3:1 ratio) b. Identifying Tactical Risks. During Test Plan writing author concentrate on identifying risks w.r.t team formation Lack of knowledge of Testers in the domain. • • • • • • Lack of Budget. Lack of Resources. Delays in Delivery. Lack of Test Data. Lack of Development Process Rigor. Lack of communication (In between testing team and development team) After completion of Team Formation and Risk Finding, Author(Test manager or Test Lead) start writing Test Plan document Here are the steps of writing the test plan. 6.2.2. Detailed test plan: This is a summary of the ANSI/IEEE Standard 829-1983. It describes a test plan as: “A document describing the scope, approach, resources, and schedule of intended testing activities. It identifies test items, the features to be tested, the testing tasks, who will do each task, and any risks requiring contingency Planning.” Arcus Infotech Pvt Ltd - 73 -

Software Testing ________________________________________________________________________ … (ANSI/IEEE Standard 829-1983) This standard specifies the following test plan outline: a. Test Plan Identifier • • • • A unique identifier Summary of the items and features to be tested Need for and history of each item (optional) References to related documents such as project authorization, project plan, QA plan, configuration management plan, relevant policies, relevant standards • • • • • • • • References to lower level test plans Test items and their version Characteristics of their transmittal media References to related documents such as requirements specification, design specification, users guide, operations guide, installation guide References to bug reports related to test items Items which are specifically not going to be tested (optional) All software features and combinations of features to be tested References to test-design specifications associated with each feature and combination of features e. Features Not to Be Tested • • • • All features and significant combinations of features which will not be tested The reasons these features won’t be tested Overall approach to testing For each major group of features of combinations of featres, specify the approach c. Test Items b. Introduction

d. Features to be tested

f. Approach

Arcus Infotech Pvt Ltd - 74 -

Software Testing ________________________________________________________________________ • • • • • • Specify major activities, techniques, and tools which are to be used to test the groups Specify a minimum degree of comprehensiveness required Identify which techniques will be used to judge comprehensiveness Specify any additional completion criteria Specify techniques which are to be used to trace requirements Identify significant constraints on testing, such as test-item availability, testing resource availability, and deadline g. Item Pass/Fail Criteria • Specify the criteria to be used to determine whether each test item has passed or failed testing

h. Test Deliverables • Identify the deliverable documents: test plan, test design specifications, test case specifications, test procedure specifications, test item transmittal reports, test logs, test incident reports, test summary reports • • • • • • • • Identify test input and output data Identify test tools (optional) Identify tasks necessary to prepare for and perform testing Identify all task interdependencies Identify any special skills required Specify the level of security required Identify special test tools needed Specify necessary and desired properties of the test environment: physical characteristics of the facilities including hardware, communications and Arcus Infotech Pvt Ltd - 75 -

i. Testing Tasks

j. Environmental Needs

Software Testing ________________________________________________________________________ system software, the mode of usage (i.e., stand-alone), and any other software or supplies needed • • • • • Identify any other testing needs Identify the source for all needs which are not currently available Identify groups responsible for managing, designing, preparing, executing, witnessing, checking and resolving Identify groups responsible for providing the test items identified in the Test Items section Identify groups responsible for providing the environmental needs identified in the Environmental Needs section l. Staffing and Training Needs • • Specify staffing needs by skill level Identify training options for providing necessary skills

k. Responsibilities

m. Schedule • • • • • • • • • Specify test milestones Specify all item transmittal events Estimate time required to do each testing task Schedule all testing tasks and test milestones For each testing resource, specify its periods of use Identify the high-risk assumptions of the test plan Specify contingency plans for each Specify the names and titles of all persons who must approve the plan Provide space for signatures and dates

n. Risks and Contingencies

o. Approvals

6.3. Test Design Phase: Arcus Infotech Pvt Ltd - 76 -

Software Testing ________________________________________________________________________ 6.3.1. Test Scenario Document: A set of test cases that ensure that the business process flows are tested from end to end. They may be independent tests or a series of tests that follow each other, each dependent on the output of the previous one. Test scenario templates: S.no Module Requirements Test scenario Test case

6.3.2. Test Case Document: It is a group of steps that is to be executed to check the functionality of a specific object. The main objective of writing test case is to validate the test coverage of an application. Test case Templates: Test case id Test case Step description Name Step Test Test input Expected Actual result result description data (or)

6.4. Execute Test Case Phase: Executing all the test cases based on the functional specification. 6.5. Test Report Phase: 6.5.1. Defect Report Document: The document that contains the information regarding accepted defects & rejected defects, defect corrected, status of the defect. IEEE format for defect report document: Arcus Infotech Pvt Ltd - 77 -

Software Testing ________________________________________________________________________ a. Defect id (or) name: A unique number or name must be given to he defect by the test engineer for future reference. b. Defect description (or) introduction: A brief summary or a brief description of the identified defects. c. Severity: It means the seriousness of the defect in terms of functionality. i. High severity: The software build is not working correctly due to occurrence of defect and not able to continue remaining testing until that defect is resolved. Eg: login. ii. Medium severity: The software build is not working correctly due to occurrence of defect but able to continue remaining testing and that defect must be resolved completely. iii. Low severity: The build is having a defect but it may or may not be resolved. Eg: unwanted options available in the application. d. Priority: The importance of the defect to be resolved in terms of severity. (Or) It is nothing but how fast to fix the bug in terms of severity. e. Reprodusable: It means during execution the defect can be reproduced again and again or not. Two options are available to mention this • Yes: the defect can be reproduced again. Attach the test procedure in this component & send it to defect tracking team.

Arcus Infotech Pvt Ltd - 78 -

Software Testing ________________________________________________________________________ • No: the defect cannot be reproduced again. Attach the test procedure & snapshot and forward to development team. f. Status: the status will be new g. Tested by: Testers name should be mentioned. h. Fixed by: Developers name should be mentioned. i. Reported on: The date in which the defect report was reported. 6.5.2. Tools Used Tools that are used to track and report defects are, a. Clear Quest (CQ) It belongs to the Rational Test Suite and it is an effective tool in Defect Management. CQ functions on a native access database and it maintains a common database of defects. With CQ the entire Defect Process can be customized. For e.g., a process can be designed in such a manner that a defect once raised needs to be definitely authorized and then fixed for it to attain the status of retesting. Such a systematic defect flow process can be established and the history for the same can be maintained. Graphs and reports can be customized and metrics can be derived out of the maintained defect repository. b. Test Director (TD): Test Director is an Automated Test Management Tool developed by Mercury Interactive for Test Management to help to organize and manage all phases of the software testing process, including planning, creating tests, executing tests, and tracking defects. Test Director enables us to manage user access to a project by creating a list of authorized users and assigning each user a password and a user group such that a perfect control can be exercised on the kinds of additions and modifications and user can make to the project. Apart from Manual Test Execution, the Win Runner automated test scripts of the project can also be executed directly from Test Director.

Arcus Infotech Pvt Ltd - 79 -

Software Testing ________________________________________________________________________ Test Director activates Win Runner, runs the tests, and displays the results. Apart form the above, it is used for • • • • To report defects detected in the software. As a sophisticated system for tracking software defects. To monitor defects closely from initial detection until resolution. To analyze our Testing Process by means of various graphs and reports.

c. Defect Tracker Defect Tracker is a tool developed by Maveric Systems Ltd. an Independent Software Testing Company in Chennai for defect management. This tool is used to manage the defect, track the defect and report the defect effectively by the testing team.

Test Closure Phase:
a. Sign Off Sign off Criteria: In order to acknowledge the completion of the test process and certify the application, the following has to be completed. • • • All passes have been completed All test cases should have been executed All defects raised during the test execution have either been closed or deferred b. Authorities The following personnel have the authority to sign off the test execution process • • • Client: The owners of the application under test Project manager: Maveric Personnel who managed the project Project Lead: Maveric Personnel who managed the test process

c. Deliverables The following are the deliverables to the Clients Arcus Infotech Pvt Ltd - 80 -

Software Testing ________________________________________________________________________ • • • • • • Test Strategy High Level Test Conditions or Scenarios and Test Conditions document Consolidated defect report Weekly Status report Traceability Matrix Test Acceptance/Summary Report.

d. Metrics i. Defect Metrics Analysis on the defect report is done for management and client information. These are categorized as ii. Defect age: Defect age is the time duration between the points of introduction of defect to the point of closure of the defect. This would give a fair idea on the defect set to be included for smoke test during regression. iii. Defect Analysis: The analysis of the defects can be done based on the severity, occurrence and category of the defects. As an example Defect density is a metric which gives the ratio of defects in specific modules to the total defects in the application. Further analysis and derivation of metrics can be done based on the various components of the defect management.

7. Types of Testing
7.1. Compliance Testing Involves test cases designed to verify that an application meets specific criteria, such as processing four-digit year dates, properly handling special data boundaries and other business requirements. 7.2. Intersystem Testing / Interface Testing “Integration testing where the interfaces between system components are tested” Arcus Infotech Pvt Ltd - 81 -

Software Testing ________________________________________________________________________ The intersystem testing is designed to check and verify the interconnection between application function correctly Applications are frequently interconnected to other systems. The interconnection may be data coming into the system from another application, leaving for another application frequently in multiple cycles .The intersystem testing involves the operations of multiple systems in test. The basic need of intersystem test arises whenever there is a change in parameters between application systems, where multiple systems are integrated in cycles. 7.3. Parallel Testing • • The process of comparing test results of processing production data concurrently in both the old and new systems. Process in which both the old and new modules run at the same time so that performance and outcomes can be compared and corrected prior to deployment; commonly done with modules like Payroll. • Testing a new or an alternate data processing system with the same source data that is used in another system. The other system is considered as the standard of comparison. 7.4. Database Testing The database component is a critical piece of any data-enabled application. Today’s intricate mix of client-server and Web-enabled database applications is extremely difficult to Test productively. Testing at the data access layer is the point at which your application communicates with the database. Tests at this level are vital to improve not only your overall Test strategy, but also your product’s quality. Database testing includes the process of validation of database stored procedures, database triggers; database APIs, backup, recovery, security and database conversion. 7.5. Manual support Testing

Arcus Infotech Pvt Ltd - 82 -

Software Testing ________________________________________________________________________ Manual support testing involves all functions performed by the people in preparing data for and using data from automated system. The objective of manual support testing is • • • Verify the manual – support procedures are documented and complete Determine the manual-support responsibilities has been assigned Determine manual support people are adequately trained.

Manual support testing involves first the evaluation of the adequacy of the process and seconds the execution of the process. The method of testing may be testing is same but the objective remains the same. 7.6. Ad-hoc Testing “Testing carried out using no recognised test case design technique.” Testing without a formal test plan or outside of a test plan. With some projects this type of testing is carried out as an adjunct to formal testing. If carried out by a skilled tester, it can often find problems that are not caught in regular testing. Sometimes, if testing occurs very late in the development cycle, this will be the only kind of testing that can be performed. Sometimes ad hoc testing is referred to as exploratory testing. 7.7. Configuration Testing Testing to determine how well the product works with a broad range of hardware/peripheral equipment configurations as well as on different operating systems and software. 7.8. Pilot Testing Testing that involves the users just before actual release to ensure that users become familiar with the release contents and ultimately accept it. Often is considered a Move-to-Production activity for ERP releases or a beta test for commercial products. Typically involves many users, is conducted over a short period of time and is tightly controlled.

Arcus Infotech Pvt Ltd - 83 -

Software Testing ________________________________________________________________________ 7.9. Automated Testing Software testing that utilizes a variety of tools to automate the testing process and when the importance of having a person manually testing is diminished. Automated testing still requires a skilled quality assurance professional with knowledge of the automation tool and the software being tested to set up the tests. 7.10. Load Testing Load Testing involves stress testing applications under real-world conditions to predict system behaviour and performance and to identify and isolate problems. Load testing applications can emulate the workload of hundreds or even thousands of users, so that you can predict how an application will work under different user loads and determine the maximum number of concurrent users accessing the site at the same time. 7.11. Stress and Volume Testing “Stress Testing: Testing conducted to evaluate a system or component at or beyond the limits of its specified requirements.” “Volume Testing: Testing where the system is subjected to large volumes of data. “ Testing with the intent of determining how well a product performs when a load is placed on the system resources that nears and then exceeds capacity. Volume Testing, as its name implies, is testing that purposely subjects a system (both hardware and software) to a series of tests where the volume of data being processed is the subject of the test. Such systems can be transactions processing systems capturing real time sales or could be database updates and or data retrieval. 7.12. Usability Testing “Testing the ease with which users can learn and use a product.” Arcus Infotech Pvt Ltd - 84 -

Software Testing ________________________________________________________________________ All aspects of user interfaces are tested: • • • • Display screens messages report formats navigation and selection problems

7.13. Environmental Testing These tests check the system’s ability to perform at the installation site. Requirements might include tolerance for • • • • • • heat humidity chemical presence portability electrical or magnetic fields Disruption of power, etc.

7.14. Active Testing:In active testing tester introduced the test data and analyzing the results. For example, we will fill the tank of a car with 1 liter petrol and see it's average.

7.15. Passive Testing:Passive testing is monitoring the results of a running system without introducing any special test data. For example, a engine is running and we are listening it's sound to note noise pollution by engine. 7.16. CLIENT / SERVER TESTING This type of testing usually done for 2 tier applications (usually developed for LAN) ere we will be having front-end and backend. The application launched on front-end will be having forms and reports which will be monitoring and manipulating data Arcus Infotech Pvt Ltd - 85 -

Software Testing ________________________________________________________________________ E.g.: applications developed in VB, VC++, Core Java, C, C++, D2K, PowerBuilder etc., The backend for these applications would be MS Access, SQL Server, Oracle, Sybase, Mysql, Quadbase

7.17. WEB TESTING This is done for 3 tier applications (developed for Internet / intranet / xtranet) Here we will be having Browser, web server and DB server. The applications accessible in browser would be developed in HTML, DHTML, XML, JavaScript etc. (We can monitor through these applications) Applications for the web server would be developed in Java, ASP, JSP, VBScript, JavaScript, Perl, Cold Fusion, PHP etc. (All the manipulations are done on the web server with the help of these programs developed) The DBserver would be having oracle, sql server, Sybase, mysql etc. (All data is stored in the database available on the DB server)

8. Review 8.1. Definition Review is a process or meeting during which a work product or set of work products, is presented to project personnel, managers, users, customers, or other interested parties for comment or approval. 8.2. Types of Reviews There are three general classes of reviews: • • • Informal / peer reviews Semiformal / walk-through Formal / inspections.

Arcus Infotech Pvt Ltd - 86 -

Software Testing ________________________________________________________________________ 8.2.1. Walkthrough “A review of requirements, designs or code characterized by the author of the material under review guiding the progression of the review. “ A 'walkthrough' is an informal meeting for evaluation or informational purposes. Little or no preparation is usually required. These are led by the author of the document, and are educational in nature. Communication is therefore predominately one-way in nature. Typically they entail dry runs of designs, code and scenarios/ test cases.

8.2.2. Inspection A group review quality improvement process for written material. It consists of two aspects; product (document itself) improvement and process improvement (of both document production and inspection). An inspection is more formalized than a 'walkthrough', typically with 3-8 people including a moderator, reader, and a recorder to take notes. The subject of the inspection is typically a document such as a requirements specification or a test plan, and the purpose is to find problems and see what's missing, not to fix anything. Attendees should prepare for this type of meeting by reading thru the document; most problems will be found during this preparation. The result of the inspection meeting should be a written report. Thorough preparation for inspections is difficult, painstaking work, but is one of the most cost effective methods of ensuring quality.

Arcus Infotech Pvt Ltd - 87 -

Software Testing ________________________________________________________________________ Led by trained moderator (not author), has defined roles, and includes metrics and formal process based on rules and checklists with entry and exit criteria. 8.2.3. Informal Review • • • Unplanned and Undocumented Useful, Cheap and widely used Contrast with walkthroughs is that communication is very much two-way in nature 8.2.4. Technical Review Technical reviews are also known as peer review as it is vital that participants are made up from the 'peer group', rather than including managers. • • • • Documented Defined fault detection process Includes peers and technical experts No management participant

8.3. Comparison of review types: Review Type Walkthrough Inspection Primary Led by Participants Peers Reader, Recorder, Author, Inspector Not defined Degree of

Purpose Education Author Finding faults Moderator and process improvement

formality Presentational Formal defined inspection process. Largely unplanned and undocumented. Arcus Infotech Pvt Ltd

Informal Review

Find Problems Not defined quickly cheaply and

- 88 -

Software Testing ________________________________________________________________________ Technical review. Finding faults Chair person Peers, technical expert Formal detection process. fault

8.4. Activities performed during review: Activities in Review Deliverables in Review Factors for pitfall of review : Planning, overview meeting, Review meeting and follow-up. : Product changes, source document changes and improvements. : Lack of training, documentation and management support. Review of the Requirements / Planning and Preparing Acceptance Test At the beginning of the project the test activities must start. These first activities are: • • • • • • • • • • • Fixing of test strategy and test concept risk analysis determine criticality expense of testing test intensity Draw up the test plan Organize the test team Training of the test team - If necessary Establish monitoring and reporting Provide required hardware resources (PC, data base, …) Provide required software resources (software version, test tools, …)

The activities include the foundations for a manageable and high-quality test process. A test strategy is determined after a risk evaluation, a cost estimate and test plan are developed and progress monitoring and reporting are established.

Arcus Infotech Pvt Ltd - 89 -

Software Testing ________________________________________________________________________ During the development process all plans must be updated and completed and all decisions must be checked for validity. In a mature development process reviews and inspections are carried out through the whole process. The review of the requirement document answers questions like: Are all customers’ requirements fulfilled? Are the requirements complete and consistent? And so on. It is a look back to fix problems before going on in development. But just as important is a look forward. Ask questions like: Are the requirements testable? Are they testable with defensible expenditure? If the answer is no, then there will be problems to implement these requirements. If you have no idea how to test some requirements then it is likely that you have no idea how to implement these requirements. At this stage of the development process all the knowledge for the acceptance tests is available and to hand. So this is the best place for doing all the planning and preparing for acceptance testing. For example one can • • • • Establish priorities of the tests depending on criticality Specify (functional and non-functional) test cases Specify and - if possible - provide the required infra-structure At this stage all of the acceptance test preparation is finished and can be achieved. 8.5. Review of the Specification / Planning and Preparing System Test: In the review meeting of the specification documents ask questions like: Is the specification testable? Are they testable with defensible expenditure? Only these kinds of specifications can be realistically implemented and be used for the next steps in the development process. There must be a re-work of the specifications if the answers to the questions are no. Here all the knowledge for the system tests is available and to hand. Tasks in planning and preparing for system testing include: • • Establishing priorities of the tests depending on criticality Specifying (functional / non-functional) system test cases

Arcus Infotech Pvt Ltd - 90 -

Software Testing ________________________________________________________________________ • Defining and establishing the required infra-structure

As with the acceptance test preparation, all of the system test preparation is finished at this early development stage. • • • Review of the Architectural Design Detailed Design Planning and Preparing Integration/Unit Test

During the review of the architectural design one can look forward and ask questions like: What is about the testability of the design? Are the components and interfaces testable? Are they testable with defensible expenditure? If the components are too expensive to test a re-work of the architectural design has to be done before going further in the development process. Also at this stage all the knowledge for integration testing is available. All preparation, like specifying control flow and data flow integration test cases, can be achieved. All accordingly activities of the review of the architectural design and the integration tests can be done here at the level of unit tests. 8.6. Roles and Responsibilities In order to conduct an effective review, everyone has a role to play. More specifically, there are certain roles that must be played, and reviewers cannot switch roles easily. The basic roles in a review are: • • • • The moderator The recorder The presenter Reviewers

8.6.1. Moderator: The moderator makes sure that the review follows its agenda and stays focused on the topic at hand. The moderator ensures that side-discussions do not derail the review, and that all reviewers participate equally. Arcus Infotech Pvt Ltd - 91 -

Software Testing ________________________________________________________________________ 8.6.2. Recorder: The recorder is an often overlooked, but essential part of the review team. Keeping track of what was discussed and documenting actions to be taken is a full-time task. Assigning this task to one of the reviewers essentially keeps them out of the discussion. Worse yet, failing to document what was decided will likely lead to the issue coming up again in the future. Make sure to have a recorder and make sure that this is the only role the person plays. 8.6.3. Presenter: The presenter is often the author of the artifact under review. The presenter explains the artefact and any background information needed to understand it (although if the artifact was not self explanatory, it probably needs some work). It’s important that reviews not become “trials” – the focus should be on the artifact, not on the presenter. It is the moderator’s role to make sure that participants (including the presenter) keep this in mind. The presenter is there to kick-off the discussion, to answer questions and to offer clarification. 8.6.4. Reviewer: Reviewers raise issues. It’s important to keep focused on this, and not get drawn into side discussions of how to address the issue. Focus on results, not the means. 9. Difference Tables: 9.1. Quality Vs Testing Quality Testing “Quality is giving more cushions for Testing is an activity done to achieve user to use system with all its expected the quality. characteristics” it is usually said as journey towards excellence.

Arcus Infotech Pvt Ltd - 92 -

Software Testing ________________________________________________________________________ 9.2. Testing Vs Debugging: Testing Testing is done to find defects Debugging Debugging is an art of fixing bugs.

9.3. Quality Assurance Vs Quality Control: Quality Assurance Quality Control Study on process followed in project Study on project for its function and development. specifications. QA is a planned and systematic set of QC is a process by which product quality activities adequate necessary to provide is compared with applicable standards, that and the action taken when nonconfidence

requirements are properly established conformance is detected. and products or services conform to specified requirements. It is an activity that establishes and It is an activity , which verifies if the evaluates the processes to produce product meets predefined standards. the products by preventing the introduction of issues or defects. QA improves the processes that are QC improves the development of a applied to multiple products that will specific product or service by identifying ever be produced by a process. • • Help establish processes. Sets up measurements to evaluate in programs processes. • Identifies weaknesses processes and improves them. It is performed during development It is performed after a work product is on key artefacts, like walkthroughs, produced against established criteria review and inspection, mentor ensuring that the product integrates feedback , training, checklists and correctly into the environment. standards. Arcus Infotech Pvt Ltd - 93 the defects, reporting them and correcting the defects.

Software Testing ________________________________________________________________________ QA is the determination of QC is a demonstration of consistency,

correctness of the finak software completeness, and correctnessof the product by a development project software at each stage and between with respect to the user needs and each stahe of the development life cycle requirements and the responsibility of and the responsibility of the tester. the entire team.

9.4. Verification Vs Validation: Verification Validation Process of determining whether output Process of determining whether a fully of one phase of development conforms developed system conforms to its SRS to its previous phase. document. Verification is concerned with phase Validation is concerned about the final containment of errors. product to be error free.

9.5. Black Box Testing Vs White Box Testing: Black Box Testing White Box Testing Test case selection that is based on an Test case selection that is based on an analysis of the specification of the analysis of the internal structure of the component without reference to its component. internal workings. It focuses on global issues of It is based on how the system is built.

workflows, configuration, performance, and so forth. It attempts to find errors in the external It applied to individual components and behaviour of the code in the following interfaces, being particularly effective at categories: • Incorrect functionality. Arcus Infotech Pvt Ltd - 94 or discovering localized errors in control missing and data flows.

Software Testing ________________________________________________________________________ • • • • Interface errors. Errors in data structures used by interfaces. Behaviour errors. Initialization and termination errors. It involves insightful test planning, It involves the creation of custom test careful design, and meticulous result data. And we can reuse such test data checking. for other kinds of tests. Skilled manual tester knows how to No matter who does the structural follow a trail of bugs, a good manual testing, they will need to understand tester also applies on the spot some fundamental test design judgement to observed results that an techniques to do a good job. automated tool can’t. 9.6. IST Vs UAT: Particulars Base line document Data Environment Orientation Tester composition Purpose 9.7. SIT Vs IST System Integration Testing Integrated System testing SIT can be done when system is on the IST need integrated system of various process of integration. unit levels of independent functionality and checks its workability after integration and compares its before integration. Integrated System User Acceptance testing Functional specification Simulated Controlled Component Testing firm Verification Testing Business requirement Live data Simulated live Business Testing firm/ users validation or performance

Arcus Infotech Pvt Ltd - 95 -

Software Testing ________________________________________________________________________ 9.8. Alpha Testing Vs Beta Testing: Component Test data Test environment To achieve Tested by Supporting document used Alpha testing Simulated Controlled Functionality Customer Functional specification Beta testing Live Uncontrolled User needs Testers and end users Customer requirement specification

9.9. Test Bed Vs Test Environment: Test Bed Test Environment Test bed holds only testing documents Test environment test data, test guidelines etc.

includes

all

which supports testing which includes supportive elements namely hardware, software, tools, browsers, servers, etc.

9.10. Re-testing Vs Regression testing: Retesting Regression testing To check for a particular bug and its To check for the added or new dependencies after it is said to be functionalities effect on the existing fixed. system.

10. Regression Testing and Re-testing “Retesting of a previously tested program following modification to ensure that faults have not been introduced or uncovered as a result of the changes made.” “Regression Testing is the process of testing the changes to computer programs to make sure that the older programs still work with the new changes.” “When making improvements on software, retesting previously tested functions to make sure adding new features has not introduced new problems.”

Arcus Infotech Pvt Ltd - 96 -

Software Testing ________________________________________________________________________ Regression testing is an expensive but necessary activity performed on modified software to provide confidence that changes are correct and do not adversely affects other system components. Four things can happen when a developer attempts to fix a bug. Three of these things are bad, and one is good:

Because of the high probability that one of the bad outcomes will result from a change to the system, it is necessary to do regression testing. A regression test selection technique chooses, from an existing test set, the tests that are deemed necessary to validate modified software. There are three main groups of test selection approaches in use: • • Minimization approaches seek to satisfy structural coverage criteria by identifying a minimal set of tests that must be rerun. Coverage approaches are also based on coverage criteria, but do not require minimization of the test set. Instead, they seek to select all tests that exercise changed or affected program components. • Safe attempt instead to select every test that will cause the modified program to produce different output than original program. 10.1. Factors favour Automation of Regression Testing • • • • • • Ensure consistency Speed up testing to accelerate releases Allow testing to happen more frequently Reduce costs of testing by reducing manual labour Improve the reliability of testing Define the testing process and reduce dependence on the few who know it

Arcus Infotech Pvt Ltd - 97 -

Software Testing ________________________________________________________________________ 10.2. Tools used in Regression testing • • • • • • Win Runner from Mercury e-tester from Empirix WebFT from Radview Silktest from Radview Rational Robot from Rational QA Run from Compuware

11. Roles & Responsibilities 11.1. Test Associate Reporting To: Team Lead of a project Responsibilities: • • • • • Design and develop test conditions and cases with associated test data based upon requirements Design test scripts Executes the test ware (Conditions, Cases, Test scripts etc.) with the test data generated Reviews test ware, record defects, retest and close defects Preparation of reports on Test progress

11.2. Test Engineer Reporting To: Team Lead of a project Responsibilities:

Arcus Infotech Pvt Ltd - 98 -

Software Testing ________________________________________________________________________ • • • • • Design and develop test conditions and cases with associated test data based upon requirements Design test scripts Executes the test ware (Conditions, Cases, Test scripts etc.) with the test data generated Reviews test ware, record defects, retest and close defects Preparation of reports on Test progress

11.3. Senior Test Engineer Reporting To: Team Lead of a project Responsibilities: • • Responsible for collection of requirements from the users and evaluating the same and send out for team discussion Preparation of the High level design document incorporating the feedback received on the high level design document and initiate on the low level design document • • • • • Assist in the preparation of test strategy document drawing up the test plan Preparation of business scenarios, supervision of test cases preparation based on the business scenarios Maintaining the run details of the test execution, Review of test condition/cases, test scripts Defect Management Preparation of test deliverable documents and defect metrics analysis report

Arcus Infotech Pvt Ltd - 99 -

Software Testing ________________________________________________________________________ 11.4. Test Lead Reporting To: Test Manager Responsibilities: • • • • • • • • • • • • • • • • • Technical leadership of the test project including test approach and tools to be used Preparation of test strategy Ensure entrance criteria prior to test start-off Ensure exit criteria prior to completion sign-off Test planning including automation decisions Review of design documents (test cases, conditions, scripts) Preparation of test scenarios and configuration management and quality plan Manage test cycles Assist in recruitment Supervise test team Resolve team queries/problems Report and follow-up test systems outrages/problems Client interface Project progress reporting Defect Management Staying current on latest test approaches and tools, and transferring this knowledge to test team Ensure test project documentation

11.5. Test Manager Reporting To: Management Arcus Infotech Pvt Ltd - 100 -

Software Testing ________________________________________________________________________ Responsibilities: • • • • • • • • • • • • • • • Liaison for interdepartmental interactions: Representative of the testing team Client interaction Recruiting, staff supervision, and staff training. Test budgeting and scheduling, including test-effort estimations. Test planning including development of testing goals and strategy. Test tool selection and introduction. Coordinating pre and post test meetings. Test program oversight and progress tracking. Use of metrics to support continual test process improvement. Test process definition, training and continual improvement. Test environment and test product configuration management. Nomination of training Cohesive integration of test and development activities. Mail Training Process for training needs, if required Review of the proposal

12. Types of defects: After receiving all the defect reports, the development team will analyse these defects for reality (i.e.) the development team will make analysis on this defects to identify the type of defects by go through the explanation provided by the test engineer for the corresponding defects. If the defect is acceptable then the defect tracking team is categorizing the defects as • • • Test Procedure related defects. Test data (or) Test Input Related defects. Coding related defects.

Arcus Infotech Pvt Ltd - 101 -

Software Testing ________________________________________________________________________ • Hardware related defects (or) Infrastructure related defects.

12.1. Test Procedure related defects: The defects related to the test steps in the test procedure. (i.e.) defects in the testing process. 12.2. Test data related defects: The data or the input values given by the test engineer to test a requirement. The input values or the data for that requirement is the specification mentioned by the client or customer to that particular requirement. (Or) Test data related defects means the defects occurred in the data given to test a requirement. 12.3. Coding related defects: The defects occurred in the programming logic. 12.4. Hardware related defects: The defects occurred in the hardware configuration. Hard wares like (Scanner, Printer, Ram, Rom etc..) 13. Types of risks & its Solutions: The types of risk the tester will face during the testing the applications are • • • Due to lack of time (Buddy Testing). Due to lack of Documentation (Exploratory testing). Due to lack of Domain Knowledge (Pair testing).

13.1. Buddy Testing (Due to lack of time): Buddy means group of programmers & testers. In this kind of testing, the coding and testing processes are going on parallely due to lack time for testing (i.e.) as the develop the programs to develop the functionality, parallely it will be tested by the tester. (Or)

Arcus Infotech Pvt Ltd - 102 -

Software Testing ________________________________________________________________________ Due to lack of time and lack of test data test engineers grouped with developers to conduct test on application as early as possible this style of testing. "Buddy means group of programmers and testers". (Or) Buddy Testing is the testing technique adapted under unplanned testing situations...Where a Developer and Tester with work together in order to test and develop the application at a rapid pace. Each will do their own job and as they work together the chance of bug’s raises/fixes will be reduced so as to meet the time deadline. 13.2. Exploratory Testing (Due to lack of Documentation): Due to lack of documentation, the testing team is depending on available documents, discussions with other in a testing team, internet browsing, previous experience to understand the customer requirements, contacting the client or customer who specifies the requirement. (Or) Its a kind of testing conducting at the time of documentation lacking. In the testing process whenever we don’t have sufficient documentation to do the testing process like preparing test scenarios, test cases then we will go for the following Techniques. 1) Getting information from the previous projects 2) Taking suggestions from the seniors and leads or managers 13.3. Pair Testing (Due to lack of Domain knowledge): Due to lack of lack of domain knowledge, the test lead is defining pairs with a senior test engineer & a junior test Engineer to share their knowledge during testing. Pair means a senior test engineer, a junior test engineer. Lack of domain knowledge for test engineer in a testing team means no training sessions are provided for them to understand the development project.

Arcus Infotech Pvt Ltd - 103 -

Software Testing ________________________________________________________________________ 14. Bug Life Cycle (or) Defect life cycle: Introduction: Bug can be defined as the abnormal behavior of the software. No software exists without a bug. The elimination of bugs from the software depends upon the efficiency of testing done on the software. A bug is a specific concern about the quality of the Application under Test (AUT). Bug Life Cycle: In software development process, the bug has a life cycle. The bug should go through the life cycle to be closed. A specific life cycle ensures that the process is standardized. The bug attains different states in the life cycle. The life cycle of the bug can be shown diagrammatically as follows:

The different states of a bug can be summarized as follows: 1. New

Arcus Infotech Pvt Ltd - 104 -

Software Testing ________________________________________________________________________ 2. Open 3. Assign 4. Test 5. Verified 6. Deferred 7. Reopened 8. Duplicate 9. Rejected and 10. Closed

Description of Various Stages: 1. New: When the bug is posted for the first time, its state will be “NEW”. This means that the bug is not yet approved. 2. Open: After a tester has posted a bug, the lead of the tester approves that the bug is genuine and he changes the state as “OPEN”. 3. Assign: Once the lead changes the state as “OPEN”, he assigns the bug to corresponding developer or developer team. The state of the bug now is changed to “ASSIGN”. 4. Test: Once the developer fixes the bug, he has to assign the bug to the testing team for next round of testing. Before he releases the software with bug fixed, he changes the state of bug to “TEST”. It specifies that the bug has been fixed and is released to testing team. 5. Deferred: The bug, changed to deferred state means the bug is expected to be fixed in next releases. The reasons for changing the bug to this state have many

Arcus Infotech Pvt Ltd - 105 -

Software Testing ________________________________________________________________________ factors. Some of them are priority of the bug may be low, lack of time for the release or the bug may not have major effect on the software. 6. Rejected: If the developer feels that the bug is not genuine, he rejects the bug. Then the state of the bug is changed to “REJECTED”. 7. Duplicate: If the bug is repeated twice or the two bugs mention the same concept of the bug, then one bug status is changed to “DUPLICATE”. 8. Verified: Once the bug is fixed and the status is changed to “TEST”, the tester tests the bug. If the bug is not present in the software, he approves that the bug is fixed and changes the status to “VERIFIED”. 9. Reopened: If the bug still exists even after the bug is fixed by the developer, the tester changes the status to “REOPENED”. The bug traverses the life cycle once again. 10. Closed: Once the bug is fixed, it is tested by the tester. If the tester feels that the bug no longer exists in the software, he changes the status of the bug to “CLOSED”. This state means that the bug is fixed, tested and approved. While defect prevention is much more effective and efficient in reducing the number of defects, most organization conducts defect discovery and removal. Discovering and removing defects is an expensive and inefficient process. It is much more efficient for an organization to conduct activities that prevent defects. 15. Test Strategy: Test strategies are two types   Static testing Dynamic testing

Arcus Infotech Pvt Ltd - 106 -

Software Testing ________________________________________________________________________ 15.1. Static testing: Source code has been checked before executed. 15.2. Dynamic testing: Source code has been executed and checked. Then the actual result are compared with the expected result.

16. Testing Process / Methodology

Arcus Infotech Pvt Ltd - 107 -

Software Testing ________________________________________________________________________ 16.1. Test initiation phase:

Input • • Signed proposal Arrange internal kick-off meeting among the team members, Test Manager, Lead - Commercial and Lead - Operations • • • • • • • Distribute the baseline documents based on the individual roles defined. Prepare a micro level schedule indicating the roles allocated for all team members with timelines in MPP Fill in the Project Details form and the Top Level Project Checklist Minutes of meeting MPP to be attached to Test Strategy (Test Planning process) Project Details form Top Level Project Checklist to Test Delivery Management Output Procedure

Arcus Infotech Pvt Ltd - 108 -

Software Testing ________________________________________________________________________ 16.2. Test planning phase:

Arcus Infotech Pvt Ltd - 109 -

Software Testing ________________________________________________________________________ Input • • • • • • • Baseline documents MPP Team members understand the functionality from baseline documents Raise and obtain resolution of clarifications Internal presentation and reverse presentation to the client TL defines test environment and request the client for the same TL identifies risks associated and monitors the project level risks throughout the project. The risks at the project delivery level are maintained by the Lead - Ops • • • TL prepares Test Strategy, review is by Lead - Commercial, Lead - Ops and Test Manager AM revises commercials if marked difference between Test Strategy and the Proposal TL prepares Configuration Management and Quality Plan, Review is by Lead - Ops and Test Manager Output • • • Clarification document Test Environment Request to client Risk Analysis document to Test Delivery Management

Procedure

Arcus Infotech Pvt Ltd - 110 -

Software Testing ________________________________________________________________________

16.3. Test design phase:

Input • • • • Test Strategy Team members prepare the Test Conditions, Test Cases and Test Script TL prepares the Test Scenarios (free form) Review of the above by the Lead - Ops, Test Manager Procedure

Arcus Infotech Pvt Ltd - 111 -

Software Testing ________________________________________________________________________ • • • • Client review records are also maintained. Lead - Ops is responsible for sign-off Team members prepare Traceability matrix if agreed in Test Strategy and updated during Test Execution with defect ids Team members prepare Data Guidelines whenever required TL sends daily status reports to clients and the Test Delivery Management team. TL sends the weekly status reports to clients, Test Manager, delivery management team and the Account Manager • TL escalates any changes in baseline documents to Delivery Management team. Output • • • Test Condition/Test Case document, Test Script, Test Scenarios (free form) Traceability matrix to the client Daily and Weekly status reports to client, Test Delivery Management and Account Management 16.4. Test Execution Process

Arcus Infotech Pvt Ltd - 112 -

Software Testing ________________________________________________________________________

16.5. Defect Management Process

Input • • • • • Test Conditions/ Test Cases document Test Scenarios document Traceability matrix Validate test environment and deal with any issues Execute first rounds of testing

Procedure

Arcus Infotech Pvt Ltd - 113 -

Software Testing ________________________________________________________________________ • • • • • • • • Update the Test Condition/Test case document (and the Test Scripts, if prepared) with actual result and status of test Log in the Defect Report, consolidate all defects and send to client, Test Manager and delivery management team Conducts defect review meetings with client (as specified in Test Strategy) Consolidate the Test Conditions/Test Cases to be executed in the subsequent round in a separate document. No review or sign-off required Carry out the test in the new version of the application Changes to baseline or scope of work escalated to Lead - Ops Complete rounds/stages of testing as agreed in the Test Strategy

Send daily and weekly status reports to clients and the Test Delivery Management team • Escalate any changes in baseline documents to Delivery Management team. Output • Defect Report • Daily and Weekly status reports to the client, Test Delivery Management and Account Management 16.6. Test Closure Phase

Arcus Infotech Pvt Ltd - 114 -

Software Testing ________________________________________________________________________

Input • • • • • • Consolidated Defect Report Team Lead in consultation with Lead - Ops decides about closure of a project (both complete and incomplete) In case of incomplete testing, decisions whether to close and release deliverables are taken by delivery management team The team prepare the Quantitative measurements TL prepares the Final Testing Checklist and Lead - Ops approves the same TL prepares the Final Test Report and Lead - Ops and Test Manager Reviews the same. Internal Review records and review records of clients are also stored. Procedure

Arcus Infotech Pvt Ltd - 115 -

Software Testing ________________________________________________________________________ • Test Manager, Lead - Ops, Lead - Comm., Account Manager, TL and team members carry out the de-brief meeting. Effort variances, % compliance to schedule are documented in Project De-brief form. If required, inputs are given to Quality Department for process improvements Output • • Final Testing Checklist and Final Test Report to the client Project De-brief to Quality Department

17. Test Deliverables Template 17.1. Project Details Form

Arcus Infotech Pvt Ltd - 116 -

Software Testing ________________________________________________________________________

Arcus Infotech Pvt Ltd - 117 -

Software Testing ________________________________________________________________________

17.2. Minutes of Meeting

17.3. Top Level Project Checklist

Arcus Infotech Pvt Ltd - 118 -

Software Testing ________________________________________________________________________

Arcus Infotech Pvt Ltd - 119 -

Software Testing ________________________________________________________________________

17.4. Test Environment Request Project Code: Project Name: Application Version No: Type of Testing: Prepared By: Date: Approved By: Date:

Arcus Infotech Pvt Ltd - 120 -

Software Testing ________________________________________________________________________ 17.5. Clarification Document

17.6. Test condition / Test Case Document

Arcus Infotech Pvt Ltd - 121 -

Software Testing ________________________________________________________________________ 17.7. Test Script Document

17.8. Traceability Matrix

17.9. Daily Status Report Project Code: Project Name: Phase of the Testing Life Cycle: Application Version No: Round: Report Date: Highlights of the Day: Arcus Infotech Pvt Ltd - 122 -

Software Testing ________________________________________________________________________ 17.9.1. Design Phase

17.9.2. Execution Phase

17.9.3. Defect Distribution

Arcus Infotech Pvt Ltd - 123 -

Software Testing ________________________________________________________________________

17.9.4. Environment Non-availability

17.9.5. Other Issues / Concerns

17.9.6. General Remarks: Prepared By: Date: Approved By: Date:

17.10. Weekly Status Report Project Code: Project Name: Phase of the Testing Life Cycle: Application Version No: Report Date: Report for Week:

Arcus Infotech Pvt Ltd - 124 -

Software Testing ________________________________________________________________________ 17.10.1. Life Cycle/Process

17.10.2. Highlights of the Week 17.10.2.1. Design Phase

Win Runner Scripting Progress

17.10.2.2. Execution Phase

17.10.2.

Defect Distribution

Arcus Infotech Pvt Ltd - 125 -

Software Testing ________________________________________________________________________ 17.10.4. Environment Non-availability Total man-hours lost during the week: 17.10.5. Other Issues / Concerns

17.10.6. General Remarks Prepared By: Date: Approved By: Date: 17.11. Defect Report

Round 2& Round 3 as same as Round 1 17.12. Final Test Checklist Project Code: Project Name: Prepared By: Date: Approved By: Date:

Arcus Infotech Pvt Ltd - 126 -

Software Testing ________________________________________________________________________

Comments and Observations: Final inspection result: Approved/ Rejected 17.13. Project De-brief Form Project Code: Project Name: Prepared By: Date: Approved By: Date: Overview of the Application: Key Challenges faced during Design or Execution: Lessons learnt: Suggested Corrective Actions:

Arcus Infotech Pvt Ltd - 127 -

Software Testing ________________________________________________________________________

18. CMM Levels: SEI known as “Software Engineering Institute” at Carnegie mellon university which was initiated by the U.S defence department to help, improve software development process. SEI invented capability maturity model (CMM). Now-a-days CMM is called as CMMI(Capability Maturity Model Integration). The software organizations can receive CMMI readings by undergoing assessments by qualified auditors. The level is given to the software organization depending upon the quality that the organization is producing the software. Level 1: In a level 1 company, in order to complete a project successfully, the individuals of that company should put more effort. The success may not be repeatable. The project may be or may not be completed. Level2: It consists of software project tracking, requirement management, and realistic planning and configuration management processes. The successful practices can be repeated. Level 3: It consists of standard software development and maintenance processes. These two are integrated throughout an organization. Level 4: In this level, measurements are used to track productivity, processes and products. In this level company the quality is high. Level 5: In this level the focus on continuous process improvement. The impact of new processes and technologies can be predicted and effectively implemented when required.

Arcus Infotech Pvt Ltd - 128 -

Software Testing ________________________________________________________________________ Here success & quality is high, testing and developing is completely satisfied

19. 10 Tips you should read before automating your testing work
I was getting too many questions on when and how to automate testing process. Instead of answering them individually I thought it would be better to have some discussion here. I will put my thoughts about when to automate, how to automate or should we automate our testing work? I know there some of our readers are smarter than me. So it would be always a good idea to start a meaningful discussion on such vast topic to get in-depth idea and thoughts from experts from different areas and their experience in automation testing.

Why Automation testing? 1) You have some new releases and bug fixes in working module. So how will you ensure that the new bug fixes have not introduced any new bug in previous working functionality? You need to test the previous functionality also. So will you test manually all the module functionality every time you have some bug fixes or new functionality addition? Well you might do it manually but then you are not doing testing effectively. Effective in terms of company cost, resources, Time etc. Here comes need of Automation. - So automate your testing procedure when you have lot of regression work.

2) You are testing a web application where there might be thousands of users interacting with your application simultaneously. How will you test such a web application? How will you create those many users manually and simultaneously? Well very difficult task if done manually. - Automate your load testing work for creating virtual users to check load capacity of your application.

Arcus Infotech Pvt Ltd - 129 -

Software Testing ________________________________________________________________________ 3) You are testing application where code is changing frequently. You have almost same GUI but functional changes are more so testing rework is more. - Automate your testing work when your GUI is almost frozen but you have lot of frequently functional changes. What are the Risks associated in Automation Testing?

There are some distinct situations where you can think of automating your testing work. I have covered some risks of automation testing here. If you have taken decision of automation or are going to take sooner then think of following scenarios first. 1) Do you have skilled resources? For automation you need to have persons having some programming knowledge. Think of your resources. Do they have sufficient programming knowledge for automation testing? If not do they have technical capabilities or programming background that they can easily adapt to the new technologies? Are you going to invest money to build a good automation team? If your answer is yes then only think to automate your work. 2) Initial cost for Automation is very high: I agree that manual testing has too much cost associated to hire skilled manual testers. And if you are thinking automation will be the solution for you, Think twice. Automation cost is too high for initial setup i.e. cost associated to automation tool purchase, training and maintenance of test scripts is very high. There are many unsatisfied customers regretting on their decision to automate their work. If you are spending too much and getting merely some good looking testing tools and some basic automation scripts then what is the use of automation? 3) Do not think to automate your UI if it is not fixed: Beware before automating user interface. If user interface is changing extensively, cost associated with script maintenance will be very high. Basic UI automation is sufficient in such cases.

Arcus Infotech Pvt Ltd - 130 -

Software Testing ________________________________________________________________________ 4) Is your application is stable enough to automate further testing work? It would be bad idea to automate testing work in early development cycle (Unless it is agile environment). Script maintenance cost will be very high in such cases. 5) Are you thinking of 100% automation? Please stop dreaming. You cannot 100% automate your testing work. Certainly you have areas like performance testing, regression testing, load/stress testing where you can have chance of reaching near to 100% automation. Areas like User interface, documentation, installation, compatibility and recovery where testing must be done manually. 6) Do not automate tests that run once: Identify application areas and test cases that might be running once and not included in regression. Avoid automating such modules or test cases. 7) Will your automation suite be having long lifetime? Every automation script suite should have enough life time that its building cost should be definitely less than that of manual execution cost. This is bit difficult to analyze the effective cost of each automation script suite. Approximately your automation suite should be used or run at least 15 to 20 times for separate builds (General assumption. depends on specific application complexity) to have good ROI. Here is the conclusion: Automation testing is the best way to accomplish most of the testing goals and effective use of resources and time. But you should be cautious before choosing the automation tool. Be sure to have skilled staff before deciding to automate your testing work. Otherwise your tool will remain on the shelf giving you no ROI. Handing over the expensive automation tools to unskilled staff will lead to frustration. Before purchasing the automation tools make sure that tool is a best fit to your requirements. You cannot have the tool that will 100% match with your requirements. So find out the limitations of the tool that is best match with your requirements and then use manual testing techniques to overcome those testing

Arcus Infotech Pvt Ltd - 131 -

Software Testing ________________________________________________________________________ tool limitations. Open source tool is also a good option to start with automation. To know more on choosing automation tools read my previous posts here and here. Instead of relying 100% on either manual or automation use the best combination of manual and automation testing. This is the best solution (I think) for every project. Automation suite will not find all the bugs and cannot be a replacement for real testers. Ad-hoc testing is also necessary in many cases.

20. Q & A General Q1: Why does software have bugs? Ans: • • Miscommunication or no communication - understand the application requirements. Software complexity - the complexity of current software applications can be difficult to comprehend for anyone without experience in modern-day software development. • • Programming errors - programmers "can" make mistakes. Changing requirements - A redesign, rescheduling of engineers, effects on other projects, etc. If there are many minor changes or any major changes, known and unknown dependencies among parts of the project are likely to interact and cause problems, and the complexity of keeping track of changes may result in errors. • Time pressures - scheduling of software projects is difficult at best, often requiring a lot of guesswork. When deadlines loom and the crunch comes, mistakes will be made. • • Poorly documented code - it's tough to maintain and modify code that is badly written or poorly documented that result as bugs. Software development tools - various tools often introduce their own bugs or are poorly documented, resulting in added bugs. Q2: What does "finding a bug" consist of? Arcus Infotech Pvt Ltd - 132 -

Software Testing ________________________________________________________________________ Ans: Finding a bug consists of number of steps that are performed: • • • • • Ans: When a program is sent for testing (or a website given) a list of any known bugs should accompany the program. If a bug is found, then the list will be checked to ensure that it is not a duplicate. Any bugs not found on the list will be assumed to be new. Q4: What's the big deal about 'requirements'? Ans: Requirements are the details describing an application's externally perceived functionality and properties. Requirements should be, clear & documented, complete, reasonably detailed, cohesive, attainable, and testable. A non-testable requirement would be, for example, 'userfriendly' (too subjective). Without such documentation, there will be no clear-cut way to determine if a software application is performing correctly. Q5: What can be done if requirements are changing continuously? Ans: It's helpful if the application's initial design allows for some adaptability so that any changes done later do not require redoing the application from scratch. To makes changes easier for the developers the code should be well commented and well documented. Use rapid prototyping whenever possible to help customers feel sure of their requirements and minimize changes. Be sure that customers and management understand the scheduling impacts, inherent risks, Searching for and locating a bug Analyzing the exact circumstances under which the bug occurs Documenting the bug found Reporting the bug and if necessary, the error is simulated Testing the fixed code to verify that the bug is really fixed

Q3: What will happen about bugs that are already known?

Arcus Infotech Pvt Ltd - 133 -

Software Testing ________________________________________________________________________ and costs of significant requirements changes. Design some flexibility into test cases (this is not easily done; the best bet might be to minimize the detail in the test cases, or set up only higher-level generic-type test plans) Q6: When to stop testing? Ans: This can be difficult to determine. Many modern software applications are so complex, and run in such an interdependent environment, so a complete testing can never be performed. Common factors in deciding when to stop testing are: • • • • • • Ans: Client/server applications can be quite complex due to the multiple dependencies among clients, data communications, hardware, and servers. Thus testing requirements can be extensive. When time is limited (as it usually is) the focus should be on integration and system testing. Additionally, load/ stress/ Performance testing may be useful in determining client/server application Limitations and capabilities. Q8: Does it matter how much the software has been tested already? Ans: No. It is up to the tester to decide how much to test it before it is tested. An initial assessment of the software is made, and it will be classified into one of three possible stability levels: • Low stability (bugs are expected to be easy to find, indicating that the program has not been tested or has only been very lightly tested) Deadlines achieved (release deadlines, testing deadlines, etc.) Test cases completed with certain percentage passed Test budget depleted Coverage of code/functionality/requirements reaches a specified point Defect rate falls below a certain level Beta or Alpha testing period ends

Q7: How does a client/server environment affect testing?

Arcus Infotech Pvt Ltd - 134 -

Software Testing ________________________________________________________________________ • • Normal stability (normal level of bugs, indicating a normal amount of programmer testing) High stability (bugs are expected to be difficult to find, indicating already well tested) Q9: How is testing affected by object-oriented designs? Ans: Well-engineered object-oriented design can make it easier to trace from code to internal design to functional design to requirements. While there will be little affect on black box testing (where an understanding of the internal design of the application is unnecessary), white-box testing can be oriented to the application's objects. If the application was well designed this can simplify test design. Q10: Will automated testing tools make testing easier? Ans: A tool set that allows controlled access to all test assets promoted better communication between all the team members, and will ultimately break down the walls that have traditionally existed between various groups. Automated testing tools are only one part of a unique solution to achieving customer success. The complete solution is based on providing the user with principles, tools, and services needed to efficiently develop software. Q11: Why outsource testing? Ans: Skill and Expertise Developing and maintaining a team that has the expertise to thoroughly test complex and large applications is expensive and effort intensive. Testing a software application now involves a variety of skills. • Focus - Using a dedicated and expert test team frees the development team to focus on sharpening their core skills in design and development, in their domain areas. • Independent assessment - Independent test team looks afresh at each test project while bringing with them the experience of earlier test assignments, for different clients, on multiple platforms and across different domain areas.

Arcus Infotech Pvt Ltd - 135 -

Software Testing ________________________________________________________________________ • • Save time - Testing can go in parallel with the software development life cycle to minimize the time needed to develop the software. Reduce Cost - Outsourcing testing offers the flexibility of having a large test team, only when needed. This reduces the carrying costs and at the same time reduces the ramp up time and costs associated with hiring and training temporary personnel. Q12: What steps are needed to develop and run software tests? Ans: The following are some of the steps needed to develop and run software tests: • • • Obtain requirements, functional design, and internal design specifications and other necessary documents Obtain budget and schedule requirements Determine project-related personnel and their responsibilities, reporting requirements, required standards and processes (such as release processes, change processes, etc.) • • • • • • • • • Identify application's higher-risk aspects, set priorities, and determine scope and limitations of tests Determine test approaches and methods - unit, integration, functional, system, load, usability tests, etc. Determine Determine test environment requirements (hardware, tools, software, coverage communications, etc.) test-ware requirements (record/playback analyzers, test tracking, problem/bug tracking, etc.) Determine test input data requirements Identify tasks, those responsible for tasks, and labor requirements Set schedule estimates, timelines, milestones Determine input equivalence classes, boundary value analyses, error classes Prepare test plan document and have needed reviews/approvals Write test cases Have needed reviews/inspections/approvals of test cases Arcus Infotech Pvt Ltd - 136 -

Software Testing ________________________________________________________________________ • Prepare test environment and test ware, obtain needed user

manuals/reference documents/configuration guides/installation guides, set up test tracking processes, set up logging and archiving processes, set up or obtain test input data • • • • • • Obtain and install software releases Perform tests Evaluate and report results Track problems/bugs and fixes Retest as needed Maintain and update test plans, test cases, test environment, and test ware through life cycle Q13: What is a Test Strategy and Test Plan? Ans: A test strategy is a statement of the overall approach to testing, identifying what levels of testing are to be applied and the methods, techniques and tools to be used. A test strategy should ideally be organization wide, being applicable to all of organizations software developments. Developing a test strategy, which efficiently meets the needs of an organization, is critical to the success of software development within the organization. The application of a test strategy to a software development project should be detailed in the projects software quality plan. The next stage of test design, which is the first stage within a software development project, is the development of a test plan. A test plan states what the items to be tested are, at what level they will be tested, what sequence they are to be tested in, how the test strategy will be applied to the testing of each item, and describes the test environment. A test plan may be project wide, or may in fact be a hierarchy of plans relating to the various levels of specification and testing:

Arcus Infotech Pvt Ltd - 137 -

Software Testing ________________________________________________________________________ • An Acceptance Test Plan, describing the plan for acceptance testing of the software. This would usually be published as a separate document, but might be published with the system test plan as a single document. • A System Test Plan, describing the plan for system integration and testing. This would also usually be published as a separate document, but might be published with the acceptance test plan. • A Software Integration Test Plan, describing the plan for integration of testes software components. This may form part of the Architectural Design Specification. • Unit Test Plan(s), describing the plans for testing of individual units of software. These may form part of the Detailed Design Specifications. The objective of each test plan is to provide a plan for verification, by testing the software, that the software produced fulfils the requirements or design statements of the appropriate software specification. In the case of acceptance testing and system testing, this means the Requirements Specification. 16.2. G.E. – Interview 1. What is Software Testing? “The process of exercising or evaluating a system or system component by manual or automated means to verify that it satisfies specified requirements or to identify differences between expected and actual results..." 2. What is the Purpose of Testing? • • • To uncover hidden error. To achieve the maximum usability of the system To demonstrate expected performance of the system.

3. What types of testing do testers perform? Black-box testing, White box testing is the basic type of testing testers Performs. Apart from that they also perform a lot of tests like • • Ad-Hoc testing Cookie testing

Arcus Infotech Pvt Ltd - 138 -

Software Testing ________________________________________________________________________ • • • • • • • • • • • • • • • • • CET (Customer Experience test) Client-Server Test Configuration Tests Compatibility testing Conformance Testing Depth Test Error Test Event-Driven Full Test Negative Test Parallel Testing Performance Testing Recovery testing Sanity Test Security Testing Smoke testing Web Testing

4. What is the Outcome of Testing? A stable application, performing its task as expected. 5. What is the need for testing? The Primary need is to match requirements get satisfied with the functionality and also to answer two questions A. Whether the system is doing what it supposes to do? B. Whether the system is not performing what it is not suppose to do? 6. What are the entry criteria for Functionality and Performance testing? Functional testing: Functional Spec/ BRS (CRS) / User Manual. An integrated application, Stable for testing Performance Testing:

Arcus Infotech Pvt Ltd - 139 -

Software Testing ________________________________________________________________________ Same above mentioned baseline document support and good and healthy application that supports drastic performance testing 7. What is test metrics? After doing the actual testing, an evaluation doing on the testing to extract some information about the application health using outputs of testing. Software metrics is any type of measurement, which relates to a software system, process or related documentation. Eg: Size of code and Found bugs on that count Number of bugs reported per day. Number of Conditions/Cases tested per day It can be • • Test Efficiency Total number of tests executed

8. Why do you go for White box testing, when Black box testing is available? A benchmark that certifies Commercial (Business) aspects and also functional (technical) aspects is objectives of black box testing. Here loops, structures, arrays, conditions, files, etc., are very micro level but they are Basement for any application, So White box takes these things in Macro level and test these things 9. What are the entry criteria for Automation testing? Application should be stable. Clear Design and Flow of the application is needed 10. When to start and Stop Testing? If we follow ‘Waterfall’ model then testing can be started after coding. If ‘V’ model is followed then testing can be started at design phase itself. Regard less of model the following criteria should considered To start: When test Environment was supportive enough for testing. When Application study was confident enough To Stop: After full coverage of Scope of testing After getting enough confidence on health of the application. 11. What is Quality? Arcus Infotech Pvt Ltd - 140 -

Software Testing ________________________________________________________________________ “Fitness to use” “A journey towards excellence” 12. What is Baseline document, Can you say any two? A baseline document, which starts the understanding of the application before the tester, starts actual testing. Functional Specification Business Requirement Document 13. What is verification? A tester uses verification method to ensure the system complies with an organization standards and processes, relying on review or non executable methods (such as software, hardware, documentation and personnel) “Are we Building the Right Product” 14. What is validation? Validation physically ensures that the system operates according to plan by Executing thesystem functions through series of tests that can be observed or evaluated. “Are we building the Product Right” 15. What is quality assurance? A planned and systematic pattern for all actions necessary to provide adequate confidence that the item or product conforms to established technical requirements. 16. What is quality control? Quality Control is defined as a set of activities or techniques whose purpose is to ensure that all quality requirements are being met. In order to achieve this purpose, processes are monitored and performance problems are solved. 17. What are SDLC and TDLC? The Flow and explanation process, which clearly pictures how a software development and testing should be done, were explained in SDLC and TDLC respectively. (Software development Life Cycle and testing development Life cycle) TDLC is a informal concept and also referred as TLC Arcus Infotech Pvt Ltd - 141 -

Software Testing ________________________________________________________________________ 18. What are the Qualities of a Tester? • • • • • • • • • • Should be perfectionist Should be tactful and diplomatic Should be innovative and creative Should be relentless Should possess negative thinking with good judgment skills Should possess the attitude to break the system Unit Testing Integration testing System Testing User Acceptance Testing

19.What are the various levels of testing?

20. Tell names of some testing type which you learnt or experienced? Any 5 or 6 types which are related to companies profile is good to say in the interview, • • • • • • • • • • • • • • Ad - Hoc testing Cookie Testing CET (Customer Experience Test) Client-Server Test Configuration Tests Compatibility testing Conformance Testing Depth Test Error Test Event-Driven Full Test Negative Test Parallel Testing Performance Testing Arcus Infotech Pvt Ltd - 142 -

Software Testing ________________________________________________________________________ • • • • • Recovery testing Sanity Test Security Testing Smoke testing Web Testing

21. What exactly is Heuristic checklist approach for unit testing? It is method of achieving the most appropriate solution of several found by alternative methods is selected at successive stages testing. The check list Prepared to Proceed is called Heuristic check list 22. After completing testing, what would you deliver to the client? • • • • • • • Test deliverables namely Test plan Test Data Test design Documents (Condition/Cases) Defect Reports Test Closure Documents Test Metrics

23. What is a Test Bed? Before Starting the Actual testing the elements which supports the testing activity such as Test data, Data guide lines. Are collectively called as test Bed. 24. What is a Data Guideline? Data Guidelines are used to specify the data required to populate the test bed and prepare test scripts. It includes all data parameters that are required to test the conditions derived from the requirement / specification The Document, which supports in preparing test data are called Data guidelines 25. Why do you go for Test Bed? When Test Condition is executed its result should be compared to Test result (expected result), as Test data is needed for this here comes the role of test Bed where Test data is made ready. 26. What is Severity and Priority and who will decide what? Arcus Infotech Pvt Ltd - 143 -

Software Testing ________________________________________________________________________ Severity: How much the Bug found is supposed to affect the systems Function/Performance, Usually we divide as Emergency, High, Medium, and Low. Priority: Which Bug should be solved fist in order of benefit of system’s health? Normally it starts from Emergency giving first Priority to Low as last Priority. 27. Can Automation testing replace manual testing? If it so, how? Automated testing can never replace manual Testing. As these tools to Follow GIGO principle of computer tools. Absence of creativity and innovative thinking. But 1. It speeds up the process. Follow a clear Process, which can be reviewed easily. Better Suited for Regression testing of Manually tested Application and Performance testing. 28. What is a test case? A Test Case gives values / qualifiers to the attributes that the test condition can have. Test cases, typically, are dependent on data / standards. A Test Case is the end state of a test condition, i.e., it cannot be decomposed or broken down further. Test Case design techniques for Black box Testing. • • • • • • Decision table Equivalence Partitioning Method Boundary Value Analysis Cause Effect Graphing State Transition Testing Syntax Testing

29. What is a test condition? A Test Condition is derived from a requirement or specification. It includes all possible combinations and validations that can be attributed to that requirement/specification. 30. What is the test script?

Arcus Infotech Pvt Ltd - 144 -

Software Testing ________________________________________________________________________ A Test Script contains the Navigation Steps, Instructions, Data and Expected Results required to execute the test case(s). Any test script should say how to drive or swim through out the application even for a new user. 31. What is the test data? The value which are given at expected places(fields) in a system to verify its functionality have been made ready in a piece of document called test data. 32. What is an Inconsistent bug? The Bug which is not occurring in a definable format or which cannot be caught, even if a process is followed. It may occur and may not when tested with same scenario. 33. What is the difference between Re-testing and Regression testing? Retest-To check for a particular bug and its dependencies after it is said to be fixed. Regression testing: To check for the added or new functionality's effect on the existing ystem 34. What are the different types of testing techniques? • • • White box Black box Gray Box

35. What are the different types of test case techniques? Test Case design techniques for Black box Testing. • • • • • • • Decision table Equivalence Partitioning Method Boundary Value Analysis Cause Effect Graphing State Transition Testing Syntax Testing Resource Risk (A. Human Resource B. Hardware resource C. Software resource) Arcus Infotech Pvt Ltd - 145 -

36. What are the risks involved in testing?

Software Testing ________________________________________________________________________ • • Technical risk Commercial Risk

37. Differentiate Test bed and Test Environment? Test bed holds only testing documents which supports testing which includes Test data, ata guidelines etc. Test environment includes all supportive elements namely hardware, software, tools, rowsers, Servers, etc., 38. What ifs the difference between defect, error, bug, failure, fault? Error: “Is an undesirable deviation from requirements?” Any problem or cause for many problems which stops the system to perform its functionality is referred as Error Bug: Any Missing functionality or any action that is performed by the system which is not upposed to be performed is a Bug. “Is an error found BEFORE the application goes into production?” Any of the following may be the reason for birth of Bug 1. Wrong functionality 2. Missing functionality 3. Extra or unwanted functionality Defect: A defect is a variance from the desired attribute of a system or application. “Is an error found AFTER the application goes into production?” Defect will be commonly categorized into two types: 1. Defect from product Specification 2. Variance from customer/user expectation. Failure: Any Expected action that is suppose to happen if not can be referred as failure or we can say Absence of expected response for any request. Fault:

Arcus Infotech Pvt Ltd - 146 -

Software Testing ________________________________________________________________________ This generally referred in hardware terminologies. A Problem, which cause the system not to perform its task or objective. 39. What is the difference between quality and testing? “Quality is giving more cushions for user to use system with all its expected characteristics” It is usually said as Journey towards Excellence. Testing is an activity done to achieve the quality. 40. What is the difference between White & Black Box Testing? White box: Structural tests verify the structure of the software itself and require complete access to the object's source code. This is known as ‘white box’ testing because you see into the internal workings of the code. Black Box: Functional tests examine the observable behavior of software as evidenced by its outputs without reference to internal functions. Hence ‘black box’ testing. If the program consistently provides the desired features with acceptable performance, then specific source code features are irrelevant. It's a pragmatic and down-to-earth assessment of software. 41. What is the difference between Quality Assurance and Quality Control? QA: Study on Process followed in Project development QC: Study on Project for its Function and Specification 42. What is the difference between Testing and debugging? Testing is done to find bugs Debugging is an art of fixing bugs. Both are done to achieve the quality 43. What is the difference between bug and defect? Bug: Any Missing functionality or any action that is performed by the system which is not supposed to be performed is a Bug. “Is an error found BEFORE the application goes into production?” Any of the following may be the reason for birth of Bug 1. Wrong functionality 2. Missing functionality 3. Extra or unwanted functionality Arcus Infotech Pvt Ltd - 147 -

Software Testing ________________________________________________________________________ Defect: A defect is a variance from the desired attribute of a system or application. “Is an error found AFTER the application goes into production?” Defect will be commonly categorized into two types: 1. Defect from product Specification 2. Variance from customer/user expectation 44. What is the difference between verification and validation? Verification: The process of determining whether of not the products of a given phase of the software development cycle meets the implementation steps and can be traced to the incoming objectives established during the previous phase. The techniques for verification are testing, inspection and reviewing. In other words we can say Verification as “Are we Building the Right Product” A tester uses verification method to ensure the system complies with an organization standards and processes, relying on review or non executable methods (such as software, hardware, documentation and personnel) Validation: The process of evaluating software at the end of the software development process to ensure compliance with software requirements. The technique for validation is testing, inspection and reviewing. In other words we can say Verification as “Are we building the Product Right” Validation physically ensures that the system operates according to plan by Executing the system functions through series of tests that can be observed or evaluated. 45. What is the difference between functional spec? And Business requirement Specification? Functional specification will be more technical, It holds properties of a field and functionality dependencies E.g.: size, Type of data whether numeric or alphabets etc. Arcus Infotech Pvt Ltd - 148 -

Software Testing ________________________________________________________________________ Business Requirement Specification will be more business oriented which throws light more on needs or requirements 46. What is the difference between unit testing and integration testing?

Unit Testing: Testing of single unit of code, module or program. it is usually done by the developer of the unit. It validates that the software performs as designed. Deliverable of the unit testing is software unit ready for testing with other system components. Integration Testing: Testing of related programs, modules or units of code. Validates that multiple parts of the system perform as per specification. Deliverable of integration testing is parts of system ready for testing with other portions of system. 47. What is the difference between Volume & Load? Testing type Load Volume Data Constant Increase till User Increase till saturation

point is reached. saturation Constant.

point is increased. 48. What is difference between Volume & Stress? Volume testing is increasing the volume of data to maximum withstand capacity of the system. Stress is the combination of both volume and load, so need not to increase in volume alone even user can also increased objective here is to check the up to which extend it can bare the increasing load and volume. 49. What is the difference between Stress & Load Testing?

Arcus Infotech Pvt Ltd - 149 -

Software Testing ________________________________________________________________________ Stress is the combination of both volume and load, so need not to increase in volume alone even user can also increased objective here is to check the up to which extend it can bare the increasing load and volume. Load Testing is increasing number of user to maximum withstand capacity of the system. 50. What is the difference between Client Server & Web Based Testing? Client server needs a Client server environment that is a system to Request and another to respond to its request. Web Based testing normally goes with 3W sites testing, done to check its stability and functionality when goes online. 51. What is the difference between Integration & System Testing? Integration testing 52. What is the Difference between Code Walkthrough & Code Review? Both are almost same except in one issue that is Walkthrough need not be done by people inside the team are who have more knowledge about the system. Review is highly recommended to be done by people of higher level in team or who have good knowledge about the application. 53. What is the difference between walkthrough and inspection? Walkthrough: In a walk through session, the material being examined is presented by a reviewed and evaluated by a team of reviewers. A walk through is generally carried out without any plan or preparation. the aim of this review is to enhance the process carried out in production environment. Inspections: Design and code inspection was first described by FAGUN. There are three separate inspection performed, they are • • • Following design, but prior to implementation. Following implementation, but prior to Unit testing. Finally inspecting Unit testing, this was not considered to be cost-effective in discovering errors. 54. What is the Difference between SIT & IST? Arcus Infotech Pvt Ltd - 150 -

Software Testing ________________________________________________________________________ • • SIT can be done when system is on the process of integration, IST need integrated System of various Unit levels of independent functionality and checks its workability after integration and compares it before integration. 55. What is the Difference between static and dynamic? · Static testing: Testing performed with expecting any response for specific request placed at that time. Done Based on structures, Algorithms, logic, etc., · Dynamic testing: Performed to the System that responds to any specific request. More than all that without executing the application this testing cannot be done. 56. What is the difference between alpha testing and beta testing? Component Test data Test environment To achieve Tested by Supporting document Used Alpha testing Simulated Controlled Functionality Only testers Functional specification Beta testing Live Uncontrolled User needs Testers and end-users Customer requirement specification.

57. What are the Minimum requirements to start testing? • • • • Baseline Documents. Stable application. Enough hardware and software support E.g. Browsers, Servers, and Tools) Optimum maintenance of resource

58. What is Smoke Testing & when it will be done? A quick-and-dirty test that the major functions of a piece of software work without bothering with finer details. Originated in the hardware testing practice of turning on a new piece of hardware for the first time and considering it a success if it does not catch on fire. 59. What is Ad hoc testing? When it can be done? Appropriate and very often syndicated when tester wants to become familiar with the product, or in the environment when technical/testing materials are not 100% Arcus Infotech Pvt Ltd - 151 -

Software Testing ________________________________________________________________________ completed. It is also largely based on general software product functionality/testing understanding and the normal 'Human Common Sense'. This can be performed even with non-availability of of Baseline documents. 60. What is cookie testing? Cookie is a text file normally written by web applications to store all your login-id, password validation and details about your session. Cookies will get stored in our machines (client).Its mainly to verify whether cookies are being written correctly. . Importance of cookie testing: • • • • To evaluate the performance of a web application To assure the health of www application where more cookies are involved To test how the system can defense itself from external attacks. How much it can with stand from breaking the system from performing its assigned task. Many critical software applications and services have integrated security measures against malicious attacks. The purpose of security testing of these systems include identifying and removing software flaws that may potentially lead to security violations, and validating the effectiveness of security measures. Simulated security attacks can be performed to find vulnerabilities. 62. What is database testing? The demonstrate the backend response for front end requests How backend, which stores and retrieve back the data and supports the front end when inneed is justified database testing. 63. What is the relation ship between Quality & Testing? Quality is a journey towards excellence; Testing is the way of achieving quality. 64. How do you determine, what to be tested? The Scope of testing should be created based on the requirements or needs given by the end user or client, based on these things the testing scope should be decided. 65. How do you go about testing a project?

61. What is security testing?

Arcus Infotech Pvt Ltd - 152 -

Software Testing ________________________________________________________________________ • • • System study Understanding the application Test environment setup

66. What is the Initial Stage of testing? Right from understanding the application testing starts with clarifying the ambiguities in the application and continues to Test initiation encloses, Test process, test data, Data guidelines Preparation and test design which is finally executed 67. What is Web Based Application Testing? Web Based testing normally goes with 3W sites testing, done to check its stability and functionality when goes online. 68. What is Client Server Application Testing? Client server needs a Client server environment that is a system to Request and another to respond to its request. 69. What is the use of Functional Specification? Functional Specification is a baseline document prepared in technical perspective, says how the system should behave in ideal scenario. Tells right from syntax to its functionality and dependencies Eg: for a password and user id fields It should accept <n>number of characters in<Type> of type of data and it gets input from <x> and gives output to <y>. 70. Why do we prepare test condition, test cases, test script (Before Starting Testing)? These are test design document which are used to execute the actual testing Without which execution of testing is impossible ,finally this execution is going to find the bugs to be fixed so we have prepare this documents. 71. Is it not waste of time in preparing the test condition, test case & Test Script? No document prepared in any process is waste of time, That too test design documents which plays vital role in test execution can never be said waste of time as without which proper testing cannot be done. 72. How do you go about testing of Web Application? Arcus Infotech Pvt Ltd - 153 -

Software Testing ________________________________________________________________________ To approach a web application testing, the first attack on the application should be on its performance behavior as that is very important for a web application and then transfer of data between web server ,front end server, security server and back end server. 73. How do you go about testing of Client Server Application? To approach a client server environment we can track back the data transfer, Check the compatibility and verify the individual behavior and then to compare as client and server. 74. What is meant by Static Testing? Structure of a program, Program Logic, Condition coverage, Code coverage etc. can be tested. Analysis of a program carried out without executing the program. 75. Can the static testing be done for both Web & Client Server Application? Yes, Can be done regardless of type of application, but Depends on the Application’s individual structure and behavior. 76. In the Static Testing, what all can be tested? • • • • • Functions, Conditions Loops Arrays Structures

77. Can test condition, test case & test script help you in performing the static testing? Static testing will be done based on Functions, Conditions, loops, arrays and structures. so hardly not needed to have These documents, By keeping this also static testing can be done. 78. What does dynamic testing mean? Any dynamic application i.e., the system that responds to request of user is tested by executing it is called dynamic testing 79. Is the dynamic testing a functional testing?

Arcus Infotech Pvt Ltd - 154 -

Software Testing ________________________________________________________________________ Yes, Regardless of static or dynamic if applications functionality's are attacked keeping in mind to achieve the need then it will come under functional testing. 80. Is the Static testing a functional testing? Yes, Regardless of static or dynamic if applications functionality's is attacked keeping in mind to achieve the need then it will come under functional testing. 81. What is the functional testing you perform? I have done Conformance testing, Regression testing, Workability testing, Function Validation and Field level validation testing. 82. What is meant by Alpha Testing? Alpha testing is testing of product or system at developer’s site by the customer. 83. What kind of Document you need for going for a Functional testing? Functional specification is the ultimate document, which expresses all the functionality's of the application and other documents like user manual and BRS are also need for functional testing. Gap analysis document will add value to understand expected and existing system. 84. What is meant by Beta Testing? User Acceptance testing which is done with the objective of achieving all users needs. In this testing usually users or testers will involve in performing. E.g.: a Product after completion given to customers for trail as Beta version and feedback from users and important suggestions which will add quality will be done before release. 85. At what stage the unit testing has to be done? After completing coding of individual functionality's unit testing can be done. Say for E.g.: if an application have 5 functionality's to work together, if they have been developed individually then unit testing can be carried out before their integration is suppose to be done. Who can perform the Unit Testing? Both developers and testers can perform this unit level testing 86. When will the Verification & Validation be done?

Arcus Infotech Pvt Ltd - 155 -

Software Testing ________________________________________________________________________ Software development How to use verification How to use validation

phases Requirements gathering Project planning

Verify completeness of Not usable. requirement.  Verify capability, applicable.  Verify completeness of  Validate correctness of changes.  Validate regression.  Validate meets user acceptance criteria.  Validate supplier’s software process correctly.  Validate software interfaces. interim project test plan  Verify correctness and completeness of  Verify contingency plan. deliverables. vendor Not usable if

Project implementation

86.What is the testing that a tester performs at the end of Unit Testing? Integration testing will be performed after unit testing to ensure that unit tested modules get integrated correctly. 87. What are the things, you prefer & Prepare before starting Testing? Arcus Infotech Pvt Ltd - 156 -

Software Testing ________________________________________________________________________ Study the application, Understanding the applications expected functionality's, Preparing Test plan, Ambiguity/Clarification Document and test design Documents. 88. What is Integration Testing? Integration testing exercises several units that have been combined to form a module, subsystem, or system. Integration testing focuses on the interfaces between units, to make sure the units work together. The nature of this phase is certainly 'white box', as we must have certain knowledge of the units to recognize if we have been successful in fusing them together in the module. 89. What is Incremental Integration Testing? Incremental Integration Testing is an approach of testing where we will integrate the modules top to bottom or on the incrementing scale of intensity. 90. What is meant by System Testing? The system test phase begins once modules are integrated enough to perform tests in a whole system environment. System testing can occur in parallel with integration test, especially with the top-down method. 91. What is meant by SIT? System Integration Testing done after the completion of Unit level testing. An application which is integrated together after assuring their individual functionality's. 92. When do you go for Integration Testing? When all Separate unit in Unit Level testing is assured to do good their performance, Then Application is recommended for integration after these unit getting integrated, application can be performed integration testing. 93. Can the System testing be done at any stage? No, The system as a whole can be tested only if all modules are integrated and all modules work correctly System testing should be done before UAT (User Acceptance testing) and Before Unit Testing. 94. What are stubs & drivers?

Arcus Infotech Pvt Ltd - 157 -

Software Testing ________________________________________________________________________ Driver programs provide emerging low-level modules with simulated inputs and the necessary resources to function. Drivers are important for bottom-up testing, where you have a complete low-level module, but nothing to test it with. Stubs simulate sub-programs or modules while testing higher-level routines. 95. What is the Concept of Up-Down & Down-Up in Testing in integration testing? There is two approach in testing an application if the functionality sequence was mapped and tracked from top to bottom then it is called top down method ,If that was done for integration testing then it is top down model testing in Integration and vice versa for Bottom up model 96. What is the final Stage of Integration Testing? All the individual units integrated together to Perform a task as a system or Part of the system as expected to do so. 97. Where in the SDLC, the Testing Starts? It depends upon the Software Model which we follow. If we use Waterfall model then testing will comes in to picture only after coding is done. If we follow V model then testing can be started at the design phase itself. UAT test cases can be written from URS/BRS and System test cases can be written from SRS. 98. What is the Outcome of Integration Testing? At the completion of integration testing all the unit level functionalities or sub modules are integrated together and finally it should work as a system as whole as expected. 99. What is meant by GUI Testing? Testing the front-end user interfaces to applications, which use GUI support systems and standard such as MS Windows. 100. What is meant by Back-End Testing? Database Testing is also called as back end testing checking whether database elements have been accessed by front end whenever required as desired. 101. What are the features, you take care in Prototype testing? Prototype testing is carrying out testing in same method reputedly to understand the system behavior; here full coverage of functionality should be taken care With the same process followed as for Prototype testing. Arcus Infotech Pvt Ltd - 158 -

Software Testing ________________________________________________________________________ 102. What is Mutation testing & when can it be done? Mutation testing is a powerful fault-based testing technique for unit level testing. Since it is a fault-based testing technique, it is aimed at testing and uncovering some specific kindsof faults, namely simple syntactic changes to a program. Mutation testing is based on two assumptions: the competent programmer hypothesis and the coupling effect. The competent programmer hypothesis assumes that competent programmers tend to write nearly "correct" programs. The coupling effect stated that a set of test data that can uncover all simple faults in a program is also capable of detecting more complex faults. Mutation testing injects faults into code to determine optimal test inputs 103. What is Compatibility Testing? Testing to ensure compatibility of an application with different browsers, Operating Systems, and hardware platforms. Compatibility testing can be performed manually or can be driven by an automated functional or regression test suite. 104. What is Usability Testing? Usability testing is a core skill because it is the principal means of finding out whether a system (see our definition below) meets its intended purpose. All other skills that we deploy or cultivate aim to make usability (and, ultimately, use) successful. It is a Process of Testing the effectiveness, efficiency, and satisfaction with which specified users could achieve specified goals in the Application. Synonymous with "ease of use". 105. What is the Importance of testing? Software Testing is more Oriented to detecting the defects or often equated to finding bugs. Testing is mainly done to make things go wrong to determine if things happen when they shouldn't or things don't happen when they should. Testing only demonstrates that the product performs each function intended & to show the product is free from defect. 106. What is meant by regression Testing? Regression testing is an expensive but necessary activity performed on modified softwareto provide confidence that changes are correct and do not adversely Arcus Infotech Pvt Ltd - 159 -

Software Testing ________________________________________________________________________ affects other system components. Four things can happen when a developer attempts to fix a bug. Three of these things are bad, and one is good:

Because of the high probability that one of the bad outcomes will result from a change to the system, it is necessary to do regression testing. 107. When we prefer Regression & what are the stages where we go for Regression Testing? We Prefer regression testing to provide confidence that changes are correct & has not affected the flow or Functionality of an application which got Modified or bugs got fixed in it. Stages where we go for Regression Testing are: • Minimization approaches seek to satisfy structural coverage criteria by identifying a minimal set of tests that must be retested, for identifying whether the application works fine. • Coverage approaches are also based on coverage criteria, but do not require minimization of the test set. Instead, they seek to select all tests that exercise changed or affected program components. • Safe attempt instead to select every test that will cause the modified program to produce different output than original program. 108. What is performance testing? An important phase of the system test, often-called load, volume or performance test, and stress tests try to determine the failure point of a system under extreme pressure. Stress tests are most useful when systems are being scaled up to larger environments or being implemented for the first time. Web sites, like any other large-scale system that requires multiple accesses and processing, contain vulnerable nodes that should be tested before deployment. Unfortunately, most stress testing can only simulate loads on various points of the system and cannot Arcus Infotech Pvt Ltd - 160 -

Software Testing ________________________________________________________________________ truly stress the entire network, as the users would experience it. Fortunately, once stress and load factors have been successfully overcome, it is only necessary to stress test again if major changes take place. A drawback of performance testing is that can easily confirm that the system can handle heavy loads, but cannot so easily determine if the system is producing the correct information. In other words, processing incorrect transactions at high speed can cause much more damage and liability than simply stopping or slowing the processing of correct transactions. Performance testing can be applied to understand your application or WWW site's scalability, or to benchmark the performance in an environment of third party products such as servers and middleware for potential purchase. This sort of testing is particularly useful to identify performance bottlenecks in high use applications. Performance testing generally involves an automated test suite as this allows easy simulation of a variety of normal, peak, and exceptional load conditions. The following three types highly influence Performance of an application. Load testing, Volume testing, Stress Testing Stress testing is the combination of both load and volume. 109. What is the Performance testing; those can be done Manually & Automatically? This sort of testing is particularly useful to identify performance bottlenecks in high use applications. Performance testing generally involves an automated test suite as this allows easy simulation of a variety of normal, peak, and exceptional load conditions. Manually: - Load, Stress, & Volume are the types of testing which are been done Manually. Automated: - Load, Stress, & Volume are the types of testing which are been done automatically, by using the Automated Skills. 110. What is Volume, Stress & Load Testing?

Arcus Infotech Pvt Ltd - 161 -

Software Testing ________________________________________________________________________ Volume testing: - Testing the Application under varying loads; keeping the Number of Users constantly & finding the Response time & the system With Standing Capability or varying the Load till saturation Point is reached Load Testing : -Testing the Application under Constant load; keeping varying the Number of Users & there by finding the Response time & the system With Standing Capability or varying the Users till saturation Point is reached Stress Testing : - Testing the Application under varying loads; keeping varying the Number of Users simultaneously & there by finding the Response time & the system With Standing Capability or varying the Load & Users till saturation Point is reached

111. What is a Bug? Bug: - “Is an error found BEFORE the application goes into production?” 112. What is a Defect? Defect: - “Is an error found AFTER the application goes into production?” 113. What is the defect Life Cycle? • • • • Test Team (Here the Defect status will be Open) Test Lead Authorize the bugs found (Here the Defect Status will be Open) Development Team reviewing the Defect (Here the Defect Status will be Open) The defect can be Authorized or Unauthorized by the development Team (Here the Status of the Defect will be Open (For Authorized Defects) & Reject (For Unauthorized Defects) Arcus Infotech Pvt Ltd - 162 -

Software Testing ________________________________________________________________________ • Development Team fixing the Defect (Here the authorized Bugs will get fixed or differed; it is done again by the Development team. Here the Status after the Development team fixing the bugs will be (Fixed) & Status will be Differed for the bugs which got Differed) • The Fixed bugs will be again Re-tested by the Test Team (Here based on the Closure of the Bug, the status will be made as closed or if the Defect still remains, it will be Re-raised again & even the new bugs with status Open will be sent to the Development team) The above-mentioned cycle flows on continuously, until all the bugs gets fixed in the application. 114. What is the Priority in fixing the Bugs? Priority: - The value will be given to the bugs, by both Testers & Developers (But Mostly the Development team will take care of this). It mainly deals with, what importance should be given to each bug by the Developer. (i.e.) like the Critical bugs should be solved first & then the Major bugs can be taken care. 115. Explain the Severity you rate for the bugs found? • • • • Emergency High (Very High & high) Medium Low (Very Low & Low)

Testers will rate severity; it is based on the Defect we find in the application. Severity can be rated as Critical or major or Minor. It is mostly done based on the nature of the defect found in the Application. Eg: - When user is not able to proceed or system gets crashes & so that tester is not able to proceed further testing (These Bugs will be rated as Critical) E.g.: - When user tries to add an record & then tries to view the same record added & if the details getting displayed to the fields are not the same which the user provided as the value to the fields (These Type of Bugs will be rated as Major Bugs)

Arcus Infotech Pvt Ltd - 163 -

Software Testing ________________________________________________________________________ E.g.: - Mostly the FLV Bugs & some functional bugs (Related the value display etc.) will be rated as Minor. 116. Difference between UAT & IST? UAT & IST UAT: 1. Done Using BRD 2. Done with the Live Data 3. Testing is done in User Style 4. Testing in done in the Client Place 5. Testing is done by the Real Users or some Third Party Testers IST: 1. Done Using FS 2. Done with the Simulated Data 3. Testing is done in a Controlled Way. 4. Testing in done in Offsite 5. Testing is done in the Testers Company 117. What is meant by UAT? Traditionally, this is where the users ‘get their first crack’ at the software. Unfortunately, by this time, it's usually too late. If the users have not seen prototypes, been involved with the design, and understood the evolution of the system, they are inevitably going to be unhappy with the result. If you can perform every test as user acceptance tests, you have a much better chance of a successful project User Acceptance testing is done to achieve the following:• • • • User specified requirements have been satisfied Functionality is doing as per supporting documents Expected performance have been achieved End user is comfortable to use the application.

118. What all are the requirements needed for UAT?

Arcus Infotech Pvt Ltd - 164 -

Software Testing ________________________________________________________________________ • • Business Requirement Document is the Documents required for performing the UAT Testing by the testers. Application should be Stable (Means, all the Modules should be tested at least once after Integrating the Modules) 119. What are the docs required for Performance Testing? Bench Mark is the Basic Document required for Performance Testing. Where the documents contains in detail about the Response Time, Transaction Time, Data Transfer Time, Virtual Memory in which the Application should work. 120. What is risk analysis? Risk Analysis is a series step that helps the Software or Testing Team to understand & manage Uncertainty. It is a process of evaluating risks, threats, controls, & vulnerabilities. Threat: - Which is capable of exploiting vulnerability in the security of a computer system or application. Vulnerability: -Is a design, implementation, or operations flaw that may be exploited by a threat? Control: -Control is anything that tends to cause the reduction of risk. 121. How to do risk management? Identifying the Risk Involved in the project & finding Mitigation for the Risk Found will do risk Management. Risk Mitigation will be a solution for the Risk Identified. 122. What are test closure documents? • • • • • • • • Test Conditions Test Case Test Plan Test Strategy Traceability Matrix Defect Reports Test Closure Document Test Data

Arcus Infotech Pvt Ltd - 165 -

Software Testing ________________________________________________________________________ (The Above Mentioned Deliverables are based on the deliverables accepted by the Testing Team & mentioned in the Test Strategy) 123. What is Traceability matrix? Traceability Matrix: Through out the testing life cycle of the project Traceability matrix has been maintained to ensure the Verification & Validation of the testing is complete. 124. What ways can be followed for defect management? • • • Reporting the Bugs through the Defect Report (Excel Template) Any in-house tool inbuilt in the company may also be used. Commonly available tools like TEST DIRECTOR can also be employed

21. Glossary Testing “The process of exercising software to verify that it satisfies specified requirements and to detect errors “ Quality Assurance “A planned and systematic pattern for all actions necessary to provide adequate confidence that the item or product conforms to established technical requirements” Quality Control “QC is a process by which product quality is compared with applicable standards, and the action taken when nonconformance is detected.” Verification “The process of evaluating a system or component to determine whether the products of the given development phase satisfy the conditions imposed at the start of that phase.” Validation Determination of the correctness of the products of software development with respect to the user needs and requirements. Static Testing Techniques Arcus Infotech Pvt Ltd - 166 -

Software Testing ________________________________________________________________________ “Analysis of a program carried out without executing the program.” Review - Definition Review is a process or meeting during which a work product or set of work products, is presented to project personnel, managers, users, customers, or other interested parties for comment or approval. Walkthrough “A review of requirements, designs or code characterized by the author of the material under review guiding the progression of the review. “ Inspection A group review quality improvement process for written material. It consists of two aspects; product (document itself) improvement and process improvement (of both document production and inspection). Dynamic Testing Techniques “The process of evaluating a system or component based upon its behaviour during execution. “ Black Box Testing: “Test case selection that is based on an analysis of the specification of the component without reference to its internal workings.” Equivalence partition testing: Equivalence partition testing: A test case design technique for a component in which test cases are designed to execute representatives from equivalence classes. Equivalence class: A portion of the component's input or output domains for which the component's behaviour is assumed to be the same from the component's specification. Boundary Value Analysis Boundary value: An input value or output value which is on the boundary between equivalence classes, or an incremental distance either side of the boundary. Boundary value analysis: A test case design technique for a component in which test cases are designed which include representatives of boundary values. Cause and Effect Graphs Arcus Infotech Pvt Ltd - 167 -

Software Testing ________________________________________________________________________ “A graphical representation of inputs or stimuli (causes) with their associated outputs (effects), which can be used to design test cases” White-Box Testing: “Test case selection that is based on an analysis of the internal structure of the component.” Statement Coverage: “A test case design technique for a component in which test cases are designed to execute statements.” Branch Testing: Branch Testing: A test case design technique for a component in which test cases are designed to execute branch outcomes. Branch : A conditional transfer of control from any statement to any other statement in a component, or an unconditional transfer of control from any statement to any other statement in the component except the next statement, or when a component has more than one entry point, a transfer of control to an entry point of the component. Path Testing Path: A sequence of executable statements of a component, from an entry point to an exit point. Path testing: A test case design technique in which test cases are designed to execute paths of a component. Data Flow-Based Testing: “Testing in which test cases are designed based on variable usage within the code.” Unit Testing “The testing of individual software components.” Integration Testing “Testing performed to expose faults in the interfaces and in the interaction between integrated components” Incremental Integration Testing

Arcus Infotech Pvt Ltd - 168 -

Software Testing ________________________________________________________________________ “Integration testing where system components are integrated into the system one at a time until the entire system is integrated” Top Down Integration “An approach to integration testing where the component at the top of the component hierarchy is tested first, with lower level components being simulated by stubs. Tested components are then used to test lower level components. The process is repeated until the lowest level components has been tested.” Bottom up Integration “An approach to integration testing where the lowest level components are tested first, then used to facilitate the testing of higher level components. The process is repeated until the component at the top of the hierarchy is tested.” Stubs: Stubs are program units that are stand-ins² for the other (more complex) program units that are directly referenced by the unit being tested. Drivers: Drivers are programs or tools that allow a tester to exercise/examine in a controlling manner the unit of software being tested. Big Bang Integration “Integration testing where no incremental testing takes place prior to all the system's components being combined to form the system.” Validation Testing Validation testing aims to demonstrate that the software functions in a manner that can be reasonably expected by the customer. Configuration review An audit to ensure that all elements of the software configuration are properly developed, catalogued, and has necessary detail to support maintenance. System Testing “System testing is the process of testing an integrated system to verify that it meets specified requirements". Requirement based Testing “Designing tests based on objectives derived from requirements for the software Arcus Infotech Pvt Ltd - 169 -

Software Testing ________________________________________________________________________ component (e.g., tests that exercise specific functions or probe the non-functional constraints such as performance or security)” Business-Process based Non-Functional Testing Testing of those requirements that do not relate to functionality. I.e. performance, usability, etc. “ Recovery testing “Testing aimed at verifying the system's ability to recover from varying degrees of failure.” Security testing “Testing whether the system meets its specified security objectives.” Stress testing “Testing conducted to evaluate a system or component at or beyond the limits of its specified requirements.” Performance testing “Testing conducted to evaluate the compliance of a system or component with specified performance requirements.” Alpha and Beta testing “Alpha testing: Simulated or actual operational testing at an in-house site not otherwise involved with the software developers.” “Beta testing: Operational testing at a site not otherwise involved with the software developers.” User Acceptance Testing “Acceptance testing: Formal testing conducted to enable a user, customer, or other authorized entity to determine whether to accept a system or component” Regression Testing and Re-testing “Retesting of a previously tested program following modification to ensure that faults have not been introduced or uncovered as a result of the changes made.” Ad-hoc Testing “Testing carried out using no recognised test case design technique.” Load Testing

Arcus Infotech Pvt Ltd - 170 -

Software Testing ________________________________________________________________________ Load Testing involves stress testing applications under real-world conditions to predict system behavior and performance and to identify and isolate problems. Load testing applications can emulate the workload of hundreds or even thousands of users, so that you can predict how an application will work under different user loads and determine the maximum number of concurrent users accessing the site at the same time. Stress and Volume Testing “Stress Testing: Testing conducted to evaluate a system or component at or beyond the limits of its specified requirements.” “Volume Testing: Testing where the system is subjected to large volumes of data. “ Usability Testing “Testing the ease with which users can learn and use a product.” Environmental Testing These tests check the system’s ability to perform at the installation site. Business Requirement It describes user’s needs for the application. Functional Specification “The document that describes in detail the characteristics of the product with regard to its intended capability.” Design Specification The Design Specification document is prepared based on the functional specification. It contains the system architecture, table structures and program specifications. System Specification The System Specification document is a combination of Functional specification and design specification. Error Guessing “A test case design technique where the experience of the tester is used to postulate what faults might occur, and to design tests specifically to expose them. “ Error Seeding

Arcus Infotech Pvt Ltd - 171 -

Software Testing ________________________________________________________________________ “The process of intentionally adding known faults to those already in a computer program for the purpose of monitoring the rate of detection and removal, and estimating the number of faults remaining in the program.” Test Plan A record of the test planning process detailing the degree of tester indedendence, the test environment, the test case design techniques and test measurement techniques to be used,and the rationale for their choice. - BS “A document describing the scope, approach, resources, and schedule of intended testing activities. It identifies test items, the features to be tested, the testing tasks, who will do each task, and any risks requiring contingency planning.” - IEEE Test Case “A set of inputs, execution preconditions, and expected outcomes developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.” Comprehensive Testing - Round I All the test scripts developed for testing are executed. Some cases the application may not have certain module(s) ready for test; hence they will be covered comprehensively in the next pass. The testing here should not only cover all the test cases but also business cycles as defined in the application. Discrepancy Testing - Round II All the test cases that have resulted in a defect during the comprehensive pass should be executed. In other words, all defects that have been fixed should be retested. Function points that may be affected by the defect should also be taken up for testing. This type of testing is called as Regression testing. Defects that are not fixed will be executed only after they are fixed. Sanity Testing - Round III This is final round in the test process. This is done either at the client's site or at Maveric depending on the strategy adopted. This is done in order to check if the system is sane enough for the next stage i.e. UAT or production as the case may be under an isolated environment. Ideally the defects that are fixed from the

Arcus Infotech Pvt Ltd - 172 -

Software Testing ________________________________________________________________________ previous phases are checked and freedom testing done to ensure integrity is conducted. Defect – Definition “Error: A human action that produces an incorrect result. “ “Fault: A manifestation of an error in software. A fault, if encountered may cause a failure. “ “Failure: Deviation of the software from its expected delivery or service. “ “A deviation from expectation that is to be tracked and resolved is termed as a defect. “ Defects Classification Showstopper A Defect which may be very critical in terms of affecting the schedule, or it may be a show stopper – that is, it stops the user from using the system further Major A Defect where a functionality/data is affected significantly but not cause a showstopping condition or a block in the test process cycles. Minor A Defect which is isolated or does not stop the user from proceeding, but causes inconvenience. Cosmetic Errors would also feature in this category Severity: How much the Bug found is supposed to affect the systems Function/Performance, Usually we divide as Emergency, High, Medium, and Low. Priority: Which Bug should be solved fist in order of benefit of system’s health? Normally it starts from Emergency giving first Priority to Low as last Priority. Test Bed Before Starting the Actual testing the elements which supports the testing activity such as Test data, Data guide lines. Are collectively called as test Bed. Data Guidelines Data Guidelines are used to specify the data required to populate the test bed and prepare test scripts. It includes all data parameters that are required to test the Arcus Infotech Pvt Ltd - 173 -

Software Testing ________________________________________________________________________ conditions derived from the requirement / specification. The Document, which supports in preparing test data are called Data guidelines Test script A Test Script contains the Navigation Steps, Instructions, Data and Expected Results required to execute the test case(s). Any test script should say how to drive or swim through out the application even for a new user. Test data The value which are given at expected places(fields) in a system to verify its functionality have been made ready in a piece of document called test data. Test environment A description of the hardware and software environment in which the tests will be run, and any other software with which the software under test interacts when under test including stubs and test drivers. Traceability Matrix Through out the testing life cycle of the project Traceability matrix has been maintained to ensure the Verification & Validation of the testing is complete.

Arcus Infotech Pvt Ltd - 174 -