Basic QA (Test) artifacts

Lecture



  • Why do we need, why exist, how much time should be spent on them
  • Tasks, approach, content:
    • Test Plan and work planning stage
    • Test Case & Test Scenario - when what we use
    • Check List - work with yes / no tasks
    • QA Reports - how and what we report

Test Artifacts

In accordance with the processes or methodologies of software development, during testing, a certain number of test artifacts (documents, models, etc.) are created and used. The most common test artifacts are:

  • A Test Plan is a document describing the entire scope of testing, starting with the description of the object, strategy, schedule, criteria for starting and ending testing, up to the equipment needed during the operation, special knowledge, and risk assessment with options for their resolution.
  • A set of test cases and tests (Test Case & Test suite) is a sequence of actions by which you can check whether the function under test meets the established requirements.
  • Defects / Bug Reports (Bug Reports / Defects) are documents describing a situation or sequence of actions that led to incorrect operation of the test object, indicating the reasons and the expected result.

Even having dealt with all the artifacts, many will not immediately be able to compile the necessary documents on their own. To do this, we decided to put in open access templates and examples of documentation. If you still could not understand some of the subtleties, we will be happy to help you. To do this, leave your message on the page Questions, suggestions and requests.

Test Plan (Test Plan)

A Test Plan is a document that describes the entire scope of testing, starting with a description of the object, strategy, schedule, criteria for starting and ending testing, up to the equipment necessary for the operation, special knowledge, and risk assessment with options for resolving them. .

Tips for writing a Test Plan

Each methodology or process tries to impose on us its own format for the design of test plans. I offer you, as an example, test plan templates from RUP (Rational Unified Process) and IEEE 829 standard :

  1. Test Plan Template RUP
  2. Test Plan Template IEEE 829

Looking closely it becomes clear that both documents describe the same thing, but in a different form. If the standard template does not suit you or you decide to create your own document format that is more suitable for you, then from experience we can say that a good test plan should at least describe the following:

  1. What should be tested?
    • description of the test object: systems, applications, equipment
  2. What will you test?
    • list of functions and description of the system being tested and its components separately
  3. How will you test?
    • testing strategy, namely: types of testing and their application in relation to the test object
  4. When will you test?
    • sequence of work: preparation (Test Preparation), testing (Testing), analysis of results (Test Result Analisys) in the context of the planned development phases
  5. Criteria for testing :
    • readiness of the test platform (test bench)
    • completeness of the development of the required functionality
    • availability of all necessary documentation
    • ...
  6. Test end criteria :
    • test results meet the product quality criteria:
      • requirements for the number of open bugs are met
      • Exposure of a certain period without changing the source code of the Code Freeze application (CF)
      • Exposure of a certain period without opening new bugs Zero Bug Bounce (ZBB)
    • ...

By answering the above questions in your test plan, you can assume that you already have a good draft of a test planning document. Further, in order for the document to acquire a less serious look, we suggest supplementing it with the following points:

  • Environment of the system under test (description of software and hardware)
  • Equipment and software necessary for testing (test bench and its configuration, software for automated testing, etc.)
  • Risks and ways to resolve them

Types of test plans

Most often, in practice, one encounters the following types of test plans:

  1. Master Plan or Master Test Plan
  2. Test Plan (Test Plan) , let's call it a detailed test plan.
  3. Acceptance Testing Plan (Product Acceptance Plan) - a document describing a set of actions related to acceptance testing (strategy, date, decision-makers, etc.) (RUP acceptance test pattern)

The apparent difference between the Master Test Plan and the simple Test Plan is that the master test plan is more static due to the fact that it contains high-level information that is not subject to frequent changes in the process of testing and revising the requirements. The very detailed test plan , which contains more specific information on strategy, types of testing, schedule of work, is a “living” document that is constantly undergoing changes, reflecting the real situation on the project.

In everyday life, a project can have one Master Test Plan and several detailed test plans describing individual modules of a single application.

Review and Approval

To increase the value of your test plan, it is recommended that it be periodically reviewed by project team members. This can be done simply by agreeing among themselves or by implementing it in the form of an “approval procedure”. As an example, we give a list of project team members, whose approval we deem necessary:

  • Lead Tester
  • Test manager (quality manager)
  • Development Manager
  • Project manager

Each of the listed project participants, before approval, will conduct a review and make their comments and suggestions that will help make your test plan more complete and high-quality.

Conclusion

Using the above tips, you will have more chances to write a good document than to invent everything yourself.

Test Case

A Test Case is an artifact that describes a set of steps, specific conditions and parameters necessary to verify the implementation of a test function or its part.

Under the test case refers to the structure of the form:

Action > Expected Result > Test Result

Example:

Action Expected Result Test result
(passed / failed / blocked)
Open page "login" Login page is opened Passed

Types of Test Cases

Test cases are divided by the expected result into positive and negative :

  • The positive test case uses only valid data and checks that the application has correctly executed the called function.
  • The negative test case operates with both correct and incorrect data (at least 1 incorrect parameter) and aims to check for exceptional situations (validation of validators), and also checks that the function called by the application is not executed when the validator is triggered.

Test Case Structure

On the Internet you can find a lot of information about the structure of test cases, their level of detail and the number of checks in them, I'm going to talk about the approach I used, and which I want to offer to use for you.

Each test case should have 3 parts:

Preconditions A list of actions that lead the system to a state suitable for conducting a basic inspection. Or a list of conditions, the fulfillment of which indicates that the system is in a condition suitable for conducting the main test.
Test Case Description The list of actions that translate the system from one state to another, to obtain a result, based on which we can conclude that the implementation is satisfied, the requirements
Postconditions The list of actions that translate the system to its original state (the state before the test - initial state)

Note : Post Conditions is not required. This is most likely - the rule of good tone: "Namusoril - clean up after you." This is especially true for automated testing, where you can fill a database with hundreds or even thousands of incorrect documents in one run.

Sample test case:

do A1, verify B1

do A2, verify B2

do A3, verify B3

In the above example, the final check is B3. This means that it is the key. Hence, A1 and A2 are actions that lead the system to a testable state. And B1 and B2 are the conditions of that the system is in a state suitable for testing. Thus we have:

Action Expected Result Test result
(passed / failed / blocked)
Preconditions
do A1 verify B1
do A2 verify B2
Test Case Description
do a3 verify B3
Postconditions

PostConditions in this example have not been described, but according to the logic of things it is necessary to perform the steps that would return the system to its original state. (for example, delete the created entry, or cancel changes made to the document)

Now we will answer the question: "Why is this partition convenient to use?"

Answer: the final test is one, i.e. if the test failed, it will be immediately clear because of what. Because if B1 and / or B2 checks fail, the test case will be blocked ( test blocked ), because it is not possible to bring the function into a testable state (state suitable for testing), but this does not mean that the function being tested is not works.

Action Expected Result Test result
(passed / failed / blocked)
Preconditions
do A1 verify B1 passed
do A2 verify B2 failed
Test Case Description :
do a3 verify B3 blocked
Postconditions

Detailing the description of test cases (Test Case Specification)

There are many different opinions about the level of detail when writing test cases, as well as the number of checks in one test case. All of them are correct in their own way. My opinion is that the level of detail of the test cases should be such as to provide a reasonable ratio of the time of passage of the test coverage. Those. as long as the test coverage of a specific functionality does not change, the test case details can be reduced.


Sample test case 1:

Check page display

Act Expected Result Test result
Open the Login page - The "Login" window is open
- Window name - Login
- Company logo is displayed in the upper right corner
- There are 2 fields on the form - Name and Password
- Login button is available
- Link "forgot password" - available
...


Sample test case 2 :

Title: Check page display
Action: Open the Login page
Check: Check that the displayed page corresponds to the page in the picture 1 (and attach the image of the "Login to page"

In example 1 and 2, the coverage will be the same, but the time it takes to pass will be different. It seems to me that the second example will be even clearer.

On our site you can also find an example of a test case design.

In addition, I would like to say that the decision on the type of test case and the details of its description is taken by the person responsible for its creation - Test Designer or Test Analyst, who has the necessary experience, and who knows the test design not by hearsay and has experience in practical application of test design techniques. In many companies, this role is not singled out separately, but is entrusted to ordinary testers, which in the case of insufficient qualifications may lead to correspondence of test cases.

Conclusion

In conclusion, I’ll say that in order for the *** testing team to work together and not be distracted on the design of test cases, everyone should have a single template or approach to writing them . What we offer is the structure of PreConditions, Test Case Description, PostConditions , and it’s already your personal business to use it or invent your own “bicycle”.

Bug Report (Bug Report)

A bug or defect report is a document that describes a situation or sequence of actions that led to incorrect operation of the test object, indicating the reasons and the expected result.

For more detailed information about the bug report, we recommend the following information to your attention, after reading which you will get a comprehensive understanding of the structure, features of writing and some other nuances necessary for writing, good bug reports:

  • Report Bug Structure
  • Defect Severity and Priority
  • Writing bug reports
  • Bug life cycle

We offer you a single developer comment:
- After reading the short description of the bug (Bug Summary), I have to understand what the problem is, after reading the detailed description of the bug (Bug Description) I must know the line of code that I edit.

You can agree or disagree with this, but the point of this statement is that you should do everything so that there are fewer questions for you on the substance of the problem described in the bug report. Since each bug report returned to you with the status "Rejected", "Not reproduced", "Required information" (Rejected, Can't Reproduce, More info) is a waste of time, both of you and the developer. And time, as you know, is the money we receive, for doing our work better than anyone!

We are confident that if in the future you take advantage of all the recommendations we offer, the quality of your reporting bugs will be high, and in the course of work you will have the least complaints from both managers and developers.

Templates and sample documents

Examining the queries of users of our site, we decided to publish the most popular documents on testing on one page.

Test Plans

  1. Test Plan IEEE 829
  2. Test Plan RUP
  3. RUP Acceptance Test Plan
  4. Load Testing Plan (RU / Russian)

Test Design Specification

  1. MSF Test Design Specification Example
  2. Sample IEEE 829-1998 Test Design Specification

Test Case

  1. An example of a test case

Basic QA (Test) artifacts

Bug report

  1. An example of a report bug

Basic QA (Test) artifacts

Basic QA (Test) artifacts


Comments


To leave a comment
If you have any suggestion, idea, thanks or comment, feel free to write. We really value feedback and are glad to hear your opinion.
To reply

Quality Assurance

Terms: Quality Assurance