7 Nguyên tắc kiểm thử

KIỂM THỬ CHỈ RA “CÒN LỖI”

  • Kiểm thử có thể cho thấy rằng phần mềm đang có lỗi
  • Kiểm thử không thể chứng minh rằng phần mềm không có lỗi.

KIỂM THỬ VÉT CẠN LÀ KHÔNG THỂ

  •  Kiểm thử với tất cả các kết hợp đầu vào và đầu ra, với tất cả các kịch bản là không thể
  • Cần phân tích rủi ro và dựa trên sự mức độ ưu tiên để tập trung việc kiểm thử vào một số điểm cần thiết, có nguy cơ lỗi cao hơn.
Kiểm thử vét cạn tất cả trường hợp là không thể

KIỂM THỬ PHẢI BẮT ĐẦU SỚM
Các hoạt động kiểm thử nên được bắt đầu càng sớm càng tốt trong qui trình phát triển (vòng đời phát triển) phần mềm

Chi phí cho việc sửa lỗi sớm tăng theo tỷ lệ 1 – 10 – 100 theo giai đoạn phát hiện lỗi

LỖI QUẦN TỤ

  • Thông thường, lỗi tập trung vào những module, thành phần chức năng chính của hệ thống (nguyên tắc perato/ nguyên tắc tổ gián)
  • Nên tập trung vào tìm kiếm lỗi quanh khu vực được xác định
80% lỗi thường nằm ở 20% tính năng của phần mềm

TEST CASE CÙN DẦN THEO THỜI GIAN

  • Hiệu quả của các trường hợp kiểm thử bắt đầu giảm xuống sau một số lần thực hiện
  • Phải luôn xem xét và sửa đổi các trường hợp kiểm thử trên một khoảng thời gian thường xuyên
Test case luôn bị cùn dần theo thời gian

KIỂM THỬ THEO CÁC NGỮ CẢNH ĐỘC LẬP

  • Việc kiểm thử phụ thuộc vào ngữ cảnh và chúng ta phải tiếp cận kiểm thử theo nhiều ngữ cảnh khác nhau
  • Mỗi ngữ cảnh cần có chiến lược kiểm thử phù hợp. Ví dụ:
    • Kiểm thử ứng dụng di động khác kiểm thử web/desktop
    • Kiểm thử cho quán cafe khác kiểm thử cho quán nhậu
Việc kiểm thử phải được đặt trong ngữ cảnh của người sử dụng

QUAN NIỆM SAI LẦM VỀ VIỆC “HẾT LỖI”

  • Việc không tìm thấy lỗi trên sản phẩm không đồng nghĩa với việc sản phẩm đã sẵn sàng để tung ra thị trường
  • Có thể dừng test khi
    • Rủi ro về chất lượng ở mức chấp nhận đượcCác mục tiêu, chỉ tiêu đưa ra trong kế hoạch test đã đạt được

Small test 4 – Fundamental of testing – answer

  1. According to ISTQB, a human being can make (A), which produces (B) in the program code, or in a document. Which of following combination is correct?
    1. A: Fault, B: Mistake
    2. A: Fault, B: Error
    3. A: Defect, B: Bug
    4. A: Error, B: Fault
  2. Which of following is NOT correct regarding of testing and quality?
    1. Testing can give confidence in the quality of the software if it finds low or no defect
    2. A properly designed test that passes reduces the overall level of risk in a system
    3. Testing should not be integrated as one of the quality assurance activities for independent
    4. By understanding the root cause of defects found in other project, testing processes can be improved
  3. Below shows several levels of independence defined. Which of following has lowest independence?
    1. Test designed by the person who wrote the software under test
    2. Tests designed by another person
    3. Tests designed by a person from a different organizational group of test specialists
    4. Tests designed by a person from different organization or company including outsourcing
  4. Which of following activity is NOT related with Test implementation and execution the fundamental test process?
    1. Creating test suites form the test procedures for efficient execution
    2. Verifying that the test environment has been set up correctly
    3. Finalizing, implementing and prioritizing test case including the identification test data
    4. Identifying necessary test data to support test condition and test cases
  5. Which of the following describes NOT related with testing objective?
    1. Gaining confidence about the level of quality
    2. Providing information for decision – making
    3. Preventing defects
    4. Making a zero defect software
  6. which of the following describes NOT related with Testing Principles?
    1. Testing can show that defects are present, but cannot prove that there are no defects
    2. Exhaustive testing is feasible when risk analysis is used
    3. To find defect early, testing activities shall be as early as possible in the software or system development life-cycle
    4. Finding and fixing defects does not help if the system built is unusable and does not fulfill the users needs and expectation
  7. If the same testes are repeated over and over again, eventually the same set of test cases will no longer find any new defects. To overcome this, test cases need to be regularly reviewed and revised. Which of the following describes this in software testing principle?
    1. Pesticide paradox
    2. Defect clustering
    3. Absence-of-errors fallacy
    4. Early testing
  8. According to ISTQB syllabus, a software error can be described as…
    1. A mismatch between the program and its specification
    2. A description of the relationship between two or more variables or set members in which the values of one does not influence the values of others
    3. The process in which developers determinate the root cause of a bug and identify possible fixes
    4. Any ill-advised, substandard, or temporary fix applied to an urgent problem in the (often misguide) belief that doing so will keep a project moving forward
  9. Which of the following is NOT “test analysis and design” activity in fundamental test process?
    1. Identifying necessary test data to support the test condition and test cases
    2. Identifying and prioritizing test conditions based on analyses of test items, the specification, behavior and structure of the software
    3. Creating bi-directional trace-ablility between test basic and test cases
    4. Creating test suites from the test procedures for efficient test execution
  10. Which is not a goal of writing effective problem/bug report?
    1. Explain how to reproduce the problem
    2. Analyze the error so you can describe it in a minimum number of steps
    3. Write a report that is complete, easy to understand, and non-antagonistic
    4. Illustrate how to fix the problem
  11. Which of the following are conclusion you could draw from the test principles?
    A: Website cannot be tested because of the vast number of combinations of platform, browser, internet route and navigation options
    B: The testing coverage required in one organization may not be appropriate in another organization
    C: Risk assessment is required to understand how much testing is enough for each system
    D: safety critical systems are likely to need more coverage than non-safety critical systems
    E: Safety critical systems are tested exhaustively
    A, D and E are reasonable conclusions the other are not
    B, C and D are reasonable conclusions the other are not
    B, C and E are reasonable conclusions the other are not
    B, D and E are reasonable conclusions the other are not
  12. Which of the following statements contains a valuable objective for a test team?
    1. The goal is to fulfill all testing activity requirements as specified in the project handbook
    2. The goal is to cause as many failures as possible so that faults can be identified and corrected
    3. The goal is to prove that all faults are identified
    4. The goal is to prove that failures may not be caused any more by the remaining defects
  13. Which of following alternative describes a general testing principle..
    1. Defect are evenly distributed through the system
    2. Repeating the same test cases over again in new version of the software finds less and less defects
    3. Testing can prove that there are no defects
    4. If testing shows no defects, then the system is useful
  14. Which is correct relation between test conditions and test cases?
    1. Test conditions explain procedures to execute of test case
    2. Test cases are design to fill test conditions
    3. Test condition are design to fill certain test cases
    4. Test conditions are equivalent to certain test cases

Small test 3 – Fundamental of testing – answer

  1. You are running a written test which other testers have run previously. In addition, this test has found bugs previously, and those bugs have been confirmed to be fixed. The test manager has encouraged you to the specific way in which you run the test, such as the order of certain actions,.. use of mouse versus hot-keys, and the particular input values, based on the way users will use in the system. Which of the following is a testing principle that could explain the test manager’s directive?
    1. Early testing
    2. Absence of error fallacy
    3. Random testing
    4. Pesticide paradox
  2. According to istqb syllabus, a human being can make (P) which produces (Q) in the program code, or in a document. If (Q) in code is executed, the system may fail to do that it should do, causing (R). Which of following is correct?
    1. P: failure, Q: defect, R: error
    2. P: Error, Q: Defect, R: Failure
    3. P: Error, Q: failure, R: Defect
    4. P: failure, Q: Error, R: Defect
  3. Deciding how much testing is enough should take account of followings except
    1. Project constraints
    2. Level of risk
    3. Decision table
    4. Safety
  4. Under limited project time and budget, testing effort shall be focused proportionally to the expected and later observed defect density of modules so that project constraints are properly managed. Which testing principle is explained?
    1. Pesticide paradox
    2. Absence of error fallacy
    3. Early testing
    4. Defect clustering
  5. Which of following is NOT test analysis and design activity in Fundamental of testing
    1. Identifying necessary test data to support the test conditions and test cases
    2. Identifying and prioritizing test conditions based on analyses of test items, the specification, behavior and structure of the software
    3. Creating bi-directional trace-ability between test basis and test cases
    4. Creating test suites from the test procedures for efficient test execution
  6. Which statement about testing is true?
    1. Testing is started after the code is written so that we have a system with which to work
    2. Testing is started as early as possible in the life cycle
    3. Testing is most economically done at the end of the life cycle
    4. Testing can only be done by an independent test team
  7. The risk of an undetected defect…
    1. Does not influence testing in anyway
    2. Has influence on test comprehensiveness and intensity
    3. Has no influence on the specification of the test cases
    4. Has no influence on the prioritization of the tests
  8. During testing…
    1. All parts of the system should be tested with the same intensity, because defects can be everywhere
    2. The user interface should primarily be tested, because failures in it are most annoying for the customer
    3. System parts where failures may cause great risks should be tested most intensively
    4. Data base access should be tested intensively, in order to prevent wrong data and inconsistencies an the database
  9. Which of the following is a true general testing principle?
    1. Defect are evenly distributed throughout the system
    2. Testing can prove that there are no defect
    3. Repeating the same test cases over again in new versions of the software finds less and less defects
    4. If testing shows no defects, then the system in useful

Small test 2 – Fundamental of testing – Answer

1.Which of the following describes NOT related with Testing Principles

  1. Automated tests allow better statements of confidence about the quality of software products than manually executed test
  2. Exhaustive testing of software is, with sufficient effort and tool support feasible for all software
  3. For a software system, it is normally impossible to test all input/output combinations
  4. The purpose of testing is demonstrating the absence of defects

2. As a test leader you are collecting measure about defects. You recognize that after the first test cycle-covering all requirements – subsystem hat a defect density that is 150% higher than the average. Subsystem A on the other hand has a defect density that is 60% lower than average. What conclusion for the next test cycle could you draw from this fact?

  1. We focus testing ob subsystem A, because we have found fewest defects in that module so far
  2. Because we are already found many defects in subsystem C we do not need to test is much more
  3. Observed defect density does not allow any conclusion about the amount of additional testing
  4. According to the testing principle “defect clustering” it is probable that subsystem C has till more hidden defects. Therefore we need to test subsystem C in more detail

3. Below situation tells one of testing principle. What is it? One tester has tested software applications running on mobile phone for 5 years. He has great deal of information for testing the application and is able to test faster than anyone else. However, he has not put sufficient efforts for deriving new test cases and modifying them to find new defects any longer, which lead to finding less and less defects. What is the basic test principle that he has ignored?

  1. Pesticide paradox
  2. Absence-of-errors fallacy
  3. Defect clustering
  4. Early testing

4. Which of following activity is not related with test implementation and execution of the fundamental test process

  1. Creating test suites from the test procedures for efficient test execution
  2. Verifying that the test environment has been set up correctly
  3. Finalizing, implementing and prioritizing test case including the identification of test data
  4. Identifying necessary test data to support test conditions and test case

5. A certain degree of independence often makes the tester more effective at finding defects and failures. Which of following is highest level of independence?

  1. Tests deigned by the person who wrote the software under test
  2. Tests deigned by another person
  3. Tests deigned by a person from a different organizational group of test specialists
  4. Tests deigned by a person from different organisation or company including outsourcing

6. Which of the following alternative describes NOT a major task of test implementation and execution

  1. Checking test logs against the exit criteria specified in test planning
  2. Develop and prioritizing test cases, creating test data, writing test procedures and optionally, preparing test harness and writing automated-test scripts.
  3. Logging the outcome of test execution and recording the identities and versions of the software under test, test tools and test ware.
  4. Verifying that the test environment has been set up correctly.

7. Which of the following alternatives describes fundamental test process

  1. Planning, follow up and control, analysis and specification, design and implementation, execution and evaluating exit criteria, reporting and test closure activities
  2. Strategic planning, specification, implementation, test environment implementation, execution and checking, follow-up and reporting
  3. Planning and control, implementation, execution, evaluation, retest and regression test, evaluating exit criteria and reporting, test closure activities
  4. Planning and control, analysis and design, implementation and execution, test completion activities

8. Which of the following statement describes correctly about software testing terms for problem in software

  1. Errors or mistakes result in faults, which, when executed, may result in failures
  2. Errors result in failure which, when executed, may result in incidents
  3. Human failure may result in defects, which, when executed, will result in incidents
  4. Problems during analysis and design may result in errors, which during execution may result in failure

9. Which of following alternative describes a general testing principle

  1. Defects are evenly distributed through the system
  2. Repeating the same test cases over again in new version of the software finds less and less defects
  3. Testing can prove that there are no defects
  4. If testing shows no defects, then the system is useful

10. Which of the following statements contains a valuable objective for a test team?

  1. The goal is to fulfill all testing activity requirements as specified in the project handbook
  2. The goal is to cause as many failures as possible so that faults can be identified and corrected
  3. The goal is to prove that all faults are identified
  4. The goal is to prove that failures may not be caused any more by the remaining defect

Small test 4 – Fundamental of testing

← Back

Thank you for your response. ✨

According to ISTQB, a human being can make (A), which produces (B) in the program code, or in a document. Which of following combination is correct?

2. Which of following is NOT correct regarding of testing and quality?

3. Below shows several levels of independence defined. Which of following has lowest independence?

4. Which of following activity is NOT related with Test implementation and execution the fundamental test process?

5. Which of the following describes NOT related with testing objective?

6. which of the following describes NOT related with Testing Principles?

7. If the same testes are repeated over and over again, eventually the same set of test cases will no longer find any new defects. To overcome this, test cases need to be regularly reviewed and revised. Which of the following describes this in software testing principle?

8. According to ISTQB syllabus, a software error can be described as…

9. Which of the following is NOT “test analysis and design” activity in fundamental test process?

10. Which is not a goal of writing effective problem/bug report?

11. Which of the following are conclusion you could draw from the test principles? A: Website cannot be tested because of the vast number of combinations of platform, browser, internet route and navigation options B: The testing coverage required in one organization may not be appropriate in another organization C: Risk assessment is required to understand how much testing is enough for each system D: safety critical systems are likely to need more coverage than non-safety critical systems E: Safety critical systems are tested exhaustively

12. Which of the following statements contains a valuable objective for a test team?

13. Which of following alternative describes a general testing principle..

14. Which is correct relation between test conditions and test cases?

Small test 3 – Fundamental of testing

← Back

Thank you for your response. ✨

1. You are running a written test which other testers have run previously. In addition, this test has found bugs previously, and those bugs have been confirmed to be fixed. The test manager has encouraged you to the specific way in which you run the test, such as the order of certain actions,… use of mouse versus hot-keys, and the particular input values, based on the way users will use in the system. Which of the following is a testing principle that could explain the test manager’s directive?

2. According to istqb syllabus, a human being can make (P) which produces (Q) in the program code, or in a document. If (Q) in code is executed, the system may fail to do that it should do, causing (R). Which of following is correct?

3. Deciding how much testing is enough should take account of followings except

4. Under limited project time and budget, testing effort shall be focused proportionally to the expected and later observed defect density of modules so that project constraints are properly managed. Which testing principle is explained?

5. Which of following is NOT test analysis and design activity in Fundamental of testing

6. Which statement about testing is true?

7. The risk of an undetected defect…

8. During testing…

9. Which of the following is a true general testing principle?

Small test 2 – Fundamental of testing

← Back

Thank you for your response. ✨

1. Which of the following describes NOT related with Testing Principles?

2. As a test leader you are collecting measure about defects. You recognize that after the first test cycle-covering all requirements – subsystem hat a defect density that is 150% higher than the average. Subsystem A on the other hand has a defect density that is 60% lower than average. What conclusion for the next test cycle could you draw from this fact?

3. Below situation tells one of testing principle. What is it? One tester has tested software applications running on mobile phone for 5 years. He has great deal of information for testing the application and is able to test faster than anyone else. However, he has not put sufficient efforts for deriving new test cases and modifying them to find new defects any longer, which lead to finding less and less defects. What is the basic test principle that he has ignored?

4. Which of following activity is not related with test implementation and execution of the fundamental test process

5. A certain degree of independence often makes the tester more effective at finding defects and failures. Which of following is highest level of independence?

6. Which of the following alternative describes NOT a major task of test implementation and execution?

7. Which of the following alternatives describes fundamental test process?

8. Which of the following statement describes correctly about software testing terms for problem in software?

9. Which of following alternative describes a general testing principle

10. Which of the following statements contains a valuable objective for a test team?

Small Test 1 – Fundamental of Testing – answer

Question #1 Which one of the following is the BEST description of a test condition?

b) An aspect of the test basis that is relevant to achieve specific test objectives.

Question #2 Which of the following statements is a valid objective for testing?

b) To find as many failures as possible so that defects can be identified and corrected.

Question #3 Which of the following statements correctly describes the difference between testing and debugging?

b) Testing shows failures caused by defects; debugging finds, analyzes, and removes the causes of failures in the software.

 Question #4 Which one of the statements below describes a failure discovered during testing or in production?

a) The product crashed when the user selected an option in a dialog box.

Question #5 Which of the following statements CORRECTLY describes one of the seven key principles of software testing?

c) It is impossible to test all input and precondition combinations in a system.

 Question #6 In what way can testing be part of Quality assurance?

b) It reduces the level of risk to the quality of the system.

Question #7 Which of the below tasks is performed during the test analysis activity of the test process?

d) Evaluating the test basis for testability.

Question #8 A company recently purchased a commercial off-the-shelf application to automate their bill-paying process. They now plan to run an acceptance test against the package prior to putting it into production. which of the following is their most likely reason for testing?

a)  To build confidence in the application. Q

Question #9 A test team consistently finds between 90% and 95% of the defects present in the system under test. While the test manager understands that this is a good defect-detection percentage for her test team and industry, senior management and executives remain disappointed in the test group, saying that the test team misses too many bugs. Given that the users are generally happy with the system and that the failures which have occurred have generally been low impact, which of the following testing principles is most likely to help the test manager explain to these managers and executives why some defects are likely to be missed?

a. Exhaustive testing is impossible

Question #10 Which of the following is most important to promote and maintain good relation- ships between testers and developers?

b)  Explaining test results in a neutral fashion.

Question #11 Which of the statements below is the best assessment of how the test principles apply across the test life cycle?

d. Test principles affect activities throughout the test life cycle.