Software Validation and Testing

TEST SUMMARY REPORT

Version 1.5
 DOCUMENT HISTORY

DOCUMENT APPROVAL HISTORY

Prepared By Mohammad Jada

Reviewed By Raya Halwani

Approved By Raya Halwani

DOCUMENT REVISION HISTORY

DOCUMENT
DATE REVISION DESCRIPTION AUTHOR
VERSION

15/12/2023 1.0 Initial Version Mohammad Hamdani

20/12/2023 1.1 Mohammad Hamdani

23/12/2023 1.2 Osama Alnajjar

29/12/2023 1.3 Mohammad Jada

01/01/2024 1.4 Osama Alnajjar

06/01/2024 1.5 Final Version Mohammad Jada

Software Validation and Testing

Table of Content

1. INTRODUCTION …………………………………………………2
1.1 Context
1.2 Purpose
1.3 Scope
1.4 Structure

2. test summary report ………………………………………….4

2.1 Functionality
2.2 Usability
2.3 Security
2.4 performance and scalability

3. test measure ………………………………………………….…12

3.1 Test Coverage

4. Defect Report …………………………………………………..13

3
Software Validation and Testing

1. Introduction
1.1 Context
In the fast-evolving landscape of e-commerce, the optimal functioning of an online store is
pivotal for delivering a satisfying and secure shopping experience to users. This introduction
sets the stage for a comprehensive test report on the MobiGo platform.

1.2 Purpose
This section outlines the primary purpose of the report: to transparently detail the testing
procedures, outcomes, and recommendations. The report spans functionality, usability,
security, and performance testing, providing stakeholders with a comprehensive
understanding of the online store's strengths and areas for improvement.

1.3 Scope
An overview of the testing scope, which encompasses MobiGo's entire functionality, from
login processes to payment gateways. The report delves into the user interface, security
protocols, and the platform's ability to handle varying levels of user traffic.

1.4 Structure
This report is structured into distinct chapters, each dedicated to a specific testing aspect,
providing a comprehensive examination of MobiGo. The following chapters guide readers
through key elements:
Chapter 2: Test Summary Report (Page 4): An in-depth exploration of the functional testing
phase, including the use of the Selenium library in Python. This chapter details the test
objectives, deviations encountered, completion evaluations, and lessons learned during the
testing process.
Chapter 3: Test Measure (Page 12): A comprehensive analysis of test metrics, including
measurements on test cases, defects found, incidents reported, test coverage, progress,
effectiveness, and resource consumption.
Chapter 4: Defect Report (Page 13): A detailed examination of the defects discovered during
testing, categorized by severity levels, and their impact on the application. This chapter also
provides insights into the resolution status of identified defects.
As we navigate through the diverse testing phases outlined in subsequent chapters, the report
will offer valuable insights into MobiGo's status and propose pathways for future optimization.

4
 Software Validation and Testing

2. Test Summary Report

2.1 Functionality

GENERAL INFORMATION
Test Level: level 1 Summary Date:
Functionality
Application:
MobieGo.com Electronics Store Website
SUMMARY
TEST PERFORMED
The test involves testing the basic functionalities of the website to ensure that it is
working as expected. This includes testing the home page, About us page, Login
feature, product page, shopping cart, checkout process, and payment gateway.
Testing every element on those pages such as buttons, pictures, sliders, and
possibilities in different fields and their constraints. Also, make sure every button and
link has a direction.
Those tests were performed via the Selenium library on Python.

DEVIATION The only deviation encountered during the testing process was a problem with the
(VARIANCES) & testing environment (selenium) which was fixed by changing the version of Python.
RESIDUAL RISK
TEST COMPLETION The test was conducted successfully without the need to suspend it; we found that we
EVALUATION needed more test cases while we were testing, so we decided to add more test cases
trying to cover as many test cases as we could.

FACTORS THAT • Lack of team management skill

BLOCKED PROGRESS • The website development took so much time.
• Time limitation
TEST MEASURES 1. Measurements on Test Cases: 51 total test cases, 50 passed and 1 failed (98.03% pass
rate).
2. Defects: 1 defect found.
3. Incidents: No incidents reported.
4. Test Coverage: All 51 test cases covered the specified test objectives.
5. Activity Progress: All 51 test cases completed.
6. Test Effectiveness: 98.03% pass rate.
7. Resource Consumption: 25 hours of testing time.

5
 Software Validation and Testing
TEST DELIVERABLES & Test Deliverables:
REUSABLE ASSETS 1. Test cases and Requirement
2. Test Plan
3. Test Scripts
4. Test Data
5. Test Reports

Reusable Assets:
Test data is stored in a database.
LESSON LEARNED • Further testing could be carried out to find any problems that were possibly
overlooked.
• Involving the testing team early in the development process is beneficial.
• Ambiguous or incomplete requirements can lead to misunderstandings and missed
defects.
• Creating comprehensive test cases is critical for thorough testing.
• Reusable test cases save time and effort in future testing cycles.

FINAL The website will be easier to use if it has more features, such a register and login
RECOMMENDATION option that allows users to save the products they have chosen in the database rather
than storing personal data like address and payment details.

6
 Software Validation and Testing

2.2 Usability
GENERAL INFORMATION
Test Level: level 1 Summary Date:
Usability
Application:
MobieGo.com Electronics Store Website
SUMMARY
TEST PERFORMED
Usability testing is a black-box testing approach.
• Usability testing was conducted on 02/01/2024 with a diverse group of 15
participants representing the target audience.
• Participants were asked to perform tasks such as logging in and registering,
choosing a product, adding items to the cart, and completing the checkout
process.
• Key observations included that the website was easy to learn and remember.

DEVIATION Cart Issues:

(VARIANCES) & During usability testing, participants encountered issues with the shopping cart
RESIDUAL RISK functionality. Adding items to the cart, updating quantities, or proceeding to checkout
exhibited unexpected behavior, leading to user confusion.
• Participants reported instances where items added to the cart were not reflected
accurately.
• The cart update process was perceived as slow, with delays in reflecting changes.
• Total amount in cart war calculated wrong.
Residual Risk:
The identified issues with the shopping cart pose a residual risk of negatively impacting
the user's confidence in the e-store. Users may experience frustration and potentially
abandon the shopping process if inaccuracies and delays persist. This could result in a
decline in conversion rates and customer satisfaction.
Login Page Issues:
Usability testing identified challenges and confusion related to the login page.
Participants experienced difficulties in locating the login link, and some reported issues
with the authentication process.
• Participants struggled to find the login link, resulting in delayed access to account-
related features.
• Authentication errors were encountered during login attempts, leading to frustration.
Residual Risk:
The issues with the login page present a residual risk of hindering user accessibility and
potentially discouraging users from engaging with the e-store. Persistent authentication
errors could result in account lockouts and impact user trust in the platform.
Website Lag:
Usability testing revealed instances of lag and performance issues across various pages
of the website. Participants reported delays in page loading times, affecting the overall
user experience.
• Pages, especially those containing dynamic content, experienced noticeable delays in
7
 Software Validation and Testing
loading.
Residual Risk:
The identified website lag introduces a residual risk of creating a suboptimal user
experience and potentially discouraging users from exploring the website further.
Prolonged loading times may lead to user frustration, impacting retention rates and
overall satisfaction.
TEST COMPLETION The usability testing objectives were largely achieved, providing valuable insights into
EVALUATION the user experience of the MobiGo website.
The identified usability issues and positive aspects will contribute to the refinement of
the website's design and functionality.
FACTORS THAT 1. There were limited resources available for testing.
BLOCKED PROGRESS 2. Limited availability of participants due to scheduling conflicts impacted on the speed
of testing sessions.
3. The checkout page had low responsiveness.
4. Technical issues with the live environment slowed down the testing process.
TEST MEASURES Test Cases: We tracked the progress and status of the test cases created.
Defects: We tracked the number of defects found on each page and the total number of
defects found.
Surveys: Usability was measured using a combination of task success rate, time to
complete tasks, and participant feedback through post-test surveys.
Test Coverage: We kept tabs on how many test cases were run through to completion
as well as how many were not.
Activity Progress: We tracked the progress of the testing activities.
Test Effectiveness: We tracked the effectiveness of the tests performed.
Resource Consumption: We tracked the resources consumed during the testing period.
TEST DELIVERABLES & Test Deliverables:
REUSABLE ASSETS 1. Test Plan
2. Test Cases
3. Test Results
4. participant demographics
5. Usability Report
Reusable Assets:
1. Test Data
2. Test Scripts
3. Test Documentation
4. Test Reports
LESSON LEARNED • Early involvement of users in the design process enhances the identification of
usability issues before the website launch.
• Clear communication channels between development and testing teams could
improve the resolution of technical issues.
FINAL • It is recommended to prioritize the resolution of identified usability issues before the
RECOMMENDATION website's official launch.
• Consider implementing user interface improvements to enhance the overall user experience.

8
 Software Validation and Testing

2.3 Security
GENERAL INFORMATION
Test Level: level 1 Summary Date:
Security
Application:
MobieGo.com Electronics Store Website
SUMMARY
TEST PERFORMED
Security testing was conducted on our e-store website to assess vulnerabilities and
ensure the protection of user data and sensitive information. The tests covered various
aspects, including authentication, authorization, data encryption, and protection against
common web application security threats.
DEVIATION We couldn’t cover all the cross-side scripting scenarios due to time constraints, however,
(VARIANCES) & more test cases are required to cover more vulnerabilities.
RESIDUAL RISK
TEST COMPLETION We can determine whether a user is protected by looking at the test case that has been
EVALUATION completed for HTTPS. The website is protected against hacker attempts to influence
customer and admin confidentiality thanks to SQL injection and XSS coding.
FACTORS THAT • Lack of team management skill
BLOCKED PROGRESS • Time limitation
• Limited cooperation from the development team in providing details about the
application's architecture and third-party integrations also impeded the testing
process.

TEST MEASURES 1. Measurements on Test Cases: 2 total test cases, 2 passed (100% pass rate).
2. Incidents: No incidents reported.
3. Test Coverage: It covered the essential attack that may come from a malicious hacker.
4. Activity Progress. All 2 test cases were completed.
5. Test Effectiveness: 100% pass rate.
6. Resource Consumption: 2 hours of testing time.

TEST DELIVERABLES & To make the injected code usable for other test cases, it was added to the test case
REUSABLE ASSETS design file.

9
 Software Validation and Testing
LESSON LEARNED LESSON LEARNED:

Our team determined the following areas of strength and improvement based on the
results of the security test.

Strengths:
• Verified that users need authentication to access the system by testing URL
manipulation.
• Verified that the login page was using HTTPS secure communication.
• SQL Injection vulnerabilities in the username field were successfully tested.

Opportunities for Improvement:

• Test additional XSS types to expand the scope of the security testing
• Conduct frequent security audits and scans to find any security flaws.

FINAL The security test was passed, but the HTTPS feature failed. The website should be
RECOMMENDATION secured using HTTPS to ensure secure communication. And more XSS types like reflected
XSS attacks

10
 Software Validation and Testing

2.4 performance and scalability

GENERAL INFORMATION
Test Level: level 1 Summary Date:
performance and scalability
Application:
MobieGo.com Electronics Store Website
SUMMARY
TEST PERFORMED
The purpose of the performance test is to assess the scalability and performance of the
website under various load scenarios. Testing in varied traffic and transaction volumes is
part of this. The test's objectives are to find any problems or performance bottlenecks
and confirm that the website can manage the anticipated demand.
Apache JMeter was used to test the performance of the website.
DEVIATION The website worked fine under an Endurance test and high load test.
(VARIANCES) & High load (20 threads, 1 ramp-up period), but on a very higher load (100 threads, 2 ramp-
RESIDUAL RISK up period the website crashed)
TEST COMPLETION The performance test was conducted with only three test cases before being suspended.
EVALUATION Overall, the performance test on a high load was a failure but helped us to identify and
address any performance issues.
FACTORS THAT • Limited cooperation from the development team in providing timely bug fixes and
BLOCKED PROGRESS updates also impacted the testing process.
•Setting up the testing environment.
•Lack of team management skills.
TEST MEASURES 1. Measurements on Test Cases: 3 total test cases, 2 passed and 1 failed (75% pass rate).
2. Defects: 1 defect found.
3. Incidents: uncompleted tests.
4. Test Coverage: All 3 test cases covered the specified test objectives.
5. Activity Progress: 2 test cases completed.
6. Test Effectiveness: 75% pass rate.
7. Resource Consumption: 3 hours of testing time.
TEST DELIVERABLES & Test plan: The test plan outlines the scope, objectives, and approach for the
REUSABLE ASSETS performance test. It may include details such as the test environment, test data, and
test cases.
Detailed Test Cases and Scripts: The steps and anticipated outcomes of the
performance test are outlined in the test cases. They might contain information on
the load levels, response times, and performance indicators that need to be
monitored.

Test environment: The test environment represents the hardware and software
setup used for the performance test in this case it was Apache JMeter.
Summary of Test Results: a report providing a summary of the functional results of
the test. A breakdown of the number of test cases run, passed, failed, and any
problems found during testing is also included.
11
 Software Validation and Testing
LESSON LEARNED • A key lesson learned is the importance of realistic load testing scenarios that accurately
reflect expected usage patterns. Additionally, early collaboration with the development
team is crucial to obtaining the necessary information for simulating real-world
conditions.

• Comprehending the website's performance attributes: it furnished discernments

regarding the website's performance attributes, including its response times and
resource allocation under varying load scenarios.

• Determining the effect of modifications on performance: It was useful in determining

the effect of modifications to the website, including the inclusion of new features or the
use of various technologies, on its functionality.

• Identifying opportunities for performance optimization.

FINAL • Resolved performance issues found: To guarantee the website performs as well as
RECOMMENDATION possible, any faults found during the performance test will be given top priority and
resolved. Optimize server configurations, enhance database performance, and
consider implementing caching mechanisms to improve response times.
• Collaborate with the development team: Collaborate closely with the development
team to implement the recommended optimizations and conduct periodic
performance testing to ensure continued optimal performance, especially during
promotional events or high-traffic periods.

APPROVALS
NAME/TITLE SIGNATURE DATE
Name/Test Manager Raya Halwani 03/01/2024

Name/Test Engineer Mohammad Hamdani/Osama Alnajjar 02/01/2024

Name/HR Raya Halwani 05/01/2024

Name/Business Manager Mohammad Jada 06/01/2024

12
 Software Validation and Testing
3. Test Measure

3.1 Test Coverage

Total Total No. Of No. Of No

Builds/ % Of No. Of % Of
Designe Test Test Test Or % Of Stopped Defects
Passed Test Cases Failed
Build Date d Test Cases Cases Stopped Test Cases Found
Test Cases Failed Test Cases
Cases Executed Passed Test Cases1
Final Build 62 62 59 95.16% 3 4.84% 0 0 3
06/01/2024

Test Coverage Prioritization Actual No. Of % Test

No. Of Possible
(e.g., Logic, Function, Feature)2 (e.g., High, Medium, Test Cases Coverage
Test Cases
Low)
Login High 10 8 80%

Register High 9 9 100%

Buttons clicks Medium 10 10 100%
View reviews Low 1 1 100%
Links Low 6 6 100%
About us Medium 1 1 100%
Adding product to the cart High 4 4 100%
View information on the Low 1 1 100%
product
Boxes data types Medium 6 6 100%

Select country/city Medium 4 4 100%

13
 Software Validation and Testing

4. Defect Report

Total No. Of Total No. Of Total No. Of

Severity
Defects Defects Closed Defects Open At Describe the Impact On the Application Of
Level Of
Found In The At The End Of The End Of The the Postponed Function
Defect
Test Level The Test Level Test Level 3
Critical 1 1 Threate Customer information security
Major 0 0 Prevent the main requirement of the
website to operate
Average 1 0 Defects that make customer's
experience harder and give some
misinformation to the admins
Minor 1 1 Don’t affect the requirments.

14
15