Quality Assurance/Testing

Introduction

Testing is critical to project success, whether your project is a small change to an existing system or a huge first release of custom built software for a new business. It is an integral part of every stage of the System Development Life Cycle. It is not something that occurs only in the final stages of development. It includes testing for functional and non-functional requirements enabling you to find bugs in the code, as well as determining the overall quality of the system. The fact is, the earlier a bug is found the cheaper it is to fix. Creating test plans early and executing them throughout the development process distinguishes a quality development organization from the rest.

What is a Unit Test?

Unit tests allow development teams to evaluate whether individual units of source code perform as expected before asking the client to perform user acceptance testing (UAT). The “units” in unit testing are meaningful code modules, such as “Login” and “View Customer Profile.” Unit tests utilize pre-created test data and standard operating procedures to ensure the specific unit of code works correctly under different scenarios. For example, you would have a unit test to examine the many different ways a user may try to login to a system:

• Valid username and valid password
• Valid username and invalid password
• Invalid username and valid password
• Invalid username and invalid password

The unit test would ensure that the system responds correctly as expected (per the requirements) in each of the above scenarios, thus testing that the code to allow a user to login works as intended.

Creating Unit Tests

To ensure that your software performs as expected (aka a "positive test"), you must create and execute unit tests during each sprint to fully test each user story being developed. The user stories table in your charter is a great place to start for creating a list of what needs to be tested and who will do the testing. Non-functional requirements should also be considered when creating unit tests. There are many tasks to be performed before a unit test can be executed:

• Identify the functional component to be tested
• Analysis: For the functional component, determine what should happen in various scenarios if the user behaves correctly (or incorrectly) when interacting with that feature
• Design: Consider what type of data/input/files will be required to test the feature
• Construct:
  • Create required test data/files
  • Write the unit test and the detailed steps to be performed
  • Write the expected results of each step
• Review: Execute test cases, and log successes and failures

The best test design is by a project manager or business analyst or another developer who did not write the code that is being tested. The developer will think of all the tests that were part of design; the developer who worked on another functional area will think of different details to test, thus catching errors in design as well as coding.

Example Unit Test for Latinitas User Story

In our class, you will need to test each feature/functional requirement that you are creating for each user story. Here is an example of a unit test for the Latinitas case from class:

User Story: I, Angie, want to securely review donations to Latinitas so that I can give feedback on our progress to the board of directors.
Functional requirement being tested: Create secure login to admin dashboard

Test Case	Description	Step	Test Steps	Expected Results
1	Login with incorrect password	1	Navigate to www.latinitas.com	User will see homepage of www.latinitas.com
		2	Click Login link in upper right corner	User is redirected to admin login page
		3	Enter "angieadmin" in username text box and "adminadmin" in password text box	Error message displayed: "The username and/or password that was entered is incorrect. Please try again."
2	Login with correct password	1	Navigate to www.latinitas.com	User will see homepage of www.latinitas.com
		2	Click Login link in upper right corner	User is redirected to admin login page
		3	Enter "angieadmin" in username text box and "admin1" in password text box	User sees admin dashboard page
3	Login with incorrect username	1	Navigate to www.latinitas.com	User will see homepage of www.latinitas.com
		2	Click Login link in upper right corner	User is redirected to admin login page
		3	Enter "angie" in username text box and "admin1" in password text box	Error message displayed: "The username and/or password that was entered is incorrect. Please try again."

Executing Unit Tests

To manage the execution of user tests in our course, utilize the Testing Template that has been added to your client folder on UT Box in the Testing folder. It is important that the person who wrote the test should NOT be the person who executes the test. This ensures that multiple people are looking at how the system works and are evaluating whether it does so correctly. Developers should also peer review each other's code as you are developing to help ensure that unit tests pass as expected. Your professors will examine your testing progress each week, and since user stories should be fully tested during each sprint, every team should be adding new unit tests to the tracker as well as execute those tests in the tracker during each sprint.

Documenting Unit Test Execution

It is vital that you document the execution of every unit test, whether it passed or failed. This means that you must provide some sort of proof that you executed the test. There are several ways to document test execution:

• Screen shot of user messages or pages displayed
• Video capture of a process to document the various steps
• Output files generated by the process
• Database queries to document before and after of a process to illustrate changes to the database

If you do not provide documentation, how can your team or your client be confident that the system works as expected? This is especially critical when delivering software for compensation, as you can be held accountable legally for the functioning of the system that you create. 

Tips for Testing

• Test data generators are a great time saver for generating real test data for your testing. Check out the test data generators listed on the Knowledge Exchange page. Even better: try to get actual data from the client’s existing system or records.
• Avoid “developer fixation;” be sure to ask business analysts and project managers to test the code. Or ensure that developers test each other’s code, so that the developer who knows an area’s detailed requirements, designed and coded the functionality, isn’t just testing what was in his or her own view of that feature.
• Write at least one test for every functional requirement, including all data validation requirements. This includes checking for the wrong kind of data, for example, a negative number of employees in a payroll program or entering numbers where letters are expected (see below for the list of top 10 negative tests).
• Check the flow of control through the entire system to ensure that the links work: that all desired paths exist and modules communicate correctly with one another 
• Test valid, invalid, and boundary values. This includes simple boundary checks, such as minimum, maximum, and off-by-one values. Verify that all error-detection routines work as expected.
• Test compound boundaries; that is, combinations of input data that might result in a computed variable that's too small or too large.
• Test data that is not supposed to change for consistency.
• Test system backup and disaster recovery procedures.
• Test for compatibility with old data.
• Test for browser compatibility, especially any older versions that are still common. (If the software does not work for older browsers or operating systems, include a test for the older products and write a clear error message for users.)

Top 10 Negative Tests

The above text talks mostly he following are very common things you should check for when performing your tests. We call them negative tests because they check for potentially "bad" things that could happen when a user interacts with your system:

• Embedded Single Quote - Can your forms handle single quotes?
• Required Data Entry - Are you allowed to submit the form without entering required fields and does it properly handle the record if you leave out a required field?  Also check that you can submit with only required fields and empty non-required fields.
• Field Type Test - Can you put currency in a date field or vice-versa?  Try to force in incorrect data type into a field to see if you application validates this is not possible?  Depending on your application you may choose to handle this at the DB level, the screen level, or both.  
• Field Size Test - Can you put in data that exceeds the field limit of the database?  
• Numeric Bounds Test - Can you enter a number that exceeds the UPPER or even LOWER bound?  Don't just try to put in a large positive number.  Try larger negative numbers too.  
• Date Bounds Test - Can you put in a date that exceeds the limits of your business logic? 
• Date Validity - Can you put in an invalid date such as Feb 31st?
• Special Characters - Can your forms handle special characters (e.g. !@#$%^&*)
• Error Messages - Can your forms throw a user-friendly error message versus a system error or something that is not easily understood?  Make sure you have proper handling for all types of errors.

Automated Testing

Testing is tedious work, especially for graphical user interfaces. Capturing the key strokes for a series of tests and then rerunning the scripts—rather than reentering the tests manually—saves time and increases the chance that all the necessary tests will be run after changes to the code. For systems with multiple releases, this is a huge time saver as well as a way to decrease the risk of introducing errors into new releases. A variety of vendors sell test tools. These testing processes are a standard part of the Development Integration Staging Environment. If you have experience with automated testing from an internship, feel free to incorporate tools to automate your tests in your client project; however, this is not a requirement, and we do not expect you to take the time to learn how to do this in our class.

User Acceptance Testing (UAT)

For more on this topic, click here.