Software Testing

# Software Testing

Validating the functionality of a codebase.

---

# Agenda

1. [Overview](#overview)
1. [Automation](#automation)
1. [Code Linting](#code-linting)
1. [Unit Testing](#unit-testing)
1. [Code Coverage](#code-coverage)
1. [Integration Testing](#integration-testing)
1. [User Acceptance Testing](#user-acceptance-testing)
1. [Regression Testing](#regression-testing)
1. [System Testing](#system-testing)
1. [Test-Driven Development](#test-driven-development)
1. [Behavior-Driven Development](#behavior-driven-development)
1. [Conclusions](#conclusions)

---

# Overview

## Substance

Testing helps verify that the software performs as expected.

- Functionality

- Speed

- Load

- Security

- etc.

---

## Style

Testing can also help verify that the software has been written well.

- Syntax

- Style

- Maintainability

---

## Sanity

Testing can also help prove that the **software is usable** and that you haven't built it all for nil.

---

# Automation

## Rise of the Robots

While much of running software tests can now be automated, a few tasks still require humans.

- Writing the tests... although this too [may become fully automated eventually](https://duckduckgo.com/?q=autogenerate+unit+tests)

- Testing whether humans are able and happy to use the software... although [some seem to think they can automate users away](https://duckduckgo.com/?q=automate+user+acceptance+testing).

---

# Code Linting

## Concept

A code linter automatically checks both **syntax** and **style** of code.

- These are _especially useful in interpreted languages_ that do not report compilation errors.

- A code linter performs a **static analysis** of the codebase, meaning the code does not need to be executed for it to be linted.

---

## Features

A code linter is a software tool that can...

- be set to follow a set of code rules and standards

- flag suspicious usage in code that does not meet the defined rules and standards

- auto-fix some syntax and style problems

- integrate with most popular code editors

---

## Advantages

Code linters offer some advantages over human code reviews, such as:

- fast

- accurate

- consistent

- impersonal

---

## Recommendations

Use [ESLint](https://github.com/eslint/eslint) for Javascript.

- An [extension](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint) exists to integrate it into the popular Visual Studio Code IDE.

- Can be configured to test against any Javascript **coding conventions** - there are two popular competing Javascript style guides: [Google's](https://google.github.io/styleguide/jsguide.html) and [AirBnb's](https://github.com/airbnb/javascript).

- There are also plenty of people who decide on their own code styles.

- Follow [these instructions](https://www.linkedin.com/learning/building-a-website-with-node-js-and-express-js-3/setting-up-eslint-and-prettier?u=2131553) - or [these](https://www.youtube.com/watch?v=SydnKbGc7W8) if you don't have a LinkedIn Learning account - to set it up ESLint with the AirBnB style guide and the Prettier reformatter.

---

# Unit Testing

## Concept

A unit is the smallest testable unit of code - typically a function.

- [Unit testing](/content/courses/software-engineering/slides/unit-testing) verifies that each unit behaves as expected, given certain inputs.

---

## Javascript Example

A unit test example in Javascript using the `mocha` unit testing framework and Node.js's built-in assertion library, `assert`:

```javascript
// use node.js's built-in assert assertion library
const assert = require("assert")

// a set of tests of array functions
describe("Array", function () {
  // one particular unit test
  describe("#indexOf()", function () {
    // assert what should be returned
    it("should return -1 when the value is not present", function () {
      // test that assertion
      assert.equal(-1, [1, 2, 3].indexOf(4))
    })
  })
})
```

---

## Python Example

A unit test example in Python using `pytest`:

```python
import pytest
from some_package import some_module

class Tests:

def test_some_function(self):
    '''
    Verify some_function() returns a non-empty string.
    '''
    actual = some_module.some_function() # get the actual return value of the function
    assert isinstance(actual, str), f"Expected some_function() to return a string. Instead, it returned {actual}"
    assert len(actual) > 0, f"Expected some_function() not to be empty. Instead, it returned a string with {len(actual)} characters"

```

---

## Features

Unit tests are performed on each unit in isolation, in the sense that they...

- run in isolation and do not depend upon one-another.

- are not concerned with user interactions or user interfaces.

- do not require any changes to the production code.

- only test code belonging to the system, not platform code, 3rd party code or external systems such as databases or APIs.

---

## Advantages

Unit testing has discrete advantages that become increasingly important as a project grows in complexity and age.

- Failed unit tests indicate bugs that **must** be fixed

- Unit tests are fast to write and faster run

- Running unit tests can be automated, with a human notified of any failure

---

## Recommendation

Use [mocha](https://mochajs.org/) and [chai](https://www.chaijs.com/) for Javascript unit testing.

- `mocha` is a Javascript unit test framework

- by default it uses Node.js's built-in `assert` assertion library.

- but `chai` is a more elaborate assertion library that is ironically often paired with mocha

---

# Integration Testing

## Concept

Whereas unit testing tests units in-situ in isolation, integration testing tests units in vivo.

- Unit tests are meant to validate each unit in isolation from any external influences.

- Integration tests are purposefully designed to test units as they interact with those outside influences.

- For example, whereas a a unit test of the front-end of a web app might make an HTTP request to a **mock server or API** and receive a **mock response** , a unit test would make an HTTP request to a **real server or API**, receivee the **real response**, to ensuree it is handled correctly.

---

## Features

Integration tests...

- test that all components of the system interact as expected, sending and receiving messages from one-another correctly in all expected circumstances

- test that the system interacts with any external dependencies, such as libraries, databases, and APIs correctly

- can be of partial or full environments, including external bits like databases and services, or not

- can include user interfaces and results of particular system interactions, such as changes to content in databases and logs that result from certain actions

- can be run on one system, or across several systems

---

# Code Coverage

## Concept

The term `code coverage` refers to the percent of all code which is executed when unit and integration tests are run.

- Code coverage tools automatically calculate this as tests are run.

- While 100% code coverage is the ideal, anywhere above 80% is pretty well-covered.

---

## Limitations

Code coverage does not indicate that the code has been tested in all possible scenarios

- High code coverage **does not** indicate that the code is good.

- 100% code coverage **does not** indicate that the code is well-tested or that it will run well.

---

## Recommendation

Use [istanbul](https://istanbul.js.org/) for Javascript code coverage analysis.

- integrates well with the mocha unit testing framework

- An example of using istanbul in a Node.js project's `package.json` script:

```javascript
{
    "scripts": {
      "test": "nyc mocha --timeout=3000"
    }
}
```

- Code coverage analysis could then be triggered from the command line.

```bash
npm test
```

---

# User Acceptance Testing

## Concept

User Acceptance Testing (UAT) ...

- tests whether users will accept the software.

- i.e., verifies that the software works as expected from an end-user's point of view

- is a form of Integration Testing, since all the parts must inter-operate in order for the system to be used.

---

## Scripts

User Acceptance Tests test users in specific use cases or user stories.

- Users are asked to run through scripts of common scenarios and interactions they might encounter using the software

- Each script includes a set of steps for the human tester to go through to perform the test, e.g.

```
step 1 - user goes to home page
step 2 - clicks reservations
step 3 - clicks on arrival date
step 4 - the date widget should pop up with default date pre-selected
```

- The tester then records whether the user successfully completed the task, how long it took them, and notes any problems the user encountered.

---

# Regression Testing

## Concept

The term `regression testing` refers to the re-running of old tests when new tests are run.

- This ensures that the entire codebase works well, even when new features are developed, bugs are fixed, etc.

---

# System Testing

## Concept

Testing of the non-functional requirements, such as ...

- **load handling** - handling the load under expected conditions

- **stress testing** - handling the load at higher-than expected conditions

- **security testing** - making sure the system and its users are safe

---

# Test-Driven Development

## Concept

Test-Driven Development (TDD) is the practice of writing tests, particularly unit tests, before production code is written.

---

## Advantages

Test-Driven Development has several touted benefits:

- **code coverage** - every bit of code has a test developed for it before the code to be tested has even been written

- **debugging** - since tests are run with every code change, it is easy to identify the new code that created a bug

- **documentation** - the tests themselves become a specification of the system

- **planning** - it forces developers to think about what they want their code to do before writig it

---

## Criticisms

Test-Driven Development has been attacked on many fronts:

- It is often seen as a bit too extreme -in fact, TDD originated from what is called the **eXtreme Programming** (XP) methodology.

- In most cases (for better or worse), the requirements of systems are not fully known until development starts

- It is not unusually for new requirements to surface late in the development cycle.

- Many developers consider it helpful to have an ongoing feedback loop between specifying, developing and testing.

---

# Behavior-Driven Development

## Concept

An alternative preferred by some to Test-Driven Development is Behavior-Driven Development (BDD).

- BDD performs all the same duties as TDD, at the same times, for the same reasons.

- The difference arises from the language used in writing the code - BDD attempts to offer testing in a more human-centric language.

- Javascript's `chai` assertion library, for example, supports either TDD or BDD style assertions, e.g.

- A TDD-style assertion stating the expected value in a variable: `assert.equal(foo, 'bar')`

- The same assertion in BDD-style: `expect(foo).to.equal('bar')` - notice the more human-friendly phrasing of the code.

---

# Conclusions

You now have experienced a nice-and-easy overview of the different categories of software testing in common practice. The next step would be to try them out!

- Thank you. Bye.