Unit Testing with Python

What is Unit Testing?

  • A unit test exercises a unit of functionality to test its behaviour
    • Test only one function
  • A unit test framework provides a standard mechanism
    • Specifying a test (setup, execution, expected result, teardown)
    • Executing a test
    • Generating test reports

It is a good idea to design your code with testing in mind

  • a code that simply reads and writes to standard input/output is harder to test than code that provides a more structured interaction

Anatomy of a unit test

Python includes a Unit Test framework called unittest

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# include module
import unittest

# A test case is a collection of tests
class TestStringMethods(unittest.TestCase):
  
  #A method is a test
  def test_upper(self):
    self.assertEqual('foo'.upper(), 'FOO')
    
  def test_isupper(self):
    # Calls to assertXXX() methods indicate test results
    self.assertTrue('FOO'.isupper())
    self.assertFalse('Foo'.isupper()) 
    
  def test_split(self):
    a = 'hello world'
    self.assertEqual(s.split(), ['hello', 'world'])
    # check that s.split fails when the separator is not a string
    with self.assertRaises(TypeError):
      s.split(2)
    
if __name__ == "__main__":
  unittest.main()

Output of the test results would look like this.

1
2
3
4
5
...
========================
Ran 3 tests in 0.000s

OK

Another Example

1
2
3
4
5
6
7
8
9
10
class TestString(unittest.TestCase):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        ...
        
    def test_example(self):
        ...
        
if __name__ == '__main__':
    unittest.main()

To test only one class or one simple test, we can:

1
2
python test.py TestString.test_example
python test.py TestString

Design for Testing

  • Factor the program into meaningful units/components
    • e.g., parser, command processor, components, data structures, etc.
  • Each unit should have a well-defined specification
    • What are legal inputs
    • What are legal outputs
    • How inputs & outputs are passed around
  • Avoid monolithic design that reads standard input & writes to standard output
  • Good design require more work
    • Additional functionality specifically for testing/debugging purposes
    • But ultimately will save time of the overall development

Coverage.py

https://coverage.readthedocs.io/en/latest/index.html

Not a standard library, required Installation

1
pip install coverage

Coverage.py is a widely used converage tool for python.

  • Test coverage is a metric identifying how much of a program has been executed by a given test (of set of inputs)
    • e.g. no. of statements executed / no. of total statements
  • Statement coverage measures the number of statements executed
  • Branch coverage measure the number of branches taken
    • A branch is covered if both true- and false- branches are taken in some execution
  • A code that is not covered is never executed; it might be (almost) complete none sense

Usage of coverage

Executes the program and monitors which statements are executed

1
coverage run -m unittest <name-of-unit-test>

Executes the program and monitors which statements are executed and which branches are followed

1
coverage run —branch -m unittest <name-of-unit-test>

Give a summary of the coverage run report

1
coverage report

Generates an HTML report showing coverage of the last run

1
coverage html
  • Can only be executed after coverage-run as shown above
  • The result is placed in htmlconv/index.html