What is unittest

Python's unittest is a testing framework included in the Python standard library. unittest allows you to write unit tests for your Python code.

unittest provides the following features

• Creation of test classes
• Creation of test methods
• Automatic execution of tests
• Using assertions
• Skipping or ignoring tests

The following is an example of writing tests using unittest.

import unittest

class MyTest(unittest.TestCase):
    def test_addition(self):
        self.assertEqual(1 + 1, 2)

    def test_subtraction(self):
        self.assertEqual(3 - 1, 2)

if __name__ == '__main__':
    unittest.main()

In this example, we define a test class named MyTest and two test methods named test_addition and test_subtraction. Each asserts that 1 + 1 equals 2 and 3 - 1 equals 2, respectively.

Finally, unittest.main() is called to automatically execute the tests.

Using unittest improves code quality and makes it easier to find bugs. It also improves the overall development process by letting developers know when they need to fix their code and that the test has failed.

unittest terminology

Python's unittest has the following terms.

unittest procedure

1. Define functions and classes to be tested.
2. Create a class that extends the TestCase class of unittest.
3. Create a test method. It is recommended that the name of the test method begin with "test_".
4. Within the test method, call the function or class to be tested.
5. Use assert statements to check if the expected results are obtained.
6. Execute the test class with unittest.main().

A concrete example is shown below. Here, we define a reverse_string function to reverse a string as the function to be tested, and create a test class to test the function.

def reverse_string(s):
    return s[::-1]

import unittest

class TestReverseString(unittest.TestCase):
    def test_reverse_string(self):
        self.assertEqual(reverse_string('hello'), 'olleh')
        self.assertEqual(reverse_string('world'), 'dlrow')

if __name__ == '__main__':
    unittest.main()

The above code defines the TestReverseString class as a test class and creates the test_reverse_string method as a test method within it. test_reverse_string uses the reverse_string function and checks if the return value is correct using the assertEqual method.

Finally, in the if name == 'main': section, we call unittest.main() to run the test if this file is executed directly.

Detailed usage of unittest

This section describes the detailed usage of unittest.

Creating a test class

When using unittest, you need to create a test class to define test cases. The test class must inherit from the unittest.TestCase class. If you define a test method, the method name must begin with test_.

import unittest

class MyTest(unittest.TestCase):
    def test_addition(self):
        self.assertEqual(1 + 1, 2)

    def test_subtraction(self):
        self.assertEqual(3 - 1, 2)

Use Assertions

unittest uses assertions to determine the success or failure of a test. Assertions are used to verify that the test runs as expected.

For example, the following assertion can be used to verify that the values are equal

import unittest

class MyTest(unittest.TestCase):
    def test_addition(self):
        self.assertEqual(1 + 1, 2)

    def test_subtraction(self):
        self.assertEqual(3 - 1, 2)

Automatically run tests

When using unittest, tests can be run automatically. The easiest way is to call unittest.main().

if __name__ == '__main__':
    unittest.main()

To do it this way, execute the following command.

$ python test_module.py

Skip or ignore tests

To skip a test, use unittest.skip(). You can also specify the reason for skipping.

import unittest

class MyTest(unittest.TestCase):
    @unittest.skip("reason for skipping here")
    def test_addition(self):
        self.assertEqual(1 + 1, 2)

Assertion method of unittest

The assert statement is used to verify that the function under test returns the expected result. For example, you can define a test case as follows.

import unittest

def add(a, b):
    return a + b

class MyTest(unittest.TestCase):
    def test_add(self):
        self.assertEqual(add(2, 3), 5)
        self.assertNotEqual(add(2, 3), 6)
        self.assertTrue(add(2, 3) > 4)
        self.assertFalse(add(2, 3) < 4)

In the above example, the add() function is defined as the function to be tested. The test case verifies that the add() function returns the expected result. For example, self.assertEqual(add(2, 3), 5) verifies that the result of add(2, 3) is 5. If all assert statements succeed, the test case passes; if any of the assert statements fail, the test case fails.

Below are the main assertion methods.

assertEqual(a, b)

Verifies that a and b are equal. Equivalent to a == b.

self.assertEqual(1 + 2, 3)

assertNotEqual(a, b)

Verifies that a and b are not equal. Equivalent to a ! = b.

self.assertNotEqual(1 + 2, 4)

assertTrue(x)

Verifies that x is True.

self.assertTrue(1 + 2 == 3)

assertFalse(x)

Verifies that x is False

self.assertFalse(1 + 2 == 4)

assertIs(a, b)

Verifies that a and b are the same object. Equivalent to a is b.

self.assertIs([], [])

assertIsNot(a, b)

Verifies that a and b are not the same object. Equivalent to a is not b.

self.assertIsNot([], [])

assertIsNone(x)

Verifies that x is None. Equivalent to x is None.

self.assertIsNone(None)

assertIsNotNone(x)

Verifies that x is not None. Equivalent to x is not None.

self.assertIsNotNone(1 + 2)

assertIn(a, b)

Verifies that a is contained in b. Equivalent to a in b.

self.assertIn(1, [1, 2, 3])

assertNotIn(a, b)

Verifies that a is not in b. Equivalent to a not in b.

self.assertNotIn(4, [1, 2, 3])

assertIsInstance(a, b)

Verifies if a is an instance of b.

self.assertIsInstance([], list)

assertNotIsInstance(a, b)

Verifies if a is not an instance of b.

self.assertNotIsInstance([], dict)

assertAlmostEqual(a, b, places=x)

Verifies if a and b are equal within the specified number of digits.

self.assertAlmostEqual(1/3, 0.3333333333333333, places=5)

assertNotAlmostEqual(a, b, places=x)

Verifies if a and b are not equal within the specified number of digits.

self.assertNotAlmostEqual(1/3, 0.333333333, places=7)

assertRaises(exc, func, *args, **kwargs)

Runs the test with the expectation that the specified exception will be raised. That is, it is used to verify that the function under test raises an exception.

The assertRaises method takes two arguments. The first argument specifies the type of exception to be expected, and the second argument is the function to be tested. The following is an example of using the assertRaises method.

import unittest

def divide(a, b):
    if b == 0:
        raise ZeroDivisionError('division by zero')
    return a / b

class MyTest(unittest.TestCase):
    def test_divide(self):
        self.assertRaises(ZeroDivisionError, divide, 1, 0)

In the above example, the divide() function is defined to raise a ZeroDivisionError exception in case of zero division. The test_divide method of the MyTest class uses the assertRaises method to ensure that the divide() function raises a ZeroDivisionError exception. The first argument of the assertRaises method is ZeroDivisionError, and the second argument is the divide() function and its arguments. In other words, we expect to run divide(1, 0) and get a ZeroDivisionError.

The assertRaises method will fail the test if the specified exception is not raised. It also returns an exception object if an exception is raised. The exception object can also be used to verify that the exception is of the correct type. For example, you can use the assertRaises method as follows.

import unittest

def divide(a, b):
    if b == 0:
        raise ZeroDivisionError('division by zero')
    return a / b

class MyTest(unittest.TestCase):
    def test_divide(self):
        with self.assertRaises(ZeroDivisionError) as cm:
            divide(1, 0)
        self.assertEqual(str(cm.exception), 'division by zero')

In the above example, the assertRaises method is used with a with statement. By using the with statement, we get the exception object, which is the return value of the assertRaises method, and verify that the exception is of the correct type. cm.exception represents the exception object itself. str(cm.exception) returns a string representing the error message of the exception object. The self.assertEqual method is used to verify that the error message in cm.exception is the expected one.

The assertRaises method can only check for a single exception. If you want to check for multiple exceptions, you must call the assertRaises method multiple times. Also, if no arguments are passed, an AssertionError is raised.

For example, you can use the assertRaises method as follows.

import unittest

def divide(a, b):
    if b == 0:
        raise ZeroDivisionError('division by zero')
    elif a < 0:
        raise ValueError('a must be non-negative')
    return a / b

class MyTest(unittest.TestCase):
    def test_divide(self):
        self.assertRaises(ZeroDivisionError, divide, 1, 0)
        self.assertRaises(ValueError, divide, -1, 2)

In the above example, the divide() function is defined to raise a division by zero or ValueError exception. The test_divide method of the MyTest class uses the assertRaises method to ensure that the divide() function raises the specified exception. Multiple calls to the assertRaises method check for multiple exceptions.

Note that the assertRaises method can also be written using the context manager as follows.

import unittest

def divide(a, b):
    if b == 0:
        raise ZeroDivisionError('division by zero')
    elif a < 0:
        raise ValueError('a must be non-negative')
    return a / b

class MyTest(unittest.TestCase):
    def test_divide(self):
        with self.assertRaises(ZeroDivisionError):
            divide(1, 0)
        with self.assertRaises(ValueError):
            divide(-1, 2)

In this case, there is no need to specify an exception in the first argument of the assertRaises method. Also, by using the context manager, only exceptions raised within the with block will be caught.

unittest test runner

unittest provides a test runner to report the results of tests. The test runner provides an interface to manage test runs and report results. Below are the main test runners available in unittest.

unittest.main()

This is the default test runner provided by unittest. With this runner, you can run tests from the command line. Tests can be run by executing the test file directly, as shown below.

$ python test_module.py

unittest.TextTestRunner()

This test runner is used to report test results in text format. The test results can be displayed in the console as follows.

import unittest

if __name__ == '__main__':
    suite = unittest.TestLoader().discover('.')
    unittest.TextTestRunner().run(suite)

unittest.HTMLTestRunner()

This test runner is designed to report test results in HTML format. You can generate a report in HTML format as follows.

import unittest
import HTMLTestRunner

if __name__ == '__main__':
    suite = unittest.TestLoader().discover('.')
    with open('test_report.html', 'wb') as f:
        HTMLTestRunner.HTMLTestRunner(stream=f).run(suite)

unittest special methods

unittest has special methods for setting up and cleaning up tests. By using those methods, you can write test code more effectively. The main special methods are listed below.

setUp()

The setUp() method defines a process to be executed before the test case is executed. For example, you can define processes to initialize objects commonly used in test cases, or to establish a database connection. The setUp() method is executed only once per test case.

import unittest

class MyTest(unittest.TestCase):
    def setUp(self):
        self.my_object = MyObject()

    def test_something(self):
        result = self.my_object.do_something()
        self.assertEqual(result, expected_result)

In the above example, the setUp() method initializes a MyObject object so that it can be used in a test case.

tearDown()

The tearDown() method defines the processing to be performed after the test case is executed. For example, you can define a process such as disconnecting the database connection. The tearDown() method is executed only once per test case.

import unittest

class MyTest(unittest.TestCase):
    def setUp(self):
        self.my_object = MyObject()

    def tearDown(self):
        self.my_object.close()

    def test_something(self):
        result = self.my_object.do_something()
        self.assertEqual(result, expected_result)

In the above example, the tearDown() method disconnects the MyObject object.

Inheritance of setUp() and tearDown()` methods

The setUp() and tearDown() methods can also use inheritance to define common pre- and post-processing for multiple test cases. For example, to perform initialization processing common to multiple test cases, create a common parent class and define the setUp() method in the parent class.

import unittest

class MyObjectTestCase(unittest.TestCase):
    def setUp(self):
        self.my_object = MyObject()

    def tearDown(self):
        self.my_object.close()

class TestSomething(MyObjectTestCase):
    def test_something(self):
        result = self.my_object.do_something()
        self.assertEqual(result, expected_result)

class TestSomethingElse(MyObjectTestCase):
    def test_something_else(self):
        result = self.my_object.do_something_else()
        self.assertEqual(result, expected_result)

In the above example, the MyObjectTestCase class is defined as the parent class for the common initialization process. The TestSomething and TestSomethingElse classes inherit from the MyObjectTestCase class respectively, and the setUp() and tearDown() methods for using MyObject objects are automatically inherited.

Thus, unittest allows flexible definition of pre- and post-processing of test cases using the setUp() and tearDown() methods. Also, by using classes and inheritance, you can easily implement common preprocessing and postprocessing for multiple test cases.

setUpClass() and tearDownClass()

setUpClass() is a method called before the test class is executed. This method is used to set up the resources used by the test class.

tearDownClass() is a method called after the test class has been executed. This method is executed after all tests have completed and is used to clean up the resources used by the test class.

The setUpClass() and tearDownClass() methods are defined as follows.

@classmethod
def setUpClass(cls):
    pass

@classmethod
def tearDownClass(cls):
    pass

The setUpClass() and tearDownClass() methods are defined as class methods. Thus, these methods are executed at the class level and can use class variables and class methods.

The setUpClass() method can be used, for example, to establish a database connection, create a file for testing, or set up other resources.

The tearDownClass() method can be used, for example, to close a database connection, delete a file for testing, or release other resources.

The following is an example of using the setUpClass() and tearDownClass() methods.

import unittest
import os

class MyTest(unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        cls.test_dir = 'test'
        os.makedirs(cls.test_dir)

    def test_example(self):
        test_file = os.path.join(self.test_dir, 'test.txt')
        with open(test_file, 'w') as f:
            f.write('test')

    @classmethod
    def tearDownClass(cls):
        os.rmdir(cls.test_dir)

In the above example, the setUpClass() method creates a directory named test. The test_example() method creates a file named test.txt in the test directory. The tearDownClass() method deletes the test directory. Thus, by setting up the resources used by the tests with the setUpClass() method, you can avoid cross influence between tests.

subTest

subTest() is a function used by Python's unittest to repeatedly run the same test using multiple input values. Typically, it is used to test a single test function using several different input values.

subTest() can be used to identify the input values that failed if the test case fails. Also, subTest() can be used to continue execution of subsequent test cases even if a test function fails.

The following is an example of using subTest().

import unittest

class TestMath(unittest.TestCase):

    def test_add(self):
        test_cases = [(2, 3, 5), (-2, 3, 1), (0, 0, 0)]
        for a, b, expected in test_cases:
            with self.subTest(a=a, b=b):
                result = a + b
                self.assertEqual(result, expected)

In this example, the test_add() function runs the same test with several different input values. By using subTest(), the test results for each input value can be displayed independently.

subTest() is used with with statements. By making assertions within the with block, the test cases are executed independently for each input value. The argument to subTest() can also be an input value passed as a keyword argument.

Notes on writing tests using unittest

The following are points to note when writing tests using unittest.

• Prepare test data properly
  When writing tests, appropriate test data must be prepared. Insufficient data or using incorrect data may result in inadequate tests. It can also make testing more complicated.
• Avoid test dependencies
  You should avoid writing tests that depend on the order in which tests are executed. They are vulnerable and may fail if the order in which the tests are executed changes.
• Test granularity
  Test granularity is very important. Writing small tests makes it easier to identify the cause of problems when they occur. You can also combine small tests to create larger tests.
• Error message clarity
  When a test fails, it may take longer to identify the problem if the error messages are difficult to understand. Efforts should be made to make error messages clear and easy to understand.
• Order of test execution
  Because unittest does not guarantee the order in which tests are executed, state may be shared among tests. Therefore, each test method should be executed independently without being affected by the previous test method.

References

https://docs.python.org/2.7/library/unittest.html
https://docs.python.org/3/library/unittest.mock-examples.html