Startup
單元測試的核心價值在于兩點：
更加精确地定義某段代碼的作用，進而使代碼的耦合性更低
避免程式員寫出不符合預期的代碼，以及因新增功能而帶來的Regression Bug
随着Test-Driven方法論的流行，測試類庫對于進階語言來說變得不可或缺。Python生态圈中的unit testing framework相當多，不同于Java幾乎隻有JUnit與TestNG二選一，Python unittest架構中較為活躍并也有較多使用者的framework就有unittest、unittest2、nose、nose2與py.test等。不計其他較小衆的工具，光是要搞懂這些工具并從中挑選一個合适的出來使用就讓人頭大了。本文是以總結了這些類庫在實戰中的作用，以便讀者在選擇時友善比對參考。
這裡是介紹Python測試的官方文檔：
本文在該文檔的基礎上删減了入門部分，增加了深入講解和實戰案例。
類庫
Unittest
Unittest的标準文檔在這裡：
Unittest是Python标準庫的一部分。它是目前最流行的固件測試架構XUnit在Python中的實作，如果你接觸過Junit，nUnit，或者CppUnit，你會非常熟悉它的API。
Unittest架構的單元測試類用例通過繼承unittest.TestCase來實作，看起來像是這樣：
import unittest
def fun(x):
return x + 1
class MyTest(unittest.TestCase):
def test(self):
self.assertEqual(fun(3), 4)
Unittest一共包含4個概念：
Test Fixture，就是Setup()和TearDown()
Test Case，一個Test Case就是一個測試用例，他們都是unittest.TestCase類的子類的方法
Test Suite，Test Suite是一個測試用例集合，基本上你用不到它，用unittest.main()或者其它發現機制來運作所有的測試用例就對了。 :)
Test runner，這是單元測試結果的呈現接口，你可以定制自己喜歡的呈現方式，比如GUI界面，基本上你也用不到它。
一些實戰中需要用到的技巧：
發現機制
python -m unittest discover -s Project/Test/Directory -p "*test*"
# 等同于
python -m unittest discover -s Project/Test/Directory
用Assert，不要用FailUnless（它們已經被廢棄）
Deprecated.png
常用的Assert
NormalAssert.png
特殊的Assert
SpecificAssert.png
For example:
assertAlmostEqual(1.1, 3.3-2.15, places=1)
SpecificEqual.png
AssertException
AssertException.png
assertRaises
assertRaises(exception, callable, *args, **kwds)
def raisesIOError(*args, **kwds):
raise IOError("TestIOError")
class FixtureTest(unittest.TestCase):
def test1(self):
self.asertRaises(IOError, raisesIOError)
if __name__ == '__main__':
unittest.main()
assertRaises(exception)
# If only the exception argument is given,
# returns a context manager so that the code
# under test can be written inline rather
# than as a function
with self.assertRaises(SomeException):
do_something()
# The context manager will store the caught
# exception object in its exception attribute.
# This can be useful if the intention is to
# perform additional checks on the exception raised
with self.assertRaises(SomeException) as cm:
do_something()
the_exception = cm.exception
self.assertEqual(the_exception.error_code, 3)
assertRaisesRegexp
self.assertRaisesRegexp(ValueError, "invalid literal for.*XYZ'$", int, 'XYZ')
# or
with self.assertRaisesRegexp(ValueError, 'literal'):
int('XYZ')
Skip，出于各種原因，你可能需要暫時跳過一些測試用例（而不是删除它們）
class MyTestCase(unittest.TestCase):
@unittest.skip("demonstrating skipping")
def test_nothing(self):
self.fail("shouldn't happen")
@unittest.skipIf(mylib.__version__ < (1, 3),
"not supported in this library version")
def test_format(self):
# Tests that work for only a certain version of the library.
pass
@unittest.skipUnless(sys.platform.startswith("win"), "requires Windows")
def test_windows_support(self):
# windows specific testing code
pass
Class level fixtures
import unittest
class Test(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls._connection = createExpensiveConnectionObject()
@classmethod
def tearDownClass(cls):
cls._connection.destroy()
Module level fixtures
# These should be implemented as functions:
def setUpModule():
createConnection()
def tearDownModule():
closeConnection()
Mock
Mock類庫是一個專門用于在unittest過程中制作（僞造）和修改（篡改）測試對象的類庫，制作和修改的目的是避免這些對象在單元測試過程中依賴外部資源（網絡資源，資料庫連接配接，其它服務以及耗時過長等）。Mock是一個如此重要的類庫，如果沒有它，Unittest架構從功能上來說就是不完整的。是以不能了解為何它沒有出現在Python2的标準庫裡，不過我們可以很高興地看到在Python3中mock已經是unittest架構的一部分。
猴子更新檔，Monkey-patching is the technique of swapping functions or methods with others in order to change a module, library or class behavior.
>>> class Class():
... def add(self, x, y):
... return x + y
...
>>> inst = Class()
>>> def not_exactly_add(self, x, y):
... return x * y
...
>>> Class.add = not_exactly_add
>>> inst.add(3, 3)
9
Mock對象
return_value: 設定Mock方法的傳回值
>>> from mock import Mock
>>> class ProductionClass(): pass
...
>>> real = ProductionClass()
>>> real.method = Mock(return_value=3)
>>> real.method(3, 4, 5, key='value')
3
>>> real.method.assert_called_with(3, 4, 5, key='value')
>>> real.method.assert_called_with(3, 4, key='value')
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/site-packages/mock/mock.py", line 937, in assert_called_with
six.raise_from(AssertionError(_error_message(cause)), cause)
File "/usr/local/lib/python2.7/site-packages/six.py", line 718, in raise_from
raise value
AssertionError: Expected call: mock(3, 4, key='value')
Actual call: mock(3, 4, 5, key='value')
side_effect:
調用Mock方法時，抛出異常
>>> mock = Mock(side_effect=KeyError('foo'))
>>> mock()
Traceback (most recent call last):
...
KeyError: 'foo'
>>> mock = Mock()
>>> mock.return_value = 42
>>> mock()
42
調用Mock方法時，根據參數得到不同的傳回值
>>> values = {'a': 1, 'b': 2, 'c': 3}
>>> def side_effect(arg):
... return values[arg]
...
>>> mock.side_effect = side_effect
>>> mock('a'), mock('b'), mock('c')
(1, 2, 3)
模拟生成器
>>> mock.side_effect = [5, 4, 3, 2, 1]
>>> mock()
5
>>> mock(), mock(), mock(), mock()
(4, 3, 2, 1)
>>> mock()
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/site-packages/mock/mock.py", line 1062, in __call__
return _mock_self._mock_call(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/mock/mock.py", line 1121, in _mock_call
result = next(effect)
File "/usr/local/lib/python2.7/site-packages/mock/mock.py", line 109, in next
return _next(obj)
StopIteration
patch：在函數（function）或者環境管理協定（with）中模拟對象，離開函數或者環境管理器範圍後模拟行為結束。
在函數中
from mock import patch
class AClass(object): pass
class BClass(object): pass
print id(AClass), id(BClass)
@patch('__main__.AClass')
@patch('__main__.BClass')
def test(x, MockClass2, MockClass1):
print id(AClass), id(BClass)
print id(MockClass1), id(MockClass2)
print AClass
print AClass()
assert MockClass1.called
print x
test(42)
# output:
"""
140254580491744 140254580492688
4525648336 4517777552
4525648336 4517777552
42
"""
在環境管理協定中
>>> class Class(object):
... def method(self):
... pass
...
>>> with patch('__main__.Class') as MockClass:
... instance = MockClass.return_value
... instance.method.return_value = 'foo'
... assert Class() is instance
... assert Class().method() == 'foo'
...
Spec Set的寫法，你應該用不到
>>> Original = Class
>>> patcher = patch('__main__.Class', spec=True)
>>> MockClass = patcher.start()
>>> instance = MockClass()
>>> assert isinstance(instance, Original)
>>> patcher.stop()
patch.object: 在函數或者環境管理協定中模拟對象，但隻mock其中一個attribute
from mock import patch
class AClass():
def method(self, *arg):
return 42
with patch.object(AClass, 'method') as mock_method:
mock_method.return_value = "Fake"
real = AClass()
print real.method(1, 2, 3) # Fake
mock_method.assert_called_once_with(1, 2, 3)
print real.method(1, 2, 3) # 42
patch.dict: patch.dict can be used to add members to a dictionary, or simply let a test change a dictionary, and ensure the dictionary is restored when the test ends.
patch.dict(in_dict, values=(), clear=False, **kwargs)
If clear is True then the dictionary will be cleared before the new values are set.
>>> from mock import patch
>>> foo = {}
>>> with patch.dict(foo, {'newkey': 'newvalue'}):
... assert foo == {'newkey': 'newvalue'}
...
>>> assert foo == {}
>>> import os
>>> with patch.dict('os.environ', {'newkey': 'newvalue'}):
... print os.environ['newkey']
...
newvalue
>>> assert 'newkey' not in os.environ
- patch.multiple: Perform multiple patches in a single call.
>>> thing = object()
>>> other = object()
>>> @patch.multiple('__main__', thing=DEFAULT, other=DEFAULT)
... def test_function(thing, other):
... assert isinstance(thing, MagicMock)
... assert isinstance(other, MagicMock)
...
>>> test_function()
>>> @patch('sys.exit')
... @patch.multiple('__main__', thing=DEFAULT, other=DEFAULT)
... def test_function(mock_exit, other, thing):
... assert 'other' in repr(other)
... assert 'thing' in repr(thing)
... assert 'exit' in repr(mock_exit)
...
>>> test_function()
MagicMock: MagicMock是Mock的子類。MagicMock is a subclass of Mock with default implementations of most of the magic methods. You can use MagicMock without having to configure the magic methods yourself.
MagicMock的功能完全cover Mock，比如：
from mock import MagicMock
thing = ProductionClass()
thing.method = MagicMock(return_value=3)
thing.method(3, 4, 5, key='value') # return 3
thing.method.assert_called_with(3, 4, 5, key='value')
MagicMock相對于Mock的優勢：
>>> from mock import MagicMock
>>> mock = MagicMock()
>>> mock.__str__.return_value = 'foobarbaz'
>>> str(mock)
'foobarbaz'
>>> mock.__str__.assert_called_once_with()
原來需要：
>>> from mock import Mock
>>> mock = Mock()
>>> mock.__str__ = Mock(return_value = 'wheeeeee')
>>> str(mock)
'wheeeeee'
create_autospec: 使Mock對象擁有和原對象相同的字段和方法，對于方法對象，則擁有相同的簽名
>>> from mock import create_autospec
>>> def function(a, b, c):
... pass
...
>>> mock_function = create_autospec(function, return_value='fishy')
>>> mock_function(1, 2, 3)
'fishy'
>>> mock_function.assert_called_once_with(1, 2, 3)
>>> mock_function('wrong arguments')
Traceback (most recent call last):
...
TypeError: () takes exactly 3 arguments (1 given)
>>> from mock import create_autospec
>>>
>>> mockStr = create_autospec(str)
>>> print mockStr.__add__("d", "e")
Unittest2
Unittest2緻力于将Python2.7及以後版本上unittest架構的新特性移植（backport）到Python2.4~Python2.6平台中。
Backport是将一個軟體更新檔應用到比該更新檔所對應的版本更老的版本的行為。
你知道這些就可以了，基本上你不會用到它。
py.test
pytest是另一種固件測試架構，它的API設計非常簡潔優雅，完全脫離了XUnit的窠臼（unittest是XUnit在Python中的實作）。但這也正是它的缺點，unittest是标準庫的一部分，用者甚衆，與之大異難免曲高和寡。
py.test功能完備，并且可擴充，但是它文法很簡單。建立一個測試元件和寫一個帶有諸多函數的子產品一樣容易，來看一個例子
# content of test_sample.py
def func(x):
return x + 1
def test_answer():
assert func(3) == 5
運作一下：
$ py.test
============= test session starts =============
platform darwin -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /Users/wuwenxiang/Documents/workspace/testPyDev, inifile:
collected 1 items
some_test.py F
================== FAILURES ===================
_________________ test_answer _________________
def test_answer():
> assert func(3) == 5
E assert 4 == 5
E + where 4 = func(3)
some_test.py:6: AssertionError
========== 1 failed in 0.01 seconds ===========
官方文檔中入門的例子在這裡，pytest也給出了unittest Style的相容寫法示例，然并X，看完之後你會發現：圈子不同，不必強融，這句話還真TM有道理。
py.test的setup/teardown文法與unittest的相容性不高，實作方式也不直覺。
我們來看一下setup/teardown的例子：
# some_test.py
import pytest
@pytest.fixture(scope='function')
def setup_function(request):
def teardown_function():
print("teardown_function called.")
request.addfinalizer(teardown_function)
print('setup_function called.')
@pytest.fixture(scope='module')
def setup_module(request):
def teardown_module():
print("teardown_module called.")
request.addfinalizer(teardown_module)
print('setup_module called.')
def test_1(setup_function):
print('Test_1 called.')
def test_2(setup_module):
print('Test_2 called.')
def test_3(setup_module):
print('Test_3 called.')
pytest建立固件測試環境（fixture）的方式如上例所示，通過顯式指定scope=''參數來選擇需要使用的pytest.fixture裝飾器。即一個fixture函數的類型從你定義它的時候就确定了，這與使用@nose.with_setup()不同。對于scope='function'的fixture函數，它就是會在測試用例的前後分别調用setup/teardown。測試用例的參數如def test_1(setup_function)隻負責引用具體的對象，它并不關心對方的作用域是函數級的還是子產品級的。
有效的 scope 參數限于：function, module, class, session，預設為function。
運作上例：$ py.test some_test.py -s。-s用于顯示print()函數
執行效果：
$ py.test -s some_test.py
============= test session starts =============
platform darwin -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /Users/wuwenxiang/Documents/workspace/testPyDev, inifile:
collected 3 items
some_test.py setup_function called.
Test_1 called.
.teardown_function called.
setup_module called.
Test_2 called.
.Test_3 called.
.teardown_module called.
========== 3 passed in 0.01 seconds ===========
這裡需要注意的地方是：setup_module被調用的位置。
Nose
nose廣為流傳，它主要用于配置和運作各種架構下的測試用例，有更簡潔友好的測試用例發現功能。nose的自動發現政策是會周遊檔案夾，搜尋特征檔案（預設是搜尋檔案名中帶test的檔案）
$ nosetests
F.
======================================================================
FAIL: some_test.test_answer
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/Users/wuwenxiang/Documents/workspace/testPyDev/some_test.py", line 6, in test_answer
assert func(3) == 5
AssertionError
----------------------------------------------------------------------
Ran 2 tests in 0.004s
FAILED (failures=1)
很可惜，官網說：Nose has been in maintenance mode for the past several years and will likely cease without a new person/team to take over maintainership. New projects should consider using Nose2, py.test, or just plain unittest/unittest2.
Nose2是Nose的原班人馬開發。nose2 is being developed by the same people who maintain nose.
Nose2是基于unittest2 plugins分支開發的，但并不支援python2.6之前的版本。Nose2緻力于做更好的Nose，它的Plugin API并不相容之前Nose的API，是以如果你migration from Nose，必須重寫這些plugin。nose2 implements a new plugin API based on the work done by Michael Foord in unittest2’s plugins branch. This API is greatly superior to the one in nose, especially in how it allows plugins to interact with each other. But it is different enough from the API in nose that supporting nose plugins in nose2 will not be practical: plugins must be rewritten to work with nose2.
然而……
Nose2的更新……也很有限……
其作者Jason Pellerin先生坦言他目前(2014年)并沒有多餘的時間進行personal projects的開發，每周對nose與nose2的實際開發時間大概隻有30分鐘，在這種情況下，nose與nose2都将很難再有大的改版與修正。
Green
不同與nose/nose2，green是單純為了強化unittest中test runner功能而出現的工具。green所提供的隻有一個功能強大、使用友善、測試報告美觀的test runner。如果你的項目中的測試都是以傳統unittest module撰寫而成的話，green會是一個很好的test runner選擇。
使用green執行測試：
pip install green
cd path/to/project
green
Doctest
Doctest的标準文檔在這裡：
Doctest看起來像是在互動式運作環境中的輸出，事實上也确實如此 :)
def square(x):
"""Squares x.
>>> square(2)
4
>>> square(-2)
4
"""
return x * x
if __name__ == '__main__':
import doctest
doctest.testmod()
Doctest的作用是作為函數/類/子產品等單元的解釋和表述性文檔。是以它們有如下特點：
隻有期望對外公開的單元會提供doctest
這些doctest通常不是很細緻
編寫doctest測試基本不需要學習新技能點，在互動式環境裡運作一下，然後把輸出結果檢查一下貼過來就可以了。
doctest還有一些進階用法，但基本上用不到，用到的時候再去查标準文檔好了。 :)
Mox
Mox是Java EasyMock架構在Python中的實作。它一個過時的，很像mock的類庫。從現在開始，你應該放棄學習Mox，在任何情況下都用Mock就對了。
Mox is a mock object framework for Python based on the Java mock object framework EasyMock.
New uses of this library are discouraged.
People are encouraged to use https://pypi.python.org/pypi/mock instead which matches the unittest.mock library available in Python 3.
Mox3 是一個非官方的類庫，是mox的Python3相容版本
Mox3 is an unofficial port of the Google mox framework (http://code.google.com/p/pymox/) to Python 3.
It was meant to be as compatible with mox as possible, but small enhancements have been made.
The library was tested on Python version 3.2, 2.7 and 2.6.
Use at your own risk ;)
其它
tox(官方文檔): 一個自動化測試架構
checking your package installs correctly with different Python versions and interpreters
running your tests in each of the environments, configuring your test tool of choice
acting as a frontend to Continuous Integration servers, greatly reducing boilerplate and merging CI and shell-based testing.
Basic example:
# content of: tox.ini , put in same dir as setup.py
[tox]
envlist = py26,py27
[testenv]
deps=pytest # install pytest in the venvs
commands=py.test # or 'nosetests' or ...
You can also try generating a tox.ini file automatically, by running tox-quickstart and then answering a few simple questions.
To sdist-package, install and test your project against Python2.6 and Python2.7, just type: tox
testr(官方文檔): 是一個test runner。
Django的Unittest(官方文檔)
官方文檔推薦用Unittest：The preferred way to write tests in Django is using the unittest module built in to the Python standard library.
django.test.TestCase繼承了unittest.TestCase
Here is an example which subclasses from django.test.TestCase, which is a subclass of unittest.TestCase that runs each test inside a transaction to provide isolation:
from django.test import TestCase
from myapp.models import Animal
class AnimalTestCase(TestCase):
def setUp(self):
Animal.objects.create(name="lion", sound="roar")
Animal.objects.create(name="cat", sound="meow")
def test_animals_can_speak(self):
"""Animals that can speak are correctly identified"""
lion = Animal.objects.get(name="lion")
cat = Animal.objects.get(name="cat")
self.assertEqual(lion.speak(), 'The lion says "roar"')
self.assertEqual(cat.speak(), 'The cat says "meow"')
Flask的Unittest
官方文檔中介紹：Flask provides a way to test your application by exposing the Werkzeug test Client and handling the context locals for you. You can then use that with your favourite testing solution. In this documentation we will use the unittest package that comes pre-installed with Python.
app.test_client()
app = flask.Flask(__name__)
with app.test_client() as c:
rv = c.get('/?tequila=42')
assert request.args['tequila'] == '42'
Unittest with Flask
class FlaskrTestCase(unittest.TestCase):
def setUp(self):
self.db_fd, flaskr.app.config['DATABASE'] = tempfile.mkstemp()
self.app = flaskr.app.test_client()
flaskr.init_db()
def tearDown(self):
os.close(self.db_fd)
os.unlink(flaskr.app.config['DATABASE'])
def test_empty_db(self):
rv = self.app.get('/')
assert b'No entries here so far' in rv.data
@app.route("/ajax/")
def some_json():
return jsonify(success=True)
class TestViews(TestCase):
def test_some_json(self):
response = self.client.get("/ajax/")
self.assertEquals(response.json, dict(success=True))
建議和總結
在項目中盡量不要mix多種功能類似的架構。
你可以選unittest + green，或者nose/nose2(依使用Python版本和項目的曆史遺留而定) ，或者pytest，但是盡量不要混合使用。
關于Unittest
如果沒有特别的原因，新項目應該用unittest。
Unittest中要用Assert，不要用FailUnless
Django和Flask中都應該用unittest架構，他們也都提供了一個unittest.TestCase的子類以便于做與WebServer想關的測試
關于Mock
如果要Mock一個對象，用MagicMock
如果要在函數或者with-statment中Mock一個對象，用patch
如果要在函數或者with-statement中Mock一個對象的屬性，用patch.object
如果要在函數或者with-statement中Mock一個字典（增加或重建一個鍵值對），用patch.dict
如果要在函數或者with-statement中一次Patch多個Mock對象，用patch.multiple
如果希望Mock對象擁有和原對象相同的字段和方法（對于方法對象，則擁有相同的簽名），用create_autospec。