marbles.core
Extends |
|
|
An extension of |
|
An extension of |
Extends |
|
|
|
|
|
The |
Extends unittest
to provide more information to the test
consumer on test failure. This additional information includes local
variables defined within the test at the time it failed, the full
assertion statement that failed, and a free-form annotation provided
by the test author. This additional information helps test consumers
respond to test failures without digging into the test code themselves.
To get this additional information in your tests, you can inherit
from marbles.core.TestCase
anywhere you would normally
inherit from unittest.TestCase
. By simply inheriting from
marbles.core.TestCase
instead, your test failures will
include the full assertion statement that failed and any local
variables in scope at the time the test failed.
Assertions on a marbles.core.TestCase
also accept an optional
note
string that will be exposed to the test consumer on test
failure. This annotation can contain whatever the test author feels is
important, but it is especially useful for communicating their intent
and any relevant context about the test. This annotation can be a
format string that will be expanded with local variables if/when the
test fails.
You can also inherit from marbles.core.AnnotatedTestCase
.
The only difference is that, if you inherit from
marbles.core.AnnotatedTestCase
, you must provide note
annotations to all assertions. Calling an assertion without the
note
parameter on a marbles.core.AnnotatedTestCase
will produce a marbles.core.AnnotationError
.
- class marbles.core.AnnotatedTestCase(methodName='runTest')[source]
An extension of
marbles.core.TestCase
.An
AnnotatedTestCase
is only different from amarbles.core.TestCase
in that it enforces thatnote
is provided for every assertion. Calling an assertion without thenote
parameter on amarbles.core.AnnotatedTestCase
will produce amarbles.core.AnnotationError
.For other details, see
marbles.core.TestCase
.
- exception marbles.core.ContextualAssertionError(*args)[source]
Extends
AssertionError
to accept and display additional information beyond the staticmsg
parameter provided byunittest
assertions.This additional information includes the full assertion statement that failed and any local variables in scope at the time the test failed.
This additional information may also include a
note
string that can explain the intent of the test, provide any relevant context, and/or describe what to do if/when the assertion fails. This string is formatted with the local context where the assertion error is raised.- property locals
A dict containing the locals defined within the test.
- property public_test_locals
A dict containing the public (a.k.a., not internal or name-mangled) locals defined within the test.
- property assert_stmt
Returns a string displaying the whole statement that failed, with a ‘>’ indicator on the line starting the expression.
- class marbles.core.TestCase(methodName='runTest')[source]
An extension of
unittest.TestCase
.Failure messages from
marbles.core.TestCase
tests contain more information thanunittest.TestCase
tests, including local variables defined within the test at the time the test failed and the full assertion statement that failed.All assert statements, e.g.,
unittest.TestCase.assertEqual()
, in addition to accepting the optional final string parametermsg
, also accept a free-formnote
annotation provided by the test author. This annotation can contain whatever the test author feels is important, but it is especially useful for communicating their intent and any relevant context about the test that will help the test consumer understand and debug the failure. For example, this annotation can be used to provide context on a specific edge case that a test exercises, without sacrificing themsg
or trying to embed that context into the test method name.The
note
string, if provided, is formatted withstr.format()
given the local variables defined within the test itself.Example:
import requests import marbles.core class ResponseTestCase(marbles.core.TestCase): def test_create_resource(self): endpoint = 'http://example.com/api/v1/resource' data = {'id': 1, 'name': 'Little Bobby Tables'} res = requests.put(endpoint, data=data) self.assertEqual( res.status_code, 201, note=res.reason )
- failureException
alias of
ContextualAssertionError
marbles.core.main
Main method for marbles.
With unittest
, you can run python -m unittest
to discover
and run your tests.
With marbles
, you can run python -m marbles
to discover and
run them, but with marbles
-style assertion failure messages,
including local variables and additional source code scope.
- class marbles.core.main.MarblesTestResult(*args, **kwargs)[source]
A TestResult which omits the traceback.
Because marbles failure messages contain all of the information included in a normal unit test traceback, we can hide the traceback to make failure messages easier to read without losing any information. Here, we remove the traceback from the failure message, unless the user has asked for verbose output.
- class marbles.core.main.MarblesTestRunner(stream=None, descriptions=True, verbosity=1, failfast=False, buffer=False, resultclass=None, warnings=None, *, tb_locals=False)[source]
A TestRunner which uses marbles-style assertion failure messages.
- resultclass
alias of
MarblesTestResult
marbles.core.log
marbles
can log information about each
assertion called.
If configured, the marbles.core.log.logger
will log a json
object for each assertion and its success or failure, as well as any
other attributes of interest specified by the test author.
The captured information includes the assertion’s args and kwargs, msg, note, local variables (for failed assertions, and also for passing assertions if verbose logging is turned on), and the result of the assertion.
Configuration is handled via the environment variables
MARBLES_LOGFILE
, MARBLES_TEST_CASE_ATTRS
,
MARBLES_TEST_CASE_ATTRS_VERBOSE
,
MARBLES_LOG_VERBOSE
, or via the
AssertionLogger.configure()
method. Environment variables
override those set with the configure()
method,
so if a marbles
program configures these programmatically, they
can always be overridden without changing the program.
Note that AssertionLogger
should not be instantiated
directly; instead, test authors should import and configure the
marbles.core.log.logger
as needed.
- class marbles.core.log.AssertionLogger[source]
The
AssertionLogger
logs json about each assertion.This module exposes a single
AssertionLogger
,marbles.core.log.logger
, that is used during a marbles test run. It can be configured withconfigure()
before running the tests or via environment variables.Example:
import marbles.core from marbles.core import log if __name__ == '__main__': log.logger.configure(logfile='/path/to/marbles.log', attrs=['filename', 'date']) marbles.core.main()
Note
If you configure logging within an
if __name__ == '__main__'
, block (as opposed to via environment variables), you must run your tests withpython /path/to/tests.py
. If you run your tests withpython -m marbles
, theif __name__ == '__main__'
block won’t get executed and the logger won’t get configured.- configure(**kwargs)[source]
Configure what assertion logging is done.
Settings configured with this method are overridden by environment variables.
- Parameters:
logfile (str or bytes or file object) – If a string or bytes object, we write to that filename. If an open file object, we just write to it. If None, disable logging. If we open the file, we open it in
'w'
mode, so any contents will be overwritten.attrs (list of str) – Capture these attributes on the TestCase being run when logging an assertion. For example, if you are testing multiple resources, make sure the resource name is a member of your TestCase, and configure marbles logging with that name. These are only captured on failure.
verbose_attrs (list of str) – Similar to attrs, but these attrs are captured even on success.
verbose (bool or list of str) – Fields (within the set {msg, note, locals}) to capture even when the test is successful. By default, those three fields are only captured on failure.