sources for impl-test.html [rev. unknown]
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
  <head>
    <title>[impl-test] </title>
    <meta content="text/html;charset=ISO-8859-1" name="Content-Type"/>
    <link href="style.css" media="screen" rel="stylesheet" type="text/css"/></head>
  <body>
    <div><a href="http://codespeak.net"><img alt="py lib" height="114" id="pyimg" src="http://codespeak.net/img/pylib.png" width="154"/></a></div>
    <div id="metaspace">
      <div class="project_title">[impl-test] </div>
      <div id="menubar"><a class="menu" href="index.html">index</a> <a class="menu" href="../../apigen/api/index.html">api</a> <a class="menu" href="../../apigen/source/index.html">source</a> <a class="menu" href="contact.html">contact</a> <a class="menu" href="download.html">download</a></div></div>
    <div id="contentspace">
      <div id="docinfoline">
        <div style="float: right; font-style: italic;"> </div></div>
<div class="document" id="implementation-and-customization-of-py-test">
<h1 class="title">Implementation and Customization of <tt class="docutils literal"><span class="pre">py.test</span></tt></h1>
<div class="contents topic">
<p class="topic-title first"><a id="contents" name="contents">Contents</a></p>
<ul class="auto-toc simple">
<li><a class="reference" href="#collecting-and-running-tests-implementation-remarks" id="id1" name="id1">1&nbsp;&nbsp;&nbsp;Collecting and running tests / implementation remarks</a><ul class="auto-toc">
<li><a class="reference" href="#collectors-and-the-test-collection-process" id="id2" name="id2">1.1&nbsp;&nbsp;&nbsp;Collectors and the test collection process</a></li>
<li><a class="reference" href="#test-items-are-collectors-as-well" id="id3" name="id3">1.2&nbsp;&nbsp;&nbsp;test items are collectors as well</a></li>
<li><a class="reference" href="#constructing-the-package-name-for-modules" id="id4" name="id4">1.3&nbsp;&nbsp;&nbsp;constructing the package name for modules</a></li>
<li><a class="reference" href="#module-collector" id="id5" name="id5">1.4&nbsp;&nbsp;&nbsp;Module Collector</a></li>
</ul>
</li>
<li><a class="reference" href="#customizing-the-testing-process" id="id6" name="id6">2&nbsp;&nbsp;&nbsp;Customizing the testing process</a><ul class="auto-toc">
<li><a class="reference" href="#writing-conftest-py-files" id="id7" name="id7">2.1&nbsp;&nbsp;&nbsp;writing conftest.py files</a><ul class="auto-toc">
<li><a class="reference" href="#adding-custom-options" id="id8" name="id8">2.1.1&nbsp;&nbsp;&nbsp;adding custom options</a></li>
</ul>
</li>
<li><a class="reference" href="#customizing-the-collecting-and-running-process" id="id9" name="id9">2.2&nbsp;&nbsp;&nbsp;customizing the collecting and running process</a><ul class="auto-toc">
<li><a class="reference" href="#example-perform-additional-rest-checks" id="id10" name="id10">2.2.1&nbsp;&nbsp;&nbsp;example: perform additional ReST checks</a></li>
</ul>
</li>
<li><a class="reference" href="#customizing-the-collection-process-in-a-module" id="id11" name="id11">2.3&nbsp;&nbsp;&nbsp;Customizing the collection process in a module</a></li>
<li><a class="reference" href="#customizing-execution-of-functions" id="id12" name="id12">2.4&nbsp;&nbsp;&nbsp;Customizing execution of Functions</a></li>
</ul>
</li>
</ul>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id1" id="collecting-and-running-tests-implementation-remarks" name="collecting-and-running-tests-implementation-remarks"><span id="basicpicture"></span>1&nbsp;&nbsp;&nbsp;Collecting and running tests / implementation remarks</a></h1>
<p>In order to customize <tt class="docutils literal"><span class="pre">py.test</span></tt> it's good to understand
its basic architure (WARNING: these are not guaranteed
yet to stay the way they are now!):</p>
<pre class="literal-block">
 ___________________
|                   |
|    Collector      |
|___________________|
       / \
        |                Item.run()
        |               ^
 receive test Items    /
        |             /execute test Item
        |            /
 ___________________/
|                   |
|     Session       |
|___________________|

                    .............................
                    . conftest.py configuration .
                    . cmdline options           .
                    .............................
</pre>
<p>The <em>Session</em> basically receives test <em>Items</em> from a <em>Collector</em>,
and executes them via the <tt class="docutils literal"><span class="pre">Item.run()</span></tt> method.  It monitors
the outcome of the test and reports about failures and successes.</p>
<div class="section">
<h2><a class="toc-backref" href="#id2" id="collectors-and-the-test-collection-process" name="collectors-and-the-test-collection-process"><span id="collection-process"></span>1.1&nbsp;&nbsp;&nbsp;Collectors and the test collection process</a></h2>
<p>The collecting process is iterative, i.e. the session
traverses and generates a <em>collector tree</em>.  Here is an example of such
a tree, generated with the command <tt class="docutils literal"><span class="pre">py.test</span> <span class="pre">--collectonly</span> <span class="pre">py/xmlobj</span></tt>:</p>
<pre class="literal-block">
&lt;Directory 'xmlobj'&gt;
    &lt;Directory 'testing'&gt;
        &lt;Module 'test_html.py' (py.__.xmlobj.testing.test_html)&gt;
            &lt;Function 'test_html_name_stickyness'&gt;
            &lt;Function 'test_stylenames'&gt;
            &lt;Function 'test_class_None'&gt;
            &lt;Function 'test_alternating_style'&gt;
        &lt;Module 'test_xml.py' (py.__.xmlobj.testing.test_xml)&gt;
            &lt;Function 'test_tag_with_text'&gt;
            &lt;Function 'test_class_identity'&gt;
            &lt;Function 'test_tag_with_text_and_attributes'&gt;
            &lt;Function 'test_tag_with_subclassed_attr_simple'&gt;
            &lt;Function 'test_tag_nested'&gt;
            &lt;Function 'test_tag_xmlname'&gt;
</pre>
<p>By default all directories not starting with a dot are traversed,
looking for <tt class="docutils literal"><span class="pre">test_*.py</span></tt> and <tt class="docutils literal"><span class="pre">*_test.py</span></tt> files.  Those files
are imported under their <a class="reference" href="#package-name">package name</a>.</p>
</div>
<div class="section">
<h2><a class="toc-backref" href="#id3" id="test-items-are-collectors-as-well" name="test-items-are-collectors-as-well"><span id="collector-api"></span>1.2&nbsp;&nbsp;&nbsp;test items are collectors as well</a></h2>
<p>To make the reporting life simple for the session object
items offer a <tt class="docutils literal"><span class="pre">run()</span></tt> method as well.  In fact the session
distinguishes &quot;collectors&quot; from &quot;items&quot; solely by interpreting
their return value.  If it is a list, then we recurse into
it, otherwise we consider the &quot;test&quot; as passed.</p>
</div>
<div class="section">
<h2><a class="toc-backref" href="#id4" id="constructing-the-package-name-for-modules" name="constructing-the-package-name-for-modules"><span id="package-name"></span>1.3&nbsp;&nbsp;&nbsp;constructing the package name for modules</a></h2>
<p>Test modules are imported under their fully qualified
name.  Given a module <tt class="docutils literal"><span class="pre">path</span></tt> the fully qualified package
name is constructed as follows:</p>
<ul class="simple">
<li>determine the last &quot;upward&quot; directory from <tt class="docutils literal"><span class="pre">path</span></tt> that
contains an <tt class="docutils literal"><span class="pre">__init__.py</span></tt> file.  Going upwards
means repeatedly calling the <tt class="docutils literal"><span class="pre">dirpath()</span></tt> method
on a path object (which returns the parent directory
as a path object).</li>
<li>insert this base directory into the sys.path list
as its first element</li>
<li>import the root package</li>
<li>determine the fully qualified name for the module located
at <tt class="docutils literal"><span class="pre">path</span></tt> ...<ul>
<li>if the imported root package has a __package__ object
then call <tt class="docutils literal"><span class="pre">__package__.getimportname(path)</span></tt></li>
<li>otherwise use the relative path of the module path to
the base dir and turn slashes into dots and strike
the trailing <tt class="docutils literal"><span class="pre">.py</span></tt>.</li>
</ul>
</li>
</ul>
<p>The Module collector will eventually trigger
<tt class="docutils literal"><span class="pre">__import__(mod_fqdnname,</span> <span class="pre">...)</span></tt> to finally get to
the live module object.</p>
<p>Side note: this whole logic is performed by local path
object's <tt class="docutils literal"><span class="pre">pyimport()</span></tt> method.</p>
</div>
<div class="section">
<h2><a class="toc-backref" href="#id5" id="module-collector" name="module-collector">1.4&nbsp;&nbsp;&nbsp;Module Collector</a></h2>
<p>The default Module collector looks for test functions
and test classes and methods. Test functions and methods
are prefixed <tt class="docutils literal"><span class="pre">test</span></tt> by default.  Test classes must
start with a capitalized <tt class="docutils literal"><span class="pre">Test</span></tt> prefix.</p>
</div>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id6" id="customizing-the-testing-process" name="customizing-the-testing-process">2&nbsp;&nbsp;&nbsp;Customizing the testing process</a></h1>
<div class="section">
<h2><a class="toc-backref" href="#id7" id="writing-conftest-py-files" name="writing-conftest-py-files">2.1&nbsp;&nbsp;&nbsp;writing conftest.py files</a></h2>
<p>You may put conftest.py files containing project-specific
configuration in your project's root directory, it's usually
best to put it just into the same directory level as your
topmost <tt class="docutils literal"><span class="pre">__init__.py</span></tt>.  In fact, <tt class="docutils literal"><span class="pre">py.test</span></tt> performs
an &quot;upwards&quot; search starting from the directory that you specify
to be tested and will lookup configuration values right-to-left.
You may have options that reside e.g. in your home directory
but note that project specific settings will be considered
first.  There is a flag that helps you debugging your
conftest.py configurations:</p>
<pre class="literal-block">
py.test --traceconfig
</pre>
<div class="section">
<h3><a class="toc-backref" href="#id8" id="adding-custom-options" name="adding-custom-options">2.1.1&nbsp;&nbsp;&nbsp;adding custom options</a></h3>
<p>To register a project-specific command line option
you may have the following code within a <tt class="docutils literal"><span class="pre">conftest.py</span></tt> file:</p>
<pre class="literal-block">
import py
Option = py.test.config.Option
option = py.test.config.addoptions(&quot;pypy options&quot;,
    Option('-V', '--view', action=&quot;store_true&quot;, dest=&quot;view&quot;, default=False,
           help=&quot;view translation tests' flow graphs with Pygame&quot;),
)
</pre>
<p>and you can then access <tt class="docutils literal"><span class="pre">option.view</span></tt> like this:</p>
<pre class="literal-block">
if option.view:
    print &quot;view this!&quot;
</pre>
<p>The option will be available if you type <tt class="docutils literal"><span class="pre">py.test</span> <span class="pre">-h</span></tt>
Note that you may only register upper case short
options.  <tt class="docutils literal"><span class="pre">py.test</span></tt> reserves all lower
case short options for its own cross-project usage.</p>
</div>
</div>
<div class="section">
<h2><a class="toc-backref" href="#id9" id="customizing-the-collecting-and-running-process" name="customizing-the-collecting-and-running-process">2.2&nbsp;&nbsp;&nbsp;customizing the collecting and running process</a></h2>
<p>To introduce different test items you can create
one or more <tt class="docutils literal"><span class="pre">conftest.py</span></tt> files in your project.
When the collection process traverses directories
and modules the default collectors will produce
custom Collectors and Items if they are found
in a local <tt class="docutils literal"><span class="pre">conftest.py</span></tt> file.</p>
<div class="section">
<h3><a class="toc-backref" href="#id10" id="example-perform-additional-rest-checks" name="example-perform-additional-rest-checks">2.2.1&nbsp;&nbsp;&nbsp;example: perform additional ReST checks</a></h3>
<p>With your custom collectors or items you can completely
derive from the standard way of collecting and running
tests in a localized manner.  Let's look at an example.
If you invoke <tt class="docutils literal"><span class="pre">py.test</span> <span class="pre">--collectonly</span> <span class="pre">py/documentation</span></tt>
then you get:</p>
<pre class="literal-block">
&lt;DocDirectory 'documentation'&gt;
    &lt;DocDirectory 'example'&gt;
        &lt;DocDirectory 'pytest'&gt;
            &lt;Module 'test_setup_flow_example.py' (test_setup_flow_example)&gt;
                &lt;Class 'TestStateFullThing'&gt;
                    &lt;Instance '()'&gt;
                        &lt;Function 'test_42'&gt;
                        &lt;Function 'test_23'&gt;
    &lt;ReSTChecker 'TODO.txt'&gt;
        &lt;ReSTSyntaxTest 'TODO.txt'&gt;
        &lt;LinkCheckerMaker 'checklinks'&gt;
    &lt;ReSTChecker 'api.txt'&gt;
        &lt;ReSTSyntaxTest 'api.txt'&gt;
        &lt;LinkCheckerMaker 'checklinks'&gt;
            &lt;CheckLink 'getting-started.html'&gt;
    ...
</pre>
<p>In <tt class="docutils literal"><span class="pre">py/documentation/conftest.py</span></tt> you find the following
customization:</p>
<pre class="literal-block">
class DocDirectory(py.test.collect.Directory):

    def run(self):
        results = super(DocDirectory, self).run()
        for x in self.fspath.listdir('*.txt', sort=True):
                results.append(x.basename)
        return results

    def join(self, name):
        if not name.endswith('.txt'):
            return super(DocDirectory, self).join(name)
        p = self.fspath.join(name)
        if p.check(file=1):
            return ReSTChecker(p, parent=self)

Directory = DocDirectory
</pre>
<p>The existence of the 'Directory' name in the
<tt class="docutils literal"><span class="pre">pypy/documentation/conftest.py</span></tt> module makes the collection
process defer to our custom &quot;DocDirectory&quot; collector.  We extend
the set of collected test items by <tt class="docutils literal"><span class="pre">ReSTChecker</span></tt> instances
which themselves create <tt class="docutils literal"><span class="pre">ReSTSyntaxTest</span></tt> and <tt class="docutils literal"><span class="pre">LinkCheckerMaker</span></tt>
items.  All of this instances (need to) follow the <a class="reference" href="#collector-api">collector API</a>.</p>
</div>
</div>
<div class="section">
<h2><a class="toc-backref" href="#id11" id="customizing-the-collection-process-in-a-module" name="customizing-the-collection-process-in-a-module">2.3&nbsp;&nbsp;&nbsp;Customizing the collection process in a module</a></h2>
<blockquote>
REPEATED WARNING: details of the collection and running process are
still subject to refactorings and thus details will change.
If you are customizing py.test at &quot;Item&quot; level then you
definitely want to be subscribed to the <a class="reference" href="http://codespeak.net/mailman/listinfo/py-dev">py-dev mailing list</a>
to follow ongoing development.</blockquote>
<p>If you have a module where you want to take responsibility for
collecting your own test Items and possibly even for executing
a test then you can provide <a class="reference" href="test.html#generative-tests">generative tests</a> that yield
callables and possibly arguments as a tuple.   This should
serve some immediate purposes like paramtrized tests.</p>
<p>The other extension possibility goes deeper into the machinery
and allows you to specify a custom test <tt class="docutils literal"><span class="pre">Item</span></tt> class which
is responsible for setting up and executing an underlying
test.  [XXX not working: You can integrate your custom <tt class="docutils literal"><span class="pre">py.test.collect.Item</span></tt> subclass
by binding an <tt class="docutils literal"><span class="pre">Item</span></tt> name to a test class.]  Or you can
extend the collection process for a whole directory tree
by putting Items in a <tt class="docutils literal"><span class="pre">conftest.py</span></tt> configuration file.
The collection process constantly looks at according names
in the <em>chain of conftest.py</em> modules to determine collectors
and items at <tt class="docutils literal"><span class="pre">Directory</span></tt>, <tt class="docutils literal"><span class="pre">Module</span></tt>, <tt class="docutils literal"><span class="pre">Class</span></tt>, <tt class="docutils literal"><span class="pre">Function</span></tt>
or <tt class="docutils literal"><span class="pre">Generator</span></tt> level.  Note that, right now, except for <tt class="docutils literal"><span class="pre">Function</span></tt>
items all classes are pure collectors, i.e. will return a list
of names (possibly empty).</p>
<p>XXX implement doctests as alternatives to <tt class="docutils literal"><span class="pre">Function</span></tt> items.</p>
</div>
<div class="section">
<h2><a class="toc-backref" href="#id12" id="customizing-execution-of-functions" name="customizing-execution-of-functions">2.4&nbsp;&nbsp;&nbsp;Customizing execution of Functions</a></h2>
<ul class="simple">
<li>Function test items allow total control of executing their
contained test method.  <tt class="docutils literal"><span class="pre">function.run()</span></tt> will get called by the
session in order to actually run a test.  The method is responsible
for performing proper setup/teardown (&quot;Test Fixtures&quot;) for a
Function test.</li>
<li><tt class="docutils literal"><span class="pre">Function.execute(target,</span> <span class="pre">*args)</span></tt> methods are invoked by
the default <tt class="docutils literal"><span class="pre">Function.run()</span></tt> to actually execute a python
function with the given (usually empty set of) arguments.</li>
</ul>
</div>
</div>
</div>
</div></body></html>