sources for pytest-overview.html [rev. unknown]
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
  <head>
    <title>[pytest-overview] </title>
    <meta content="text/html;charset=ISO-8859-1" name="Content-Type"/>
    <link href="../style.css" media="screen" rel="stylesheet" type="text/css"/></head>
  <body>
    <div><a href="http://codespeak.net"><img alt="py lib" height="114" id="pyimg" src="http://codespeak.net/img/pylib.png" width="154"/></a></div>
    <div id="metaspace">
      <div class="project_title">[pytest-overview] </div>
      <div id="menubar"><a class="menu" href="../index.html">index</a> <a class="menu" href="../../../apigen/api/index.html">api</a> <a class="menu" href="../../../apigen/source/index.html">source</a> <a class="menu" href="../contact.html">contact</a> <a class="menu" href="../download.html">download</a></div></div>
    <div id="contentspace">
      <div id="docinfoline">
        <div style="float: right; font-style: italic;"> </div></div>
<div class="document" id="py-test-flexible-and-powerful-automated-testing">
<h1 class="title">py.test - flexible and powerful automated testing</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Authors:</th>
<td>Holger Krekel
<br />merlinux GmbH</td></tr>
<tr><th class="docinfo-name">Date:</th>
<td>13th March 2006</td></tr>
</tbody>
</table>
<!-- Definitions of interpreted text roles (classes) for S5/HTML data. -->
<!-- This data file has been placed in the public domain. -->
<!-- Colours
======= -->
<!-- Text Sizes
========== -->
<!-- Display in Slides (Presentation Mode) Only
========================================== -->
<!-- Display in Outline Mode Only
============================ -->
<!-- Display in Print Only
===================== -->
<!-- Incremental Display
=================== -->
<div class="section">
<h1><a id="intro-benefits-of-automated-testing" name="intro-benefits-of-automated-testing">Intro: Benefits of Automated Testing</a></h1>
<ul class="simple">
<li>prove that code changes actually fix a certain issue</li>
<li>minimizing Time to Feedback for developers</li>
<li>reducing overall Time to Market</li>
<li>document usage of plugins</li>
<li>tests as a means of communication</li>
<li>easing entry for newcomers</li>
</ul>
</div>
<div class="section">
<h1><a id="py-test-purposes-goals" name="py-test-purposes-goals">py.test Purposes &amp; Goals</a></h1>
<ul class="simple">
<li>automated cross-project open source testing tool</li>
<li>flexible per-project customization</li>
<li>reusing test methods/reporting across projects</li>
<li>various iterative test collection methods</li>
<li>support for distributed testing</li>
<li>py lib is a development support library</li>
</ul>
</div>
<div class="section">
<h1><a id="what-is-python" name="what-is-python">What is Python?</a></h1>
<ul class="simple">
<li>easy-to-learn flexible OO high level language</li>
<li>glue-language for connecting C++, Java and scripting</li>
<li>used e.g. by Google for deployment/testing/implementation</li>
<li>used by BIND (mainstream DNS internet server) for testing</li>
<li>Jython provides Python for JVM</li>
<li>IronPython provides Python for .NET</li>
<li>CPython is mainstream C-based platform</li>
<li>PyPy - Python in Python implementation</li>
</ul>
</div>
<div class="section">
<h1><a id="main-drivers-of-py-test-development" name="main-drivers-of-py-test-development">Main drivers of py.test development</a></h1>
<ul class="simple">
<li>PyPy project testing needs (part of EU project)</li>
<li>needs by individual (freely contributing) projects</li>
<li>at least 20 projects using py.test and py lib</li>
<li>higher level innovation plans by merlinux &amp; experts</li>
<li>commercial needs</li>
<li>almost three years of (non-fulltime) development</li>
</ul>
</div>
<div class="section">
<h1><a id="authors-copyrights" name="authors-copyrights">Authors &amp; copyrights</a></h1>
<ul class="simple">
<li>initial: Holger Krekel, Armin Rigo</li>
<li>major contributors: Jan Balster, Brian Dorsey, Grig
Gheorghiu</li>
<li>many others with small patches</li>
<li>MIT license</li>
</ul>
</div>
<div class="section">
<h1><a id="who-is-merlinux" name="who-is-merlinux">who is merlinux?</a></h1>
<ul class="simple">
<li>small company founded in 2004 by Holger Krekel and Laura
Creighton</li>
<li>purpose: research and development / open source technologies</li>
<li>7 employees (no win32 experts!), 6 freelancers</li>
<li>three larger projects:<ul>
<li>PyPy - next generation Python implementation</li>
<li>mailwitness - digital invoicing/signatures</li>
<li>provider of development servers</li>
</ul>
</li>
<li>technologies: virtualization, deployment and testing</li>
</ul>
</div>
<div class="section">
<h1><a id="main-features-of-py-test" name="main-features-of-py-test">Main Features of py.test</a></h1>
<ul class="simple">
<li>simplest possible <tt class="docutils literal"><span class="pre">assert</span></tt> approach</li>
<li>clean setup/teardown semantics</li>
<li>stdout/stderr capturing per test</li>
<li>per-project/directory cmdline options (many predefined)</li>
<li>test selection support</li>
<li>customizable auto-collection of tests</li>
<li><a class="reference" href="../test.html#features">more features</a> ...</li>
</ul>
</div>
<div class="section">
<h1><a id="main-user-level-entry-points" name="main-user-level-entry-points">Main User-Level entry points</a></h1>
<ul class="simple">
<li><tt class="docutils literal"><span class="pre">py.test.raises(Exc,</span> <span class="pre">func,</span> <span class="pre">*args,</span> <span class="pre">**kwargs)</span></tt></li>
<li><tt class="docutils literal"><span class="pre">py.test.fail(msg)</span></tt> -&gt; fail a test</li>
<li><tt class="docutils literal"><span class="pre">py.test.skip(msg)</span></tt> -&gt; skip a test</li>
<li><tt class="docutils literal"><span class="pre">py.test.ensuretemp(prefix)</span></tt> -&gt; per-test session temporary directory</li>
<li><tt class="docutils literal"><span class="pre">conftest.py</span></tt> can modify almost arbitrary testing aspects
(but it's a bit involved)</li>
</ul>
</div>
<div class="section">
<h1><a id="some-py-lib-components" name="some-py-lib-components">some py lib components</a></h1>
<ul class="simple">
<li><tt class="docutils literal"><span class="pre">py.execnet</span></tt> provides ad-hoc means to distribute programs</li>
<li><tt class="docutils literal"><span class="pre">py.path</span></tt> objects abstract local and svn files</li>
<li><tt class="docutils literal"><span class="pre">py.log</span></tt> offers (preliminary) logging support</li>
<li><tt class="docutils literal"><span class="pre">py.xml.html</span></tt> for programmatic html generation</li>
<li>lazy import ...``import py`` is enough</li>
</ul>
</div>
<div class="section">
<h1><a id="py-test-implementation" name="py-test-implementation">py.test Implementation</a></h1>
<ul class="simple">
<li><a class="reference" href="../test.html">basic picture</a></li>
<li>Session objects (Terminal and Tcl-GUI)</li>
<li>reporting hooks are on session objects</li>
<li>Collector hierarchy yield iteratively tests</li>
<li>uses py lib extensively (py.path/py.execnet)</li>
<li>&quot;conftest.py&quot; per-directory configuration mechanism</li>
</ul>
</div>
<div class="section">
<h1><a id="session-objects" name="session-objects">Session objects</a></h1>
<ul class="simple">
<li>responsible for driving the testing process</li>
<li>make use of iterative Collector hierarchies</li>
<li>responsible for reporting (XXX)</li>
<li>can be split to a Frontend and BackendSession
for distributed testing (GUI frontend uses it)</li>
</ul>
</div>
<div class="section">
<h1><a id="collector-objects" name="collector-objects">Collector objects</a></h1>
<ul class="simple">
<li>Collectors / Test Items form a tree</li>
<li>the tree is build iteratively (driven from Sessions)</li>
<li>collector tree can be viewed with <tt class="docutils literal"><span class="pre">--collectonly</span></tt></li>
<li><tt class="docutils literal"><span class="pre">run()</span></tt> returns list of (test) names or runs the test</li>
<li><tt class="docutils literal"><span class="pre">join(name)</span></tt> returns a sub collector/item</li>
<li>various helper methods to e.g. determine file/location</li>
</ul>
</div>
<div class="section">
<h1><a id="extensions-rest-documentation-checking" name="extensions-rest-documentation-checking">Extensions: ReST documentation checking</a></h1>
<ul class="simple">
<li><a class="reference" href="../conftest.py">py/documentation/conftest.py</a> provides test
items for checking documentation and link integrity</li>
<li>uses its own collector/testitem hierarchy</li>
<li>invokes <tt class="docutils literal"><span class="pre">docutils</span></tt> processing, reports errors</li>
</ul>
</div>
<div class="section">
<h1><a id="extensions-distributed-testing" name="extensions-distributed-testing">Extensions: Distributed Testing</a></h1>
<ul class="simple">
<li>using py.execnet to dispatch on different python versions</li>
<li>using py.execnet to dispatch tests on other hosts/platforms</li>
<li>currently: Popen, SSH and Socket gateways</li>
<li>missing support pushing tests to &quot;the other side&quot;</li>
<li>missing for deployment on multiple machines</li>
<li>but it's already possible ...</li>
</ul>
</div>
<div class="section">
<h1><a id="example-using-pywinauto-from-linux" name="example-using-pywinauto-from-linux">Example using pywinauto from linux</a></h1>
<ul class="simple">
<li>start socketserver.py on windows</li>
<li>connect a SocketGateway e.g. from linux</li>
<li>send tests, execute and report tracebacks through the
gateway</li>
<li>remotely use pywinauto to automate testing of GUI work flow</li>
<li>interactive example ...</li>
</ul>
</div>
<div class="section">
<h1><a id="status-of-py-lib" name="status-of-py-lib">Status of py lib</a></h1>
<ul class="simple">
<li>mostly developed on linux/OSX</li>
<li>basically all tests pass on win32 as well</li>
<li>but missing some win32 convenience</li>
<li>some support for generation of html/ReST/PDFs reports</li>
<li>py.execnet works rather reliably (pending deeper win32 testing)</li>
<li>flexible configuration but sometimes non-obvious/documented
(requires understanding of internals)</li>
</ul>
</div>
<div class="section">
<h1><a id="next-steps-py-lib-py-test" name="next-steps-py-lib-py-test">Next Steps py lib / py.test</a></h1>
<ul class="simple">
<li>refined py.execnet distribution of programs</li>
<li>more configurable and customizable reporting</li>
<li>implement support for testing distribution</li>
<li>explore refined win32 support</li>
<li>automated collection of unittest.py based tests</li>
<li>make spawning processes/gateways more robust</li>
<li>doctest support</li>
<li>unify logging approaches (py.log.*)</li>
<li>...</li>
</ul>
</div>
</div>
<div class="footer">
<hr class="footer" />
Holger Krekel (merlinux) &#8226; &#8226; 13th March 2006
</div>
</div></body></html>