Prev Next

Chapter 2. PHPUnit's Goals

So far, we only have two tests for the array built-in and the sizeof() function. When we start to test the numerous array_*() functions PHP offers, we will need to write a test for each of them. We could write the infrastructure for all these tests from scratch. However, it is much better to write a testing infrastructure once and then write only the unique parts of each test. PHPUnit is such an infrastructure.

A framework such as PHPUnit has to resolve a set of constraints, some of which seem always to conflict with each other. Simultaneously, tests should be:

Easy to learn to write.

If it's hard to learn how to write tests, developers will not learn to write them.

Easy to write.

If tests are not easy to write, developers will not write them.

Easy to read.

Test code should contain no extraneous overhead so that the test itself does not get lost in noise that surrounds it.

Easy to execute.

The tests should run at the touch of a button and present their results in a clear and unambiguous format.

Quick to execute.

Tests should run fast so so they can be run hundreds or thousands of times a day.

Isolated.

The tests should not affect each other. If the order in which the tests are run changes, the results of the tests should not change.

Composable.

We should be able to run any number or combination of tests together. This is a corollary of isolation.

There are two main clashes between these constraints:

Easy to learn to write versus easy to write.

Tests do not generally require all the flexibility of a programming language. Many testing tools provide their own scripting language that only includes the minimum necessary features for writing tests. The resulting tests are easy to read and write because they have no noise to distract you from the content of the tests. However, learning yet another programming language and set of programming tools is inconvenient and clutters the mind.

Isolated versus quick to execute.

If you want the results of one test to have no effect on the results of another test, each test should create the full state of the world before it begins to execute and return the world to its original state when it finishes. However, setting up the world can take a long time: for example connecting to a database and initializing it to a known state using realistic data.

PHPUnit attempts to resolve these conflicts by using PHP as the testing language. Sometimes the full power of PHP is overkill for writing little straight-line tests, but by using PHP we leverage all the experience and tools programmers already have in place. Since we are trying to convince reluctant testers, lowering the barrier to writing those initial tests is particularly important.

PHPUnit errs on the side of isolation over quick execution. Isolated tests are valuable because they provide high-quality feedback. You do not get a report with a bunch of test failures, which were really caused because one test at the beginning of the suite failed and left the world messed up for the rest of the tests. This orientation towards isolated tests encourages designs with a large number of simple objects. Each object can be tested quickly in isolation. The result is better designs and faster tests.

PHPUnit assumes that most tests succeed and it is not worth reporting the details of successful tests. When a test fails, that fact is worth noting and reporting. The vast majority of tests should succeed and are not worth commenting on except to count the number of tests that run. This is an assumption that is really built into the reporting classes, and not into the core of PHPUnit. When the results of a test run are reported, you see how many tests were executed, but you only see details for those that failed.

Tests are expected to be fine-grained, testing one aspect of one object. Hence, the first time a test fails, execution of the test halts, and PHPUnit reports the failure. It is an art to test by running in many small tests. Fine-grained tests improve the overall design of the system.

When you test an object with PHPUnit, you do so only through the object's public interface. Testing based only on publicly visible behaviour encourages you to confront and solve difficult design problems earlier, before the results of poor design can infect large parts of the system.

Prev Next
1. Automating Tests
2. PHPUnit's Goals
3. Installing PHPUnit
4. Writing Tests for PHPUnit
Test Dependencies
Data Providers
Testing Exceptions
Testing PHP Errors
5. The Command-Line Test Runner
6. Fixtures
More setUp() than tearDown()
Variations
Sharing Fixture
Global State
7. Organizing Tests
Composing a Test Suite Using the Filesystem
Composing a Test Suite Using XML Configuration
Using the TestSuite Class
8. TestCase Extensions
Testing Output
9. Database Testing
Data Sets
Flat XML Data Set
XML Data Set
CSV Data Set
Replacement Data Set
Operations
Database Testing Best Practices
10. Incomplete and Skipped Tests
Incomplete Tests
Skipping Tests
11. Test Doubles
Stubs
Mock Objects
Stubbing and Mocking Web Services
Mocking the Filesystem
12. Testing Practices
During Development
During Debugging
13. Test-Driven Development
BankAccount Example
14. Behaviour-Driven Development
BowlingGame Example
15. Code Coverage Analysis
Specifying Covered Methods
Ignoring Code Blocks
Including and Excluding Files
16. Other Uses for Tests
Agile Documentation
Cross-Team Tests
17. Skeleton Generator
Generating a Test Case Class Skeleton
Generating a Class Skeleton from a Test Case Class
18. PHPUnit and Selenium
Selenium RC
PHPUnit_Extensions_SeleniumTestCase
19. Logging
Test Results (XML)
Test Results (TAP)
Test Results (JSON)
Code Coverage (XML)
20. Build Automation
Apache Ant
Apache Maven
Phing
21. Continuous Integration
Atlassian Bamboo
CruiseControl
phpUnderControl
22. PHPUnit API
Overview
PHPUnit_Framework_Assert
assertArrayHasKey()
assertClassHasAttribute()
assertClassHasStaticAttribute()
assertContains()
assertContainsOnly()
assertEqualXMLStructure()
assertEquals()
assertFalse()
assertFileEquals()
assertFileExists()
assertGreaterThan()
assertGreaterThanOrEqual()
assertLessThan()
assertLessThanOrEqual()
assertNull()
assertObjectHasAttribute()
assertRegExp()
assertSame()
assertSelectCount()
assertSelectEquals()
assertSelectRegExp()
assertStringEndsWith()
assertStringEqualsFile()
assertStringStartsWith()
assertTag()
assertThat()
assertTrue()
assertType()
assertXmlFileEqualsXmlFile()
assertXmlStringEqualsXmlFile()
assertXmlStringEqualsXmlString()
PHPUnit_Framework_Test
PHPUnit_Framework_TestCase
PHPUnit_Framework_TestSuite
PHPUnit_Framework_TestResult
Package Structure
23. Extending PHPUnit
Subclass PHPUnit_Framework_TestCase
Assert Classes
Subclass PHPUnit_Extensions_TestDecorator
Implement PHPUnit_Framework_Test
Subclass PHPUnit_Framework_TestResult
Implement PHPUnit_Framework_TestListener
New Test Runner
A. Assertions
B. Annotations
@assert
@backupGlobals
@backupStaticAttributes
@covers
@dataProvider
@depends
@expectedException
@group
@outputBuffering
@runTestsInSeparateProcesses
@runInSeparateProcess
@test
@testdox
@ticket
C. The XML Configuration File
PHPUnit
Test Suites
Groups
Including and Excluding Files for Code Coverage
Logging
Test Listeners
Setting PHP INI settings, Constants and Global Variables
Configuring Browsers for Selenium RC
D. Index
E. Bibliography
F. Copyright