Help:Using PHPUnit

From semantic-mediawiki.org
Jump to: navigation, search
Testing
Using PHPUnit in connection with SMW
Image / Video collection
Keywords
PHPUnit · Testing · Tests · PHP
Table of Contents

Contents

Tests are commonly divided into a manual (without using any tool or automated script) and automated test approach.

Manual testing

If you want to run some manual tests (either as scripted or exploratory test procedure) then you just have to:

  1. Download a related branch using composer require "mediawiki/semantic-media-wiki:dev-foo (where foo refers to the branch name) or in case you want to test the current master, use @dev or dev-master as version together with the minimum-stability: dev flag so that the branch/master can be fetched without any stability limitations.
  2. Run composer dump-autoload to ensure that all registered classes are correctly initialized before starting any test procedure.

Automated testing (PHPUnit)

For the automated approach, Semantic MediaWiki relies on PHPUnit as scripted testing methodology. Scripted tests are used to verify that an expected behaviour occurs for codified requirements on the given conditions.

  • Unit test refers to a script that verifies results for a unit, module, or class against an expected outcome in an isolated environment
  • Integration test (functional test) normally combines multiple components into a single process and verifies the results in a production like environment (DB access, sample data etc.)
  • System test (and its individual modules) is treated as "black-box" to observe behaviour as a whole rather than its units

Running tests

  1. Verify that PHUnit is installed and in case it is not use composer require phpunit/phpunit:~4.8 --update-with-dependencies to add the package
  2. Verify that your MediaWiki installation comes with its test files and folders (e.g. /myMediawikiFolder/tests ) in order for Semantic MediaWiki to have access to registered MW-core classes. If the tests folder is missing then download a matchable content from the release source.
  3. Run composer phpunit from the Semantic MediaWiki base directory (e.g. /extensions/SemanticMediaWiki) using a standard command line tool which should output something like:
$ composer phpunit

Semantic MediaWiki: 2.5.0-alpha (SMWSQLStore3, sqlite)
MediaWiki:          1.28.0-alpha (Extension vendor autoloader)
Site language:      en

Execution time:     2015-01-01 01:00
Xdebug:             Disabled (or not installed)

PHPUnit 4.8.27 by Sebastian Bergmann and contributors.

Runtime:        PHP 5.6.8
Configuration:  /home/travis/build/SemanticMediaWiki/mw/extensions/SemanticMediaWiki/phpunit.xml.dist

.............................................................   61 / 4069 (  1%)
.............................................................  122 / 4069 (  2%)

Information about PHPUnit in connection with MediaWiki can be found at smw.org and mediawiki.org.

Writing tests

Writing meaningful tests isn't difficult but requires some diligence on how to setup a test and its environment. One simple rule is to avoid the use of hidden expectations or inheritance as remedy for the "less code is good code" aesthetics. Allow the code to be readable and if possible follow the arrange, act, assert pattern.

For a short introduction on "How to write a test for Semantic MediaWiki", have a look at this video.

Test cases

The use of MediaWikiTestCase is discouraged (as its binds tests and the test environment to MediaWiki) and it is best to rely on PHPUnit_Framework_TestCase and where a MW database connection is required, use the MwDBaseUnitTestCase instead.

  • QueryPrinterTestCase base class for all query and result printers
  • SpecialPageTestCase derives from SemanticMediaWikiTestCase

Integration tests

Integration tests are vital to confirm the behaviour of a component from an integrative perspective that occurs through an interplay with its surroundings. SMW\Tests\Integration\ contains most of the tests that target the validation of reciprocity with MediaWiki and/or other services such as:

  • SPARQLStore ( fuseki, virtuoso, blazegraph, or sesame )
  • Extensions such as SM, SESP, SBL etc.

Some details about the integration test environment can be found here.

Write integration tests using json script

The best practise approach in Semantic MediaWiki is to write integration tests as pseudo JSON script in combination with a specialized TestCaseRunner that handles the necessary object setup and tear down process for each test execution.

The script like test definition was introduced to lower the barrier of understanding of what is being tested by using a wikitext notation (which internally is transformed into PHPUnit with the help of the JsonTestCaseScriptRunner to run and execute the actually test) and to design test cases quicker.

A test file (which can contain different test cases) will be automatically loaded by a TestCaseRunner from the specified location.

The JSON script follows the previous mentioned arrange, act, assert approach, with the setup section to contain object definitions that are planned to be used during a test. The section expects that a entityt name and content (generally the page content in wikitext, annotations etc.) are specified.

"setup": [
    {
        "name": "Has text",
        "namespace":"SMW_NS_PROPERTY",
        "contents": "[[Has type::Text]]"
    },
    {
        "name": "Property:Has number",
        "contents": "[[Has type::Number]]"
    },
    {
        "name": "Example/S0009/1",
        "namespace":"NS_MAIN",
        "contents": "[[Has text::Some text to search]]"
    }
],

The test result assertion provides simplified string comparison methods (mostly for output related assertion but expressive enough for users to understand the test objective and its expected results). For example, verifying that a result printer does output a certain string, one has to the define an expected output in terms of:

"tests": [
    {
        "type": "format",
        "about": "#0 this case is expected to output ...",
        "subject": "Example/Test/1",
        "assert-output": {
            "to-contain": [
                "123"
            ],
            "not-contain": [
                "abc"
            ]
        }
    }
}

Each test case should describe "about" what the test is expected to test which may help during a failure to identify potential conflicts or hints on how to resolve an issue. Different types of test case definitions provide specialized assertion methods:

  • query, concept, and format
  • parser
  • rdf
  • special

It can happen that an output is mixed with language dependent content (site vs. page content vs. user language) and therefore it is recommended to fix those settings for a test by adding something like:

"settings": {
    "wgContLang": "en",
    "wgLang": "en",
    "smwgNamespacesWithSemanticLinks": {
        "NS_MAIN": true,
        "SMW_NS_PROPERTY": true
    }
}

A complete list of available json test files can be found here.

Benchmark tests

For details, please have a look at the benchmark guide document.

JavaScript (QUnit)

Running qunit tests in connection with MediaWiki requires to execute Special:JavaScriptTest. QUnit tests are currently not executed on Travis (see #136).

Miscellaneous

See also

For details on how to setup, and run PHP unit tests, see:

For details on how to write unit tests, see: