.. _`non-python tests`:

Working with non-python tests
====================================================

a basic example for specifying tests in Yaml files
--------------------------------------------------------------

.. _`pytest-yamlwsgi`: http://bitbucket.org/aafshar/pytest-yamlwsgi/src/tip/pytest_yamlwsgi.py
.. _`PyYAML`: http://pypi.python.org/pypi/PyYAML/

Here is an example ``conftest.py`` (extracted from Ali Afshnars special purpose `pytest-yamlwsgi`_ plugin).   This ``conftest.py`` will  collect ``test*.yml`` files and will execute the yaml-formatted content as custom tests:

.. include:: nonpython/conftest.py
    :literal:

You can create a simple example file:

.. include:: nonpython/test_simple.yml
    :literal:

and if you installed `PyYAML`_ or a compatible YAML-parser you can
now execute the test specification::

    nonpython $ py.test test_simple.yml
    =========================== test session starts ============================
    platform linux2 -- Python 2.6.5 -- pytest-2.0.0.dev17
    test path 1: test_simple.yml
    
    test_simple.yml .F
    
    ================================= FAILURES =================================
    ______________________________ usecase: hello ______________________________
    usecase execution failed
       spec failed: 'some': 'other'
       no further details known at this point.
    ==================== 1 failed, 1 passed in 0.37 seconds ====================

You get one dot for the passing ``sub1: sub1`` check and one failure.
Obviously in the above ``conftest.py`` you'll want to implement a more
interesting interpretation of the yaml-values.  Note that ``reportinfo()``
is used for representing the test location and is also consulted for
reporting in ``verbose`` mode::

    nonpython $ py.test -v
    =========================== test session starts ============================
    platform linux2 -- Python 2.6.5 -- pytest-2.0.0.dev17 -- /home/hpk/venv/0/bin/python
    test path 1: /home/hpk/p/pytest/doc/example/nonpython
    
    test_simple.yml:1: usecase: ok PASSED
    test_simple.yml:1: usecase: hello FAILED
    
    ================================= FAILURES =================================
    ______________________________ usecase: hello ______________________________
    usecase execution failed
       spec failed: 'some': 'other'
       no further details known at this point.
    ==================== 1 failed, 1 passed in 0.07 seconds ====================

While developing your custom test collection and execution it's also
interesting to just look at the collection tree::

    nonpython $ py.test --collectonly
    <Directory 'nonpython'>
      <YamlFile 'test_simple.yml'>
        <UsecaseItem 'ok'>
        <UsecaseItem 'hello'>