Commit Graph

197 Commits

Author SHA1 Message Date
Anthony Sottile 4df529e5b9 Clean up __future__ and coding: in tests 2019-06-03 12:08:02 -03:00
Anthony Sottile a91fe1fedd pre-commit run pyupgrade --all-files 2019-06-03 12:08:02 -03:00
Anthony Sottile 3f1ec520fc pre-commit run reorder-python-imports --all-files 2019-06-03 12:08:01 -03:00
Anthony Sottile 5034399d7a pre-commit run fix-encoding-pragma --all-files 2019-06-03 12:08:01 -03:00
Bruno Oliveira 4d49ba6529 Drop Python 2.7 and 3.4 support
* Update setup.py requires and classifiers
* Drop Python 2.7 and 3.4 from CI
* Update docs dropping 2.7 and 3.4 support
* Fix mock imports and remove tests related to pypi's mock module
* Add py27 and 34 support docs to the sidebar
* Remove usage of six from tmpdir
* Remove six.PY* code blocks
* Remove sys.version_info related code
* Cleanup compat
* Remove obsolete safe_str
* Remove obsolete __unicode__ methods
* Remove compat.PY35 and compat.PY36: not really needed anymore
* Remove unused UNICODE_TYPES
* Remove Jython specific code
* Remove some Python 2 references from docs

Related to #5275
2019-06-02 14:39:11 -03:00
Bruno Oliveira b10f28949d
Add the junit_log_passing_tests ini flag to skip logging output for passing tests. (#5052)
Add the junit_log_passing_tests ini flag to skip logging output for passing tests.
2019-05-29 20:56:13 -03:00
Bruno Oliveira 73bbff2b74 Introduce record_testsuite_property fixture
This exposes the functionality introduced in fa6acdc as a session-scoped fixture.

Plugins that want to remain compatible with the `xunit2`
standard should use this fixture instead of `record_property`.

Fix #5202
2019-05-10 19:44:27 -03:00
Bruno Oliveira 8f23e19bcb Emit a warning for record_property when used with xunit2
"property" elements cannot be children of "testsuite" according to the schema, so it is incompatible with xunit2

Related to #5202
2019-05-05 09:01:13 -03:00
Samuel Searles-Bryant a37d1df089 Show XFail reason as part of JUnitXML message field
Fixes #4907
2019-04-11 22:26:05 +01:00
Jeffrey Rackauckas da3f836ee3 Added the junit_log_passing_tests ini flag. 2019-04-04 20:26:48 -07:00
Daniel Hahler a50b92ea67 pytester: set HOME only with inline_run/popen
Ref: https://github.com/pytest-dev/pytest/issues/4955
2019-03-20 01:15:51 +01:00
Daniel Hahler c75dd10671 pytester: testdir: set $HOME to tmpdir
This avoids loading user configuration, which might interfere with test
results, e.g. a `~/.pdbrc.py` with pdb++.

Also sets USERPROFILE, which will be required with Python 3.8 [1].

1: https://bugs.python.org/issue36264
2019-03-18 20:55:39 +01:00
Joseph Hunkeler 4ecf29380a
Adds xunit2 version of test_record_attribute 2019-01-14 22:58:41 -05:00
Joseph Hunkeler af2ee1e80a
Emit JUnit compatible XML
* Remove non-standard testcase elements: 'file' and 'line'
* Replace testcase element 'skips' with 'skipped'
* Time resolution uses the standard format: 0.000
* Tests use corrected XML output with proper attributes
2019-01-14 22:58:40 -05:00
Bruno Oliveira 197fd69ddc Use mocking to make test_junit_duration_report deterministic
Fix #4563
2018-12-19 17:13:12 -02:00
Bruno Oliveira 84390acccc
Merge pull request #4553 from nicoddemus/junit-durations
Rename "junit_time" to "junit_duration_report" option
2018-12-17 10:08:20 -02:00
Daniel Hahler 8e287c5c77 Merge master into features 2018-12-14 16:44:43 +01:00
Bruno Oliveira 231863b133 Rename "junit_time" to "junit_duration_report" option
Just realized while reading the changelog that "junit_time" is not a very good
name, so I decided to open this PR renaming it to "junit_duration_report" which
I believe conveys the meaning of the option better
2018-12-14 12:56:26 -02:00
Andrey Paramonov 5d79baf3f8 Fix flaky test attempt 2 2018-12-12 15:33:29 +03:00
Andrey Paramonov 316cca204f Switch to ini config parameter 2018-12-12 13:19:39 +03:00
Andrey Paramonov 0bccfc44a7 Fix flaky test 2018-12-12 12:14:14 +03:00
Andrey Paramonov cfbd387a5d Add --junittime=call option 2018-12-11 19:29:31 +03:00
Daniel Hahler 64ee1ee81b tests: fix tests that require PYTEST_DISABLE_PLUGIN_AUTOLOAD to be unset
Fix pytest's own tests with PYTEST_DISABLE_PLUGIN_AUTOLOAD=1.
2018-12-09 12:06:18 +01:00
Anthony Sottile b3700f61ba Fix formatting of print() calls 2018-11-22 00:15:14 -08:00
Anthony Sottile 2368fbb63c Apply reorder-python-imports to all files 2018-10-25 00:01:29 -07:00
Vincent Barbaresi f55ded20a9 fix #3533: properly escape raw XML object
Using string formatting with the raw escaped object lead to string evaluation
"<py._xmlgen.raw object>"
Format the unescaped string first, then use the XML escape method as a last step.
2018-10-16 14:46:39 +02:00
Niklas JQ c14a23d4e4 Fix #4093: multiple string literals on a line 2018-10-10 19:28:31 +02:00
Bruno Oliveira 1a323fbd3c Show a warning when non-str is given to Monkeypatch.setenv 2018-10-01 20:07:07 -03:00
Bruno Oliveira 56d414177a Remove nodeid from messages for warnings generated by standard warnings
Standard warnings already contain the proper location, so we don't need
to also print the node id
2018-09-04 11:35:34 -03:00
Bruno Oliveira 78ac7d99f5 Deprecate Config.warn and Node.warn, replaced by standard warnings 2018-09-04 11:35:34 -03:00
Anthony Sottile 0fc4a806e5 py.builtins._totext -> string literals or six.text_type 2018-08-22 23:00:04 -07:00
Ronny Pfannschmidt cc6eb9f83c move test reports to own file 2018-06-26 22:09:15 +02:00
Anthony Sottile cbaa7dd56a Upgrade pre-commit hooks except pyupgrade 2018-06-26 06:35:27 -07:00
Bruno Oliveira 1b16d649f9 Fix formatting of merged files 2018-05-25 18:21:48 -03:00
Bruno Oliveira 6f8547cc1a
Merge branch 'master' into pre-commit-hook 2018-05-25 18:14:43 -03:00
Jeffrey Rackauckas b4e0265622 Removed whitespace from empty line. 2018-05-23 20:15:28 -07:00
Jeffrey Rackauckas 6c519b1280 Fixed a bug where stdout and stderr were logged twice by junitxml. 2018-05-23 19:34:45 -07:00
Ronny Pfannschmidt 703e4b11ba run black 2018-05-23 16:48:46 +02:00
Ronny Pfannschmidt 9aacb4635e run pyupgrade 2018-05-23 16:45:09 +02:00
Ronny Pfannschmidt 2018cf12b1 fix the record_property test 2018-04-05 14:05:26 +02:00
Ronny Pfannschmidt 7c0c91a7a2 fix record_property test 2018-04-03 08:00:57 +02:00
Carlos Jenkins 8b49ddfa58 Renamed the fixture record_xml_property to record_property and adapted logic so that the properties are passed to the TestReport object and thus allow compatibility with pytest-xdist. 2018-02-20 15:46:26 -06:00
Thomas Hisch c0ef4a4d35 Add captured log msgs to junit xml file
For each test this adds the captured log msgs to a system-* tag in the junit
xml output file. The destination of the system-* tag is specified by
junit_logging ini option.
2018-02-03 09:39:29 +01:00
Raphael Castaneda a5e60b6a2d implement #3130 - adding record_xml_attribute fixture
update incorrect expected attribute value in test_record_attribute

attr names must be strings

Update CHANGELOG formatting

update usage documentation

Fix versionadded for record_xml_attribute

Indent the xml schema properly inside the warning box in the docs
2018-01-22 15:14:53 -08:00
Andras Tim b840622819 Fixed E302 flake8 errors
expected 2 blank lines, found 0
2017-07-17 01:44:23 +02:00
Andras Tim 6af2abdb53 Fixed flake8 warnings
W191 indentation contains tabs
W292 no newline at end of file
W293 blank line contains whitespace
W391 blank line at end of file
2017-07-17 01:28:16 +02:00
Bruno Oliveira f39f416c5d Improve tests a bit
Use a normal function instead of a lambda
Parametrize test about suite name option
2017-05-12 17:52:50 -03:00
Dmitri Pribysh c9282f9e94 Transition to using ini option for suite name 2017-05-12 17:48:50 -03:00
Dmitri Pribysh fe7d89f033 Add '--junit-suite-name' CLI option 2017-05-12 17:48:50 -03:00
Bruno Oliveira 9f85584656 Merge remote-tracking branch 'upstream/features' into integrate-pytest-warnings 2017-03-20 19:59:05 -03:00
Bruno Oliveira 42a5d6bdfa Add __future__ imports to all pytest modules
This prevents silly errors from creeping in Python 2 when testing in Python 3
2017-03-16 22:45:40 -03:00
Bruno Oliveira 78194093af Improve warning representation in terminal plugin and fix tests 2017-03-16 21:57:32 -03:00
Bruno Oliveira 0baed781fe Merge remote-tracking branch 'upstream/features' into integrate-pytest-warnings 2017-03-16 20:02:06 -03:00
Bruno Oliveira 337f891d78 Fixed tests 2017-03-16 20:01:47 -03:00
Bruno Oliveira 1e0cf5ce4d Merge remote-tracking branch 'upstream/master' into merge-master-into-features
# Conflicts:
#	AUTHORS
#	CHANGELOG.rst
#	_pytest/pytester.py
2017-03-10 15:54:05 -03:00
Katerina Koukiou 26e50f1162 junitxml: adjust junitxml output file to comply with JUnit xsd
Change XML file structure in the manner that failures in call and errors
in teardown in one test will appear under separate testcase elements in
the XML report.
2017-03-02 15:10:25 +01:00
Katerina Koukiou d3a6be4130 junitxml: Fix double system-out tags per testcase
In the xml report we now have two occurences for the system-out tag if
the testcase writes to stdout both on call and teardown and fails in
teardown.
This behaviour is against the xsd.
This patch makes sure that the system-out section exists only
once per testcase.
2017-02-22 16:39:20 +01:00
Michal Wajszczuk fb0b90646e New-style classes implemented for python 2.7 - #2147 2017-02-16 20:28:17 +01:00
Bruno Oliveira 402fbe503a Merge branch 'master' into merge-master-into-features 2016-12-27 23:31:26 -02:00
Luke Murphy 4e1609b12e Add `type` validation.
Argparse driven argument type validation is added for the
`--junit-xml` and `--confcutdir` arguments.

The commit partially reverts #2080. Closes #2089.
2016-11-28 02:16:01 +01:00
Bruno Oliveira b30a6d22c5 Merge branch 'master' into merge-master-into-features 2016-11-27 17:30:40 -02:00
Luke Murphy 6c5475660a Add test case for #595.
This new test proves that reports do not capture stdout
by default when skipped.
2016-11-26 18:49:24 +01:00
Bruno Oliveira efc54b2e56 Merge branch 'master' into merge-master-features 2016-11-11 18:56:53 -02:00
Georgy Dyuldin e2bb4f893b Fix teardown error message in generated xUnit XML
It was "test setup failure" even error happens on test teardown.
2016-11-02 15:50:32 +03:00
John Towler 1b259f70f3 Testcase reports with a url attribute will now properly write this to junitxml 2016-08-25 13:08:51 -07:00
Bruno Oliveira 463e6572c5 Merge branch 'master' into merge-master-into-features
Preparing for 3.0
2016-08-17 22:39:23 -03:00
Raphael Pierzina 767c28d422 Fix broken test in test_junitxml 2016-08-17 22:32:27 +01:00
Raphael Pierzina bb3d6d87b6 Merge branch 'master' into fix-report-outcome-for-xpass 2016-08-17 22:15:29 +01:00
Raphael Pierzina ea379e0e4f Fix test in test_junitxml and add one for strict 2016-08-17 22:02:54 +01:00
Raphael Pierzina 55ec1d7f56 Update test_junitxml.py to interpret XPASS as passed 2016-08-15 23:58:16 +01:00
Christian Boelsen e4028b4505 Fix #1798 to include errors in total tests in junit xml output. 2016-08-08 13:35:49 +01:00
Bruno Oliveira 458ecae1df Replace all usages of "pytest_funcarg__" for @pytest.fixture 2016-07-11 22:21:52 -03:00
Omar Kohl d81f23009b Raise CollectError if pytest.skip() is called during collection
pytest.skip() must not be used at module level because it can easily be
misunderstood and used as a decorator instead of pytest.mark.skip, causing the
whole module to be skipped instead of just the test being decorated.

This is unexpected for users used to the @unittest.skip decorator and therefore
it is best to bail out with a clean error when it happens.

The pytest equivalent of @unittest.skip is @pytest.mark.skip .

Adapt existing tests that were actually relying on this behaviour and add a
test that explicitly test that collection fails.

fix #607
2016-06-24 20:56:21 +02:00
Dave Hunt ef9dd14963 Introduce pytest command as recommended entry point
Fixes #1629
2016-06-21 16:16:57 +02:00
Ronny Pfannschmidt eab762ea99 Merge branch 'master' into merge-master 2016-05-13 19:37:41 +02:00
John Towler 0d07b64571 Fixes Issue 1549 2016-05-05 11:29:05 -07:00
Ceridwen 491b30c5d9 Add Hypothesis test for _idval and fix bug it found 2016-04-01 22:45:44 -04:00
Ceridwen b631fc0bc1 Fix test_escaped_parametrized_names_xml 2016-04-01 21:30:45 -04:00
Bruno Oliveira 5fcce8a7d6 Merge branch 'master' into merge-master-into-features 2016-03-18 18:26:56 -03:00
Tareq Alayan fa6acdcfd4 junitxml: add properties node in testsuite level
The commit allow users to add a properties node in testsuite level see
example below:

<testsuite errors="0" failures="0" name="pytest" skips="1" tests="1"
time="11.824">
  <properties>
    <property name="ARCH" value="PPC"/>
    <property name="OS" value="RHEL 7.2"/>
    <property name="TestPlanURL" value="https://url.."/>
    <property name="Automated" value="True"/>
  </properties>
  <testcase classname="git.....>
  </testcase>
</testsuite>

The current situation is that properties node can be added to every
testcase node. However, sometimes we need some global properties that
applies to all testcases and give better description for the testsuite
itself.
2016-03-16 13:24:33 +02:00
TomV 3d2b7aeea5 issue469: junit parsing nodeid, add method test 2016-03-03 09:12:56 +00:00
TomV 28530836c9 issue469: add fix junit double colon split issue 2016-03-02 22:18:57 +00:00
Lukas Bednar 7e758a9dc6 junit: allow multiple properties with same name
It might happen that test can be affected by two or more bugs.
I need to be able to track them all.
2016-02-29 16:00:26 +01:00
Georgy Dyuldin b28b3cc271 Add captured stdout to jUnit report on setup error 2016-01-20 20:13:01 +03:00
Ronny Pfannschmidt ec5a429c77 junitxml tests: extend with extra items 2015-12-17 22:30:27 +01:00
Ronny Pfannschmidt 713069ebd4 implement review comments for #1266 2015-12-17 22:27:01 +01:00
Ronny Pfannschmidt 8bf7e7cc4b fixes #1259 - take finalized nodes out of the mapping
this allows double node id usage for file based items
2015-12-16 19:09:44 +01:00
Ronny Pfannschmidt 7b7737bf96 handle duplicate test ids via collection and xdist each reporting 2015-12-07 21:54:25 +01:00
Ronny Pfannschmidt 04e9ae75c8 add xfailing test for double test id failure 2015-12-07 21:54:25 +01:00
Ronny Pfannschmidt 5eef6a2821 junitxml: fix python3 compat of the tests 2015-12-07 21:54:25 +01:00
Ronny Pfannschmidt 5f5a7995b9 reintroduce junitxml report order and debug cleanups 2015-12-07 21:54:24 +01:00
Ronny Pfannschmidt 0528e5b45f junitxml: intermediate, move testcase generation to NodeReporter 2015-12-07 21:54:24 +01:00
Ronny Pfannschmidt 9b04958303 junitxml: keep track of custom property insert order
+ review: should we allow the same key multile times
2015-12-07 21:54:24 +01:00
Ronny Pfannschmidt d0107c898e junitxml restrucutre stat generation - use node tags 2015-12-07 21:54:24 +01:00
Ronny Pfannschmidt 9128fec4c4 junitxml: simplify the api used for testing junitml 2015-12-07 21:54:24 +01:00
Ronny Pfannschmidt 80bcf8d624 junitxml: simplify tests by creating a api wrapper 2015-12-07 21:54:24 +01:00
Ronny Pfannschmidt b8df5446c0 junitxml: yapf clean the tests 2015-12-07 21:54:24 +01:00
Bruno Oliveira 748da0e5d7 Fix xml generation when used with pytest-xdist
pytest_runtest_logreport must be careful to not depend on setup/call/teardown
being called sequentially in that order, as xdist will call them as they are reported
from the slaves

Fix #1064
2015-09-26 03:21:24 -03:00
Lukas Bednar 02a2272cfe junitxml: move custom properties to properties element
<testsuite>
  <testcase>
    <properties>
      <property name="ABC" value="XYZ" />
      <property name="DEF" value="ZYX" />
    </properties>
  </testcase>
</testsuite>
2015-09-17 15:56:41 +02:00
Ronny Pfannschmidt b25e41e348 Merge pull request #956 from nicoddemus/record-xml-property
add preliminary support for extended junit xml properties
2015-08-24 09:13:04 +02:00