From 74933daf8da4f79e4eb8cbce77e59d7fb99b2936 Mon Sep 17 00:00:00 2001 From: Mike Salvatore Date: Thu, 14 Jan 2021 14:00:28 -0500 Subject: [PATCH 1/2] ci: Use pytest-cov instead of coverage For some unknown reason, running `coverage` omits some python files from the coverage report. It also runs the test suite a second time, which is inefficient. By using pytest-cov, tests are only run once and coverage data is more complete. --- .travis.yml | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/.travis.yml b/.travis.yml index ca03d20b1..26482dcd5 100644 --- a/.travis.yml +++ b/.travis.yml @@ -24,7 +24,7 @@ install: # Python - pip freeze - pip install -r monkey/monkey_island/requirements.txt # for unit tests -- pip install flake8 pytest dlint isort # for next stages +- pip install flake8 pytest pytest-cov dlint isort # for next stages - pip install coverage # for code coverage - pip install -r monkey/infection_monkey/requirements.txt # for unit tests - pip install pipdeptree @@ -75,12 +75,9 @@ script: ## Check import order - python -m isort ./monkey --settings-file ./ci_scripts/isort.cfg -## Run unit tests +## Run unit tests and generate coverage data - cd monkey # This is our source dir -- python -m pytest # Have to use `python -m pytest` instead of `pytest` to add "{$builddir}/monkey/monkey" to sys.path. - -## Calculate Code Coverage -- coverage run -m pytest +- python -m pytest --cov=. # Have to use `python -m pytest` instead of `pytest` to add "{$builddir}/monkey/monkey" to sys.path. # Check JS code. The npm install must happen AFTER the flake8 because the node_modules folder will cause a lot of errors. - cd monkey_island/cc/ui From d008e3d52a22f76f7ab36490a609524c6bfcdba2 Mon Sep 17 00:00:00 2001 From: Mike Salvatore Date: Sun, 17 Jan 2021 19:15:07 -0500 Subject: [PATCH 2/2] ci: add .coveragerc to omit unit test code from coverage report The code coverage report was including the unit tests themselves in the coverage report. This resulted in an artifically inflated code coverage metric, as code coverage tools will naturally report test code to be very highly "covered". --- monkey/.coveragerc | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 monkey/.coveragerc diff --git a/monkey/.coveragerc b/monkey/.coveragerc new file mode 100644 index 000000000..722e8d71a --- /dev/null +++ b/monkey/.coveragerc @@ -0,0 +1,4 @@ +[run] +omit = + */test_*.py + */*_test.py