b69916428b | ||
---|---|---|
.. | ||
analyzers | ||
config_templates | ||
island_client | ||
log_handlers | ||
tests | ||
utils | ||
README.md | ||
__init__.py | ||
conftest.py | ||
gcp_test_machine_list.py | ||
pytest.ini | ||
start_all_gcp_machines.py | ||
stop_all_gcp_machines.py | ||
test_blackbox.py |
README.md
Automatic blackbox tests
Prerequisites
- Download google sdk: https://cloud.google.com/sdk/docs/
- Download service account key for MonkeyZoo project (if you deployed MonkeyZoo via terraform scripts then you already have it).
GCP console -> IAM -> service accounts(you can use the same key used to authenticate terraform scripts).
Place the key in
envs/monkey_zoo/gcp_keys/gcp_key.json
. - Deploy the relevant branch + complied executables to the Island machine on GCP.
Running the tests
In order to execute the entire test suite, you must know the external IP of the Island machine on GCP. You can find
this information in the GCP Console Compute Engine/VM Instances
under External IP.
Running in command line
Either run pytest from /monkey
directory or set PYTHONPATH
environment variable to
/monkey
directory so that blackbox tests can import other monkey code.
Blackbox tests have following parameters:
--island=IP
Sets island's IP--no-gcp
(Optional) Use for no interaction with the cloud (local test).--quick-performance-tests
(Optional) If enabled performance tests won't reset island and won't send telemetries, instead will just test performance of endpoints in already present island state.
Example run command:
monkey\monkey>python -m pytest -s --island=35.207.152.72:5000 ..\envs\monkey_zoo\blackbox\test_blackbox.py
Running in PyCharm
Configure a PyTest configuration with the additional arguments -s --island=35.207.152.72:5000
, and to run from
directory monkey\envs\monkey_zoo\blackbox
.
Running telemetry performance test
Before running performance test make sure browser is not sending requests to island!
To run telemetry performance test follow these steps: 0. Set no password protection on the island. Make sure the island parameter is an IP address(not localhost) as the name resolution will increase the time for requests.
- Gather monkey telemetries.
- Enable "Export monkey telemetries" in Configuration -> Internal -> Tests if you don't have exported telemetries already.
- Run monkey and wait until infection is done.
- All telemetries are gathered in
monkey/telem_sample
. If not, restart the island process.
- Run telemetry performance test.
- Move directory
monkey/telem_sample
toenvs/monkey_zoo/blackbox/tests/performance/telemetry_sample
- (Optional) Use
envs/monkey_zoo/blackbox/tests/performance/telem_sample_parsing/sample_multiplier/sample_multiplier.py
to multiply telemetries gathered.- Run
sample_multiplier.py
script with working directory set tomonkey\envs\monkey_zoo\blackbox
- Pass integer to indicate the multiplier. For example running
telem_parser.py 4
will replicate telemetries 4 times. - If you're using pycharm check "Emulate terminal in output console" on debug/run configuration.
- Run
- Add a
--run-performance-tests
flag to blackbox scripts to run performance tests as part of BlackBox tests. You can run a single test separately by adding-k 'test_telem_performance'
option.
- Move directory