2019-08-27 23:12:14 +08:00
|
|
|
# Automatic blackbox tests
|
|
|
|
|
### Prerequisites
|
|
|
|
|
1. Download google sdk: https://cloud.google.com/sdk/docs/
|
2019-10-01 15:42:51 +08:00
|
|
|
2. Download service account key for MonkeyZoo project (if you deployed MonkeyZoo via terraform scripts then you already have it).
|
2020-07-30 17:41:25 +08:00
|
|
|
GCP console -> IAM -> service accounts(you can use the same key used to authenticate terraform scripts).
|
|
|
|
|
Place the key in `envs/monkey_zoo/gcp_keys/gcp_key.json`.
|
2019-10-06 20:05:34 +08:00
|
|
|
3. Deploy the relevant branch + complied executables to the Island machine on GCP.
|
|
|
|
|
|
|
|
|
|
### Running the tests
|
|
|
|
|
In order to execute the entire test suite, you must know the external IP of the Island machine on GCP. You can find
|
|
|
|
|
this information in the GCP Console `Compute Engine/VM Instances` under _External IP_.
|
|
|
|
|
|
|
|
|
|
#### Running in command line
|
2020-05-13 15:44:04 +08:00
|
|
|
Blackbox tests have following parameters:
|
|
|
|
|
- `--island=IP` Sets island's IP
|
|
|
|
|
- `--no-gcp` (Optional) Use for no interaction with the cloud (local test).
|
|
|
|
|
- `--quick-performance-tests` (Optional) If enabled performance tests won't reset island and won't send telemetries,
|
|
|
|
|
instead will just test performance of endpoints in already present island state.
|
|
|
|
|
|
|
|
|
|
Example run command:
|
2019-10-06 20:05:34 +08:00
|
|
|
|
2021-03-08 22:47:16 +08:00
|
|
|
`monkey\monkey>python -m pytest -s --island=35.207.152.72:5000 ..\envs\monkey_zoo\blackbox\test_blackbox.py`
|
2019-10-06 20:05:34 +08:00
|
|
|
|
|
|
|
|
#### Running in PyCharm
|
2021-03-08 21:54:15 +08:00
|
|
|
Configure a PyTest configuration with the additional arguments `-s --island=35.207.152.72:5000`, and to run from
|
2020-07-30 17:41:25 +08:00
|
|
|
directory `monkey\envs\monkey_zoo\blackbox`.
|
2020-04-24 00:44:24 +08:00
|
|
|
|
|
|
|
|
### Running telemetry performance test
|
2020-05-23 02:22:29 +08:00
|
|
|
|
|
|
|
|
**Before running performance test make sure browser is not sending requests to island!**
|
|
|
|
|
|
2020-04-24 00:44:24 +08:00
|
|
|
To run telemetry performance test follow these steps:
|
2020-06-23 17:24:02 +08:00
|
|
|
0. Set `server_config.json` to "standard" (no password protection) setting.
|
2020-04-24 00:44:24 +08:00
|
|
|
1. Gather monkey telemetries.
|
|
|
|
|
1. Enable "Export monkey telemetries" in Configuration -> Internal -> Tests if you don't have
|
|
|
|
|
exported telemetries already.
|
|
|
|
|
2. Run monkey and wait until infection is done.
|
2020-05-11 21:32:18 +08:00
|
|
|
3. All telemetries are gathered in `monkey/telem_sample`
|
2020-04-24 00:44:24 +08:00
|
|
|
2. Run telemetry performance test.
|
|
|
|
|
1. Move directory `monkey/test_telems` to `envs/monkey_zoo/blackbox/tests/performance/test_telems`
|
|
|
|
|
2. (Optional) Use `envs/monkey_zoo/blackbox/tests/performance/utils/telem_parser.py` to multiply
|
|
|
|
|
telemetries gathered.
|
2020-05-11 21:32:18 +08:00
|
|
|
1. Run `telem_parser.py` script with working directory set to `monkey\envs\monkey_zoo\blackbox`
|
2020-04-24 00:44:24 +08:00
|
|
|
2. Pass integer to indicate the multiplier. For example running `telem_parser.py 4` will replicate
|
|
|
|
|
telemetries 4 times.
|
2020-04-28 22:54:12 +08:00
|
|
|
3. If you're using pycharm check "Emulate terminal in output console" on debug/run configuraion.
|
2020-05-11 21:32:18 +08:00
|
|
|
3. Performance test will run as part of BlackBox tests or you can run it separately by adding
|
|
|
|
|
`-k 'test_telem_performance'` option.
|
