From fbbce0cd996645087a81b7524af2e6d87a957d40 Mon Sep 17 00:00:00 2001 From: VakarisZ Date: Thu, 13 May 2021 09:14:50 +0300 Subject: [PATCH 1/3] Small improvements in readme of blackbox tests. --- envs/monkey_zoo/blackbox/README.md | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/envs/monkey_zoo/blackbox/README.md b/envs/monkey_zoo/blackbox/README.md index 808a0a5cb..f372244ce 100644 --- a/envs/monkey_zoo/blackbox/README.md +++ b/envs/monkey_zoo/blackbox/README.md @@ -1,20 +1,22 @@ # Automatic blackbox tests ### Prerequisites 1. Download google sdk: https://cloud.google.com/sdk/docs/ -2. Download service account key for MonkeyZoo project (if you deployed MonkeyZoo via terraform scripts then you already have it). +2. Download service account key for MonkeyZoo project (if you deployed MonkeyZoo via terraform scripts then you already have it). GCP console -> IAM -> service accounts(you can use the same key used to authenticate terraform scripts). Place the key in `envs/monkey_zoo/gcp_keys/gcp_key.json`. -3. Deploy the relevant branch + complied executables to the Island machine on GCP. +3. Deploy the relevant branch + complied executables to the Island machine on GCP. ### Running the tests -In order to execute the entire test suite, you must know the external IP of the Island machine on GCP. You can find -this information in the GCP Console `Compute Engine/VM Instances` under _External IP_. +In order to execute the entire test suite, you must know the external IP of the Island machine on GCP. You can find +this information in the GCP Console `Compute Engine/VM Instances` under _External IP_. #### Running in command line +Either run pytest from `/monkey` directory or set `PYTHONPATH` environment variable to +`/monkey` directory so that blackbox tests could import other monkey code. Blackbox tests have following parameters: - `--island=IP` Sets island's IP - `--no-gcp` (Optional) Use for no interaction with the cloud (local test). -- `--quick-performance-tests` (Optional) If enabled performance tests won't reset island and won't send telemetries, +- `--quick-performance-tests` (Optional) If enabled performance tests won't reset island and won't send telemetries, instead will just test performance of endpoints in already present island state. Example run command: @@ -27,22 +29,22 @@ directory `monkey\envs\monkey_zoo\blackbox`. ### Running telemetry performance test -**Before running performance test make sure browser is not sending requests to island!** +**Before running performance test make sure browser is not sending requests to island!** To run telemetry performance test follow these steps: 0. Set `server_config.json` to "standard" (no password protection) setting. 1. Gather monkey telemetries. - 1. Enable "Export monkey telemetries" in Configuration -> Internal -> Tests if you don't have + 1. Enable "Export monkey telemetries" in Configuration -> Internal -> Tests if you don't have exported telemetries already. 2. Run monkey and wait until infection is done. 3. All telemetries are gathered in `monkey/telem_sample` 2. Run telemetry performance test. 1. Move directory `monkey/test_telems` to `envs/monkey_zoo/blackbox/tests/performance/test_telems` - 2. (Optional) Use `envs/monkey_zoo/blackbox/tests/performance/utils/telem_parser.py` to multiply + 2. (Optional) Use `envs/monkey_zoo/blackbox/tests/performance/utils/telem_parser.py` to multiply telemetries gathered. 1. Run `telem_parser.py` script with working directory set to `monkey\envs\monkey_zoo\blackbox` 2. Pass integer to indicate the multiplier. For example running `telem_parser.py 4` will replicate telemetries 4 times. 3. If you're using pycharm check "Emulate terminal in output console" on debug/run configuraion. - 3. Performance test will run as part of BlackBox tests or you can run it separately by adding + 3. Performance test will run as part of BlackBox tests or you can run it separately by adding `-k 'test_telem_performance'` option. From 2e2fd0a53c5beee1dd0269905f535abcd3161f7b Mon Sep 17 00:00:00 2001 From: VakarisZ Date: Thu, 13 May 2021 09:15:33 +0300 Subject: [PATCH 2/3] Changed blackbox tests to fail and stop if they can't connect to the island. --- envs/monkey_zoo/blackbox/test_blackbox.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/envs/monkey_zoo/blackbox/test_blackbox.py b/envs/monkey_zoo/blackbox/test_blackbox.py index 70b133468..20f495151 100644 --- a/envs/monkey_zoo/blackbox/test_blackbox.py +++ b/envs/monkey_zoo/blackbox/test_blackbox.py @@ -159,7 +159,8 @@ class TestMonkeyBlackbox: return os.path.abspath(LOG_DIR_PATH) def test_server_online(self, island_client): - assert island_client.get_api_status() is not None + if not island_client.get_api_status(): + pytest.exit("BB tests couldn't reach the Island server, quiting.") def test_ssh_exploiter(self, island_client): TestMonkeyBlackbox.run_exploitation_test(island_client, Ssh, "SSH_exploiter_and_keys") From b79ef1680c1b33dcf0fc4cdfb9f3a1a9ce226f55 Mon Sep 17 00:00:00 2001 From: VakarisZ <36815064+VakarisZ@users.noreply.github.com> Date: Mon, 17 May 2021 14:39:30 +0300 Subject: [PATCH 3/3] Update envs/monkey_zoo/blackbox/README.md Co-authored-by: Shreya Malviya --- envs/monkey_zoo/blackbox/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/envs/monkey_zoo/blackbox/README.md b/envs/monkey_zoo/blackbox/README.md index f372244ce..b6a12333f 100644 --- a/envs/monkey_zoo/blackbox/README.md +++ b/envs/monkey_zoo/blackbox/README.md @@ -12,7 +12,7 @@ this information in the GCP Console `Compute Engine/VM Instances` under _Externa #### Running in command line Either run pytest from `/monkey` directory or set `PYTHONPATH` environment variable to -`/monkey` directory so that blackbox tests could import other monkey code. +`/monkey` directory so that blackbox tests can import other monkey code. Blackbox tests have following parameters: - `--island=IP` Sets island's IP - `--no-gcp` (Optional) Use for no interaction with the cloud (local test).