p41275903
/
monkey
forked from p15670423/monkey
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
monkey/envs/monkey_zoo/blackbox
History
Mike Salvatore 087c8f2cf8 BB: Remove internet_services from BaseTemplate 2021-08-19 14:16:44 -04:00
..
analyzers zoo: Remove unused import from zerologon_analyzer.py 2021-04-07 13:51:47 -04:00
config_templates BB: Remove internet_services from BaseTemplate 2021-08-19 14:16:44 -04:00
island_client BB performance tests: update the outdated README.md and other small improvements 2021-07-31 19:40:19 -04:00
log_handlers Reformat all python with black v20.8b1 2021-04-06 09:20:18 -04:00
tests BB: Don't wait so long for monkeys to die 2021-08-04 15:06:11 -04:00
utils Reverted back to fetching file directory first when resolving GCP keys. This is to make gcp key file relative to utils directory, not the current file. This will make it less confusing, because people usually navigate directories, not files. 2021-05-12 16:58:46 +03:00
README.md BB performance tests: update the outdated README.md and other small improvements 2021-07-31 19:40:19 -04:00
__init__.py Basic skeleton for blackbox script 2019-08-27 15:52:39 +03:00
conftest.py Changed --no-performance-tests to --run-performance-tests for convenience (skipping performance tests by default) and documented changes in CHANGELOG.md 2021-04-30 09:41:20 +03:00
gcp_test_machine_list.py BB: Move GCP_TEST_MACHINE_LIST to its own file 2021-08-02 07:09:37 -04:00
pytest.ini Added module + filename + line number to logs output in Blackbox 2019-10-07 13:49:13 +03:00
start_all_gcp_machines.py BB: Add utility scripts to start and stop all GCP machines 2021-08-02 07:10:50 -04:00
stop_all_gcp_machines.py BB: Add utility scripts to start and stop all GCP machines 2021-08-02 07:10:50 -04:00
test_blackbox.py BB: Fix "Strtuts2" typo 2021-08-04 13:15:49 -04:00

README.md

Automatic blackbox tests

Prerequisites

  1. Download google sdk: https://cloud.google.com/sdk/docs/
  2. Download service account key for MonkeyZoo project (if you deployed MonkeyZoo via terraform scripts then you already have it). GCP console -> IAM -> service accounts(you can use the same key used to authenticate terraform scripts). Place the key in envs/monkey_zoo/gcp_keys/gcp_key.json.
  3. Deploy the relevant branch + complied executables to the Island machine on GCP.

Running the tests

In order to execute the entire test suite, you must know the external IP of the Island machine on GCP. You can find this information in the GCP Console Compute Engine/VM Instances under External IP.

Running in command line

Either run pytest from /monkey directory or set PYTHONPATH environment variable to /monkey directory so that blackbox tests can import other monkey code. Blackbox tests have following parameters:

  • --island=IP Sets island's IP
  • --no-gcp (Optional) Use for no interaction with the cloud (local test).
  • --quick-performance-tests (Optional) If enabled performance tests won't reset island and won't send telemetries, instead will just test performance of endpoints in already present island state.

Example run command:

monkey\monkey>python -m pytest -s --island=35.207.152.72:5000 ..\envs\monkey_zoo\blackbox\test_blackbox.py

Running in PyCharm

Configure a PyTest configuration with the additional arguments -s --island=35.207.152.72:5000, and to run from directory monkey\envs\monkey_zoo\blackbox.

Running telemetry performance test

Before running performance test make sure browser is not sending requests to island!

To run telemetry performance test follow these steps: 0. Set no password protection on the island. Make sure the island parameter is an IP address(not localhost) as the name resolution will increase the time for requests.

  1. Gather monkey telemetries.
    1. Enable "Export monkey telemetries" in Configuration -> Internal -> Tests if you don't have exported telemetries already.
    2. Run monkey and wait until infection is done.
    3. All telemetries are gathered in monkey/telem_sample. If not, restart the island process.
  2. Run telemetry performance test.
    1. Move directory monkey/telem_sample to envs/monkey_zoo/blackbox/tests/performance/telemetry_sample
    2. (Optional) Use envs/monkey_zoo/blackbox/tests/performance/telem_sample_parsing/sample_multiplier/sample_multiplier.py to multiply telemetries gathered.
      1. Run sample_multiplier.py script with working directory set to monkey\envs\monkey_zoo\blackbox
      2. Pass integer to indicate the multiplier. For example running telem_parser.py 4 will replicate telemetries 4 times.
      3. If you're using pycharm check "Emulate terminal in output console" on debug/run configuration.
    3. Add a --run-performance-tests flag to blackbox scripts to run performance tests as part of BlackBox tests. You can run a single test separately by adding -k 'test_telem_performance' option.