diff --git a/CHANGELOG.md b/CHANGELOG.md index 3cfb18eae..61c2a177e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,7 @@ Changelog](https://keepachangelog.com/en/1.0.0/). ### Changed - "Communicate as Backdoor User" PBA's HTTP requests to request headers only and include a timeout. #1577 +- The setup procedure for custom server_config.json files to be simpler. #1576 ### Removed - The VSFTPD exploiter. #1533 diff --git a/build_scripts/docker/server_config.json b/build_scripts/docker/server_config.json index ea464f181..7f95b67ee 100644 --- a/build_scripts/docker/server_config.json +++ b/build_scripts/docker/server_config.json @@ -1,9 +1,5 @@ { "data_dir": "/monkey_island_data", - "log_level": "DEBUG", - "environment": { - "server_config": "password" - }, "mongodb": { "start_mongodb": false } diff --git a/docs/content/FAQ/_index.md b/docs/content/FAQ/_index.md index 49cc4e0b8..76fedf3a4 100644 --- a/docs/content/FAQ/_index.md +++ b/docs/content/FAQ/_index.md @@ -16,9 +16,8 @@ Below are some of the most common questions we receive about the Infection Monke - [Should I run the Infection Monkey continuously?](#should-i-run-the-infection-monkey-continuously) - [Which queries does the Infection Monkey perform to the internet exactly?](#which-queries-does-the-infection-monkey-perform-to-the-internet-exactly) - [Logging and how to find logs](#logging-and-how-to-find-logs) - - [Monkey Island server](#monkey-island-server) - - [Infection Monkey agent](#infection-monkey-agent) - - [How do I change the log level of the Monkey Island logger?](#how-do-i-change-the-log-level-of-the-monkey-island-logger) + - [Monkey Island server logs](#monkey-island-server-logs) + - [Infection Monkey agent logs](#infection-monkey-agent-logs) - [Running the Infection Monkey in a production environment](#running-the-infection-monkey-in-a-production-environment) - [How much of a footprint does the Infection Monkey leave?](#how-much-of-a-footprint-does-the-infection-monkey-leave) - [What's the Infection Monkey Agent's impact on system resources usage?](#whats-the-infection-monkeys-impact-on-system-resources-usage) @@ -174,6 +173,10 @@ The log enables you to see which requests were requested from the server and ext 2019-07-23 10:52:24,027 - report.py:580 - get_domain_issues() - INFO - Domain issues generated for reporting ``` +It's also possible to change the default log level by editing `log_level` value in a [server configuration file](../../reference/server_configuration). +`log_level` can be set to `info`(default, less verbose) or `debug`(more verbose). + + ### Infection Monkey agent logs The Infection Monkey agent log file can be found in the following paths on machines where it was executed: @@ -197,26 +200,6 @@ The logs contain information about the internals of the Infection Monkey agent's 2019-07-22 19:16:45,013 [77598:140654230214464:DEBUG] connectionpool._make_request.396: https://updates.infectionmonkey.com:443 "GET / HTTP/1.1" 200 61 ``` -### How do I change the log level of the Monkey Island logger? - -The log level of the Monkey Island logger is set in the `log_level` field -in the `server_config.json` file (located in the [data directory]({{< ref "/reference/data_directory" >}})). -Make sure to leave everything else in `server_config.json` unchanged: - -```json -{ - ... - "log_level": "DEBUG", - ... -} -``` - -Logging levels correspond to [the logging level constants in python](https://docs.python.org/3.7/library/logging.html#logging-levels). - -To apply the changes, reset the Monkey Island process. -On Linux, use `sudo systemctl restart monkey-island.service`. -On Windows, restart the program. - ## Running the Infection Monkey in a production environment ### How much of a footprint does the Infection Monkey leave? diff --git a/docs/content/reference/data_directory.md b/docs/content/reference/data_directory.md index 2ab7ca78e..2fc6605cd 100644 --- a/docs/content/reference/data_directory.md +++ b/docs/content/reference/data_directory.md @@ -22,7 +22,7 @@ On Windows, the default path is `%AppData%\monkey_island`. The location of the data directory is set in the `data_dir` field in the `server_config.json` file. -1. Create a custom `server_config.json` file and set the `data_dir` field. Its +1. [Create a custom server_config.json file](../server_configuration) and set the `data_dir` field. Its contents will look like: ```json diff --git a/docs/content/reference/server_configuration.md b/docs/content/reference/server_configuration.md new file mode 100644 index 000000000..9e470a19b --- /dev/null +++ b/docs/content/reference/server_configuration.md @@ -0,0 +1,47 @@ +--- +title: "Server configuration" +date: 2021-11-26T12:00:19+02:00 +draft: true +pre: ' ' +weight: 1 +--- + +## Configuring the Island + +The Island Server(C&C) is configured by creating a `server_config.json` file. + +### Creating a configuration file + +Here's an example `server_config.json` with all options specified: +```json +{ + "log_level": "DEBUG", + "ssl_certificate": { + "ssl_certificate_file": "", + "ssl_certificate_key_file": "" + }, + "mongodb": { + "start_mongodb": true + }, + "data_dir": "/monkey_island_data" +} +``` + +Only relevant options can be specified, for example: +```json +{ + "ssl_certificate": { + "ssl_certificate_file": "", + "ssl_certificate_key_file": "" + } +} +``` + +### Configuration options + +See setup instructions for your operating system to understand how to apply these. + + - `log_level` - can be set to `"DEBUG"`(verbose), `"INFO"`(less verbose) or `"ERROR"`(silent, except errors). + - `ssl_certificate` - contains paths for files, required to run the Island server with custom certificate. + - `data_dir` - path to a writeable directory where the Island will store the database and other files. + - `mongodb` - options for MongoDB. Should not be changed unless you want to run your own instance of MongoDB. diff --git a/docs/content/setup/docker.md b/docs/content/setup/docker.md index db5979fc6..5de3c9fb5 100644 --- a/docs/content/setup/docker.md +++ b/docs/content/setup/docker.md @@ -49,12 +49,12 @@ any MongoDB containers or volumes associated with the previous version. mongo:4.2 ``` -### 3a. Start Monkey Island with default certificate +### 3. Start Monkey Island with default certificate By default, Infection Monkey comes with a [self-signed SSL certificate](https://aboutssl.org/what-is-self-sign-certificate/). In enterprise or other security-sensitive environments, it is recommended that the user [provide Infection Monkey with a -certificate](#3b-start-monkey-island-with-user-provided-certificate) that has +certificate](#start-monkey-island-with-user-provided-certificate) that has been signed by a private certificate authority. 1. Run the Monkey Island server @@ -67,23 +67,57 @@ been signed by a private certificate authority. guardicore/monkey-island:VERSION ``` -### 3b. Start Monkey Island with user-provided certificate -{{% notice info %}} -If you are upgrading the Infection Monkey to a new version, be sure to remove -any volumes associated with the previous version. -{{% /notice %}} +### 4. Accessing Monkey Island -1. Create a directory named `monkey_island_data`. If you already have it, - **make sure it's empty**. This will serve as the location where Infection - Monkey stores its configuration and runtime artifacts. +After the Monkey Island docker container starts, you can access Monkey Island by pointing your browser at `https://localhost:5000`. +## Configuring the server + +You can configure the server by mounting a volume and specifying a + [server configuration file](../../reference/server_configuration): + +1. Create a directory for server configuration file, e.g. `monkey_island_data`: ```bash mkdir ./monkey_island_data chmod 700 ./monkey_island_data ``` +1. Move your `server_config.json` file to `./monkey_island_data` directory. +1. Run the container with a mounted volume, specify the path to the `server_config.json`: +```bash +sudo docker run \ + --rm \ + --name monkey-island \ + --network=host \ + --user "$(id -u ${USER}):$(id -g ${USER})" \ + --volume "$(realpath ./monkey_island_data)":/monkey_island_data \ + guardicore/monkey-island:VERSION --setup-only --server-config="/monkey_island_data/server_config.json" +``` -1. Run Monkey Island with the `--setup-only` flag to populate the `./monkey_island_data` directory with a default `server_config.json` file. +### Start Monkey Island with user-provided certificate +By default, Infection Monkey comes with a [self-signed SSL +certificate](https://aboutssl.org/what-is-self-sign-certificate/). In +enterprise or other security-sensitive environments, it is recommended that the +user provide Infection Monkey with a certificate that has been signed by a +private certificate authority. + +1. Terminate the docker container if it's already running. +1. Move your `.crt` and `.key` files to `./monkey_island_data` (directory created for the volume). +1. Make sure that your `.crt` and `.key` files are readable only by you. + ```bash + chmod 600 + chmod 600 + ``` +1. Modify the [server configuration file](../../reference/server_configuration) and add the following lines: + ```json + { + "ssl_certificate": { + "ssl_certificate_file": "/monkey_island_data/my_cert.crt", + "ssl_certificate_key_file": "/monkey_island_data/my_key.key" + } + } + ``` +1. Run the container with a mounted volume, specify the path to the `server_config.json`: ```bash sudo docker run \ --rm \ @@ -91,55 +125,32 @@ any volumes associated with the previous version. --network=host \ --user "$(id -u ${USER}):$(id -g ${USER})" \ --volume "$(realpath ./monkey_island_data)":/monkey_island_data \ - guardicore/monkey-island:VERSION --setup-only + guardicore/monkey-island:VERSION --setup-only --server-config="/monkey_island_data/server_config.json" ``` +1. Access the Monkey Island web UI by pointing your browser at + `https://localhost:5000`. -1. Move your `.crt` and `.key` files to `./monkey_island_data`. +### Change logging level -1. Make sure that your `.crt` and `.key` files are readable and writeable only by you. - - ```bash - chmod 600 ./monkey_island_data/ - chmod 600 ./monkey_island_data/ - ``` - -1. Edit `./monkey_island_data/server_config.json` to configure Monkey Island - to use your certificate. Your config should look something like this: - - ```json {linenos=inline,hl_lines=["11-14"]} +1. Stop the docker container if it's already running. +1. Modify the [server configuration file](../../reference/server_configuration) by adding the following lines: + ```json { - "data_dir": "/monkey_island_data", - "log_level": "DEBUG", - "environment": { - "server_config": "password", - "deployment": "docker" - }, - "mongodb": { - "start_mongodb": false - }, - "ssl_certificate": { - "ssl_certificate_file": "/monkey_island_data/", - "ssl_certificate_key_file": "/monkey_island_data/" - } + "log_level": "INFO" } ``` - -1. Start the Monkey Island server: - +1. Run the container with a mounted volume, specify the path to the `server_config.json`: ```bash sudo docker run \ - --tty \ - --interactive \ + --rm \ --name monkey-island \ --network=host \ --user "$(id -u ${USER}):$(id -g ${USER})" \ --volume "$(realpath ./monkey_island_data)":/monkey_island_data \ - guardicore/monkey-island:VERSION + guardicore/monkey-island:VERSION --setup-only --server-config="/monkey_island_data/server_config.json" ``` - -### 4. Accessing Monkey Island - -After the Monkey Island docker container starts, you can access Monkey Island by pointing your browser at `https://localhost:5000`. +1. Access the Monkey Island web UI by pointing your browser at + `https://localhost:5000`. ## Upgrading diff --git a/docs/content/setup/linux.md b/docs/content/setup/linux.md index 275330c2c..c39ec75bc 100644 --- a/docs/content/setup/linux.md +++ b/docs/content/setup/linux.md @@ -46,6 +46,14 @@ do, see the [FAQ]({{< ref >}}) for more information. {{% /notice %}} +## Configuring the server + +You can configure the server by creating +a [server configuration file](../../reference/server_configuration) and +providing a path to it via command line parameters: + +`./InfectionMonkey-v1.12.0.AppImage --server-config="/path/to/server_config.json"` + ### Start Monkey Island with user-provided certificate By default, Infection Monkey comes with a [self-signed SSL @@ -54,13 +62,7 @@ enterprise or other security-sensitive environments, it is recommended that the user provide Infection Monkey with a certificate that has been signed by a private certificate authority. -1. Run the Infection Monkey AppImage package with the `--setup-only` flag to - populate the `$HOME/.monkey_island` directory with a default - `server_config.json` file. - - ```bash - ./InfectionMonkey-v1.12.0.AppImage --setup-only - ``` +1. Terminate the Island process if it's already running. 1. (Optional but recommended) Move your `.crt` and `.key` files to `$HOME/.monkey_island`. @@ -72,30 +74,42 @@ private certificate authority. chmod 600 ``` -1. Edit `$HOME/.monkey_island/server_config.json` to configure Monkey Island - to use your certificate. Your config should look something like this: +1. Create a [server configuration file and provide the path to the certificate](../../reference/server_configuration). +The server configuration file should look something like: - ```json {linenos=inline,hl_lines=["11-14"]} + ```json { - "data_dir": "~/.monkey_island", - "log_level": "DEBUG", - "environment": { - "server_config": "password", - "deployment": "linux" - }, - "mongodb": { - "start_mongodb": true - }, - "ssl_certificate": { - "ssl_certificate_file": "", - "ssl_certificate_key_file": "" - } + "ssl_certificate": { + "ssl_certificate_file": "$HOME/.monkey_island/my_cert.crt", + "ssl_certificate_key_file": "$HOME/.monkey_island/my_key.key" + } } ``` 1. Start Monkey Island by running the Infection Monkey AppImage package: ```bash - ./InfectionMonkey-v1.12.0.AppImage + ./InfectionMonkey-v1.12.0.AppImage --server-config="/path/to/server_config.json" + ``` + +1. Access the Monkey Island web UI by pointing your browser at + `https://localhost:5000`. + +### Change logging level + +1. Terminate the Island process if it's already running. + +1. Create a [server configuration file](../../reference/server_configuration). +The server configuration file should look something like: + + ```json + { + "log_level": "INFO" + } + ``` + +1. Start Monkey Island by running the Infection Monkey AppImage package: + ```bash + ./InfectionMonkey-v1.12.0.AppImage --server-config="/path/to/server_config.json" ``` 1. Access the Monkey Island web UI by pointing your browser at diff --git a/docs/content/setup/windows.md b/docs/content/setup/windows.md index f9fd5acaf..b78712e71 100644 --- a/docs/content/setup/windows.md +++ b/docs/content/setup/windows.md @@ -26,6 +26,13 @@ do, see the [FAQ]({{< ref "/faq/#i-updated-to-a-new-version-of-the-infection-monkey-and-im-being-asked-to-delete-my-existing-data-directory-why" >}}) for more information. {{% /notice %}} +> +## Configuring the server + +You can configure the server by editing [the configuration +file](../../reference/server_configuration) located in installation directory. +The default path is +`C:\Program Files\Guardicore\Monkey Island\monkey\monkey_island\cc\server_config.json`. ### Start Monkey Island with user-provided certificate @@ -34,32 +41,35 @@ enterprise or other security-sensitive environments, it is recommended that the user provide Infection Monkey with a certificate that has been signed by a private certificate authority. -1. If you haven't already, run the Monkey Island by clicking on the desktop - shortcut. This will populate MongoDB, as well as create and populate - `%AppData%\monkey_island`. 1. Stop the Monkey Island process. 1. (Optional but recommended) Move your `.crt` and `.key` files to `%AppData%\monkey_island`. -1. Edit `%AppData%\monkey_island\server_config.json` to configure Monkey Island - to use your certificate. Your config should look something like this: - - ```json {linenos=inline,hl_lines=["11-14"]} +1. Modify the `server_config.json` (by default located in `C:\Program Files\Guardicore\Monkey Island\monkey\monkey_island\cc\server_config.json`) by adding the following lines: + ```json { - "log_level": "DEBUG", - "environment": { - "server_config": "password", - "deployment": "windows" - }, - "mongodb": { - "start_mongodb": true - }, + ... "ssl_certificate": { - "ssl_certificate_file": "", - "ssl_certificate_key_file": "" - } + "ssl_certificate_file": "%AppData%\\monkey_island\\my_cert.crt", + "ssl_certificate_key_file": "%AppData%\\monkey_island\\my_key.key" + }, + ... } ``` 1. Run the Monkey Island by clicking on the desktop shortcut. +1. Access the Monkey Island web UI by pointing your browser at + `https://localhost:5000`. +### Change logging level + +1. Stop the Island server. +1. Modify the `server_config.json` (by default located in `C:\Program Files\Guardicore\Monkey Island\monkey\monkey_island\cc\server_config.json`) by adding the following lines: + ```json + { + ... + "log_level": "INFO", + ... + } + ``` +1. Run the Monkey Island by clicking on the desktop shortcut. 1. Access the Monkey Island web UI by pointing your browser at `https://localhost:5000`. diff --git a/monkey/monkey_island/cc/environment/__init__.py b/monkey/monkey_island/cc/environment/__init__.py deleted file mode 100644 index e69de29bb..000000000 diff --git a/monkey/monkey_island/cc/environment/server_config_handler.py b/monkey/monkey_island/cc/environment/server_config_handler.py deleted file mode 100644 index 363b7c2e6..000000000 --- a/monkey/monkey_island/cc/environment/server_config_handler.py +++ /dev/null @@ -1,27 +0,0 @@ -import json -import os -from pathlib import Path - -from monkey_island.cc.server_utils.consts import DEFAULT_SERVER_CONFIG_PATH, SERVER_CONFIG_FILENAME -from monkey_island.cc.setup.island_config_options import IslandConfigOptions - - -def create_default_server_config_file(data_dir: str) -> str: - config_file_path = os.path.join(data_dir, SERVER_CONFIG_FILENAME) - if not os.path.isfile(config_file_path): - write_default_server_config_to_file(config_file_path) - - return config_file_path - - -def write_default_server_config_to_file(path: str) -> None: - default_config = Path(DEFAULT_SERVER_CONFIG_PATH).read_text() - Path(path).write_text(default_config) - - -def load_server_config_from_file(server_config_path) -> IslandConfigOptions: - with open(server_config_path, "r") as f: - config_content = f.read() - config = json.loads(config_content) - - return IslandConfigOptions(config) diff --git a/monkey/monkey_island/cc/server_config.json b/monkey/monkey_island/cc/server_config.json index 7ae515179..ff6eb9a7f 100644 --- a/monkey/monkey_island/cc/server_config.json +++ b/monkey/monkey_island/cc/server_config.json @@ -1,8 +1,5 @@ { "log_level": "DEBUG", - "environment": { - "server_config": "password" - }, "mongodb": { "start_mongodb": true } diff --git a/monkey/monkey_island/cc/server_setup.py b/monkey/monkey_island/cc/server_setup.py index 65617774d..a3c0cf750 100644 --- a/monkey/monkey_island/cc/server_setup.py +++ b/monkey/monkey_island/cc/server_setup.py @@ -3,14 +3,13 @@ import json import logging import sys from pathlib import Path -from sys import exit from threading import Thread -from typing import Tuple import gevent.hub from gevent.pywsgi import WSGIServer from monkey_island.cc.server_utils.consts import ISLAND_PORT +from monkey_island.cc.setup.config_setup import get_server_config # Add the monkey_island directory to the path, to make sure imports that don't start with # "monkey_island." work. @@ -18,7 +17,6 @@ MONKEY_ISLAND_DIR_BASE_PATH = str(Path(__file__).parent.parent) if str(MONKEY_ISLAND_DIR_BASE_PATH) not in sys.path: sys.path.insert(0, MONKEY_ISLAND_DIR_BASE_PATH) -import monkey_island.cc.setup.config_setup as config_setup # noqa: E402 from common.version import get_version # noqa: E402 from monkey_island.cc.app import init_app # noqa: E402 from monkey_island.cc.arg_parser import IslandCmdArgs # noqa: E402 @@ -34,7 +32,7 @@ from monkey_island.cc.services.initialize import initialize_services # noqa: E4 from monkey_island.cc.services.reporting.exporter_init import populate_exporter_list # noqa: E402 from monkey_island.cc.services.utils.network_utils import local_ip_addresses # noqa: E402 from monkey_island.cc.setup import island_config_options_validator # noqa: E402 -from monkey_island.cc.setup.data_dir import IncompatibleDataDirectory # noqa: E402 +from monkey_island.cc.setup.data_dir import IncompatibleDataDirectory, setup_data_dir # noqa: E402 from monkey_island.cc.setup.gevent_hub_error_handler import GeventHubErrorHandler # noqa: E402 from monkey_island.cc.setup.island_config_options import IslandConfigOptions # noqa: E402 from monkey_island.cc.setup.mongo import mongo_setup # noqa: E402 @@ -45,12 +43,13 @@ logger = logging.getLogger(__name__) def run_monkey_island(): island_args = parse_cli_args() - config_options, server_config_path = _setup_data_dir(island_args) + config_options = _extract_config(island_args) + _setup_data_dir(config_options.data_dir) _exit_on_invalid_config_options(config_options) _configure_logging(config_options) - _initialize_globals(config_options, server_config_path) + _initialize_globals(config_options.data_dir) mongo_db_process = None if config_options.start_mongodb: @@ -58,22 +57,24 @@ def run_monkey_island(): _connect_to_mongodb(mongo_db_process) - _configure_gevent_exception_handling(Path(config_options.data_dir)) + _configure_gevent_exception_handling(config_options.data_dir) _start_island_server(island_args.setup_only, config_options) -def _setup_data_dir(island_args: IslandCmdArgs) -> Tuple[IslandConfigOptions, str]: +def _extract_config(island_args: IslandCmdArgs) -> IslandConfigOptions: try: - return config_setup.setup_server_config(island_args) - except OSError as ex: - print(f"Error opening server config file: {ex}") - exit(1) + return get_server_config(island_args) except json.JSONDecodeError as ex: print(f"Error loading server config: {ex}") - exit(1) + sys.exit(1) + + +def _setup_data_dir(data_dir_path: Path): + try: + setup_data_dir(data_dir_path) except IncompatibleDataDirectory as ex: print(f"Incompatible data directory: {ex}") - exit(1) + sys.exit(1) def _exit_on_invalid_config_options(config_options: IslandConfigOptions): @@ -81,7 +82,7 @@ def _exit_on_invalid_config_options(config_options: IslandConfigOptions): island_config_options_validator.raise_on_invalid_options(config_options) except Exception as ex: print(f"Configuration error: {ex}") - exit(1) + sys.exit(1) def _configure_logging(config_options): @@ -89,8 +90,8 @@ def _configure_logging(config_options): setup_logging(config_options.data_dir, config_options.log_level) -def _initialize_globals(config_options: IslandConfigOptions, server_config_path: str): - initialize_services(config_options.data_dir) +def _initialize_globals(data_dir: Path): + initialize_services(data_dir) def _start_mongodb(data_dir: Path) -> MongoDbProcess: diff --git a/monkey/monkey_island/cc/server_utils/consts.py b/monkey/monkey_island/cc/server_utils/consts.py index ab9261140..29fe78933 100644 --- a/monkey/monkey_island/cc/server_utils/consts.py +++ b/monkey/monkey_island/cc/server_utils/consts.py @@ -36,8 +36,6 @@ MONGO_EXECUTABLE_PATH = ( ) MONGO_CONNECTION_TIMEOUT = 15 -DEFAULT_SERVER_CONFIG_PATH = str(Path(MONKEY_ISLAND_ABS_PATH, "cc", SERVER_CONFIG_FILENAME)) - DEFAULT_LOG_LEVEL = "INFO" DEFAULT_START_MONGO_DB = True diff --git a/monkey/monkey_island/cc/setup/config_setup.py b/monkey/monkey_island/cc/setup/config_setup.py index b1e76b51d..6835dfc61 100644 --- a/monkey/monkey_island/cc/setup/config_setup.py +++ b/monkey/monkey_island/cc/setup/config_setup.py @@ -1,42 +1,39 @@ -from typing import Tuple +import json +from logging import getLogger +from pathlib import Path from common.utils.file_utils import expand_path from monkey_island.cc.arg_parser import IslandCmdArgs -from monkey_island.cc.environment import server_config_handler -from monkey_island.cc.server_utils.consts import DEFAULT_SERVER_CONFIG_PATH -from monkey_island.cc.setup.data_dir import setup_data_dir +from monkey_island.cc.server_utils.consts import MONKEY_ISLAND_ABS_PATH, SERVER_CONFIG_FILENAME from monkey_island.cc.setup.island_config_options import IslandConfigOptions +logger = getLogger(__name__) + +PACKAGE_CONFIG_PATH = Path(MONKEY_ISLAND_ABS_PATH, "cc", SERVER_CONFIG_FILENAME) + + +def get_server_config(island_args: IslandCmdArgs) -> IslandConfigOptions: + config = IslandConfigOptions() + + _update_config_from_file(config, PACKAGE_CONFIG_PATH) -def setup_server_config(island_args: IslandCmdArgs) -> Tuple[IslandConfigOptions, str]: if island_args.server_config_path: - return _setup_config_by_cmd_arg(island_args.server_config_path) + path_to_config = expand_path(island_args.server_config_path) + _update_config_from_file(config, path_to_config) - return _setup_default_config() + return config -def _setup_config_by_cmd_arg(server_config_path) -> Tuple[IslandConfigOptions, str]: - server_config_path = expand_path(server_config_path) - config = server_config_handler.load_server_config_from_file(server_config_path) - - # TODO refactor like in https://github.com/guardicore/monkey/pull/1528 because - # there's absolutely no reason to be exposed to IslandConfigOptions extraction logic - # if you want to modify data directory related code. - setup_data_dir(config.data_dir) - - return config, server_config_path +def _update_config_from_file(config: IslandConfigOptions, config_path: Path): + try: + config_from_file = _load_server_config_from_file(config_path) + config.update(config_from_file) + logger.info(f"Server config updated from {config_path}") + except OSError: + logger.warn(f"Server config not found in path {config_path}") -def _setup_default_config() -> Tuple[IslandConfigOptions, str]: - default_config = server_config_handler.load_server_config_from_file(DEFAULT_SERVER_CONFIG_PATH) - default_data_dir = default_config.data_dir - - # TODO refactor like in https://github.com/guardicore/monkey/pull/1528 because - # there's absolutely no reason to be exposed to IslandConfigOptions extraction logic - # if you want to modify data directory related code. - setup_data_dir(default_data_dir) - - server_config_path = server_config_handler.create_default_server_config_file(default_data_dir) - config = server_config_handler.load_server_config_from_file(server_config_path) - - return config, server_config_path +def _load_server_config_from_file(server_config_path) -> dict: + with open(server_config_path, "r") as f: + config_content = f.read() + return json.loads(config_content) diff --git a/monkey/monkey_island/cc/setup/data_dir.py b/monkey/monkey_island/cc/setup/data_dir.py index af01da050..124b2a6f7 100644 --- a/monkey/monkey_island/cc/setup/data_dir.py +++ b/monkey/monkey_island/cc/setup/data_dir.py @@ -15,14 +15,14 @@ class IncompatibleDataDirectory(Exception): pass -def setup_data_dir(data_dir_path: Path) -> None: +def setup_data_dir(data_dir_path: Path): logger.info(f"Setting up data directory at {data_dir_path}.") if _is_data_dir_old(data_dir_path): logger.info("Version in data directory does not match the Island's version.") _handle_old_data_directory(data_dir_path) create_secure_directory(str(data_dir_path)) write_version(data_dir_path) - logger.info("Data directory set up.") + logger.info(f"Data directory set up in {data_dir_path}.") def _is_data_dir_old(data_dir_path: Path) -> bool: diff --git a/monkey/monkey_island/cc/setup/island_config_options.py b/monkey/monkey_island/cc/setup/island_config_options.py index 66a49306a..27df897e8 100644 --- a/monkey/monkey_island/cc/setup/island_config_options.py +++ b/monkey/monkey_island/cc/setup/island_config_options.py @@ -10,24 +10,41 @@ from monkey_island.cc.server_utils.consts import ( DEFAULT_START_MONGO_DB, ) +_DATA_DIR = "data_dir" +_SSL_CERT = "ssl_certificate" +_SSL_CERT_FILE = "ssl_certificate_file" +_SSL_CERT_KEY = "ssl_certificate_key_file" +_MONGODB = "mongodb" +_START_MONGODB = "start_mongodb" +_LOG_LEVEL = "log_level" + class IslandConfigOptions: - def __init__(self, config_contents: dict): - self.data_dir = expand_path(config_contents.get("data_dir", DEFAULT_DATA_DIR)) + def __init__(self, config_contents: dict = None): + if not config_contents: + config_contents = {} + self.data_dir = config_contents.get(_DATA_DIR, DEFAULT_DATA_DIR) - self.log_level = config_contents.get("log_level", DEFAULT_LOG_LEVEL) + self.log_level = config_contents.get(_LOG_LEVEL, DEFAULT_LOG_LEVEL) self.start_mongodb = config_contents.get( - "mongodb", {"start_mongodb": DEFAULT_START_MONGO_DB} - ).get("start_mongodb", DEFAULT_START_MONGO_DB) + _MONGODB, {_START_MONGODB: DEFAULT_START_MONGO_DB} + ).get(_START_MONGODB, DEFAULT_START_MONGO_DB) - self.crt_path = expand_path( - config_contents.get("ssl_certificate", DEFAULT_CERTIFICATE_PATHS).get( - "ssl_certificate_file", DEFAULT_CRT_PATH - ) + self.crt_path = config_contents.get(_SSL_CERT, DEFAULT_CERTIFICATE_PATHS).get( + _SSL_CERT_FILE, DEFAULT_CRT_PATH ) - self.key_path = expand_path( - config_contents.get("ssl_certificate", DEFAULT_CERTIFICATE_PATHS).get( - "ssl_certificate_key_file", DEFAULT_KEY_PATH - ) + self.key_path = config_contents.get(_SSL_CERT, DEFAULT_CERTIFICATE_PATHS).get( + _SSL_CERT_KEY, DEFAULT_KEY_PATH ) + + self._expand_paths() + + def _expand_paths(self): + self.data_dir = expand_path(str(self.data_dir)) + self.crt_path = expand_path(str(self.crt_path)) + self.key_path = expand_path(str(self.key_path)) + + def update(self, target: dict): + self.__dict__.update(target) + self._expand_paths() diff --git a/monkey/monkey_island/main.py b/monkey/monkey_island/main.py index 19cf07d9f..ca91c054b 100644 --- a/monkey/monkey_island/main.py +++ b/monkey/monkey_island/main.py @@ -1,8 +1,10 @@ # This import patches other imports and needs to be first -import monkey_island.setup.gevent_setup # noqa: F401 isort:skip +import sys from monkey_island.cc.server_utils.island_logger import setup_default_failsafe_logging +import monkey_island.setup.gevent_setup # noqa: F401 isort:skip + def main(): # This is here in order to catch EVERYTHING, some functions are being called on @@ -11,7 +13,7 @@ def main(): setup_default_failsafe_logging() except Exception as ex: print(f"Error configuring logging: {ex}") - exit(1) + sys.exit(1) from monkey_island.cc.server_setup import run_monkey_island # noqa: E402 diff --git a/monkey/tests/unit_tests/monkey_island/cc/environment/__init__.py b/monkey/tests/unit_tests/monkey_island/cc/environment/__init__.py deleted file mode 100644 index e69de29bb..000000000 diff --git a/monkey/tests/unit_tests/monkey_island/cc/server_utils/test_consts.py b/monkey/tests/unit_tests/monkey_island/cc/server_utils/test_consts.py index 26d7bc583..8d0b0b1b4 100644 --- a/monkey/tests/unit_tests/monkey_island/cc/server_utils/test_consts.py +++ b/monkey/tests/unit_tests/monkey_island/cc/server_utils/test_consts.py @@ -1,5 +1,4 @@ import os -import platform from monkey_island.cc.server_utils import consts @@ -7,12 +6,3 @@ from monkey_island.cc.server_utils import consts def test_monkey_island_abs_path(): assert consts.MONKEY_ISLAND_ABS_PATH.endswith("monkey_island") assert os.path.isdir(consts.MONKEY_ISLAND_ABS_PATH) - - -def test_default_server_config_file_path(): - if platform.system() == "Windows": - server_file_path = f"{consts.MONKEY_ISLAND_ABS_PATH}\\cc\\{consts.SERVER_CONFIG_FILENAME}" - else: - server_file_path = f"{consts.MONKEY_ISLAND_ABS_PATH}/cc/{consts.SERVER_CONFIG_FILENAME}" - - assert consts.DEFAULT_SERVER_CONFIG_PATH == server_file_path diff --git a/monkey/tests/unit_tests/monkey_island/cc/setup/test_config_setup.py b/monkey/tests/unit_tests/monkey_island/cc/setup/test_config_setup.py new file mode 100644 index 000000000..88e1850a7 --- /dev/null +++ b/monkey/tests/unit_tests/monkey_island/cc/setup/test_config_setup.py @@ -0,0 +1,96 @@ +import json +from json import dumps +from pathlib import Path + +import pytest + +import monkey_island.cc.setup.config_setup # noqa: F401 +from monkey_island.cc.arg_parser import IslandCmdArgs +from monkey_island.cc.server_utils.file_utils import is_windows_os +from monkey_island.cc.setup.config_setup import get_server_config +from monkey_island.cc.setup.island_config_options import IslandConfigOptions + + +@pytest.fixture +def cmd_server_config_path(tmp_path) -> Path: + # Represents the config that user can provide via cmd arguments + return tmp_path / "fake_server_config.json" + + +@pytest.fixture +def deployment_server_config_path(tmp_path) -> Path: + # Represents the config that is built in, deployment specific + return tmp_path / "fake_server_config3.json" + + +def create_server_config(config_contents: str, server_config_path: Path): + with open(server_config_path, "w") as file: + file.write(config_contents) + + +@pytest.fixture(autouse=True) +def mock_deployment_config_path(monkeypatch, deployment_server_config_path): + monkeypatch.setattr( + "monkey_island.cc.setup.config_setup.PACKAGE_CONFIG_PATH", + deployment_server_config_path, + ) + + +@pytest.fixture +def empty_cmd_args(): + return IslandCmdArgs(setup_only=False, server_config_path=None) + + +@pytest.fixture +def cmd_args_with_server_config(cmd_server_config_path): + return IslandCmdArgs(setup_only=False, server_config_path=cmd_server_config_path) + + +def test_extract_config_defaults(empty_cmd_args): + expected = IslandConfigOptions({}) + assert expected.__dict__ == get_server_config(empty_cmd_args).__dict__ + + +def test_deployment_config_overrides_defaults(deployment_server_config_path, empty_cmd_args): + expected = IslandConfigOptions({"log_level": "/log_level_2"}) + create_server_config(dumps({"log_level": "/log_level_2"}), deployment_server_config_path) + assert expected.__dict__ == get_server_config(empty_cmd_args).__dict__ + + +def test_cmd_config_overrides_everything( + deployment_server_config_path, cmd_server_config_path, cmd_args_with_server_config +): + expected = IslandConfigOptions({"log_level": "/log_level_3"}) + create_server_config(dumps({"log_level": "/log_level_2"}), deployment_server_config_path) + create_server_config(dumps({"log_level": "/log_level_3"}), cmd_server_config_path) + extracted_config = get_server_config(cmd_args_with_server_config) + assert expected.__dict__ == extracted_config.__dict__ + + +def test_not_overriding_unspecified_values( + deployment_server_config_path, cmd_server_config_path, cmd_args_with_server_config +): + expected = IslandConfigOptions({"log_level": "/log_level_2", "data_dir": "/data_dir1"}) + create_server_config(dumps({"data_dir": "/data_dir1"}), deployment_server_config_path) + create_server_config(dumps({"log_level": "/log_level_2"}), cmd_server_config_path) + extracted_config = get_server_config(cmd_args_with_server_config) + assert expected.__dict__ == extracted_config.__dict__ + + +def test_paths_get_expanded(deployment_server_config_path, empty_cmd_args): + if is_windows_os(): + path = "%temp%/path" + else: + path = "$HOME/path" + create_server_config(dumps({"data_dir": path}), deployment_server_config_path) + extracted_config = get_server_config(empty_cmd_args) + assert not extracted_config.data_dir == path + + +BAD_JSON = '{"data_dir": "C:\\test\\test"' + + +def test_malformed_json(cmd_server_config_path, cmd_args_with_server_config): + create_server_config(BAD_JSON, cmd_server_config_path) + with pytest.raises(json.JSONDecodeError): + get_server_config(cmd_args_with_server_config)