Workflows¶
Updating the Greenbone Community Containers¶
Important
Please always ensure to use the latest version of the docker-compose.yml file when following these steps. The file might got updates and important changes since your last download.
To update the Greenbone Community Containers to the latest version, it is required to pull the images and restart the containers which have new images. This can be done with:
docker compose -f $DOWNLOAD_DIR/docker-compose.yml pull
docker compose -f $DOWNLOAD_DIR/docker-compose.yml up -d
Performing a Feed Synchronization¶
For the actual vulnerability scanning, Vulnerability Tests, security information like CVEs, port lists and scan configurations are required. All this data is provided by the Greenbone Community Feed via dedicated data container images.
A feed synchronization always consists of two parts:
Downloading the changes via pulling new container images
Loading the changes into memory and a database by a daemon
Both steps may take a while, from several minutes up to hours, especially for the initial synchronization. Only if both steps are finished, the synchronized data is up-to-date and can be used.
The first step is done via the docker compose pull. The second step is done automatically when the daemons are running.
Downloading the Feed Changes¶
The data of the Greenbone Community Feed is provided via several container images. When these images are started, they copy the data into the used docker volumes automatically. Afterwards, the data is picked up from the volumes by the running daemons.
To download the latest feed data container images run
docker compose -f $DOWNLOAD_DIR/docker-compose.yml pull notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats data-objects
To copy the data from the images to the volumes run
docker compose -f $DOWNLOAD_DIR/docker-compose.yml up -d notus-data vulnerability-tests scap-data dfn-cert-data cert-bund-data report-formats data-objects
Loading the Feed Changes¶
Important
When feed content has been downloaded, the new data must be loaded by the corresponding daemons. This may take several minutes up to hours, especially for the initial loading of the data. Without loaded data, scans will contain incomplete and erroneous results.
After the Greenbone Community Containers have been started, the running daemons will always pick up the feed content and load the data automatically.
Vulnerability Tests Data¶
If the log (of ospd-openvas) contains the following output, the OpenVAS Scanner starts to load the new VT data:
Loading VTs. Scans will be [requested|queued] until VTs are loaded. This may
take a few minutes, please wait...
The loading of the VT data is finished if the log message can be found:
Finished loading VTs. The VT cache has been updated from version X to Y.
After the scanner is aware of the VT data, the data will be requested by gvmd. This will result in the following log message:
OSP service has different VT status (version X) from database (version (Y), Z VTs). Starting update ...
When gvmd has finished loading all VTs, the following message appears:
Updating VTs in database ... done (X VTs).
SCAP Data¶
gvmd starts loading the SCAP data containing CPE and CVE information when the following message can be found in the logs:
update_scap: Updating data from feed
The SCAP data is loaded and the synchronization is finished when the (gvmd) log contains the following message:
update_scap_end: Updating SCAP info succeeded
CERT Data¶
gvmd starts loading the CERT data containing DFN-CERT and CERT-Bund advisories when the following message can be found in the logs:
sync_cert: Updating data from feed
The CERT data is loaded and the synchronization is finished when the (gvmd) log contains the following message:
sync_cert: Updating CERT info succeeded.
GVMD Data¶
The log contains several messages when the gvmd data is loaded. For port lists, these messages are similar to:
Port list All IANA assigned TCP (33d0cd82-57c6-11e1-8ed1-406186ea4fc5) has been created by admin
For report formats:
Report format XML (a994b278-1f62-11e1-96ac-406186ea4fc5) has been created by admin
Hint
Scan Configs can only be loaded if the VT data is available in gvmd and a Feed Import Owner is set.
For scan configs:
Scan config Full and fast (daba56c8-73ec-11df-a475-002264764cea) has been created by admin
Accessing the Web Interface Remotely¶
When using the docker compose file, the web server is configured to listen only on the local address of the host (127.0.0.1). To allow remote access on all interfaces of the host, the compose file must be modified to configure the web server nginx to listen on all network interfaces.
The following change of the docker compose file can be applied (it also changes to the default http port 80 as an example):
...
nginx:
image: nginx
environment:
NGINX_HOST: "localhost"
- NGINX_HTTP_PORT: 9392
+ NGINX_HTTP_PORT: 80
NGINX_HTTPS_PORT: 443
NGINX_SERVER_CERT: "/etc/nginx/certs/server.cert.pem"
NGINX_SERVER_KEY: "/etc/nginx/certs/server.key"
NGINX_ACCESS_CONTROL_ALLOW_ORIGIN_HEADER: "https://localhost"
NGINX_X_FRAME_OPTIONS_HEADER: "SAMEORIGIN"
NGINX_CONTENT_SECURITY_POLICY_HEADER: "default-src 'none'; object-src 'none'; base-uri 'none'; connect-src 'self'; script-src 'self'; script-src-elem 'self' 'unsafe-inline';frame-ancestors 'none'; form-action 'self'; style-src-elem 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; font-src 'self';img-src 'self' blob: data:;"
NGINX_STRICT_TRANSPORT_SECURITY_HEADER: "max-age=31536000; includeSubDomains;"
ports:
- - 127.0.0.1:443:443
- - 127.0.0.1:9392:9392
+ - 80:80
+ - 443:443
volumes:
- nginx_templates_vol:/etc/nginx/templates:ro
- nginx_certificates_vol:/etc/nginx/certs:ro
- gsa_data_vol:/usr/share/nginx/html:ro
depends_on:
gvm-config:
condition: service_completed_successfully
gsa:
condition: service_healthy
gsad:
condition: service_started
...
Starting from Scratch¶
To start from scratch, the containers must be stopped. Afterwards, the containers and volumes must be removed to delete all data. All this can be done by running:
docker compose -f $DOWNLOAD_DIR/docker-compose.yml down -v
Gaining a Terminal for a Container¶
If you want to debug something in a container, install additional software, take a look at the file content, or change some configuration, it is possible to gain shell access to a container.
To access a container with a bash shell as a root user, you can run:
docker compose -f $DOWNLOAD_DIR/docker-compose.yml exec <container-name> /bin/bash
Afterwards, you can execute standard bash commands within the running container.
Using gvm-tools for CLI access¶
To query data or control gvmd and ospd-openvas via CLI, gvm-tools can be used. gvm-tools is provided in the gvm-tools container. This container can be started with:
docker compose -f $DOWNLOAD_DIR/docker-compose.yml run --rm gvm-tools
Afterwards, a bash shell is provided and gvm-cli, gvm-pyshell or gvm-script
can be run. For example:
gvm-cli --gmp-username admin socket --pretty --xml "<get_version/>"
Exposing gvmd Unix socket for GMP Access¶
To enable the use of the protocol GMP provided by gvmd from the docker
host, a bind mount
must be used for the /run/gvmd directory. To make the gvmd Unix socket
available, a directory must be created first and the permissions must
be adjusted.
mkdir -p /tmp/gvm/gvmd
chmod -R 777 /tmp/gvm
In the next step, the docker compose file must be changed as follows:
gvmd:
image: greenbone/gvmd:stable
restart: on-failure
volumes:
- gvmd_data_vol:/var/lib/gvm
- vt_data_vol:/var/lib/openvas
- psql_data_vol:/var/lib/postgresql
- - gvmd_socket_vol:/run/gvmd
+ - /tmp/gvm/gvmd:/run/gvmd
- ospd_openvas_socket_vol:/run/ospd
- psql_socket_vol:/var/run/postgresql
depends_on:
- pg-gvm
...
gsad:
image: registry.community.greenbone.net/community/gsad:stable
restart: on-failure
environment:
GSAD_ARGS: "--listen=0.0.0.0 --http-only --api-only -f"
volumes:
- - gvmd_socket_vol:/run/gvmd
+ - /tmp/gvm/gvmd:/run/gvmd
depends_on:
gvmd:
condition: service_started
After restarting the containers with
docker compose -f $DOWNLOAD_DIR/docker-compose.yml up -d
the Unix socket should be available at /tmp/gvm/gvmd/gvmd.sock. For example,
to use the socket with gvm-tools, the
following command can be executed:
gvm-cli socket --socketpath /tmp/gvm/gvmd/gvmd.sock --pretty --xml "<get_version/>"
Setting up a mail transport agent inside Docker container¶
gvmd uses the msmtp client as MTA. It can be configured with environment variables
within compose file or command line. Available variables (for detailed explanation refer
to msmtp documentation, note that not all
mstmp options implemented in gvmd container):
MTA_HOST: The SMTP server to send the mail to. (Mandatory parameter).MTA_PORT: The port that the SMTP server listens on. (default = ‘25’).MTA_TLS: Enable or disable TLS (on|off’).MTA_STARTTLS: TLS variant: start TLS from within the session (‘on’, default), or nnel the session through TLS (‘off’).MTA_TLS_CERTCHECK: Enable or disable checks of the server certificate. (WARNING: When the checks are disabled, TLS sessions will not be secure!) (‘on’|’off’).MTA_AUTH: Enable or disable authentication and optionally choose a method to use ‘on’|’off’|’method’_).MTA_USER: Username for authentication.MTA_PASSWORD: Password for authentication.MTA_FROM: Set the envelope-from address.MTA_LOGFILE: Enable logging to the specified file.
Examples:
...
gvmd:
image: registry.community.greenbone.net/community/gvmd:stable
environment:
- MTA_HOST=postfix-server.example.org
- MTA_PORT=25
- MTA_TLS=off
- MTA_STARTTLS=off
- MTA_AUTH=off
...
...
gvmd:
image: registry.community.greenbone.net/community/gvmd:stable
environment:
- MTA_HOST=smtp.gmail.com
- MTA_PORT=587
- MTA_TLS=on
- MTA_STARTTLS=on
- MTA_AUTH=on
- MTA_USER=<username>
- MTA_FROM=<username>@gmail.com
- MTA_PASSWORD=<some_password>
...
Performing a Manual Feed Sync¶
It is possible to do a manual feed sync using rsync instead of pulling the current feed content via the newest container images.
Warning
Please be aware that the manually synced data will be overridden if the data containers are (re-)started.
For the manual feed sync, the greenbone-feed-sync
script will be used. The greenbone-feed-sync script is also provided via a
container image. Using the container image requires extending the docker compose
file as follows:
...
greenbone-feed-sync:
image: registry.community.greenbone.net/community/greenbone-feed-sync
volumes:
- vt_data_vol:/var/lib/openvas/plugins
- notus_data_vol:/var/lib/notus
- gvmd_data_vol:/var/lib/gvm
- scap_data_vol:/var/lib/gvm/scap-data
- cert_data_vol:/var/lib/gvm/cert-data
- data_objects_vol:/var/lib/gvm/data-objects/gvmd
...
Syncing Vulnerability Tests¶
VT data contains .nasl and .notus files for creating results
during a vulnerability scan.
docker compose -f $DOWNLOAD_DIR/docker-compose.yml \
run --rm greenbone-feed-sync greenbone-feed-sync --type nasl
Syncing SCAP, CERT and GVMD Data¶
SCAP data contains CPE and CVE information.
docker compose -f $DOWNLOAD_DIR/docker-compose.yml \
run --rm greenbone-feed-sync greenbone-feed-sync --type scap
CERT data contains vulnerability information from the German DFN-CERT and CERT-Bund agencies.
docker compose -f $DOWNLOAD_DIR/docker-compose.yml \
run --rm greenbone-feed-sync greenbone-feed-sync --type cert
gvmd data (or also called data-objects) are scan configurations, compliance policies, port lists and report formats.
docker compose -f $DOWNLOAD_DIR/docker-compose.yml \
run --rm greenbone-feed-sync greenbone-feed-sync --type gvmd-data
Setting up SSL/TLS for GSA¶
Enabling SSL/TLS for the the web interface (GSA) requires generating a private key and public certificate. By default a self signed certificate is generated automatically. To provide an own certificate the following steps are necessary.
As of September 2020, the maximum validity period for publicly trusted SSL/TLS certificates is 398 days. An expiration date of more than 397 days is not valid and may cause some browsers to block the connection. OpenSSL can be used to generate the private key and certificate:
openssl req -x509 -newkey rsa:4096 -keyout server.key -out servercert.cert.pem -nodes -days 397
The user that executes the docker compose command must have read access to the
private key and certificate. So, they must be placed in an appropriate location
such as the user’s home directory or the tmp directory.
mkdir $HOME/.ssl && mv server.key server.cert.pem $HOME/.ssl
Finally, the nginx configuration in the docker-compose.yml file must be
modified to use the new certificate files.
Sample nginx service settings to use own TLS certificate files:
nginx:
image: nginx
environment:
NGINX_HOST: "localhost"
NGINX_HTTP_PORT: 9392
NGINX_HTTPS_PORT: 443
NGINX_SERVER_CERT: "/etc/nginx/certs/server.cert.pem"
NGINX_SERVER_KEY: "/etc/nginx/certs/server.key"
NGINX_ACCESS_CONTROL_ALLOW_ORIGIN_HEADER: "https://localhost"
NGINX_X_FRAME_OPTIONS_HEADER: "SAMEORIGIN"
NGINX_CONTENT_SECURITY_POLICY_HEADER: "default-src 'none'; object-src 'none'; base-uri 'none'; connect-src 'self'; script-src 'self'; script-src-elem 'self' 'unsafe-inline';frame-ancestors 'none'; form-action 'self'; style-src-elem 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; font-src 'self';img-src 'self' blob: data:;"
NGINX_STRICT_TRANSPORT_SECURITY_HEADER: "max-age=31536000; includeSubDomains;"
ports:
- 127.0.0.1:443:443
- 127.0.0.1:9392:9392
volumes:
- nginx_templates_vol:/etc/nginx/templates:ro
- nginx_certificates_vol:/etc/nginx/certs:ro
+ - /home/<username>/.ssl/:/etc/nginx/certs:ro
- gsa_data_vol:/usr/share/nginx/html:ro
depends_on:
gvm-config:
condition: service_completed_successfully
gsa:
condition: service_healthy
gsad:
condition: service_started
After modifying the docker-compose.yml file, restart the containers to enable
the changes.
docker compose -f $DOWNLOAD_DIR/docker-compose.yml up -d