Configuring Anchore

1 - General Configuration

Initial Configuration

A single configuration file config.yaml is required to run Anchore - by default, this file is embedded in the Enterprise container image, located in /config/config.yaml. The default configuration file is provided as a way to get started, which is functional out of the box, without modification, when combined with either the Helm method or docker-compose method of installing Enterprise. The default configuration is set up to use environment variable substitutions so that configuration values can be controlled by setting the corresponding environment variables at deployment time (see Using Environment Variables in Anchore.

Each environment variable (starting with ANCHORE_) in the default config.yaml is set (either the baseline as set in the Dockerfile, or an override in docker-compose or Helm) to ensure that the system comes up with a fully populated configuration.

Some examples of useful initial settings follow.

• Default admin credentials: A default admin email and password is required to be defiend in the catalog service for the initial bootstrap of enterprise to succeed, which are both set through the default config file using the ANCHORE_ADMIN_PASSWORD and ANCHORE_ADMIN_EMAIL environment variables respectively. The Dockerfile defines a default email admin@myanchore, but does not define a default password. If using the default config file, the user must set a value for ANCHORE_ADMIN_PASSWORD in order to succeed the initial bootstrap of the system. To set the default password or to override the default email, simply add overrides for ANCHORE_ADMIN_PASSWORD and ANCHORE_ADMIN_EMAIL environment variables, set to your preferred values prior to deploying Anchore Enterprise. After the initial bootstrap, this can be removed if desired. The docker-compose file referenced in the quickstart installation guide has set ANCHORE_ADMIN_PASSWORD to foobar on the catalog service already

default_admin_password: '${ANCHORE_ADMIN_PASSWORD}'
default_admin_email: '${ANCHORE_ADMIN_EMAIL}'

• Log level: Anchore Enterprise is configured to run at the INFO log level by default. The full set of options are FATAL, ERROR, WARN, INFO, and DEBUG (in ascending order of log output verbosity). To set the log level of Anchore Enterprise Services, add an override for ANCHORE_LOG_LEVEL prior to deploying Anchore Enterprise.

log_level: '${ANCHORE_LOG_LEVEL}'

• Postgres Database: Anchore Enterprise requires access to a PostgreSQL database to operate. The database can be run as a container with a persistent volume or outside of your container environment (which is set up automatically if the example docker-compose.yaml is used). If you wish to use an external Postgres Database, the elements of the connection string in the config.yaml can be specified as environment variable overrides. The default configuration is set up to connect to a postgres DB that is deployed alongside Anchore Enterprise Services when using docker-compose or Helm, to the internal host anchore-db on port 5432 using username postgres with password mysecretpassword and db postgres. If an external database service is being used then you will need to provide the use, password, host, port and DB name environment variables, as shown below.

db_connect: 'postgresql://${ANCHORE_DB_USER}:${ANCHORE_DB_PASSWORD}@${ANCHORE_DB_HOST}:${ANCHORE_DB_PORT}/${ANCHORE_DB_NAME}'

Manual Configuration File Override

While Anchore Enterprise is set up to run out of the box without modifications, and many useful values can be overriden using environment variables as described above, one can always opt to have full control over the configuration by providing a config.yaml file explicitly, typically by generating the file and making it available from an external mount/configmap/etc. at deployment time. A good method to start if you wish to provide your own config.yaml is to extract the default config.yaml from the Anchore Enterprise container image, modify it, and then override the embedded /config/config.yaml at deployment time. For example:

• Extract the default config file from the anchore/enterprise container image:

# docker pull docker.io/anchore/enterprise:latest
# docker create --name ae docker.io/anchore/enterprise:latest
# docker cp ae:/config/config.yaml ./my_config.yaml
# docker rm ae

• Modify the configuration file to your liking.

• Set up your deployment to override the embedded /config/config.yaml at run time (below example shows how to achieve this with docker-compose). Edit the docker-compose.yaml to include a volume mount that mounts your my_config.yaml over the embedded /config/config.yaml, resulting in a volume section for each Anchore Enterprise service definition.

...
  api:
...
    volumes:
     - /path/to/my_config.yaml:/config/config.yaml:z
...
  catalog:
...
    volumes:
     - /path/to/my_config.yaml:/config/config.yaml:z
...
  simpleq:
...
    volumes:
     - /path/to/my_config.yaml:/config/config.yaml:z
...
  policy-engine:
...
    volumes:
     - /path/to/my_config.yaml:/config/config.yaml:z
...
  analyzer:
...
    volumes:
     - /path/to/my_config.yaml:/config/config.yaml:z
...

Now, each service will come up with your external my_config.yaml mounted over the embedded /config/config.yaml.

2 - Running Anchore Enterprise in an Air-Gapped Environment

Anchore Enterprise can run in an isolated environment with no outside internet connectivity. It does require a network connection to its own components and must be able to reach the Docker image registries (v2 API compatible) where the images to be analyzed are hosted.

Components

• Private Network
• Public Network (internet is reachable)
• Anchore Enterprise
• Anchore Enterprise Feeds
• Anchore Enterprise Feeds in Read-Only Mode
• Docker Image Registry (any registry that is compatible with the Docker Registry v2 API)

Assumptions

• The docker images to be analyzed are available within the Private Network.
• Anchore Enterprise will be accessed from within the private network by the components in the infrastructure that need to query for analysis results.
• There exists a way to move a data file from the Public Network to the Private Network.

Installation

1. Refer to feed data migration for configuring a Read-Only Feeds in Private Network.
2. Install Anchore Enterprise in Private Network.
3. Configure the Anchore Enterprise to use the Read-Only Feeds installation, see configuration.
4. Start Anchore Enterprise.

Periodically Updating Feed Data

To ensure that the Anchore Enterprise installation has up-to-date vulnerability data from the vulnerability sources, you need to update the Read-Only Feed Service with data from the feed service running on the public network. This is essentially the same process that was used at installation to initialize the Read-Only Feed Service. It should be done on a regular schedule, or when the Public Network Feed Service task execution indicates new data was detected.

3 - Configuring AnchoreCTL

The anchorectl command can be configured with command-line arguments, environment variables, and/or a configuration file. Typically, a configuration file should be created to set any static configuration parameters (your Anchore Enterprise’s URL, logging behavior, cataloger configurations, etc), so that invocations of the tool only require you to provide command-specific parameters as environment/cli options. However, to fully support stateless scripting, a configuration file is not strictly required (settings can be put in environment/cli options).

In this document, we focus on the anchorectl configuration file setup and options.

API Version Compatibility

It is highly recommended that the version of AnchoreCTL you are using is supported by the deployed version of Enterprise. Please refer to the Enterprise Release Notes for the supported version of AnchoreCTL.

The release of Anchore Enterprise v4.9.x, contains both the deprecated V1 API and the new V2 API.

• AnchoreCTL v1.8.0 is compatible with the Anchore Enterprise v4.9.x V1 API.
• AnchoreCTL v4.9.0 is compatible with the Anchore Enterprise v4.9.x V2 API.

The release of Anchore Enterprise v5.0.0, contains only the new V2 API.

• AnchoreCTL v5.0.0 is compatible with the Anchore Enterprise v5.0.0 V2 API.

In AnchoreCTL v1.8.0, a version check was introduced to prevent accidental use of AnchoreCTL v1.8.x with an incompatible API endpoint. If the check fails but you are sure that the configured URL you have set is a V1 API endpoint, you can bypass the check by setting the skip-api-version-check field in the configuration to boolean true as in th example below.

url: "http://localhost:8228"
username: "admin"
password: "foobar"
skip-api-version-check: true
...

This can also be done with the environment variable: ANCHORECTL_SKIP_API_VERSION_CHECK=true

Config File Search Locations

The anchorectl tool will seach for an avialable configuration file using the following search order, until it finds a match:

.anchorectl.yaml, anchorectl.yaml, .anchorectl/config.yaml, ~/.anchorectl.yaml, ~/anchorectl.yaml, $XDG_CONFIG_HOME/anchorectl/config.yaml

Generating Your First .anchorectl.yaml

For the most basic functional invocation of anchorectl, the only parameters that should be set are listed here:

  # the URL to the Anchore Enterprise API (env var: "ANCHORECTL_URL")
  url: ""

  # the Anchore Enterprise username (env var: "ANCHORECTL_USERNAME")
  username: ""

  # the Anchore Enterprise user's login password (env var: "ANCHORECTL_PASSWORD")
  password: ""

For example, with a quickstart deployment of Anchore Enterprise running on your local system:

  url: "http://localhost:8228"
  username: "admin"
  password: "foobar"

A convenient way to see your changes taking effect is to instruct anchorectl to output DEBUG level logs to the screen using the -vv flag, which will display the full configuration that the tool is using (including the options you set, plus all the defaults and additional configuration file options available).

# anchorectl -vv
[0000]  INFO anchorectl version: 1.6.0
[0000] DEBUG application config:
  url: http://localhost:8228
  username: admin
  password: '******'
  account: ""
  skip-api-version-check: false
  config: /your/homedir/.anchorectl.yaml
  update:
    check: true
    url: https://anchorectl-releases.anchore.io/anchorectl/releases/latest/metadata.json
  quiet: false
  log:
    level: debug
    file: ""
  debug:
    api: false
    events: false
  http:
    tls-insecure: false
    timeout: 180
    headers: {}
  dev:
    profile-cpu: false
    profile-mem: false
  registry:
    insecure-skip-tls-verify: false
    insecure-use-http: false
    auth: []
  secret-search:
    cataloger:
      enabled: false
      scope: AllLayers
    additional-patterns: {}
    exclude-pattern-names: []
    reveal-values: false
    skip-files-above-size: 1048576
  content-search:
    cataloger:
      enabled: false
      scope: AllLayers
    patterns: {}
    reveal-values: false
    skip-files-above-size: 1048576
  file-contents:
    cataloger:
      enabled: false
      scope: Squashed
    skip-files-above-size: 1048576
    globs: []

NOTE: if you would like to capture the full default configuration as displayed when running with -vv, you can paste that output as the contents of your .anchorectl.yaml, and then work with the settings for full control.

Once you verify that your baseline configuration parameters are set, the tools is correctly consuming them, we recommend a quick test to confirm that your enterprise deployment is accessible and the tool is ready to go:

# anchorectl system status
 ✔ Status system                                                                                                                                                                                                                                                           
┌─────────────────┬────────────────────┬─────────────────────────────┬──────┬────────────────┬────────────┬──────────────┐
│ SERVICE         │ HOST ID            │ URL                         │ UP   │ STATUS MESSAGE │ DB VERSION │ CODE VERSION │
├─────────────────┼────────────────────┼─────────────────────────────┼──────┼────────────────┼────────────┼──────────────┤
│ rbac_manager    │ anchore-quickstart │ http://rbac-manager:8228    │ true │ available      │ 500        │ 5.0.0        │
│ analyzer        │ anchore-quickstart │ http://analyzer:8228        │ true │ available      │ 500        │ 5.0.0        │
│ notifications   │ anchore-quickstart │ http://notifications:8228   │ true │ available      │ 500        │ 5.0.0        │
│ policy_engine   │ anchore-quickstart │ http://policy-engine:8228   │ true │ available      │ 500        │ 5.0.0        │
│ reports         │ anchore-quickstart │ http://reports:8228         │ true │ available      │ 500        │ 5.0.0        │
│ reports_worker  │ anchore-quickstart │ http://reports-worker:8228  │ true │ available      │ 500        │ 5.0.0        │
│ rbac_authorizer │ anchore-quickstart │ http://rbac-authorizer:8228 │ true │ available      │ 500        │ 5.0.0        │
│ simplequeue     │ anchore-quickstart │ http://queue:8228           │ true │ available      │ 500        │ 5.0.0        │
│ apiext          │ anchore-quickstart │ http://api:8228             │ true │ available      │ 500        │ 5.0.0        │
│ catalog         │ anchore-quickstart │ http://catalog:8228         │ true │ available      │ 500        │ 5.0.0        │
└─────────────────┴────────────────────┴─────────────────────────────┴──────┴────────────────┴────────────┴──────────────┘

Configuring Catalogers for Distributed Analysis Mode

If you intend to use anchorectl in Distributed Analysis mode, then you’ll need to enable two additional catalogers (secret-search, and file-contents) to mirror the behavior of Anchore Enterprise defaults, when performing an image analysis in Centralized Analysis mode. Below are the settings to mirror the Anchore Enterprise defaults.

  secret-search:
    cataloger:
      enabled: true
      scope: Squashed
    additional-patterns: {}
    exclude-pattern-names: []
    reveal-values: false
    skip-files-above-size: 10000
  content-search:
    cataloger:
      enabled: false
      scope: Squashed
    patterns: {}
    reveal-values: false
    skip-files-above-size: 10000
  file-contents:
    cataloger:
      enabled: true
      scope: Squashed
    skip-files-above-size: 1048576
    globs: ['/etc/passwd']

For more information on using anchorectl in Distributed Analysis mode, see Concepts: Image Analysis and AnchoreCTL Usage: Images.

4 - Content Hints

For an overview of the content hints and overrides features, see the feature overview

Enabling Content Hints

This feature is disabled by default to ensure that images may not exercise this feature without the admin’s explicit approval.

In each analyzer’s config.yaml file (by default at /config/config.yaml):

Set the enable_hints: true setting in the analyzer service section of config.yaml.

If using the default config.yaml included in the image, you may instead set an environment variable (e.g for use in our provided config for Docker Compose):

ANCHORE_HINTS_ENABLED=true environment variable for the analyzer service.

For Helm: see the Helm installation instructions for enabling the hints file mechanism when deploying with Helm.

5 - Custom Certificate Authority

If a custom CA certificate is required to access an external resource then the Trust Store in the Anchore container needs to be updated in two places.

1. The operating system provided trust store.
2. The Certifi trust store.

The operating system trust store is read by the skopeo utility and python requests library that is used to access container registries to read manifests and pull image layers.

If your container registry users a custom CA then you can update the trust store to trust the certificate or use the –insecure option when configuring the registry.

To add a certificate to the operating system trust store the CA certificate should be placed in the /etc location that is appropriate for the container image being used.

• For anchore 0.2.X and earlier, the base container is CentOS 7, which stores certs in /etc/pki/ca-trust/source/anchors/ and requires user to run update-ca-trust command as root to update the system certs.
• For anchore 0.3.X, the base container is Ubuntu 18.04, which stores certs in /usr/local/share/ca-certificates/<new_directory>/<new_cert_file>.crt and requires the user to run update-ca-certificates command as root to update the system certs.
• For anchore 0.4.X and newer, the base container is Red Hat Universal Base Image 7, which stores certs in /etc/pki/ca-trust/source/anchors/ and requires user to run update-ca-trust command as root to update the system certs.
• For anchore 0.7.X and newer, the base container is Red Hat Universal Base Image 8, which stores certs in /etc/pki/ca-trust/source/anchors/ and requires user to run update-ca-trust command as root to update the system certs.
• For anchore 4.5.X and newer, the base container is Red Hat Universal Base Image 9, which stores certs in /etc/pki/ca-trust/source/anchors/ and requires user to run update-ca-trust command as root to update the system certs.

Certifi is a curated list of trusted certificate authorities that is used by the Python requests HTTP client library. The Python requests library is used by Anchore for all HTTP interactions, including when communicating with Anchore Feed service, when webhooks are sent to a TLS enabled endpoint and inbetween Anchore services if TLS has been configured. To update the Certifi trust store the CA certificate should be appended onto the cacert.pem file provided by the Certifi library.

• For Enterprise 2.3.x and newer, the base container is Red Hat Universal Base Image 8, certifi’s cacert.pem is installed in /usr/local/lib/python3.6/site-packages/certifi/cacert.pem

• For Enterprise 3.0.x and newer, the Python version was updated from 3.6 to 3.8, certifi’s cacert.pem is installed in /usr/local/lib/python3.8/site-packages/certifi/cacert.pem

• For Enterprise 4.4.x and newer, the Python version was updated from 3.8 to 3.9, certifi’s cacert.pem is installed in /usr/local/lib/python3.9/site-packages/certifi/cacert.pem

• For Enterprise 4.5.x and newer, the base image was upgraded to RHEL UBI9 and Python was moved to a virtual environment, certifi’s cacert.pem is installed in /home/anchore/venv/lib/python3.9/site-packages/certifi/cacert.pem

The following Dockerfile illustrates an example of how this general process can be automated to produce your own container with a new custom CA cert installed.

Dockerfile

FROM docker.io/anchore/anchore-engine:v0.4.0

USER root:root
COPY ./custom-ca.pem /usr/local/lib/python3.6/site-packages/certifi/
RUN update-ca-trust
RUN /usr/bin/cat /usr/local/lib/python3.6/site-packages/certifi/custom-ca.pem >> /usr/lib/python3.6/site-packages/certifi/cacert.pem
USER anchore:anchore

Build Custom Image

sudo docker build -t anchore/anchore-engine:v0.4.0custom .

6 - Environment Variables

Environment variable references may be used in the Anchore config.yaml file to set values that need to be configurable during deployment.

Using this mechanism a common configuration file can be used with multiple Anchore instances with key values being passed using environment variables.

The config.yaml configuration file is read by Anchore and any references to variables prefixed with ANCHORE will be replaced by the value of the matching environment variable.

For example in the sample configuration file the host_id parameter is set be appending the ANCHORE_HOST_ID variable to the string dockerhostid

host_id: 'dockerhostid-${ANCHORE_HOST_ID}'

Notes:

1. Only variables prefixed with ANCHORE will be replaced
2. If an environment variable is referenced in the configuration file but not set in the environment then a warning will be logged
3. It is recommend to use curly braces, for example ${ANCHORE_PARAM} to avoid potentially ambiguous cases

Passing Environment Variables as a File

Environment Variables may also be passed as a file contained key value pairs.

ANCHORE_HOST_ID=myservice1
ANCHORE_LOG_LEVEL=DEBUG

The system will check for an environment variable named ANCHORE_ENV_FILE if this variable is set then Anchore will attempt to read a file at the location specified in this variable.

The Anchore environment file is read before any other Anchore environment variables so any ANCHORE variables passed in the environment will override the values set in the environment file.

7 - False Positive Management

When Anchore analyzes an image, it reports a Software Bill of Materials (SBOM) to be stored and later scanned in order to match package metadata against known vulnerabilities. One aspect of the SBOM is a best effort guess of the CPE (Common Platform Enumeration) for a given package. The Anchore analyzer builds a list of CPEs for each package based on the metadata that is available (ex. for Java packages, the manifest, which contains multiple different version specifications among other metadata), but sometimes gets this wrong.

For example, Java Spring packages are generally reported as follows:

• Spring Core, version 5.1.4
  • cpe:2.3:a:*:spring-core:5.1.4:*:*:*:*:*:*:*

However, since Spring is a framework built by Pivotal Software, the CPE referenced in the NVD database looks more like:

• cpe:2.3:a:pivotal_software:spring_security:5.1.4:*:*:*:*:*:*:*

To facilitate this correction, Anchore provides the False Positive Management feature. Now, a user can provide a correction that will update a given package’s metadata so that attributes (including CPEs) can be corrected when Anchore does a vulnerability scan

Using the above example, a user can add a correction as using anchorectl or via HTTP POST to the /corrections endpoint:

{
  "description": "Update Spring Core CPE",
  "match": {
    "type": "java",
    "field_matches": [
        {
            "field_name": "package",
            "field_value": "spring-core"
        },
        {
            "field_name": "implementation-version",
            "field_value": "5.1.4.RELEASE"
        }
    ]
  },
  "replace": [
      {
          "field_name": "cpes",
          "field_value": "cpe:2.3:a:pivotal_software:spring_security:5.1.4:*:*:*:*:*:*:*"
      }
  ],
  "type": "package"
}

JSON Reference:

• description: A description of the correction being added (for note taking purposes)
• type: The type of correction being added. Currently only “package” is supported
• match:
  • type: The type of package to match upon. Supported values are based on the type of content available to images being analyzed (ex. java, gem, python, npm, os, go, nuget)
  • field_matches: A list of name/value pairs based on which package metadata fields to match this correction upon
    • The schema of the fields to match can be found by outputting the direct JSON content for the given content type:
      • Ex. Java Package Metadata JSON:

      {
          "cpes": [
              "cpe:2.3:a:*:spring-core:5.1.4.RELEASE:*:*:*:*:*:*:*",
              "cpe:2.3:a:*:spring-core:5.1.4.RELEASE:*:*:*:*:java:*:*",
              "cpe:2.3:a:*:spring-core:5.1.4.RELEASE:*:*:*:*:maven:*:*",
              "cpe:2.3:a:spring-core:spring-core:5.1.4.RELEASE:*:*:*:*:*:*:*",
              "cpe:2.3:a:spring-core:spring-core:5.1.4.RELEASE:*:*:*:*:java:*:*",
              "cpe:2.3:a:spring-core:spring-core:5.1.4.RELEASE:*:*:*:*:maven:*:*"
          ],
          "implementation-version": "5.1.4.RELEASE",
          "location": "/app.jar:BOOT-INF/lib/spring-core-5.1.4.RELEASE.jar",
          "maven-version": "N/A",
          "origin": "N/A",
          "package": "spring-core",
          "specification-version": "N/A",
          "type": "JAVA-JAR"
      }
      

• replace: a list of field name/value pairs to replace.

Note: if a new field is specified here, it will be added to the content output when the correction is matched. See below for additional functionality around CPEs.

To add the above JSON using anchorectl the following command can be used anchorectl correction add -i path-to-file.json

The result of the correction can be checked using the image content command of anchorectl. For example to see our above java correction we would run anchorectl image content -t java Image_sha256_ID -o json We would see the spring-core package returned as having the CPE cpe:2.3:a:pivotal_software:spring_security:5.1.4:*:*:*:*:*:*:*

Note: Don’t forget to replace the Image_sha256_ID with the image ID you’re trying to test.

Corrections may be updated and deleted via the API as well. Creation of a Correction generates a UUID that may be used to reference that Correction later. Refer to the Enterprise Swagger spec for more details.

CPE Templating

CPE replacement can be templated based on the other fields of the package as well. In the above example, a replacement could have been provided as follows:

{
  "field_name": "cpes",
  "field_value": "cpe:2.3:a:pivotal_software:spring_security:{implementationVersion}:*:*:*:*:*:*:*" 
}

For the “cpes” field only, Anchore Enterprise can recognize a templated field via curly braces “{}”. Package JSON keys contained here will be replaced with their corresponding value.

Vulnerability Matching Configuration

Search by CPE can be globally configured per supported ecosystem via the anchore enterprise policy engine config. The default enables search by cpe for all ecocystems except for javascript (since NPM package vulnerability reports are exhaustively covered by the GitHub Security Advisory Database).

A fully-specified default config is as below:

policy_engine:
    vulnerabilities:
      matching:
        default:
          search:
            by_cpe:
              enabled: true
        ecosystem_specific:
          dotnet:
            search:
              by_cpe:
                enabled: true
          golang:
            search:
              by_cpe:
                enabled: true
          java:
            search:
              by_cpe:
                enabled: true
          javascript:
            search:
              by_cpe:
                enabled: false
          python:
            search:
              by_cpe:
                enabled: true
          ruby:
            search:
              by_cpe:
                enabled: true
          stock:
            search:
              by_cpe:
                # Disabling search by CPE for the stock matcher will entirely disable binary-only matches 
                # and is *NOT ADVISED*
                enabled: true

A shorter form of the default config is:

policy_engine:
    vulnerabilities:
      matching:
        default:
          search:
            by_cpe:
              enabled: true
        ecosystem_specific:
          javascript:
            search:
              by_cpe:
                enabled: false

If disabling search by CPE for all GitHub covered ecosystems is desired, the config would look like:

policy_engine:
    vulnerabilities:
      matching:
        default:
          search:
            by_cpe:
              enabled: false
        ecosystem_specific:
          stock:
            search:
              by_cpe:
                enabled: true

It is important to note that the GitHub Provider must be enabled in the feed service when disabling search by CPE or else there will be no vulnerability reporting for those ecosystems.

8 - Feeds

Introduction

In this section, you’ll learn about the requirements for installing Anchore Enterprise Feeds and configuring its components. You can read more about how Feeds works in the feature overview.

Requirements

Network Ingress

Anchore Enterprise Feeds exposes a RESTful API by default on port 8228 however this port can be remapped.

The policy engine service(s) in your Anchore deployment will use this endpoint and API to get the latest updates aggregated into Anchore-specific formats.

Network Egress

Anchore Enterprise Feeds requires access to the upstream data feeds from supported Linux distributions and package registries. See Feeds Endpoints for the full list of the endpoints.

Database

Ruby Gems project publishes package data as a PostgreSQL dump. Enabling the gem driver in Anchore Enterprise Feeds will increase the load on the PostgreSQL database used by the service. We recommend using a different PostgreSQL instance for the gem driver to avoid load spikes and interruptions to the service. The database endpoint for the gem driver can be configured using services.feeds.drivers.gem.db_connect parameter in config.yaml

Vulnerability Provider Configuration

Some of the vulnerability data providers will require additional configuration steps, like retrieving an API or token key from a provider.

GitHub Provider

The GitHub provider requires generation of a Personal Access Token (PAT) which is associated with a GitHub Account. To generate this token, the user will be required to log in to GitHub and use the following url: https://github.com/settings/tokens/new

There are no special permissions needed for the token, so there is no need to select any scopes. A note may be added as a reminder for what the token is being used. At the bottom of the tokens page, click the Generate token button which will show a new page displaying the generated token. Make sure you copy that token and use it to configure the GitHub provider, the token will only be seen once in the GitHub interface.

Use the new token in config.yaml in the feeds section: for all components of the deployment:

  feeds:
      ...
      github:
        enabled: true
        token: "****************************************"

It is also possible to set the token with the following environment variable: ANCHORE_ENTERPRISE_FEEDS_GITHUB_DRIVER_TOKEN. Note that this is only valid when using the config.yaml provided in the image due to that file referencing them explicitly as replacement values.

Microsoft Security Response Center (MSRC) Provider

The MSRC provider is not enabled by default so must be explicitly enabled in order to execute. This provider is required in order for Anchore to get data about vulnerabilities and fixes for Microsoft Windows and its associated KBs (patches).

• For quickstart and deployments using docker-compose.yaml, find the feeds service definition and uncomment or add the following environment variables:

  services:
    ...
    feeds:
    ...
      environment:
      ...
      - ANCHORE_ENTERPRISE_FEEDS_MSRC_DRIVER_ENABLED=true
  

• For deployments using config.yaml, update the feeds configuration section:

  services:
    ...
    feeds:
    ...
      drivers:
        msrc:
          enabled: true
  

Anchore Vulnerability Match Exclusion Provider

The Anchore Vulnerability Match Exclusion provider is available to users that meet specific license requirements. Please contact Anchore Support for details.

• For quickstart and deployments using docker-compose.yaml, find the feeds service definition and uncomment or add the following environment variables:

  services:
    ...
    feeds:
    ...
      environment:
      ...
      - ANCHORE_ENTERPRISE_FEEDS_VULN_MATCH_EXCLUSION_DRIVER_ENABLED=true
  

• For deployments using config.yaml, update the feeds configuration section:

  services:
    ...
    feeds:
    ...
      drivers:
        anchore_match_exclusion:
          enabled: true
  

Ubuntu Provider

The Ubuntu https endpoint is heavily throttled and can be seen as unreliable.
A configuration option is provided to switch the Ubuntu provider’s access to the git protocol endpoint instead.
This endpoint has improved reliability but is not validated or encrypted and uses a different port than https. Please contact Anchore Support for additional details.

NVD Provider

The NVD provider uses the v2 NVD API and it is recommended passing an API key to avoid more aggressive throttling.
Request an API key via https://nvd.nist.gov/developers/request-an-api-key.

Use the new api key in config.yaml in the feeds section: for all components of the deployment:

  feeds:
      ...
      nvd:
        enabled: true
        api_key: "****************************************"

It is also possible to set the token with the following environment variable: ANCHORE_ENTERPRISE_FEEDS_NVD_PROVIDER_API_KEY. Note that this is only valid when using the config.yaml provided in the image due to that file referencing them explicitly as replacement values.

8.1 - Migrating Data Between On-Premises Anchore Enterprise Feeds Installations

The On-Premises Anchore Enterprise Feeds has two high level functions:

• Gather vulnerability data from external sources, normalize and persist the data
• Serve persisted normalized vulnerability data from via an API

These two high level functions are decoupled and can be executed independently of each other. This design allows the service to operate in a Read-Only mode and service API requests with the vulnerability data in the service’s database. It is used for running Anchore Enterprise in Air-Gapped mode

Warning: The following process deletes the database of the Read-Only installation. Ensure the database is not used by other services such as Anchore Engine

Start with the public network

1. Install Anchore Enterprise Feeds in a public network with access to internet, see network requirements
2. Start the service and use the tasks API to query the status of the FeedSyncTask that should have started
3. Wait for the FeedSyncTask to complete and populate the database
4. Stop the service
5. Create a database dump using pg_dump tool pg_dump -h <public-network-db-hostname> -p 5432 -U postgres -Fc -v -f anchore-enterprise-feeds.dump postgres
6. Copy it to a location where it can be restored from in the following steps

Next, switch to the private network where Anchore Engine is installed

1. Install Anchore Enterprise Feeds in the private network alongside Anchore Engine, don’t start the service.

2. Configure Read-Only mode by setting api_only: True property in the config.yaml

feeds:
  ...
  api_only: True
  ...

3. Restore the database from the anchore-enterprise-feeds.dump file created in the previous section

pg_restore -h <private-network-db-hostname> -p 5432 -U postgres -C -d postgres anchore-enterprise-feeds.dump

4. Start the service

Note: PostgreSQL version must be the same across both databases for the pg_dump and pg_restore utility to work correctly

9 - Integration

Integrations that Anchore Supports

• CI/CD Integration
• Kubernetes Integration
• Image Registry Integrations

9.1 - CI / CD Integration

Anchore Enterprise can be integrated into CI/CD pipelines such as Jenkins to secure the CI/CD pipeline by adding image scanning including not just CVE based security scans but policy based scans that can include checks around security, compliance and operational best practices.

As part of the CI/CD workflow images are submitted to the Anchore Enterprise for analysis. If an image does not pass the policy checks then the build can be failed.

Images that pass the policy check can be promoted to the production registry.

Some users may implement a gating workflow where policy violations fail the build, other users may prefer to allow the build to proceed but provide a warning to the developer and operations team about the violations present in the image.

A plugin is available for Jenkins that supports Freestyle jobs and Pipeline jobs.

For other CI/CD systems Integration is as simple as making two API or CLI calls to the system

1. Submit image for analysis
2. Retrieve image policy status.

These calls can be made using the Anchore CLI or through the Anchore REST API.

9.1.1 - CircleCI

Integrating Anchore Enterprise into your CircleCI pipeline can be accomplished seamlessly using the official Anchore Orb.

Orb source code & usage examples can be found on our GitHub Repository.

To read more about CircleCI orbs, view the official documentation - CircleCi Orbs

9.1.2 - Jenkins

We keep the latest Anchore/Jenkins integration documentation hosted alongside the official Anchore Jenkins plugin module, here.

9.1.3 - GitHub

Image Scanning can be easily integrated into your GitHub Actions pipeline using the official Anchore Scan Action.

Documentation on how to use the action can be found in the official GitHub repository anchore scan-action

Check out our Anchore Scan Action webinar demo

9.1.4 - GitLab

Adding Anchore Scanning to GitLab

The Anchore Enterprise GitLab Scan integration enables vulnerability scanning via Anchore Enterprise in GitLab CI. This includes automatic conversion from the Anchore default vulnerability report format to GitLab’s container scanning format, for display in the GitLab Security & Compliance Vulnerability UI.

Requirements

1. Anchore Enterprise is deployed in your environment, with the API accessible from your GitLab CI environment.
2. Credentials for your GitLab Container Registry are added to Anchore Enterprise, under the Anchore account that you intend to use with GitLab CI. See Registries. For information on what registry/credentials must be added to allow Anchore Enterprise to access your GitLab Container Registry, see https://docs.gitlab.com/ee/user/packages/container_registry/.

Usage

1. In your GitLab project repository, ensure that the following variables are set in Settings > CI/CD > Variables:

• ANCHORECTL_URL
• ANCHORECTL_USERNAME
• ANCHORECLT_PASSWORD

This allows the integration to access your Anchore Enterprise depoyment. The ANCHORECTL_PASSWORD variable should have protected and masked options set to prevent exposure of the variable in job logs.

2. In your project’s .gitlab-ci.yml, include the following snippet anywhere in your CI flow after your target container image has been built and pushed to the GitLab Container Registry that you have made accessible from your Anchore Enterprise deployment.

containerscan:anchorescan:
  stage: test
  image: docker.io/anchore/enterprise-gitlab-scan:v3
  variables:
    ANCHORE_IMAGE: "${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}"
  script:
    - anchore-gitlab-scan > gl-container-scanning-report.json
  artifacts:
    reports:
      container_scanning: gl-container-scanning-report.json

3. After a successful scan, results will be available for review and management using GitLab’s native security features. Navigate to your project’s Security & Compliance > Vulnerability Report UI to review any discovered vulnerabilities in your container image.

Example

For this example, we’ll add two sections to the GitLab CI .gitlab-ci.yml file; the first runs Docker to build an image and push it to the GitLab Container Registry for the project. The second runs Anchore Enterprise vulnerability scanning on the image and converts the output to GitLab’s container scanning format.

Add Dockerfile and .gitlab-ci.yml To Your GitLab Project

Example Dockerfile:

FROM alpine:latest

CMD /bin/sh

Example .gitlab-ci.yml:

# Build our image and push it to our project's container repo
containerbuild:
  stage: build
  image: docker:19.03.12
  variables:
    DOCKER_TLS_CERTDIR: "/certs"
  services:
    - docker:19.03.12-dind
  # See note below on Access Tokens for authenticating to the GitLab Container Registry
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker pull $CI_REGISTRY_IMAGE:latest || true
    - docker build --cache-from $CI_REGISTRY_IMAGE:latest --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA --tag $CI_REGISTRY_IMAGE:latest .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest

# Scan with Anchore Enterprise and convert the results for GitLab's native security display
containerscan:anchorescan:
  stage: test
  image: docker.io/anchore/enterprise-gitlab-scan:v3
  variables:
    ANCHORE_IMAGE: "${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}"
  script:
    - anchore-gitlab-scan > gl-container-scanning-report.json
  artifacts:
    reports:
      container_scanning: gl-container-scanning-report.json

Navigate To Settings > CI/CD; Expand the Variables Section

Set Or Edit Anchore Variables

Trigger GitLab CI

Make a change in the repository that will trigger GitLab CI so that your pipeline runs.

Then, navigate to CI/CD > Pipelines, where you can see details of your pipelines:

Review Scan Report

Navigate to Security & Compliance > Vulnerability Report to view the scan results:

Access Tokens

Authentication with the GitLab Container Registry can use a personal access token or a deploy token. Please see GitLab’s documentation on authentication options.

9.2 - Kubernetes Integration

For installation see - Kubernetes Installation

Anchore Enterprise can be integrated with Kubernetes to ensure that only certified images are started within a Kubernetes POD.

Kubernetes can be configured to use an Admission Controller to validate that the container image is compliant with the user’s policy.

The admission controller can be configured to make a webhook call into Anchore Enterprise. Anchore Enterprise exports a Kubernetes-specific API endpoint and will return the pass of fail response in the form of an ImageReview response.

This approach allows the Kubernetes system to make the final decision on running an container image and does not require installation of any per-node plugins into Kubernetes.

Using native Kubernetes features allows this approach to be used in both on-prem and cloud hosted Kubernetes environments.

9.3 - Registry Integrations

Image Registry Integrations

It makes sense in many cases to integrate a container scanner directly into the container registry so that results are visible directly to the image consumers that may or may not have access to Anchore directly.

Anchore is able to scan images from nearly all image registries, but here we curate a list of registries for which there is a direct integration with Anchore such that the registry can either directly initiate scans on Anchore and/or present those results directly to its users.

• Harbor Project and the Harbor Scanner Adapter for Anchore

10 - Malware Scans of Images

Malware Scanning Overview

See Malware Scanning for an overview of the feature and how it works. This section is for configuration of scan behavior.

Customizing the analyzer_config.yaml requires a restart of the analyzer container. The typical process is to mount it externally into /anchore_service/analyzer_config.yaml from a host volume or as a ConfigMap in Kubernetes and all analyzers in the deployment share the same configuration.

Enabling & Disabling Malware Scans

Each analyzer needs to have it’s analyzer_config.yaml file updated to include:

malware:
  clamav:
    enabled: true
    db_update_enabled: true

malware.clamav.enabled = true will enable the analyzer that runs the scan. If not enabled, the analyzer will run but will not execute a ClamAV scan so no scan results will be reported.

:warning: Malware Scanning currently only supports image sizes up to 4gb.

Disabling DB Updates for ClamAV

The db_update_enabled property of the malware.clamav object shown above in the analyzer_config.yaml controls whether the analyzer will invoke a refreshclam call prior to each analysis execution. By default it is enabled and should be left on for up-to-date scan results. The db version is returned in the metadata section of the scan results available from the Anchore Enterprise API.

You can disable the update if you want to mount an external volume to provide the db data in /home/anchore/clamav/db inside the container (must be read-write for the anchore user) This can be used to cache or share a db across multiple analyzers (e.g. using AWS EFS) or to support air-gapped deployments where the db cannot be automatically updated from deployment itself.

Advanced Configuration

The path for the db and db update configuration are also available as environment variables inside the analyzer containers. These should not need to be used in most cases, but for air-gapped or other installation where the default configuration is not sufficient they are available for customization.

For most cases, anchore uses the default values for the clamscan and freshclam invocations. If you would like to override any of the default values of those commands or replace existing ones, you can add the following to the analyzer_config.yaml:

malware:
  clamav:
    clamscan_args:
      - max-filesize=1000m
      - max-scansize=1000m
    freshclam_args:
      - datadir=/tmp/different/datadir

Please note that the value above will be passed directly to the corresponding commands, e.g.:

clamscan --suppress-ok-results --infected --recursive --allmatch --archive-verbose --tempdir={tempdir} --database={database} --max-filesize=1000m --max-scansize=1000m <path_to_tar>

11 - Max Image Size

Setting Size Filter

As of v3.0, Anchore Enterprise can be configured to have a size limit for images being added for analysis. Images that exceed the configured maximum size will not be added to Anchore and the catalog service will log an error message providing details of the failure. This size limit is applied when adding images to anchore via the api/anchoreclt, tag subscriptions, and repository watchers.

The max size feature is disabled by default but can be enabled via max_compressed_image_size_mb in the configuration file, which represents the size limit in MB of the compressed image. Values less than 0 will disable the feature and allow images of any size to be added to Anchore. A value of 0 will be enforced and prevent any images from being added. Non-integer values will cause bootstrap of the service to fail. If using compose with the default config, this can be set through the ANCHORE_MAX_COMPRESSED_IMAGE_SIZE_MB env variable on the catalog service. If using helm, it can be defined in the values file via anchoreGlobal.maxCompressedImageSizeMB

12 - Monitoring

Anchore Enterprise exposes prometheus metrics in the API of each service if the config.yaml that is used by that service has the metrics.enabled key set to true.

Each service exports its own metrics and is typically scraped by a Prometheus installation to gather the metrics. Anchore does not aggregate or distribute metrics between services. You should configure your Prometheus deployment or integration to check each Anchore service’s API using the same port it exports for the /metrics route.

Example with Quickstart docker-compose.yaml

The following example prometheus.yaml file is used to configure prometheus jobs that corresponds to the docker-compose.yaml that is bundled in the Enterprise container.

To use this example, do the following.

1. Uncomment the anchore-prometheus service in the docker-compose.yaml, as shown.

anchore-prometheus:
      image: docker.io/prom/prometheus:latest
      depends_on:
       - engine-api
      volumes:
       - ./anchore-prometheus.yml:/etc/prometheus/prometheus.yml:z
      logging:
       driver: "json-file"
       options:
        max-size: 100m
      ports:
       - "9090:9090"

2. Enable metrics for each enterprise service as shown.

sed -i 's/ANCHORE_ENABLE_METRICS=false/ANCHORE_ENABLE_METRICS=true/g' docker-compose.yaml

3. Create an anchore-prometheus.yaml file in the same directory as the docker-compose.yaml with the following content:

# This Prometheus configuration corresponds to the service names and ports in the docker-compose.yaml provided in the Enterprise docker image. Adjust names and ports accordingly for other environments.

global:
  scrape_interval: 15s
  scrape_timeout: 10s
  evaluation_interval: 15s
alerting:
  alertmanagers:
  - static_configs:
    - targets: []
    scheme: http
    timeout: 10s
scrape_configs:
- job_name: anchore-api
  scrape_interval: 15s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  static_configs:
  - targets:
    - api:8228
  basic_auth:
    username: admin
    password: foobar

- job_name: anchore-catalog
  scrape_interval: 15s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  static_configs:
  - targets:
    - catalog:8228
  basic_auth:
    username: admin
    password: foobar

- job_name: anchore-simplequeue
  scrape_interval: 15s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  static_configs:
  - targets:
    - queue:8228
  basic_auth:
    username: admin
    password: foobar

- job_name: anchore-analyzer
  scrape_interval: 15s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  static_configs:
  - targets:
    - analyzer:8228
  basic_auth:
    username: admin
    password: foobar

- job_name: enterprise-rbac-manager
  scrape_interval: 15s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  static_configs:
  - targets:
    - rbac-manager:8228
  basic_auth:
    username: admin
    password: foobar

- job_name: anchore-policy-engine
  scrape_interval: 15s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  static_configs:
  - targets:
    - policy-engine:8228
  basic_auth:
    username: admin
    password: foobar

- job_name: enterprise-feeds
  scrape_interval: 15s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  static_configs:
  - targets:
    - feeds:8228
  basic_auth:
    username: admin
    password: foobar

- job_name: enterprise-ui
  scrape_interval: 15s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  static_configs:
  - targets:
    - ui:3000

4. Type: docker-compose up -d.
5. Open a browser to: http://localhost:9090 to see metrics from the default prometheus console.

Monitoring in Kubernetes and/or Helm Chart

Prometheus is very commonly used for monitoring Kubernetes clusters. Prometheus is supported by core Kuberenetes services. There are many guides on using Prometheus to monitor a cluster and services deployed within, and also many other monitoring systems can consume Prometheus metrics.

The Anchore Helm Chart includes a quick way to enable the Prometheus metrics on each service container:

• Set: helm install --name myanchore anchore/anchore-engine --set anchoreGlobal.enableMetrics=true

• Or, set it directly in your customized values.yaml

The specific strategy for monitoring services with prometheus is outside the scope of this document. But, because Anchore exposes metrics on the /metrics route of all service ports, it should be compatible with most monitoring approaches (daemon sets, side-cars, etc).

Metrics of Note

Anchore services export a range of metrics. The following list shows some Anchore services that can help you determine the health and load of an Anchore deployment.

• anchore_queue_length, specifically for queuename: “images_to_analyze”
  • This is the number of images pending analysis, in the not_analyzed state.
  • As this number grows you can expect longer analysis times.
  • Adding more analyzers to a system can help drain the queue faster and keep wait times to a minimum.
  • Example: anchore_queue_length{instance=“engine-simpleq:8228”,job=“anchore-simplequeue”,queuename=“images_to_analyze”}.
  • This metric is exported from all simplequeue service instances, but is based on the database state so they should all present a consistent view of the length of the queue.
• anchore_monitor_runtime_seconds_count
  • These metrics, one for each monitor, record the duration of the async processes as they execute on a duty cycle.
  • As the system grows, these will become longer to account for more tags to check for updates, repos to scan for new tags, and user notifications to process.
• anchore_tmpspace_available_bytes
  • This metric tracks the available space in the “tmp_dir” location for each container. This is most important for the instances that are analyzers where this can indicate how much disk is being used for analysis and how much overhead there is for analyzing large images.
  • This is expected to be consumed in cycles, with usage growing during analysis and then flushing upon completion. A consistent growth pattern here may indicate left over artifacts from analysis failures or a large layer_cache setting that is not yet full. The layer cache (see Layer Caching) is located in this space and thus will affect the metric.
• process_resident_memory_bytes
  • This is the memory actually consumed by the instance, where each instance is a service process of Anchore. Anchore is fairly memory intensive for large images and in deployments with lots of analyzed images due to lots of json parsing and marshalling, so monitoring this metric will help inform capacity requirements for different components based on your specific workloads. Lots of variables affect memory usage, so while we give recommendations in the Capacity Planning document, there is no substitute for profiling and monitoring your usage carefully.

13 - Network Proxies

As covered in the Network sections of the requirements document, Anchore requires three categories of network connectivity.

• Registry Access Network connectivity, including DNS resolution, to the registries from which Anchore needs to download images.

• Feed Service Anchore synchronizes feed data such as operating system vulnerabilities (CVEs) from Anchore Cloud Service. See Feeds Overview for the full list of endpoints.

• Access to Anchore Internal Services Anchore is composed of six smaller services that can be deployed in a single container or scaled out to handle load. Each Anchore service should be able to connect the other services over the network.

In environments were access to the public internet is restricted then a proxy server may be required to allow Anchore to connect to Anchore Cloud Feed Service or to a publicly hosted container registry.

Anchore can be configured to access a proxy server by using environment variables that are read by Anchore at run time.

• https_proxy: Address of the proxy service to use for HTTPS traffic in the following form: {PROTOCOL}://{IP or HOSTNAME}:{PORT} eg. https://proxy.corp.example.com:8128

• http_proxy:
  Address of the proxy service to use for HTTP traffic in the following form: {PROTOCOL}://{IP or HOSTNAME}:{PORT}
  eg. http://proxy.corp.example.com:8128

• no_proxy:
  Comma delimited list of hostnames or IP address which should be accessed directly without using the proxy service. eg. localhost,127.0.0.1,registry,example.com

Environment Variables to Control Proxy Behavior

• Setting the endpoints to HTTP proxy:
  • Set both HTTP_PROXY and http_proxy environment variables for regular HTTP protocol use.
  • Set both HTTPS_PROXY and https_proxy environment variables for HTTP + TLS (HTTPS) protocol use.
• Setting endpoints to exclude from proxy use:
  • Set both NO_PROXY and no_proxy environment variables to exclude those domains from proxy use defined in the preceding proxy configurations.

If using Docker Compose these need to be set in each service entry.

If using Helm Chart, set these in the extraEnv entry for each service.

Notes:

• Do not use double quotes (") around the proxy variable values.

Authentication

For proxy servers that require authentication the username and password can be provided as part of the URL:

eg. https_proxy=https://user:[email protected]:8128

If the username or password contains and non-url safe characters then these should be urlencoded.

For example:

The password F@oBar! would be encoded as F%40oBar%21

Setting Environment Variables

Docker Compose: https://docs.docker.com/compose/environment-variables/

Kubernetes: https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/

Deployment Architecture Notes

When setting up a network proxy, keep in mind that you will need to explicitly allow inter-service communication within the anchore engine deployment to bypass the proxy, and potentially other hostnames as well (e.g. internal registries) to ensure that traffic is directed correctly. In general, all anchore engine service endpoints (the URLs for enabled services in the output of an ‘anchorectl system status’ command) as well as any internal registries (the hostnames you may have set up with ‘anchorectl registry add –username …’ or as part of an un-credentialed image add ‘anchorectl image add registry:port/….’), should not be proxied (i.e. added to the no_proxy list, as described above).

If you wish to tune this further, below is a list of each component that makes an external URL fetch for various purposes:

• Catalog: makes connections to image registries (any host added via ‘anchorectl registry add’ or directly via ‘anchorectl image add’)
• Analyzer: same as catalog
• Policy Engine: by default, makes HTTPS connection to https://toolbox.anchore.io for the public GrypeDB data unless a local feed service is deployed.
• Feeds: connects to public and Anchore-private data sources for downloading vulnerability data. See Feed Drivers for the full list.

14 - Notifications

Glossary

Event

An information packet generated by Anchore Enterprise service to indicate some activity.

Endpoint

External tool capable of receiving messages such as Slack, GitHub, Jira, MS Teams, email or webhook.

Endpoint Configuration

Connection information of the endpoint used for sending messages.

Selector

Criteria for selecting events to be notified.

Overview

Anchore Enterprise includes Notifications service to alert external endpoints about the system’s activity. Services that make up Anchore Enterprise generate events to record significant activity, such as an update, in the policy evaluation result or vulnerabilities for a tag, or an error analyzing an image. This service provides a mechanism to selectively notify events of interest to supported external endpoints. The actual notification itself depends on the endpoint - formatted message to Slack, email and MS Teams endpoints, tickets in GitHub and Jira endpoints, and JSON payload to webhook endpoint.

Installation

Anchore Enterprise Notifications is included with Anchore Enterprise, and is installed by default when deploying a trial quickstart with Docker Compose, or a production deployment Kubernetes.

Configuration

Enterprise Notifications Service

The service loads configuration from the notifications section of the config.yaml. See the following snippet of the configuration:

...
services:
  notifications:
    enabled: true
    require_auth: true
    endpoint_hostname: "<hostname>"
    listen: '0.0.0.0'
    port: 8228
    cycle_timers:
      notifications: 30
    # Set Anchore Enterprise UI base URL to receive UI links in notifications
    ui_url: "<enterprise-ui-url>"
    authorization_handler: external
    authorization_handler_config:
      endpoint: "<rbac-authorizer-endpoint>"

The cycle_timers -> notifications controls how often the service checks for events in the system that need to be processed for notifications. Default is every 30 seconds.

The ui_url is used for constructing links to Enterprise UI pages in the notifications. Configure this property to the Enterprise UI’s base URL. This URL should be accessible from the endpoint receiving the notification for the links to work correctly. If the value is not set, the notification message is still be sent to the endpoint, but it won’t contain a clickable link to the Enterprise UI.

Note: Any changes to the configuration requires a restart of the service for the updates to take effect.

RBAC Permissions

In an Anchore Enterprise RBAC-enabled deployment, the table below lists the required actions and containing roles:

External Tools

To send notifications to an external tool/endpoint, the service requires connection information to that endpoint. See the following for the required information for each endpoint:

• Slack
• GitHub
• Jira
• SMTP
• Microsoft Teams

Concepts

Endpoint Status

All endpoints in the Notifications service can be toggled as Enabled or Disabled. The endpoint status reflects the eabled or disabled state. By default, the status for all endpoints is enabled by default. Setting the endpoint status to disabled stops all notifications from going out to any configurations of that specific endpoint. This is a system-wide setting that can only be updated by the admin account. It is read-only for remaining accounts.

Endpoint Configuration

The endpoint configuration is the connection information such as URL, user credentials, and so on for an endpoint. The service allows multiple configurations per endpoint. Endpoint configurations are scoped to the account.

Selector

The services provides a mechanism to selectively choose notifications and route them to a configured endpoint. This is achieved using a Selector, which is a collection of filtering criteria. Each event is processed against a Selector to determine whether it is a match or not. If the Selector matches the event, a notification is sent to the configured endpoint by the service.

For a quick list of useful notifications and associated Selector configurations, see Quick Selection.

A Selector encapsulates the following distinct filtering criteria: scope, level, type, and resource type. Some allow a limited set of values, and others wildcards. The value for each criteria has to be set for the matching to compute correctly.

Scope

Allowed values: account, global

Events are scoped to the account responsible for the event creation.

• account scope matches events associated with a user account.
• global scope matches events of any and all users. global scope is limited to admin account only. Non-admin account users can only specify account as the scope.

Level

Allowed values: info, error, *

Events are associated with a level that indicates whether the underlying activity is informational or resulted in an error. - info matches informational events such as policy evaluation or vulnerabilities update, image analysis completion and so on.

• error matches failures such as image analysis failure.
• * will match all events.

Type

Allowed values: strings with or without regular expressions

Event types have a structured format <category>.<subcategory>.<event>. Thus, * matches all types of events. Category is indicative of the origin of the event.

• system.* matches all system events.
• user.* matches events that are relevant to individual consumption.
• Omitting an asterisk will do an exact match. See the GET /event_types route definition in the external API for the list of event types.

Resource Type

Allowed values: *, image_digest, image_tag, image_reference, repository, feeds, feed, feed_group, *

In most cases, events are generated during an operation involving a resource. Resource type is metadata of that resource. For instance image_tag is the resource type in a policy evaluation update event. * matches all resource types if you are uncertain what resource type to use.

Quick Selection

The following Selector configurations are for notifying a few interesting events.

See it in action

Enterprise UI

To learn more about configuring Notifications service in the Enterprise UI, see Notifications.

14.1 - Slack

Notifications to Slack are in the form of messages to a channel.

Requirements

Do the following to receive Slack notifications.

1. An Incoming Webhook URL is required. Follow instructions to create one.
2. Copy the Incoming Webhook URL.
3. Create a Slack endpoint configuration in the Notifications service either via Enterprise UI or the API directly.

14.2 - GitHub

Notifications to GitHub are in the form of new issues in a repository.

Requirements

Do the following to receive GitHub notifications.

1. Provide the following required GitHub account and repository-related information.
   • Username of the account for creating issues.
   • Token with repository access. Follow instructions to create a new Access Token in the account used for creating issues.
   • Name of the repository where the issues are created.
   • Owner of the repository.
   • (Optional) GitHub API URL, defaults to https://api.github.com if not specified.
   • (Optional) milestone assigned to the issue.
   • (Optional) one or more labels assigned to the issue.
   • (Optional) one or more GitHub users assigned to the issue.
2. Create a GitHub endpoint configuration in the Notifications service either via Enterprise UI, or the API directly.

14.3 - Jira

Notifications to Jira are in the form of new issues in a project.

Requirements

Do the following to receive Jira notifications.

1. Provide the following required Jira account and project related information.
   • URL of the Jira project.
   • Username of the account for creating issues.
   • API token or password depending on the Jira project.
     • For Jira Cloud projects an API token is required. Follow instructions to create a new API token for the account creating issues.
     • For Jira Self-managed projects password of the account creating issues is required.
   • Project key, same as the prefix in the issue identifier. For instance, issue TEST-1 has the project key TEST.
   • Type of the issue created such as Bug, Task, Story, and so on.
   • (Optional) priority assigned to the issue such as Low, Medium, High, and so on.
   • (Optional) one or more labels to be assigned to the issue.
   • (Optional) Jira user to be assigned to the issue.
2. Create a Jira endpoint configuration in the Notifications service either via Enterprise UI, or the API directly.

14.4 - SMTP

Notifications to SMTP server are in the form of html and plain text message packets.

Requirements

Do the following to receive SMTP notifications.

1. Provide the following required SMTP server information.
   • Host name/address
   • Port number
   • From address
   • To address
   • (Optional) username
   • (Optional) password
   • (Optional) use tls to encrypt connection
2. Create an SMTP endpoint configuration in the Notifications service either via Enterprise UI, or the API directly.

14.5 - Microsoft Teams

Notifications to Microsoft Teams are in the form of messages to a channel.

Requirements

Do the following to receive Microsoft Teams notifications.

1. An incoming webhook URL is required. See the Microsoft instructions to create one.
2. Copy the incoming webhook URL.
3. Create a Microsoft Teams endpoint configuration in the Notifications service either via Enterprise UI, or the API directly.

15 - Pipeline Image Analysis

Pipeline Image Analysis and Scanning

Anchore now supports analysis of images at build time with no requirement to push images up to a registry in order for them to be analyzed and added to the system.

This feature works by executing anchorectl inside your pipeline and giving it an endpoint and credentials to upload the results to and Anchore deployment. It will analyze the image locally for package artifacts and upload the analysis and container metadata to Anchore. The system then loads the result after which the image analysis is available for vulnerability queries and policy evaluations using AnchoreCTL or direct API operations.

The analysis import is processed by the analyzer services, so you will see the image enter the not_analyzed state when first uploaded, then analyzing and analyzed. Once in the analyzing state the proces is usually very fast (seconds) since it only is operating on the provided package manifest rather than having to pull any image data or perform significant IO to unpack an image.

Example

❯ syft -o json ubuntu:latest | anchorectl image add ubuntu:latest --wait --from -
 ✔ Loaded image
 ✔ Parsed image
 ✔ Cataloged packages      [101 packages]
Image:
  status:           analyzed (active)
  tag:              docker.io/ubuntu:latest
  digest:           sha256:33bca6883412038cc4cbd3ca11406076cf809c1dd1462a144ed2e38a7e79378a
  id:               sha256:df5de72bdb3b711aba4eca685b1f42c722cc8a1837ed3fbd548a9282af2d836d
  distro:           [email protected] (amd64)
  layers:           1

❯ anchorectl image get ubuntu:latest
Tag: docker.io/ubuntu:latest
Digest: sha256:33bca6883412038cc4cbd3ca11406076cf809c1dd1462a144ed2e38a7e79378a
ID: sha256:df5de72bdb3b711aba4eca685b1f42c722cc8a1837ed3fbd548a9282af2d836d
Analysis: analyzed
Status: active

❯ anchorectl image vulnerabilities ubuntu:latest -t all
...

Next Steps

Install anchorectl. to scan local images and generate software Bill-of-Materials to upload into your Anchore deployment.

After uploading the analysis, you’ll need to use AnchoreCTL or the UI to view vulnerabilities or policy evaluations using the enterprise feed data and policy features such as base-image diffs or false positive management

16 - Role-Based Access Control

Overview

Anchore Enterprise includes support for using Role-Based Access Control (RBAC) to control the permissions that a specific user has to a specific set of resources in the system. This allows administrators to configure specific, limited permissions on user enabling limited access usage for things like CI/CD automation accounts, read-only users for browsing analysis output, or security team users that can modify policy but not change image analysis configuration.

Anchore Enterprise provides a predefined and immutable set of roles:

1. Full Control - All permissions on resources in the account
2. Read Write- Standard user permissions
3. Read Only - Read but no updates on all resources
4. Policy Editor - Create/Update/Delete policies and view policy evaluations on images
5. Account Users Admin - Add/delete/update users in the account
6. Image Analyzer - Analyze, view, and evaluate policy on images

The Enterprise UI contains an enumeration of the specific permissions granted to users that are members of each of the roles.

Roles, Users, and Accounts

Roles are applied within the existing account and user frameworks defined in Anchore Enterprise. Resources are still scoped to the account namespace and accounts provide full resource isolation (e.g. an image must be analyzed within an account to be referenced in that account). Roles allow users to be granted permissions in both the account to which they belong as well as external accounts to facilitate resource-sharing.

Terminology

User: An authenticated identity (a principal in rbac-speak).

Account: A resource namespace and user grouping that also defines an authorization domain to which permissions are applied.

Role: A named set of permissions.

Permission: An action to grant an operation on a set of resources.

Action: The operation to be executed, such as listImages, createRegistry, and updateUser.

Target: The resource to be operated on, such as an image digest.

Role Membership: Mapping a username to a role within a specific account. This confers the permissions defined by the role to resources in the specified account to the specified user. The user is not required to be a member of the account itself.

Constraints

1. A user may be a member of a role within one or more accounts.
2. A user may be a member of many roles, or no roles.
3. There is no default role set on a user when a user is created. Membership must be explicitly set.
4. Roles are immutable. The set of actions they grant is static.
5. Creating and deleting accounts is only available to users in the admin account. The scope of accounts and roles is different than other resources because they are global. The authorization domain for those resources is not the account name but rather the global domain: system.

Role Summary and Permissions

Enabling RBAC using the Helm Chart

The chart provides a set of flags to enable to use Enterprise, and RBAC is configured by default when those are set.

The flags are:

1. anchoreEnterpriseGlobal.enabled=true
2. anchoreEnterpriseRbac.enabled=true

This will have the effect of adding some containers to the anchoreAPI deployment pods that run the RBAC Manager and RBAC Authorizer services as well as configuring the api endpoints to use the authorizer correctly.

NOTE: if you have already installed Anchore and have created accounts and users, enabling RBAC will cause all non-admin users to have no permissions or API access to authorization-gated routes (most API calls) until you explicitly assign each user one or more roles. The configuration and install process for RBAC does not automatically assign roles to existing users.

Granting Cross-Account Access

The Anchore API supports a specific mechanism for allowing a user to make requests in another account’s namespace, the x-anchore-account header. By including x-anchore-account: "desiredaccount" on a request, a user can attempt that request in the namespace of the other account. This is subject to full authorization and RBAC.

To grant a username the ability to execute operations in another account, simply make the username a member of a role in the desired account. This can be accomplished in the UI or via API against the RBAC Manager service endpoint. For example, using curl:

curl -u admin:foobar -X POST -H "Content-Type: application/json" -d '{"username": "someuser", "for_account": "some_other_account"}' http://localhost:8229/roles/policy-editor/members

This should be done with caution as there is currently no support for resource-specific access controls. A user will have the permitted actions on all resources in the other account (based on the permissions of the role). For example, making a user a member of policy-editor role for another account will enable full ability to create, delete, and update that account’s policy bundles.

WARNING: Because roles do not currently provide custom target/resource definitions, assigning users to the Account User Admin role for an account other than their own is dangerous because there are no guards against that user then removing permissions of the granting user (unless that user is an ‘admin’ account user), so use with extreme caution.

NOTE: admin account users are not subject to RBAC constraints and therefore always have full access to add/remove users to roles in any account. So, while it is possible to grant them role membership, the result is a no-op and does not change the permissions of the user in any way.

16.1 - RBAC Service Components

RBAC in Anchore Enterprise is implemented as a pair of services: the RBAC Management API and the RBAC Authorizer.

RBAC Manager

The manager is a user-facing component that proves a REST API for managing roles, role permissions, and role membership of users. It stores its state in the Anchore Enterprise schema of the DB and is consumed by the Enteprise UI for the rbac-related management controls. See: Managing Accounts, Users, and Roles in the UI. Because this API is itself subject to RBAC controls, the manager invokes the RBAC Authorizer for authorization decisions on api calls.

As with other Anchore API services, it has the following utility routes in the API:

• /version - Returns version information
• /health - Returns 200 OK if service is running
• /swagger.json - Obtain the API spec directly

For information on configuring and running the service, see: Quickstart or Configuring RBAC

RBAC Authorizer

The RBAC Authorizer is called directly by user-facing API services to provide authorization decisions. The Authorizer utilizes the same DB as the Manager and while its workload is predominantly read-only, there are a few write operations involved around managing the lifecycles of usernames and accounts.

NOTE: This is an internal component, it should never be exposed on the open network or have its ports exposed outside the host. Typically, this means it is run with no host-port mappings and relies on Docker network/links to other local-host containers or runs within the same Kubernetes pod as the API components and also does not have a port on the pod IP.

17 - Reports

Overview

Anchore Enterprise Reports aggregates data to provide insightful analytics and metrics for account-wide artifacts. The service employs GraphQL to expose a rich API for querying the aggregated data and metrics.

NOTE: This service captures a snapshot of artifacts in Anchore Enterprise at a given point in time. Therefore, analytics and metrics computed by the service are not in real time, and may not reflect most up-to-date state in Anchore Enterprise.

Installation

Anchore Enterprise Reports is included with Anchore Enterprise, and is installed by default when deploying a trial quickstart with Docker Compose, or a production deployment Kubernetes.

How it works

One of the main functions of Anchore Enterprise Reports is aggregating data. The service keeps a summary of all current and historical images and tags for every account known to Anchore Enterprise. It also maintains vulnerability reports and policy evaluations generated using the active bundle for all the images and tags respectively.

WARNING: Anchore Enterprise Reports shares a persistence layer with Anchore Enterprise. Ensure sufficient storage is provisioned.

Configuration

Anchore Enterprise Reports service loads it’s configuration from the reports section of the config.yaml. The following is a snippet:

...
services:
  reports:
    enabled: true
    require_auth: true
    endpoint_hostname: "<hostname>"
    listen: '0.0.0.0'
    port: 8228
    # GraphiQL is a GUI for editing and testing GraphQL queries and mutations.
    # Set enable_graphiql to true and open http://<host>:<port>/v2/reports/graphql in a browser for reports API
    enable_graphiql: true
    # Set enable_data_ingress to true for periodically syncing data from anchore engine into the reports service
    enable_data_ingress: true
    cycle_timers:
      reports_data_load: 600 # images and tags synced every 10 minutes
      reports_data_refresh: 7200 # policy evaluations and vulnerabilities refreshed every 2 hours
      reports_metrics: 3600 # metrics generated every hour
    authorization_handler: external
    authorization_handler_config:
      endpoint: "<rbac-authorizer-endpoint>"

NOTE: Any changes to the configuration requires a restart of the service for the updates to take effect.

In an Anchore Enterprise RBAC-enabled deployment, any non-admin account user must at least have listImages permission to execute queries against Reports API.

Data ingress

Reports service handles data ingress from Anchore Enterprise via the following asynchronous processes triggered periodically:

• Loader: Compares the working set of images and tags in Anchore Enterprise with its own records. Based on the difference, images and tags along with the vulnerability report and policy evaluations are loaded into the service. Artifacts deleted from Anchore Enterprise are marked inactive in the service.

  This process is triggered periodically every 10 minutes (600 seconds). To modify the loader interval, update cycle_timers -> reports_data_load in the config.yaml snippet above. In a quickstart deployment, add ANCHORE_ENTERPRISE_REPORTS_DATA_LOAD_INTERVAL_SEC=<interval-in-seconds> to the environment variables section of the reports service in docker-compose.yaml.

• Refresher: Refreshes the vulnerability report and policy evaluations of all the images and tags actively maintained by the service.

  This process is triggered periodically every 2 hours (7200 seconds). To modify the refresher interval, update cycle_timers -> reports_data_refresh in the config.yaml snippet above. In a quickstart deployment, add ANCHORE_ENTERPRISE_REPORTS_DATA_REFRESH_INTERVAL_SEC=<interval-in-seconds> to the environment variables section of the reports service in docker-compose.yaml.

WARNING: Reports service may miss updates to artifacts if they are added and deleted in between consecutive ingress processes.

Data ingress is enabled by default. It can be turned off with enable_data_ingress: false in the config.yaml snippet shown previously. In a quickstart deployment, add ANCHORE_ENTERPRISE_REPORTS_ENABLE_DATA_INGRESS=false to the environment variables section of the reports service in docker-compose.yaml. When the ingress is turned off, Reports service will no longer aggregate data from Anchore Enterprise, metric computations will also come to a halt. However, the service will continue to serve API requests/queries with the existing data.

Metrics

Reports service comes loaded with a few pre-defined/canned metric definitions. A metric definition consists of an identifier, readable name, description and the type of the metric. The type is loosely based on statsd metric types. Currently, all the pre-defined metrics are of type ‘counter’ - a measure of the number of items that match certain criteria. A value for each of these metric definitions is computed using the data aggregated by the service.

All metric values are computed periodically every hour (3600 seconds). To modify the interval, update cycle_timers -> reports_metrics in the config.yaml snippet above. In a quickstart deployment, add ANCHORE_ENTERPRISE_REPORTS_METRICS_INTERVAL_SEC=<interval-in-seconds> to the environment variables section of the reports service in docker-compose.yaml.

See it in action

To see Reports service in the Enterprise UI, see Dashboard or Reports view. The dashboard view utilizes metrics generated by the service and renders customizable widgets. The reports view employs graphQL queries and aggregates the results into multiple formats (CSV, JSON, and so on).

For using the API directly, see API Access.

18 - Runtime Inventory

Overview

Using Anchore’s runtime inventory agents provides Anchore Enterprise access to what images are being used in your deployments. This can help give insight into where vulnerabilities or policy violations are in your production workloads.

Agents

Anchore provides agents for collecting the inventory of different container runtime environments:

• Kubernetes Runtime Inventory
• ECS Runtime Inventory

General Runtime Configuration

Inventory Time-To-Live

As part of reporting on your runtime environment, Anchore maintains an active record of the containers, the images they run, and other related metadata based on time they were last reported by an inventory agent.

The configuration setting below allow you to specify how long inventory should remain part of the Catalog Service’s working set. These are the default settings found in the values file.

services:
  catalog:
    runtime_inventory:
      inventory_ingest_overwrite: false
      inventory_ttl_days: 120

Below are a few examples on how you may want to use this feature.

Keep most recently reported inventory

      inventory_ingest_overwrite: true
      inventory_ttl_days: 7

For each cluster/namespace reported from the inventory agent, the system will delete any previously reported containers and images and replace it with the new inventory.

Note: The inventory_ttl_days is still needed to remove any cluster/namespaces that are no longer reported as well as some of the supporting metadata (ie. pods, nodes). This value should be configured to be long enough that inventory isn’t incorrectly removed in case of an outage from the reporting agent. The exact value depends on each deployment, but 7 days is a reasonable value here.

Keep inventory reported over a time period

      inventory_ingest_overwrite: false
      inventory_ttl_days: 14

This will delete any container and image that has not been reported by an agent in the last 14 days. This includes its supporting metadata (ie. pods, nodes).

Keep inventory indefinitely

      inventory_ingest_overwrite: false
      inventory_ttl_days: 0

This will keep any containers, images, and supporting metadata reported by an inventory agent indefinitely.

Deleting Inventory via API

Where it is not desirable to wait for the Image TTL to remove runtime inventory images it is possible to manually delete inventory items via the API by issuing a DELETE to /v2/inventories with the following query parameters.

• inventory_type (required) - either ecs or kubernetes
• context (required) - it must match a context as seen by the output of anchorectl inventory list
  • Kubernetes - this is a combination of cluster name (as defined by the anchore-k8s-inventory config) and a namespace containing running containers e.g. cluster1/default.
  • ECS - this is the cluster ARN e.g. arn:aws:ecs:eu-west-2:123456789012:cluster/myclustername
• image_digest (optional) - set if you only want to remove a specific image

e.g. DELETE /v2/inventories?inventory_type=<string>&context=<string>&image_digest=<string>

Using curl: curl -X DELETE -u username:password "http://{servername:port}/v2/inventories?inventory_type=&context=&image_digest=

18.1 - Kubernetes Runtime Inventory

Overview

Anchore uses a go binary called anchore-k8s-inventory that leverages the Kubernetes Go SDK to reach out and list containers in a configurable set of namespaces to determine which images are running.

anchore-k8s-inventory can be deployed via its helm chart, embedded within your Kubernetes cluster as an agent. It will require access to the Anchore API.

KAI (Kubernetes Automated Inventory)

Prior to the v1.0.0 release of anchore-k8s-inventory Kubernetes inventory was called kai. A future release of Anchore Enterprise will remove support for kai. However, Anchore Enterprise v4.7 will support both kai and anchore-k8s-inventory. For documentation related to the configuration of kai please refer to the Anchore Enterprise v4.6 documentation.

Deployment

The most common way to track inventory is to install anchore-k8s-inventory as an agent in your cluster. To do this you will need to configure credentials and information about your deployment in the values file. It is recommended to first configure a specific robot user for the account where you’ll want to track your Kubernetes inventory.

As an agent anchore-k8s-inventory is installed using helm and the helm chart is hosted as part of the https://charts.anchore.io repo. It is based on the anchore/k8s-inventory docker image.

To install the helm chart, follow these steps:

1. Configure your username, password, Anchore URL and cluster name in the values file.

k8sInventory:
  # Path should not be changed, cluster value is used to tell Anchore which cluster this inventory is coming from
  kubeconfig:
    cluster: <unique-name-for-your-cluster>

  anchore:
    url: <URL for your>

    # Note: recommend using the inventory-agent role
    user: <user>
    password: <password>

2. Run helm install in the cluster(s) you wish to track

$ helm repo add anchore https://charts.anchore.io
$ helm install <release> -f <values.yaml> anchore/k8s-inventory

anchore-k8s-inventory must be able to resolve the Anchore URL and requires API credentials. Review the anchore-k8s-inventory logs if you are not able to see the inventory results in the UI.

Note: the Anchore API Password can be provided via a Kubernetes secret, or injected into the environment of the anchore-k8s-inventory container

• For injecting the environment variable, see: injectSecretsViaEnv
• For providing your own secret for the Anchore API Password, see: useExistingSecret. K8s Inventory creates it’s own secret based on your values.yaml file for key k8sInventory.anchore.password, but the k8sInventory.useExistingSecret key allows you to create your own secret and provide it in the values file. See the K8s Inventory repo for more information about the K8s Inventory specific configuration

Usage

To verify that you are tracking Kubernetes Inventory you can access inventory results with the command anchorectl inventory list and look for results where the TYPE is kubernetes.

The UI also displays the Kubernetes Inventory and allows operators to visually navigate the images, vulnerability results, and see the results of the policy evaluation.

For more details about watching clusters, and reviewing policy results see the Using Kubernetes Inventory section.

18.2 - ECS Runtime Inventory

Overview

Anchore uses a go binary called anchore-ecs-inventory that leverages the AWS Go SDK to gather an inventory of containers and their images running on ECS and report back to Anchore.

Deployment

Via Helm Chart

You can install the chart via:

helm repo add anchore https://charts.anchore.io
helm install <release-name> -f <values.yaml> anchore/ecs-inventory

A basic values file can always be found here. The key configurations are in the ecsInventory section.

Anchore ECS Inventory creates it’s own secret based on your values.yaml file for the following keys that are required for successfully deploying and connecting the ecs-inventory service to the Anchore Platform and AWS ECS Service:

• ecsInventory.awsAccessKeyId
• ecsInventory.awsSecretAccessKey

Using your own secrets

The (ecsInventory.useExistingSecret and ecsInventory.existingSecretName) or ecsInventory.injectSecretsViaEnv keys allows you to create your own secret and provide it in the values file or place the required secret into the pod via different means such as injecting the secrets into the pod using hashicorp vault.

For example:

• Create a secret in kubernetes:

  apiVersion: v1
  kind: Secret
  metadata:
    name: ecs-inventory-secrets
  type: Opaque
  stringData:
    ANCHORE_ECS_INVENTORY_ANCHORE_PASSWORD: foobar
    AWS_ACCESS_KEY_ID: someKeyId
    AWS_SECRET_ACCESS_KEY: someSecretAccessKey
  

• Provide it to the helm chart via the values file:

  ecsInventory:
      useExistingSecret: true
      existingSecretName: "ecs-inventory-secrets"
  

The Anchore API Password and required AWS secret values can also be injected into the environment of the ecs-inventory container. For injecting the environment variable

# set
ecsInventory:
  injectSecretsViaEnv=true

See the ecs-inventory repo for more information about the ECS Inventory specific configuration

Via ECS

It is also possible to deploy the ecs-inventory container on ECS. Here is an sample task definition that could be used to deploy ecs-inventory with a default configuration:

{
    "family": "anchore-ecs-inventory-example-task-definition",
    "containerDefinitions": [
        {
            "name": "ecs-inventory",
            "image": "docker.io/anchore/ecs-inventory:latest",
            "cpu": 0,
            "essential": true,
            "environment": [
                {
                    "name": "ANCHORE_ECS_INVENTORY_ANCHORE_URL",
                    "value": "https://anchore.url"
                },
                {
                    "name": "ANCHORE_ECS_INVENTORY_ANCHORE_USER",
                    "value": "admin"
                },
                {
                    "name": "ANCHORE_ECS_INVENTORY_ANCHORE_ACCOUNT",
                    "value": "admin"
                },
                {
                    "name": "ANCHORE_ECS_INVENTORY_REGION",
                    "value": "us-east-2"
                }
            ],
            "secrets": [
                {
                    "name": "ANCHORE_ECS_INVENTORY_ANCHORE_PASSWORD",
                    "valueFrom": "arn:aws:ssm:${region}:${aws_account_id}:parameter/ANCHORE_ADMIN_PASS"
                },
                {
                    "name": "AWS_ACCESS_KEY_ID",
                    "valueFrom": "arn:aws:ssm:${region}:${aws_account_id}:parameter/ECS_INVENTORY_AWS_ACCESS_KEY_ID"
                },
                {
                    "name": "AWS_SECRET_ACCESS_KEY",
                    "valueFrom": "arn:aws:ssm:${region}:${aws_account_id}:parameter/ECS_INVENTORY_AWS_SECRET_ACCESS_KEY"
                }
            ],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-create-group": "true",
                    "awslogs-group": "/anchore/ecs-inventory",
                    "awslogs-region": "us-east-2",
                    "awslogs-stream-prefix": "ecs"
                }
            }
        }
    ],
    "executionRoleArn": "arn:aws:iam::${aws_account_id}:role/ecsTaskExecutionRole",
    "networkMode": "awsvpc",
    "requiresCompatibilities": [
        "FARGATE"
    ],
    "cpu": "512",
    "memory": "1024",
    "runtimePlatform": {
        "cpuArchitecture": "X86_64",
        "operatingSystemFamily": "LINUX"
    }

Usage

To verify that you are tracking ECS Inventory you can access inventory results with the command anchorectl inventory list and look for results where the TYPE is ecs.

Watching ECS Inventory to auto analyze

It is possible to create a subscription to watch for new ECS Inventory that is reported to Anchore and automatically schedule those images for analysis.

1. Create the subscription

A subscription can be created by sending a POST to /v1/subscriptions with the following payload:

{
  "subscription_key": "<SUBSCRIPTION_KEY>",
  "subscription_type": "runtime_inventory"
}

Curl example:

curl -X POST -u USERNAME:PASSWORD --url ANCHORE_URL/v1/subscriptions --header 'Content-Type: application/json' --data '{
  "subscription_key": "arn:aws:ecs:eu-west-2:123456789012:cluster/myclustername",
  "subscription_type": "runtime_inventory"
}'

The subscription_key can be set to any part of an ECS ClusterARN. For example setting the subscription_key to the:

• full ClusterARN arn:aws:ecs:us-east-1:012345678910:cluster/telemetry will create a subscription that only watches this cluster
• partial ClusterARN arn:aws:ecs:eu-west-2:988505687240 will result in a subscription that watches every cluster within the account 988505687240

2. Activate the subscription

After a subscription has been created it needs to be activated. This can be achieved with anchorectl.

anchorectl subscription activate <SUBSCRIPTION_KEY> runtime_inventory

19 - SSO Support

Overview

Anchore Enterprise can be configured to support user login to the UI using identities from external identity providers that support SAML 2.0. Anchore never stores any credentials for the users, only their usernames and Anchore permissions. All UI access is gated through a user’s valid login into the identity provider. Anchore uses the external provider to verify username identity and initialize a username, account, and roles on first login for a new user. Once a user’s identity is initialized in Anchore, the Anchore administrator can manage user permissions by managing the roles associated with the user’s identity in Anchore itself.

Terms

SAML Terms:

• Identity Provider (IDP) - The service that stores the identity database and provides identity and authentication services to Anchore.
• Service Provider (SP) - The service providing resources to the end user, in this case, the Anchore Enterprise deployment.
• Assertion Consumer Service (ACS) - The consumer of SAML assertions generated by the Identity Provider. For Anchore Enterprise, the UI proxies the SAML assertion to the Anchore Enterprise RBAC Manager service which consumes it, but the UI is the network endpoint the user’s browser interacts with.

Anchore Terms:

• Native User - A user that exists in the Anchore DB and has login credentials (passwords).
• SAML User - A user that exists in the Anchore DB only with a username and permissions, but no credentials. This prevents any username conflicts. SAML users will also be associated with a single Identity Provider. This prevents overlapping usernames from multiple Identity Providers gaining access to unexpected users or accounts.

How SAML integration works

When a user logs into the Anchore Enterprise UI, they can choose which Identity Provider to authenticate with. User credentials are never passed to Anchore Enterprise. Instead, other information about the user is passed from the Identity Provider to Anchore. Some information used by Anchore during login include the username, authenticating Identity Provider, associated account, and initial RBAC permissions.
After the initial login, RBAC permissions can be adjusted for this user directly by an Anchore administrator. This allows the Anchore administrator the ability to control access of Anchore users without having to gain access to the corporate IDP system.

Dynamic SAML User Provisioning

The first time a SAML User logs into Anchore, if the username is not found within the Anchore DB, a record will be automatically created for the user. If the user’s associated account is not found within the Anchore DB, an account record will also be automatically created at this time. This is often referred to as Just In Time Provisioning (JIT).

Explicit Creation of SAML Users

An Anchore administrator has the ability to create other users with administrator privileges. This includes Native and SAML Users. When creating a SAML Administrator User, the username and the Identity Provider’s name will be required. Upon SSO login by this new user, it will be associated with account admin and have all the permissions of an Anchore administrator.

A global configuration mode is also available if SSO is the preferred method of login, but the Anchore administrator would like explicit control over which users can gain access to Anchore Enterprise.

`sso_require_existing_users: ${ANCHORE_SSO_REQUIRES_EXISTING_USERS}`

When this configuration mode is set to true, any users who have permissions to create other users, will now have the ability to explicitly create SAML Users. As stated above, when creating a SAML User, the username and the Identity Provider’s name will be required. In addition, an RBAC role will also be needed for each SAML User creation. Upon SSO login by this new user, it will be associated with the account it was created in and have all the RBAC permissions provided for it. When this configuration mode is set to true, SSO logins are only permitted within Anchore for users who have existing SAML user records fround in Anchore DB.

When explicitly creating SAML Users, the account and RBAC role provided will take precedent over any default values or IDP Attributes which may be configured in the SAML Configuration described below. For more information, please see Mapping.

Note: Any users that have previously authenticated via SSO will continue to have access regardless of the configuration mode setting. If you wish to prevent future access when setting sso_require_existing_users to true, simply delete the user record in Anchore.

SSO Login Validation

During subsequent SSO logins, Anchore will find an existing user record in the Anchore DB. The following information will be validated:

• The user record must be a SAML User. If the user was previously configured as a Native User and you want to convert it to a SAML User, simply delete the user record in Anchore and have the user log in again via SSO.
• The user record must be authenticating from the same Identity Provider. If the user has been changed to authenticate via a different Identity Provider, simply delete the user record in Anchore and have the user log in again via SSO.

Configuration Overview

In order to use SAML SSO, the Anchore Enterprise deployment must:

1. Have Oauth enabled. This is required so that Anchore can issue bearer tokens for subsequent API usage by the UI to the system APIs.

2. Using hashed passwords is optional but highly recommended. See User Credential Storage for more information on configuring OAuth and hashed password storage.

3. Be able to reach the IDP login URL from the user’s browser.

4. Be able to reach the metadata XML endpoint in the IDP (if using url).

Configuration of SAML SSO is done via API/UI operations but requires configuration both in your Identity Provider and in Anchore.

In the IDP:

• Must support HTTP Redirect binding
• Should support signed assertions and signed documents
• Must allow unsigned client requests from Anchore
• Must allow unencrypted requests and responses

Anchore IDP Configuration Fields are as follows.

• Name - The name to use for this configuration. It will be part of the UI’s /service/auth/sso/ route as well as the RBAC Manager’s /saml/sso/ and /saml/login/ routes that are used to implement SSO.
• Enabled - Whether auth via this configuration is allowed.
• ACS HTTPS Port - HTTPS port if non-standard. If omitted or -1, 443 is used.
• SP Entity ID - The identity for the Anchore system to present when requesting auth from the SAML IDP. This is typically a URL identifying the Anchore deployment.
• ACS URL - The URL to which SAML Responses should be sent from the IDP. For UI usage, this should be the hostname and port of the UI deployment and a path: /service//sso/auth/{idp config name}.
• Default Account - If set, this is the account that SSO users will be initialized to be members of upon sign-in the first time. This property or IDP Account Attribute must be set.
• Default Role - The role that will be granted to new users on first sign-in. Use this setting to apply a consistent role to all users if you do not want that data imported from the IDP. This property or IDP Role Attribute must be set.
• IDP Account Attribute - A SAML assertion attribute name from the SAML responses that Anchore will use to determine the account name to initialize the user into. If the account does not exist, it is created. For more information on the initialization process see Initializing User Identities below. This property or Default Account must be set.
• IDP Username Attribute - A SAML assertion attribute name from the SAML responses that Anchore will use to determine the username for the anchore identity. This is optional and typically will not need to be set. If omitted, the SAML Subject is used and this should meet most needs.
• IDP Metadata URL - URL to retrieve the IDP metadata xml from. This value is mutually exclusive with IDP Metadata XML, so only one of the two properties may be specified.
• IDP Metadata XML - Raw XML for the IDP metadata. This value is mutually exclusive with IDP Metadata URL, so only one of the two properties may be specified.
• IDP Role Attribute - The SAML assertion attribute from the SAML responses that Anchore will use to initialize a user’s roles. This may be a multi-value property. This property or Default Account must be set.
• Require Signed Assertions - If true, require the individual assertions in the SAML response to be signed by the IDP.
• Require Signed Response - If true, require the SAML response document to be signed.

Using SAML Attributes to Initialize Users and Account in Anchore

The properties of the user including the account it belongs to, the roles it has in that account as well as any other accounts the user has role access to are all initialized based on the combination of the Anchore IDP configuration and the SAML response presented by the IDP at the user’s first login.

See Mapping for more information on that process and how the configuration works.

Deleting SAML SSO Configuration

An Anchore administrator has the ability to create, modify, and delete the SAML Configuration. During deletion of the SAML Configuration, any user that was created with this Identity Provider, either dynamically or explicitly, will also be deleted.

19.1 - Mapping SSO Identities into Anchore

Overview of Mapping External Identities into Anchore

Anchore SSO support provides a way to keep users’ credentials centrally stored in a corporate Identity Provider and outside of the Anchore deployment. Anchore’s SSO approach uses the external identity store to authenticate a user and bootstrap its permissions using Anchore’s RBAC system. For each login, the user’s identity is verified against the external provider but once the identity is initialized inside the Anchore deployment, its access to resources is controlled by the Anchore Enterprise RBAC system. This approach allows Anchore admins to manage user access without having to require administrator access to a corporate or central IT identity system while still being able to leverage that system for defining identity and securing access credentials.

The identity mapping process has two distinct phases:

1. Initial identity bootstrap - Occurs on the first login of a user to Anchore causing dynamic construction of an Anchore user record.
2. Identity verification and assertion validation - Validates the administrators requirements against the external identity record on each login.

Defining the Username

By default, with SSO, the SAML Assertion’s “Subject” attribute is used to define the username. Using the subject is the right solution in most situations, but in extreme cases it may be necessary to provide the username in some other form via attributes in the SAML Response. This behavior can be configured by setting: idp_username_attribute in the SAML Configuration within Anchore. This should only be used when the subject either cannot be used due to it being a transient ID as configured by the IDP itself, or you want the username to map to some form other than the IDP’s username, email, or persistent ID.

If the idp_username_attribute is set to an attribute name, and that attribute is not found or has no value in the SAML Response presented during login, then that user login will be rejected by Anchore.

If idp_username_attribute is an empty string or null, then the SAML Response’s subject attribute is used as the username. This is the default behavior.

Defining the Account the User Belongs To

In Anchore, all users belong to an Account. When an SSO user logs into Anchore UI for the first time, the identity is initialized with the username (as defined above), but the account to which the user belongs is configurable via a separate pair of configuration properties in the SAML Configuration within Anchore. These configuration properties are mutually exclusive.

idp_account_attribute - If set in the SAML Configuration, this attribute must be found within the SAML Response during each login for every user. The attribute value received is the ‘account name’. It must also be valid. A valid value must be greater than three characters and must not be a reserved account name such as ‘admin’ or ‘anchore-system’ If the attribute is not found within the SAML Response or the value is not valid, the login is rejected.

default_account - If set in the SAML Configuration, it’s value is the account all users that login from this IDP will be assigned.

• In both cases, on the initial login by the user, if the account does not already exist within Anchore, an external account with that name is created.

Defining the User’s Initial Roles

In Anchore, all users are allowed to have one or more Roles that describe a set of access permissions.
Roles are assigned to the user via a separate pair of configuration properties in the SAML Configuration within Anchore.
These configuration properties are mutually exclusive.

idp_role_attribute - If set in the SAML Configuration, the attribute must be found within the SAML Response during each login for every user. The attribute value received is one or more ‘role name’. The value must also be valid. If the attribute is not found within the SAML Response or the value is not valid, the login is rejected.

default_role - If set in the SAML Configuration, it’s value will be the single role set for all users that login with this IDP. During a user’s first login, this role will be set on the account during user identity initialization. On subsequent logins for this user, the value will be ignored.

Revoking SSO User Access

1. Disable the Anchore account. Any user, SSO or otherwise, that is a member of a disabled account cannot log in or perform API operations.
2. If using idp_account_attribute or idp_role_attribute, simply remove or zero that attribute at the IDP for that user or group. All affected users will no longer be able to log in to Anchore.

Changing the Anchore SAML Configuration

Initialization of identities and roles occurs on the user’s first login. Once initialized, the configuration must match the SAML Response presented during each login for the user to log in.

Thus, changes to the SAML Configuration within Anchore may affect subsequent logins for your users. For instance, if you change the SAML Configuration within Anchore to start using attributes instead of defaults, a user’s SAML Response will need to contain the same attributes. Failure to find the correct attribute(s) with valid values will prevent the user’s login.

Examples

Identity Mapping Example Configurations

Examples of configuration values for a single SAML Configuration within Anchore to achieve different behavior:

1. All SSO users in one account with the same read-write permissions:

   • default_account = ‘account’
   • default_role = ‘read-write’

2. SSO users in accounts managed by the IDP property, for example a ‘groups’ property:

   • default_account = null
   • default_role = null
   • idp_account_attribute = "primary_group"
   • idp_role_attribute = "roles"
   • Will initialize the user with the roles specified in the ‘roles’ attribute for the accounts in the ‘primary_group’ attribute.
     • E.g. if ‘primary_group’ = [’testers’], and ‘roles’ = [‘read-only’] for a user, [email protected]
     • [email protected] will be initialized at login as username = [email protected] in account testers with role read-only
     • Because the account is from an attribute, [email protected] might have ‘primary_group’ = [‘security_engineers’] and thus be initialized in a different account in Anchore.

19.2 - SSO Examples

Examples of SSO configuration between Anchore and an external Identity Provider

These examples provide simple defaults and identity mappings. More advanced configurations are possible as outlined in the identity mappings documentation.

• Okta
• KeyCloak

19.2.1 - KeyCloak SAML Example

Configuring SAML SSO for Anchore with KeyCloak

The JBoss KeyCloak system is a widely used and open-source identity management system that supports integration with applications via SAML and OpenID Connect. It also can operate as an identity broker between other providers such as LDAP or other SAML providers and applications that support SAML or OpenID Connect.

The following is an example of how to configure a new client entry in KeyCloak and configure Anchore to use it to permit UI login by KeyCloak users that are granted access via KeyCloak configuration.

Configuring KeyCloak

Anchore supports multiple IDP configurations, each given a name. For this example we’ll choose the name “keycloak” for our configuration. This important as that name is used in several URL paths to ensure that the correct configuration is used for validating responses, so make sure you pick a name that is meaningful to your users (they will see it in the login screen) and also that is url friendly.

Some config choices and assumptions specifically for this example:

1. Let’s assume that you are running Anchore Enterprise locally. Anchore Enterprise UI is available at: https://localhost:3000. Replace with the appropriate url as needed.
2. We’re going to choose keycloak as the name of this saml/sso configuration within Anchore. This will identify the specific configuration and is used in urls.
3. Based on that, the Single-SignOn URL for this deployment will be: https://localhost:3000/service/sso/auth/keycloak
4. Our SP Entity ID will use the same url: http://localhost:3000/service/sso/auth/keycloak

Add a Client entry in KeyCloak

1. See SAML Clients in KeyCloak documentation

2. For this example, set the following values in “Add Client” screen (these are specific to the settings in this example described above):

   1. Client ID - http://localhost:3000/service/sso/auth/keycloak - This will be the SP Entity ID used in the Anchore configuration later
   2. Client Protocol: “saml”
   3. Client SAML Endpoint: “http://localhost:3000/service/sso/auth/keycloak”

3. In the next screen, Client Settings

   

   1. Name - “Anchore Enterprise”. This is only used to display a friendly name to Keycloak users in the KeyCloak UI. Can use any name you like.

   2. Enabled - Select on

   3. Include Authn Statement - Select on

   4. Sign Documents - Select on

   5. Client Sign Authn Requests - Select Off

   6. Sign Assertions - Select off

   7. Encrypt Assertions - Select off

   8. Client Signature Required - Select off

   9. Force Post Binding - Select off. Anchore requires the HTTP Redirect Binding to work, so this setting must be off to enable that.

   10. Force Name ID Format - Select on

   11. Name ID Format - Select username or email (transient uses a generated UUID per login and persistent use the Keycloak user’s UUID)

   12. Root URL - Leave empty

   13. Valid Redirect URIs - Add http://localhost:3000/service/sso/auth/keycloak

   14. Base URL - Leave empty

   15. Master SAML Processing URL - http://localhost:3000/service/sso/auth/keycloak

   16. Fine Grain SAML Endpoint Configuration

       1. Assertion Consumer Service Redirect Binding URL - http://localhost:3000/service/sso/auth/keycloak

   17. Save the configuration

   

4. Download the metadata xml to import into Anchore

   1. Select ‘Installation’ tab.
   2. Select Format
   • Keycloak <= 5.0.0
   1. Select Format Option - SAML Metadata IDPSSODescriptor
   • Keycloak 6.0.0+
   1. Select Format Option - Mod Auth Mellon files
   2. Unzip the downloaded .zip and locate idp-metadata.xml
   3. Download or copy the XML to save in the Anchore configuration

Configure Anchore Enterprise to use the KeyCloak

1. You’ll need the following information from keycloak in order to configure the SAML record within Anchore:

2. The name to use fo the configuration, in this example keycloak

3. Metadata XML downloaded or copied from the previous section

4. In the Anchore UI, create an SSO IDP Configuration:

5. Login as admin

6. Select “Configuration” Tab on the top

7. Select “SSO” on the left-side menu

8. Click “Let’s Add One” in the configuration listing

1. Enter the values:
   1. Name: “keycloak” - This is the name of the configuration and will be referenced in login and sso URLs, so we use the value chosen at the beginning of this example
   2. Enabled: True - This controls whether or not users will be able to login with this configuration. We’ll enable it for the example but can disable later if no longer needed.
   3. ACS HTTPS Port: -1 or 443 - This is the port to use for HTTPS to the ACS (Assertion Consumer Service, in this case the UI). It is only needed if you need to use a non-standard https port
   4. SP Entity ID: http://localhost:3000/service/sso/auth/keycloak (NOTE: this must match the Client ID you used for the Client in the KeyCloak setup
   5. ACS URL: http://localhost:3000/service/sso/auth/keycloak
   6. Default Account: keycloakusers for this example, but can be any account name (existing or not) that you’d like the users to be members of. See Mappings for more information on how this
   7. Default Role: read-write for this example so that the users have full access to the account to analyze images, setup policies, etc.
   8. IDP Metadata XML: Paste the downloaded or copied XML from KeyCloak in step 4.3 above
   9. Require Signed Assertions - Select off
   10. Require Signed Response - Select on
   11. Save the configuration

You should now see a ‘keycloak’ option in the login screen for the Anchore Enterprise UI. This will redirect users to login to the KeyCloak instance for their username/password and will create a new user in Anchore in the keycloakusers account with read-write role.

19.2.2 - Okta SAML Example

Configuring SAML SSO for Anchore with Okta

Some config choices and assumptions specifically for this example:

1. Anchore UI endpoint: http://localhost:3000. Replace with the appropriate url as needed.
2. IDP Config Name: okta. This will identify the specific configuration and is used in urls, and can be any url-safe string you’d like.
3. The Single Sign-on URL (also called the Assertion Consumer Service/ACS URL) for this deployment will be: http://localhost:3000/service/sso/auth/okta. This is constructed with the UI endpoint and path /service/sso/auth/{IDP Config Name}
4. Our SP Entity ID will use the same url: http://localhost:3000/service/sso/auth/okta. This could be different but for simplicity we use the same value.

Configure Okta: Add an Application

See Okta SAML config to see how to create a new application authorization server. The following steps are used during specific steps of that walk-thru

1. In step #6
   1. Single sign on URL, this is the URL Okta will direct users to. This must be a URL in the Anchore Enterprise UI based on the name you chose for the configuration. In our example: http://localhost:3000/service/sso/auth/okta
   2. Set the Use this for Recipient URL and Destination URL checkbox as well. a1. Set the Audience URI(SP Entity ID) to a URI that will identify the anchore installation. This can be the same as the single-sign-on URL for simplicity. We’ll need to enter this value later in the Anchore config as well.
   3. Leave Default RelayState empty
   4. Name ID format can be left “Unspecified” or set to an email or username format.
   5. Choose the application username that makes sense for your install. Anchore can support a regular username or email address for the usernames.
2. In step #7, these attribute statements are not required but can be set. This is, however, where you can set additional attributes that you want Anchore to use to initialize the user’s Anchore account or permission set. Later in the SAML Configuration you can specify attributes that Anchore will look for to extract the account name and roles for initializing a user that doesn’t yet exist in Anchore.
3. In step #9, be sure to copy the metadata URL link so you have that. Anchore will need that value.
   1. Right-click here and copy the link address: The URL should be something like: https://<youraccount>.okta.com/app/<appid>/sso/saml/metadata
4. Finish the setup and save the Application entry.
5. Important: To allow Okta users to login to Anchore you need to assign the Okta user to this new Application entry. See Assign and unassign apps to users for information on this process.

Configure Anchore Enterprise to use the Okta Identity Provider

You’ll need the following information from okta to enter in the Anchore UI:

• The name chosen for the configuration: okta in this case
• Metadata XML URL (from “configuring okta” step 3.1 above)
• The Single Sign-on/ACS URL described in Step 3

In the Anchore UI, create an SSO Idp Configuration:

1. Login as admin
2. Select “Configuration” Tab on the top
3. Select “SSO” on the left-side menu
4. Click “Let’s Add One” in the configuration listing

And…

1. Enter the values:

   • Name: okta - This is the name of the configuration and will be referenced in login and sso URLs, so we use the value chosen at the beginning of this example
   • Enabled: True - This controls whether or not users will be able to login with this configuration. We’ll enable it for the example but can disable later if no longer needed.
   • ACS HTTPS Port: -1 or 443 - This is the port to use for HTTPS to the ACS (Assertion Consumer Service, in this case the UI). It is only needed if you need to use a non-standard https port
   • ACS URL: http://localhost:3000/service/sso/auth/okta
   • Default Account - The account to add all okta users to when they login, for this example we use oktausers
   • Default Role - The role to grant okta users when they login in initially. For this example, we use read-write, the standard user type that has most abilities except user management.
   • IDP Metadata URL - The url from “Configure Okta” step 3.1
   • Require Signed Assertions - Select On
   • Require Signed Response - Select On

2. Save the configuration, configuration is complete when you see a login with ‘okta’ option on the login screen. Users can now log in to your Anchore deployment using this Okta endpoint.

See: Mapping Users and Roles in SSO for more information on using the account and role defaults, IDP attribute values and understanding how identities are mapped into Anchore’s RBAC system.

20 - Storage Overview

Storage During Analysis

Scratch Space

Anchore uses a local directory for image analysis operations including downloading layers and unpacking the image content for the analysis process. This space is necessary on each analyzer worker service and should not be shared. The scratch space is ephemeral and can have its lifecycle bound to that of the service container.

Layer Cache

The layer cache is an extension of the analyzer’s scratch space that is used to cache layer downloads to reduce analysis time and network usage during the analysis process itself. For more informaiton, see, Layer Caching.

Storing Analysis Results

Anchore Enterprise is a data intensive system and uses external storage systems for all data persistence. None of the services are stateful in themselves.

For structured data that must be quickly queried and indexed, Anchore relies on PostgreSQL as its primary data store. Any database that is compatible with PostgresSQL 13 or higher should work, such as Amazon Aurora and Google Cloud SQL.

For more information, see, Database

For less structured data, Anchore implements an internal object store that can be overlayed on different backend providers, but defaults to also using the main postgres db to reduce the out-of-the-box dependencies. However, S3 and Swift APIs are both supported for leveraging external systems.

For more information on configuration and requirements for the core database and object stores see, Object Storage.

Analysis Archive

To aid in capacity management, Anchore provides a separate storage location where completed image analysis can be moved to. This reduces consumption of database capacity and primary object storage. It also removes the analysis from most API actions but makes it available to restore into the primary storage systems as needed. The analysis archive is configured as an alternate object store. For more information, see: Configuring Analysis Archive.

20.1 - Analysis Archive Storage Configuration

For information on what the analysis archive is and how it works, see Concepts: Analysis Archive

The Analysis Archive is an object store with specific semantics and thus is configured as an object store using the same configuration options, just with a different config key: analysis_archive

Example configuration snippet for using the db for working set object store and S3 for the analysis archive:

...
services:
  ...
  catalog:
  ...
  object_store:
    compression:
      enabled: false
      min_size_kbytes: 100
    storage_driver:
      name: db
      config: {}      
  analysis_archive:
      compression:
        enabled: False
        min_size_kbytes: 100
      storage_driver:
        name: 's3'
        config:
          access_key: 'MY_ACCESS_KEY'
          secret_key: 'MY_SECRET_KEY'
          #iamauto: True
          url: 'https://S3-end-point.example.com'
          region: False
          bucket: 'anchorearchive'
          create_bucket: True

Default Configuration

By default, if no analysis_archive config is found or the property is not present in the config.yaml, the analysis archive will use the object_store or archive (for backwards compatibility) config sections and those defaults (e.g. db if found).

Anchore stores all of the analysis archive objects in an internal logical bucket: analysis_archive that is distinct in the configured backends (e.g a key prefix in the s3 bucket or swift container)

Changing Configuration

Unless there are image analyses actually in the archive, there is no data to move if you need to update the configuration to use a different backend, but once an image analysis has been archived to update the configuration you must follow the object storage data migration process found here. As noted in that guide, if you need to migrate to/from an analysis_archive config you’ll need to use the –from-analysis-archive/–to-analysis-archive options as needed to tell the migration process which configuration to use in the source and destination config files used for the migration.

Common Configurations

1. Single shared object store backend: omit the analysis_archive config, or set it to null or {}

2. Different bucket/container: the object_store and analysis_archive configurations are both specified and identical with the exception of the bucket or container values for the analysis_archive so that its data is split into a different backend bucket to allow for lifecycle controls or cost optimization since its access is much less frequent (if ever).

3. Primary object store in DB, analysis_archive in external S3/Swift: this keeps latency low as no external service is needed for the object store and active data but lets you use more scalable external object storage for archive data. This approach is most beneficial if you can keep the working set of images small and quickly transition old analysis to the archive to ensure the db is kept small and the analysis archive handles the data scaling over time.

20.2 - Database Storage

Anchore stores all metadata in a structured format in a PostgreSQL database to support API operations and searches.

Examples of data persisted in the database:

• Image metadata (distro, version, layer counts, …)
• Image digests to tag mapping (docker.io/nginx:latest is hash sha256:abcd at time t)
• Image analysis content indexed for policy evaluation (files, packages, ..)
• Feed data
  • vulnerability info
  • package info from upstream (gem/npm)
• Accounts, users…
• …

If the object store is not explicitly set to an external provider, then that data is also persisted in the database but can be migrated

Reducing Database Storage Usage

Beyond enabling a non-DB object store there are some configuration options to reduce database storage and IO used by Anchore.

Configuration of Indexed DB Storage for Package DB File Entries

There is a configuration option for the policy engine service to disable the usage of the database for storing indexed package database entries from each analyzed image. This data represents the files in each distro package and their metadata (digests and permissions) from each scanned image in the image_package_db_entries table. That table is only used by the policy engine to deliver the policy trigger ‘packages.verify’, but if you do not use that trigger then the use of the storage can be disabled thereby reducing database load and resource usage. The data can be quite large, often in the thousands of rows per analyzed image, so for some customers that do not use this data for policy, disabling the loading of this data can reduce database consumption significantly.

Disabling Indexed DB Storage for Package DB File Entries

In each policy engine’s config.yaml file, change:

enable_package_db_load: true

to

enable_package_db_load: false

Note that disabling the table usage will also disable support for the packages.verify trigger and any policies that have the trigger in a rule will be considered invalid and return errors on evaluation. Any new policies that attempt to use the trigger will be rejected on upload as invalid if the trigger is included.

Once this configuration is set, you may delete data in that db table to reclaim some database storage capacity. If you’re interested in this option please contact support for guidance on this process.

Enabling Indexed DB Storage for Package DB File Entries

If you find that you do need the trigger, you can change the configuration to use the table then support will be restored. However, any images analyzed while the setting was ‘false’ will need to be re-analyzed in order to populate their data in that table correctly.

20.3 - Layer Caching

Once an image is submitted to Anchore Enterprise for analysis the system will attempt to retrieve metadata about the image from the Docker registry and if successful will download the image and queue the image for analysis.

Anchore Enterprise can run one or more analyzer services to scale out processing of images. The next available analyzer worker will process the image.

Docker Images are made up of one or more layers, which are described in the manifest. The manifest lists the layers which are typically stored as gzipped compressed TAR files.

As part of image analysis Anchore Enterprise will:

• Download all layers that comprise an image
• Extract the layers to a temporary file system location
• Perform analysis on the contents of the image including:
  • Digest of every file (SHA1, SHA256 and MD5)
  • File attributes (size, owner, permissions, etc)
  • Operating System package manifest
  • Software library package manifest (NPM, GEM, Java, Python, NuGet)
  • Scan for secret materials (api keys, private keys, etc

Following the analysis the extracted layers and downloaded layer tar files are deleted.

In many cases the images will share a number of common layers, especially if images are built form a consistent set of base images. To speed up Anchore Enterprise can be configure to cache image layers to eliminate the need to download the same layer for many different images. The layer cache is displayed in the default Anchore Enterprise configuration. To enable the cache the following changes should be made:

1. Define temporary directory for cache data

It is recommended that the cache data is stored in an external volume to ensure that the cache does not use up the ephemeral storage space allocated to the container host.

By default Anchore Enterprise uses the /tmp directory within the container to download and extract images. Configure a volume to be mounted into the container at a specified path and configure this path in config.yaml

tmp_dir: '/scratch'

In this example a volume has been mounted as /scratch within the container and config.yaml updated to use /scratch as the temporary directory for image analysis.

With the cache disabled the temporary directory should be sized to at least 3 times the uncompressed image size to be analyzed. To enable layer caching the layer_cache_enable parameter and layer_cache_max_gigabytes parameter should be added to the analyzer section of the Anchore Enterprise configuration file config.yaml.

analyzer:
    enabled: True
    require_auth: True
    cycle_timer_seconds: 1
    analyzer_driver: 'nodocker'
    endpoint_hostname: '${ANCHORE_HOST_ID}'
    listen: '0.0.0.0'
    port: 8084
    layer_cache_enable: True
    layer_cache_max_gigabytes: 4

In this example the cache is set to 4 gigabytes. The temporary volume should be sized to at least 3 times the uncompressed image size + 4 gigabytes.

• The minimum size for the cache is 1 gigabyte.
• The cache users a least recently used (LRU) policy.
• The cache files will be stored in the anchore_layercache directory of the /tmp_dir volume.

20.4 - Object Storage

Anchore Enterprise uses a PostgreSQL database to store structured data for images, tags, policies, subscriptions and metdata about images, but other types of data in the system are less structured and tend to be larger pieces of data. Because of that, there are benefits to supporting key-value access patterns for things like image manifests, analysis reports, and policy evaluations. For such data, Anchore has an internal object storage interface that, while defaulted to use the same Postgres database for storage, can be configured to use external object storage providers to support simpler capacity management and lower costs. The options are:

• PostgreSQL database (default)
• Filesystem
• S3 Object Store
• Swift Object Store

The configuration for the object store is set in the catalog’s service configuration in the config.yaml.

20.4.1 - Migrating Data to New Drivers

Overview

To cleanly migrate data from one archive driver to another, Anchore Enterprise includes some tooling that automates the process in the ‘anchore-manager’ tool packaged with the system.

The migration process is an offline process; Anchore Enterprise is not designed to handle an online migration.

For the migration process you will need:

1. The original config.yaml used by the services already, if services are split out or using different config.yaml for different services, you need the config.yaml used by the catalog services
2. An updated config.yaml (named dest-config.yaml in this example), with the archive driver section of the catalog service config set to the config you want to migrate to
3. The db connection string from config.yaml, this is needed by the anchore-manager script directly
4. Credentials and resources (bucket etc) for the destination of the migration

At a high-level the process is:

1. Shutdown all anchore enterprise services and components. The system should be fully offline, but the database must be online and available. For a docker-compose install, this is achieved by simply stopping the engine container, but not deleting it.
2. Prepare a new config.yaml that includes the new driver configuration for the destination of the migration (dest-config.yaml) in the same location as the existing config.yaml
3. Test the new dest-config.yaml to ensure correct configuration
4. Run the migration
5. Get coffee… this could take a while if you have a lot of analysis data
6. When complete, view the results
7. Ensure the dest-config.yaml is in place for all the components as config.yaml
8. Start anchore-engine

Migration Example Using Docker Compose Deployed Anchore Engine

The following is an example migration for an anchore-engine deployed via docker-compose on a single host with a local postgresql container–basically the example used in ‘Installing Anchore Engine’ documents. At the end of this section, we’ll cover the caveats and things to watch for a multi-node install of anchore engine.

This process requires that you run the command in a location that has access to both the source archive driver configuration and the new archive driver configuration.

Step 1: Shutdown all services

All services should be stopped, but the postgresql db must still be available and running.

docker-compose stop anchore-engine

Step 2: Prepare a new config.yaml

Both the original and new configurations are needed, so create a copy and update the archive driver section to the configuration you want to migrate to

cd config
cp config.yaml dest-config.yaml
<edit dest-config.yaml>

Step 3: Test the destination config

Assuming that config is dest-config.yaml:

[user@host aevolume]$ docker-compose run anchore-engine /bin/bash
[root@3209ad44d7bb ~]# anchore-manager objectstorage --db-connect ${db} check /config/dest-config.yaml 
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB params: {"db_pool_size": 30, "db_connect": "postgresql+pg8000://postgres:postgres.dev@postgres-dev:5432/postgres", "db_connect_args": {"ssl": false, "connect_timeout": 120, "timeout": 30}, "db_pool_max_overflow": 100}
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB connection configured: True
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB attempting to connect...
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB connected: True
[MainThread] [anchore_manager.cli.objectstorage/check()] [INFO] Using config file /config/dest-config.yaml
[MainThread] [anchore_engine.subsys.object_store.operations/initialize()] [INFO] Archive initialization complete
[MainThread] [anchore_manager.cli.objectstorage/check()] [INFO] Checking existence of test document with user_id = test, bucket = anchorecliconfigtest and archive_id = cliconfigtest
[MainThread] [anchore_manager.cli.objectstorage/check()] [INFO] Creating test document with user_id = test, bucket = anchorecliconfigtest and archive_id = cliconfigtest
[MainThread] [anchore_manager.cli.objectstorage/check()] [INFO] Checking document fetch
[MainThread] [anchore_manager.cli.objectstorage/check()] [INFO] Removing test object
[MainThread] [anchore_manager.cli.objectstorage/check()] [INFO] Archive config check completed successfully

Step 3a: Test the current config.yaml

If you are running the migration for a different location than one of the anchore engine containers

Same as above but using /config/config.yaml as the input to check (skipped in this instance since we’re running the migration from the same container)

Step 4: Run the Migration

By default, the migration process will remove data from the source once it has confirmed it has been copied to the destination and the metadata has been updated in the anchore db. To skip the deletion on the source, use the ‘–nodelete’ option. it is the safest option, but if you use it, you are responsible for removing the data later.

[root@3209ad44d7bb ~]# anchore-manager objectstorage --db-connect ${db} migrate /config/config.yaml /config/dest-config.yaml 
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB params: {"db_pool_size": 30, "db_connect": "postgresql+pg8000://postgres:postgres.dev@postgres-dev:5432/postgres", "db_connect_args": {"ssl": false, "connect_timeout": 120, "timeout": 30}, "db_pool_max_overflow": 100}
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB connection configured: True
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB attempting to connect...
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB connected: True
[MainThread] [anchore_manager.cli.objectstorage/migrate()] [INFO] Loading configs
[MainThread] [anchore_manager.cli.objectstorage/migrate()] [INFO] Migration from config: {
  "storage_driver": {
    "config": {}, 
    "name": "db"
  }, 
  "compression": {
    "enabled": false, 
    "min_size_kbytes": 100
  }
}
[MainThread] [anchore_manager.cli.objectstorage/migrate()] [INFO] Migration to config: {
  "storage_driver": {
    "config": {
      "access_key": "9EB92C7W61YPFQ6QLDOU", 
      "create_bucket": true, 
      "url": "http://minio-ephemeral-test:9000/", 
      "region": false, 
      "bucket": "anchore-engine-testing", 
      "prefix": "internaltest", 
      "secret_key": "TuHo2UbBx+amD3YiCeidy+R3q82MPTPiyd+dlW+s"
    }, 
    "name": "s3"
  }, 
  "compression": {
    "enabled": true, 
    "min_size_kbytes": 100
  }
}
Performing this operation requires *all* anchore-engine services to be stopped - proceed? (y/N)y
[MainThread] [anchore_engine.subsys.object_store.migration/initiate_migration()] [INFO] Initializing migration from {'storage_driver': {'config': {}, 'name': 'db'}, 'compression': {'enabled': False, 'min_size_kbytes': 100}} to {'storage_driver': {'config': {'access_key': '9EB92C7W61YPFQ6QLDOU', 'create_bucket': True, 'url': 'http://minio-ephemeral-test:9000/', 'region': False, 'bucket': 'anchore-engine-testing', 'prefix': 'internaltest', 'secret_key': 'TuHo2UbBx+amD3YiCeidy+R3q82MPTPiyd+dlW+s'}, 'name': 's3'}, 'compression': {'enabled': True, 'min_size_kbytes': 100}}
[MainThread] [anchore_engine.subsys.object_store.migration/migration_context()] [INFO] Initializing source object_store: {'storage_driver': {'config': {}, 'name': 'db'}, 'compression': {'enabled': False, 'min_size_kbytes': 100}}
[MainThread] [anchore_engine.subsys.object_store.migration/migration_context()] [INFO] Initializing dest object_store: {'storage_driver': {'config': {'access_key': '9EB92C7W61YPFQ6QLDOU', 'create_bucket': True, 'url': 'http://minio-ephemeral-test:9000/', 'region': False, 'bucket': 'anchore-engine-testing', 'prefix': 'internaltest', 'secret_key': 'TuHo2UbBx+amD3YiCeidy+R3q82MPTPiyd+dlW+s'}, 'name': 's3'}, 'compression': {'enabled': True, 'min_size_kbytes': 100}}
[MainThread] [anchore_engine.subsys.object_store.migration/initiate_migration()] [INFO] Migration Task Id: 1
[MainThread] [anchore_engine.subsys.object_store.migration/initiate_migration()] [INFO] Entering main migration loop
[MainThread] [anchore_engine.subsys.object_store.migration/initiate_migration()] [INFO] Migrating 7 documents
[MainThread] [anchore_engine.subsys.object_store.migration/initiate_migration()] [INFO] Deleting document on source after successful migration to destination. Src = db://admin/policy_bundles/2c53a13c-1765-11e8-82ef-23527761d060
[MainThread] [anchore_engine.subsys.object_store.migration/initiate_migration()] [INFO] Deleting document on source after successful migration to destination. Src = db://admin/manifest_data/sha256:0873c923e00e0fd2ba78041bfb64a105e1ecb7678916d1f7776311e45bf5634b
[MainThread] [anchore_engine.subsys.object_store.migration/initiate_migration()] [INFO] Deleting document on source after successful migration to destination. Src = db://admin/analysis_data/sha256:0873c923e00e0fd2ba78041bfb64a105e1ecb7678916d1f7776311e45bf5634b
[MainThread] [anchore_engine.subsys.object_store.migration/initiate_migration()] [INFO] Deleting document on source after successful migration to destination. Src = db://admin/image_content_data/sha256:0873c923e00e0fd2ba78041bfb64a105e1ecb7678916d1f7776311e45bf5634b
[MainThread] [anchore_engine.subsys.object_store.migration/initiate_migration()] [INFO] Deleting document on source after successful migration to destination. Src = db://admin/manifest_data/sha256:a0cd2c88c5cc65499e959ac33c8ebab45f24e6348b48d8c34fd2308fcb0cc138
[MainThread] [anchore_engine.subsys.object_store.migration/initiate_migration()] [INFO] Deleting document on source after successful migration to destination. Src = db://admin/analysis_data/sha256:a0cd2c88c5cc65499e959ac33c8ebab45f24e6348b48d8c34fd2308fcb0cc138
[MainThread] [anchore_engine.subsys.object_store.migration/initiate_migration()] [INFO] Deleting document on source after successful migration to destination. Src = db://admin/image_content_data/sha256:a0cd2c88c5cc65499e959ac33c8ebab45f24e6348b48d8c34fd2308fcb0cc138
[MainThread] [anchore_engine.subsys.object_store.migration/initiate_migration()] [INFO] Migration result summary: {"last_state": "running", "executor_id": "3209ad44d7bb:37:139731996518208:", "archive_documents_migrated": 7, "last_updated": "2018-08-15T18:03:52.951364", "online_migration": null, "created_at": "2018-08-15T18:03:52.951354", "migrate_from_driver": "db", "archive_documents_to_migrate": 7, "state": "complete", "migrate_to_driver": "s3", "ended_at": "2018-08-15T18:03:53.720554", "started_at": "2018-08-15T18:03:52.949956", "type": "archivemigrationtask", "id": 1}
[MainThread] [anchore_manager.cli.objectstorage/migrate()] [INFO] After this migration, your anchore-engine config.yaml MUST have the following configuration options added before starting up again:
compression:
  enabled: true
  min_size_kbytes: 100
storage_driver:
  config:
    access_key: 9EB92C7W61YPFQ6QLDOU
    bucket: anchore-engine-testing
    create_bucket: true
    prefix: internaltest
    region: false
    secret_key: TuHo2UbBx+amD3YiCeidy+R3q82MPTPiyd+dlW+s
    url: http://minio-ephemeral-test:9000/
  name: s3

Note: If something goes wrong you can reverse the parameters of the migrate command to migrate back to the original configuration (e.g. … migrate /config/dest-config.yaml /config/config.yaml)

Step 5: Get coffee!

The migration time will depend on the amount of data and the source and destination systems performance.

Step 6: View results summary

[root@3209ad44d7bb ~]# anchore-manager objectstorage --db-connect ${db} list-migrations
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB params: {"db_pool_size": 30, "db_connect": "postgresql+pg8000://postgres:postgres.dev@postgres-dev:5432/postgres", "db_connect_args": {"ssl": false, "connect_timeout": 120, "timeout": 30}, "db_pool_max_overflow": 100}
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB connection configured: True
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB attempting to connect...
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB connected: True
id         state                  start time                         end time                 from        to        migrated count        total to migrate               last updated               
1         complete        2018-08-15T18:03:52.949956        2018-08-15T18:03:53.720554         db         s3              7                      7                2018-08-15T18:03:53.724628   

This lists all migrations for the service and the number of objects migrated. If you’ve run multiple migrations you’ll see multiple rows in this response.

Step 7: Replace old config.yaml with updated dest-config.yaml

[root@3209ad44d7bb ~]# cp /config/config.yaml /config/config.old.yaml
[root@3209ad44d7bb ~]# cp /config/dest-config.yaml /config/config.yaml

Step 8: Restart anchore-engine services

[user@host aevolume]$ docker-compose start anchore-engine

The system should now be up and running using the new configuration! You can verify with the anchorectl command by fetching a policy, which will have been migrated:

[root@d8d3f49d9328 /]# anchorectl policy list
 ✔ Fetched policies
┌─────────────────────────┬──────────────────────────────────────┬────────┬──────────────────────┐
│ NAME                    │ POLICY ID                            │ ACTIVE │ UPDATED              │
├─────────────────────────┼──────────────────────────────────────┼────────┼──────────────────────┤
│ Default bundle          │ 2c53a13c-1765-11e8-82ef-23527761d060 │ true   │ 2022-07-14T22:52:27Z │
│ anchore_security_only   │ anchore_security_only                │ false  │ 2022-07-14T22:52:27Z │
│ anchore_cis_1.13.0_base │ anchore_cis_1.13.0_base              │ false  │ 2022-07-14T22:52:27Z │
└─────────────────────────┴──────────────────────────────────────┴────────┴──────────────────────┘

[root@d8d3f49d9328 /]# anchorectl -o json-raw policy get 2c53a13c-1765-11e8-82ef-23527761d060 
[ 
  {
    "blacklisted_images": [], 
    "comment": "Default bundle", 
    "id": "2c53a13c-1765-11e8-82ef-23527761d060", 
... <lots of json>

If that returns the content properly, then you’re all done!

Things to Watch for in a Multi-Node Anchore Engine Installation

• Before migration: Ensure all services are down before starting migration
• At migration: Ensure the place you’re running the migration from has the same db access and network access to the archive locations
• After migration: Ensure that all components get the update config.yaml. Strictly speaking, only containers that run the catalog service need the update configuration, but its best to ensure that any config.yaml in the system which has a services.catalog definition also has the proper and up-to-date configuration to avoid confusion or accidental reverting of the config.

Example Process with docker-compose

# ls docker-compose.yaml 
docker-compose.yaml

# docker-compose ps
              Name                            Command               State                       Ports                     
--------------------------------------------------------------------------------------------------------------------------
aevolumepy3_anchore-db_1           docker-entrypoint.sh postgres    Up      5432/tcp                                      
aevolumepy3_anchore-engine_1       /bin/sh -c anchore-engine        Up      0.0.0.0:8228->8228/tcp, 0.0.0.0:8338->8338/tcp
aevolumepy3_anchore-minio_1        /usr/bin/docker-entrypoint ...   Up      0.0.0.0:9000->9000/tcp                        
aevolumepy3_anchore-prometheus_1   /bin/prometheus --config.f ...   Up      0.0.0.0:9090->9090/tcp                        
aevolumepy3_anchore-redis_1        docker-entrypoint.sh redis ...   Up      6379/tcp                                      
aevolumepy3_anchore-ui_1           /bin/sh -c node /home/node ...   Up      0.0.0.0:3000->3000/tcp                        

# docker-compose stop anchore-engine
Stopping aevolume_anchore-engine_1 ... done

# docker-compose run anchore-engine anchore-manager objectstorage --db-connect postgresql+pg8000://postgres:mysecretpassword@anchore-db:5432/postgres check /config/config.yaml.new
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB params: {"db_connect": "postgresql+pg8000://postgres:mysecretpassword@anchore-db:5432/postgres", "db_connect_args": {"timeout": 30, "ssl": false}, "db_pool_size": 30, "db_pool_max_overflow": 100}
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB connection configured: True
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB attempting to connect...
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB connected: True
[MainThread] [anchore_manager.cli.objectstorage/check()] [INFO] Using config file /config/config.yaml.new
...
...

# docker-compose run anchore-engine anchore-manager objectstorage --db-connect postgresql+pg8000://postgres:mysecretpassword@anchore-db:5432/postgres migrate /config/config.yaml /config/config.yaml.new
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB params: {"db_connect": "postgresql+pg8000://postgres:mysecretpassword@anchore-db:5432/postgres", "db_connect_args": {"timeout": 30, "ssl": false}, "db_pool_size": 30, "db_pool_max_overflow": 100}
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB connection configured: True
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB attempting to connect...
[MainThread] [anchore_manager.cli.utils/connect_database()] [INFO] DB connected: True
[MainThread] [anchore_manager.cli.objectstorage/migrate()] [INFO] Loading configs
[MainThread] [anchore_engine.configuration.localconfig/validate_config()] [WARN] no webhooks defined in configuration file - notifications will be disabled
[MainThread] [anchore_engine.configuration.localconfig/validate_config()] [WARN] no webhooks defined in configuration file - notifications will be disabled
[MainThread] [anchore_manager.cli.objectstorage/migrate()] [INFO] Migration from config: {
  "compression": {
    "enabled": false,
    "min_size_kbytes": 100
  },
  "storage_driver": {
    "name": "db",
    "config": {}
  }
}
[MainThread] [anchore_manager.cli.objectstorage/migrate()] [INFO] Migration to config: {
  "compression": {
    "enabled": true,
    "min_size_kbytes": 100
  },
  "storage_driver": {
    "name": "s3",
    "config": {
      "access_key": "Z54LPSMFKXSP2E2L4TGX",
      "secret_key": "EMaLAWLVhUmV/f6hnEqjJo5+/WeZ7ukyHaBKlscB",
      "url": "http://anchore-minio:9000",
      "region": false,
      "bucket": "anchorearchive",
      "create_bucket": true
    }
  }
}
Performing this operation requires *all* anchore-engine services to be stopped - proceed? (y/N) y
...
...
...
[MainThread] [anchore_engine.subsys.object_store.migration/initiate_migration()] [INFO] Migration result summary: {"last_updated": "2018-08-14T22:19:39.985250", "started_at": "2018-08-14T22:19:39.984603", "last_state": "running", "online_migration": null, "archive_documents_migrated": 500, "migrate_to_driver": "s3", "id": 9, "executor_id": "e9fc8f77714d:1:140375539468096:", "ended_at": "2018-08-14T22:20:03.957291", "created_at": "2018-08-14T22:19:39.985246", "state": "complete", "archive_documents_to_migrate": 500, "migrate_from_driver": "db", "type": "archivemigrationtask"}
[MainThread] [anchore_manager.cli.objectstorage/migrate()] [INFO] After this migration, your anchore-engine config.yaml MUST have the following configuration options added before starting up again:
...
...

# cp config/config.yaml config/config.yaml.original

# cp config/config.yaml.new config/config.yaml

# docker-compose start anchore-engine
Starting anchore-engine ... done

Migrating Analysis Archive Data

The object storage migration process migrates any data stored in the source config to the destination configuration, if the analysis archive is configured to use the same storage backend as the primary object store then that data is migrated along with all other data, but if the source or destination configurations define different storage backends for the analysis archive than that which is used by the primary object store, then additional paramters are necesary in the migration commands to indicate which configurations to migrate to/from.

The most common migration patterns are:

1. Migrate from a single backend configuration to a split configuration to move analysis archive data to an external system (db -> db + s3)

2. Migrate from a dual-backend configuration to a single-backend configuration with a different config (e.g. db + s3 -> s3)

Migrating a single backend to split backend

For example, moving from unified db backend (default config) to a db + s3 configuration with s3 for the analysis archive .

source-config.yaml snippet:

...
services:
...
  catalog:
    ...
    object_store:
      compression:
        enabled: false
        min_size_kbytes: 100
      storage_driver:
        name: db
        config: {}        
...

dest-config.yaml snippet:

...
services:
...
  catalog:
    ...
    object_store:
      compression:
        enabled: false
        min_size_kbytes: 100
      storage_driver:
        name: db
        config: {}
    analysis_archive:
      enabled: true
      compression:
        enabled: false
        min_size_kbytes: 100
      storage_driver:
        name: s3
        config: 
          access_key: 9EB92C7W61YPFQ6QLDOU
          secret_key: TuHo2UbBx+amD3YiCeidy+R3q82MPTPiyd+dlW+s
          url: 'http://minio-ephemeral-test:9000'
          region: null
          bucket: analysisarchive
      ...   

Anchore stores its internal data in logical ‘buckets’ that are overlayed onto the storage backed in a driver-specific way, so to migrate specific internal buckets (effectively these are classes of data), use the –bucket option in the manager cli. This should generally not be necessary, but for specific kinds of migrations it may be needed.

The following command will execute the migration. Note that the –bucket option is for an internal Anchore logical-bucket, not and actual bucket in S3:

anchore-manager objectstorage --db-connect migrate --to-analysis-archive --bucket analysis_archive source-config.yaml dest-config.yaml

Migrating from dual object storage backends to a single backend

For example, migrating from a db + s3 backend to a single s3 backend in a different bucket:

Example source-config.yaml snippet:

...
services:
...
  catalog:
    ...
    object_store:
      compression:
        enabled: false
        min_size_kbytes: 100
      storage_driver:
        name: db
        config: {}
    analysis_archive:
      enabled: true
      compression:
        enabled: false
        min_size_kbytes: 100
      storage_driver:
        name: s3
        config: 
          access_key: 9EB92C7W61YPFQ6QLDOU
          secret_key: TuHo2UbBx+amD3YiCeidy+R3q82MPTPiyd+dlW+s
          url: 'http://minio-ephemeral-test:9000'
          region: null
          bucket: analysisarchive        
...

The dest config is a single backend. In this case, note the S3 bucket has changed so all data must be migrated.

Example dest-config.yaml snippet:

...
services:
...
  catalog:
    ...
    object_store:
      enabled: true
      compression:
        enabled: false
        min_size_kbytes: 100
      storage_driver:
        name: s3
        config: 
          access_key: 9EB92C7W61YPFQ6QLDOU
          secret_key: TuHo2UbBx+amD3YiCeidy+R3q82MPTPiyd+dlW+s
          url: 'http://minio-ephemeral-test:9000'
          region: null
          bucket: newanchorebucket
      ...

First, migrate the object data in the db on the source:

anchore-manager objectstorage --db-connect migrate source-config.yaml dest-config.yaml

Next, migrate the object data in the analysis archive from the old config (s3 bucket ‘analysisarchive’ to the new config (s3 bucket ’newanchorebucket’):

anchore-manager objectstorage --db-connect migrate --from-analysis-archive source-config.yaml dest-config.yaml  

20.4.2 - Database Driver

The default object store driver is the PostgreSQL database driver which stores all object store documents within the PostgreSQL database.

Compression is not supported for this driver since the underlying database will handle compression.

There are no configuration options required for the Database driver.

The embedded configuration for anchore enterprise includes the default configuration for the db driver.

object_store:
  compression:
    enabled: False
    min_size_kbytes: 100
  storage_driver:
    name: db
    config: {}

20.4.3 - Filesystem Driver

Using the file system driver object store documents can be stored on a filesystem volume passed to the Anchore Enterprise container.

The filesystem driver does not handle distribution or replication. To replicate the file system across nodes for performance and redundancy a clustered / shared filesystem such as Gluster, CephFS or Amazon EFS should be used.

WARNING: This driver is not recommended for scale-out production deployments

For environments who do not want to utilize the default PostgresSQL storage and need scale or redundancy an object store such as S3 or Swift will provide a better solution than the filesystem driver.

Compression

The localfs (filesystem) driver supports compression of object_store documents. The object_store documents are JSON formatted and will see significant reduction in size through compression there is an overhead incurred by running compression and decompression on every access of these documents. The Anchore Enterprise can be configured to only compress documents above a certain size to reduce unnecessary overhead. In the example below any document over 100kb in size will be compressed.

object_store:
  compression:
    enabled: True
    min_size_kbytes: 100
  storage_driver:
    name: localfs
    config:
      archive_data_dir: '/object_store'

20.4.4 - S3 Object Store Driver

Using the S3 driver, data can be stored using Amazon’s S3 storage or any Amazon S3 API compatible system.

object_store:
  compression:
    enabled: False
    min_size_kbytes: 100
  storage_driver:
    name: 's3'
    config:
      access_key: 'MY_ACCESS_KEY'
      secret_key: 'MY_SECRET_KEY'
      #iamauto: True
      url: 'https://S3-end-point.example.com'
      region: False
      bucket: 'anchorearchive'
      create_bucket: True

Compression

The S3 driver supports compression of documents. The documents are JSON formatted and will see significant reduction in size through compression there is an overhead incurred by running compression and decompression on every access of these documents. Anchore Enterprise can be configured to only compress documents above a certain size to reduce unnecessary overhead. In the example below any document over 100kb in size will be compressed.

Authentication

Anchore Enterprise can authenticate against the S3 service using one of two methods:

• Amazon Access Keys Using this method an Access Key and Secret Access key that have access to read and write to the bucket. Parameters: access_key and secret_key

• Inherit IAM Role Anchore Enterprise can be configured to inherit the IAM role from the EC2 or ECS instance that is running Anchore Enterprise. When launching the EC2 instance that will run Anchore Enterprise you need to specify a role that includes the ability to read and write from the archive bucket. To use IAM roles to authenticate the access_key and secret_access configurations should be replaced by iamauto: True Parameters: iamauto

Bucket

• The url parameter points to the endpoint for the Amazon S3 bucket
• The region parameter should be set to the AWS region hosting the bucket or False for an AWS compatible service that does not support regions
• The bucket parameter should be set to the name of the bucket that will be used to store the archive documents.
• The create_bucket parameter is used to configure if Anchore Enterprise attempts to create a bucket. If this option is set then ensure that the IAM role or Access Keys have sufficient access to create a new bucket.

20.4.5 - Swift Archive Driver

Using the Swift driver archive documents can be stored in an OpenStack Swift Object storage service.

The Swift driver supports three authentication methods:

• Keystone V3
• Keystone V2
• Legacy (username / password)

Common Configuration Options

Compression

The Swift driver supports compression of archive documents. The archive documents are JSON formatted and will see significant reduction in size through compression there is an overhead incurred by running compression and decompression on every access of these documents. Anchore Enterprise can be configured to only compress documents above a certain size to reduce unnecessary overhead. In the example below any document over 100kb in size will be compressed.

Container

• The container parameter specifies the name of the container to be used.
• The create_container parameter is used to configure if Anchore Enterprise attempts to create a container. If this option is set then ensure that the user has the appropriate rights to create a container.

Legacy Authentication

object_store:
  compression:
    enabled: True
    min_size_kbytes: 100
  storage_driver:
    name: 'swift'
    config:
      user: 'user:password'
      auth: 'http://swift.example.com:8080/auth/v1.0'
      key:  'anchore'
      container: 'anchorearchive'
      create_container: True

• The user configuration option should include the colon delimited username and password. eg. ‘admin:secret’
• The auth parameter specifies the authentication end point for the Swift Service
• The key parameter specifies the key for the Swift Service

Keystone V3

object_store:
  compression:
    enabled: True
    min_size_kbytes: 100
  storage_driver:
     name: 'swift'
     config:
        auth_version: '3'
        os_username: 'myusername'
        os_password: 'mypassword'
        os_project_name: myproject
        os_project_domain_name: example.com
        os_auth_url: 'foo.example.com:8000/auth/etc'
       container: 'anchorearchive'
       create_container: True

• The auth_version configuration option specified Keystone V3 authentication
• The os_username parameter specifies the username
• The os_password parameter specifies the password
• The os_project_name parameter specifies the OpenStack project name under which the Swift service is configured
• The os_project_domain_name parameter specifies the domain name for the OpenStack project
• The os_auth_url parameter specifies the URL to the OpenStack Keystone service

Keystone V2

object_store:
  compression:
    enabled: true
    min_size_kbyte: 100
  storage_driver:    
    name: 'swift'
    config:
      auth_version: '2'
      os_username: 'myusername'
      os_password: 'mypassword'
      os_tenant_name: 'mytenant'
      os_auth_url: 'foo.example.com:8000/auth/etc'

• The auth_version configuration option specified Keystone V3 authentication
• The os_username parameter specifies the username
• The os_password parameter specifies the password
• The os_tenant_name parameter specifies the name of the OpenStack tenant under which the Swift service is configured
• The os_auth_url parameter specifies the URL to the OpenStack Keystone service

Note

The Anchore Enterprise archive drivers users the OpenStack Python SwiftClient library. The config section is passed to the SwiftClient library allowing any advanced parameters supported by the Swift library to be configured.

21 - Working with Subscriptions

Introduction

Anchore Enterprise supports 7 types of subscriptions:

• Tag Update
• Policy Update
• Vulnerability Update
• Analysis Update
• Alerts
• Repository Update
• Runtime Inventory

Enabling some of these will generate a notification when the event is triggered while others may have a more significant impact on the system.

Tag Update

When the tag_update subscription is enabled, a background process, called a “watcher”, will periodically query the repository for any new image digests with the same tag. For each new image digest found:

• it will be pulled into the catalog and analyzed
• a Tag Update Notification will be triggered.

Policy Updates

This class of notification is triggered if a Tag to which a user has subscribed has a change in its policy evaluation status. The policy evaluation status of an image can be one of two states: Pass or Fail. If an image that was previously marked as Pass changes status to Fail or vice-versa then the policy update notification will be triggered.

The policy status of a Tag may be changed by a number of methods.

• Change to policy
  • If an policy was changed, for example adding, editing or removing a policy check, then the policy status of an image may be effected. For example adding policy rule that denylists a specific package that is present in a given Tag may cause the Tag’s policy status to move to Fail.
• Changes to Allowlist
  • If a allowlist is changed to add or remove a CVE then this may cause a policy status change. For example if an image contains a package that is vulnerable to Crticial Severity CVE-2017-9999 then this image may fail in it’s policy evaluation. If CVE-2017-9999 is added to a CVE Allowlist that is mapped to the subscribed Tag then the policy status may change from Fail to Pass.
• Change in Policy / Allowlist Mapping
  • If the policy mapping is changed then a new policy or allowlist may be applied to an image which may change the status of the image. For example changing the mapping to add a more restrictive policy may change an Tag’s status from Pass to Fail.
• Change in Package or Vulnerability Data
  • Some policy checks make use of data from external feeds. For example vulnerability checks use CVE data feeds. Changes in data within these feed may change the policy status, such as adding a new CVE vulnerability.

Vulnerability / CVE Update

This class of notification is triggered if the list of CVEs or other security vulnerabilities in the image changes.

For example, a user was subscribed to the library/nginx:latest tag. On the 12th of September 2017 a new vulnerability was added to the Debian 9 vulnerability feed which matched a package in the library/nginx:latest image, triggering a notification.

Based on the changes made by the upstream providers of CVE data (operating system vendors and NIST) CVEs may be added, removed or modified – for example a CVE initially marked as severity level Unknown may be upgraded to a higher severity level.

Note: A change to the CVE list in a Tag may not trigger a policy status change based on the policy rules configured for an image. In the example above the CVE had an unknown severity level which may not be tested by the policy mapped to this image.

Analysis Update

This class of notification is triggered when an image has been analyzed. Typically, this is triggered when a new Tag has been added to the catalog. A common use case for this trigger is to alert an external system that a new Tag was added and has been successfully analyzed. Forcing a re-analysis on an existing image will also cause this notification to be generated.

Alerts

The UI and API use stateful alerts that will be raised for policy violations on tags to which you are subscribed for alerts. This raises a clear notification in the UI to help initiate the remediation workflow and address the violations via the remediation feature. Once all findings are addressed the alert is closed, allowing an efficient workflow for users to bring their image’s into compliance with their policy.

Repository Update

This subscription, when enabled, will query the provided repository for any new tags. Any tag not already managed with in Anchore, will be added. This subscription also provides the ability to determine if the tag_update subscription should be enabled for any new tag added to Anchore.

Please Note: Enabling this subscription may add a large number of tags to the system.

Runtime Inventory

This subscription, when enabled, will find any newly reported images from the runtime inventory and add them to Anchore to be analyzed.

Please Note: Enabling this subscription may add a large number of tags to the system.

22 - TLS / SSL

Communication with the Anchore Engine and between Anchore Engine service can be secured with TLS/SSL. This can be performed in two ways

• Externally through a load balancing, ingress controller or reverse proxy such as NGINX
• Natively within the Anchore Engine

For most use cases an external service such as a proxy or load balancer will provide the simplest approach, especially when keys need to be rotated, new instances added, etc.

The Anchore Engine is comprised of 6 services, typically only the external API service (apiext) and Kubernetes webhook service are published externally, all other services are used for internal communication.

Transport Level Security (TLS/SSL) is enabled on a per-service basis. In the sample configuration the SSL/TLS configuration options are commented out.

In the following example the external API service is configured to listen on port 443 and is configured with a certificate for its external hostname anchore.example.com

Each service published in the Anchore Engine configuration (apiext, catalog, simplequeue, analyzer, policy_engine and kubernetes_webhook) can be configured to use transport level security.

services:
  apiext:
    enabled: True
    endpoint_hostname: 'anchore.example.com'
    listen: '0.0.0.0'
    port: 443
    ssl_enable: True
    ssl_cert: '/config/anchore-ex.crt'
    ssl_key: '/config/anchore-ex.key'
    ssl_chain: '/config/anchore-ex.crt'

The certificate files should be placed on a path accessible to the Anchore Engine service, for example in the /config directory which is typically mapped as a volume into the container. Note that the location outside the container will depend on your configuration - for example if you are volume mounting ‘/path/to/aevolume/config/’ on the docker host to ‘/config’ within the container, you’ll need to place the ssl files in ‘/path/to/aevolume/config/’ on the docker host, so that they are accessible in ‘/config/’ inside the container, before starting the service.

The ssl_chain file is optional and may be required by some certificate authorities. If your certificate authority provides a chain certificate then include it within the configuration.

Note: While a certificate may be purchased from a well-known and trusted certificate authority in some cases the certificate is signed by an intermediate certificate which is not included within a TLS/SSL clients trust stores. In these cases the intermediate certificate is signed by the certificate authority however without the full ‘chain’ showing the provenance and integrity of the certificate the TLS/SSL client may not trust the certificate.

Any certificates used by the Anchore Engine services need to be trusted by all other Anchore Engine services.

If an internal certificate authority is used the root certificate for the internal CA can be added to the Anchore engine using the following procedure or SSL verification can be disabled by setting the following parameter:

internal_ssl_verify: True

23 - Enterprise UI Configuration

The Enterprise UI service has some static configuration options that are read from /config/config-ui.yaml inside the UI container image when the system starts up.

The configuration is designed to not require any modification when using the quickstart (docker-compose) or production (Helm) methods of deploying Anchore Enterprise. If modifications are desired, the options, their meanings, and environment overrides are listed below for reference:

• The (required) license_path key specifies the location of the local system folder containing the license.yaml license file required by the Anchore Enterprise UI web service for product activation. This value can be overridden by using the ANCHORE_LICENSE_PATH environment variable.

  license_path: '/'
  

• The (required) engine_uri key specifies the address of the Anchore Engine service. The value must be a string containing a properly-formed ‘http’ or ‘https’ URI. This value can be overridden by using the ANCHORE_ENGINE_URI environment variable.

  engine_uri: 'http://engine-api:8228/v1'
  

• The (required) redis_uri key specifies the address of the Redis service. The value must be a string containing a properly-formed ‘http’, ‘https’, or redis URI. Note that the default configuration uses the REdis Serialization Protocol (RESP). This value can be overridden by using the ANCHORE_REDIS_URI environment variable.

  redis_uri: 'redis://enterprise-ui-redis:6379'
  

• The (required) appdb_uri key specifies the location and credentials for the postgres DB endpoint used by the UI. The value must contain the host, port, DB user, DB password, and DB name. This value can be overridden by using the ANCHORE_APPDB_URI environment variable.

  appdb_uri: 'postgres://<db-user>:<db-pass>@<db-host>:<db-port>/<db-name>'
  

• The (required) reports_uri key specifies the address of the Reports service. The value must be a string containing a properly-formed ‘http’ or ‘https’ URI and can be overridden by using the ANCHORE_REPORTS_URI environment variable.

  Note that the presence of an uncommented reports_uri key in this file (even if unset, or set with an invalid value) instructs the Anchore Enterprise UI web service that the Reports feature must be enabled.

  reports_uri: 'http://enterprise-reports:8228/v1'
  

• The (optional) rbac_uri key specifies the address of the Role-Based Authentication Control (RBAC) service. The value must be a string containing a properly-formed ‘http’ or ‘https’ URI, and can be overridden by using the ANCHORE_RBAC_URI environment variable.

  Note that the presence of an uncommented rbac_uri key in this file (even if unset, or set with an invalid value) instructs the Anchore Enterprise UI web service that the RBAC feature must be enabled. If the RBAC service cannot subsequently be reached by the web service, the communication failure will be handled in the same manner as an Anchore Engine service outage.

  rbac_uri: 'http://enterprise-rbac-manager:8228/v1'
  

• The (optional) enable_ssl key specifies if SSL operations should be enabled within in the web app runtime. When this value is set to True, secure cookies will be used with a SameSite value of None. The value must be a Boolean, and defaults to False if unset.

  Note: Only enable this property if your UI deployment configured to run within an SSL-enabled environment (for example, behind a reverse proxy, in the presence of signed certs etc.)

  This value can be overridden by using the ANCHORE_ENABLE_SSL environment variable.

  enable_ssl: False
  

• The (optional) enable_proxy key specifies whether to trust a reverse proxy when setting secure cookies (via the X-Forwarded-Proto header). The value must be a Boolean, and defaults to False if unset. In addition, SSL must be enabled for this to work. This value can be overridden by using the ANCHORE_ENABLE_PROXY environment variable.

  enable_proxy: False
  

• The (optional) allow_shared_login key specifies if a single set of user credentials can be used to start multiple Anchore Enterprise UI sessions; for example, by multiple users across different systems, or by a single user on a single system across multiple browsers.

  When set to False, only one session per credential is permitted at a time, and logging in will invalidate any other sessions that are using the same set of credentials. If this property is unset, or is set to anything other than a Boolean, the web service will default to True.

  Note that setting this property to False does not prevent a single session from being viewed within multiple tabs inside the same browser. This value can be overridden by using the ANCHORE_ALLOW_SHARED_LOGIN environment variable.

  allow_shared_login: True
  

• The (optional) redis_flushdb key specifies if the Redis datastore containing user session keys and data is emptied on application startup. If the datastore is flushed, any users with active sessions will be required to re-authenticate.

  If this property is unset, or is set to anything other than a Boolean, the web service will default to True. This value can be overridden by using the ANCHORE_REDIS_FLUSHDB environment variable.

  redis_flushdb: True
  

• The (optional) custom_links key allows a list of up to 10 external links to be provided (additional items will be excluded). The top-level title key provided the label for the menu (if present, otherwise the string “Custom External Links” will be used instead).

  Each link entry must have a title of greater than 0-length and a valid URI. If either item is invalid, the entry will be excluded.

  custom_links:
    title: Custom External Links
    links:
    - title: Example Link 1
      uri: https://example.com
    - title: Example Link 2
      uri: https://example.com
    - title: Example Link 3
      uri: https://example.com
    - title: Example Link 4
      uri: https://example.com
    - title: Example Link 5
      uri: https://example.com
    - title: Example Link 6
      uri: https://example.com
    - title: Example Link 7
      uri: https://example.com
    - title: Example Link 8
      uri: https://example.com
    - title: Example Link 9
      uri: https://example.com
    - title: Example Link 10
      uri: https://example.com
  

• The (optional) force_websocket key specifies if the WebSocket protocol must be used for socket message communications. By default, long-polling is initially used to establish the handshake between client and web service, followed by a switch to WS if the WebSocket protocol is supported.

  If this value is unset, or is set to anything other than a Boolean, the web service will default to False.

  This value can be overridden by using the ANCHORE_FORCE_WEBSOCKET environment variable.

  force_websocket: False
  

• The (optional) authentication_lock keys specify if a user should be temporarily prevented from logging in to an account after one or more failed authentication attempts. For this feature to be enabled, both values must be whole numbers greater than 0. They can be overridden by using the ANCHORE_AUTHENTICATION_LOCK_COUNT and ANCHORE_AUTHENTICATION_LOCK_EXPIRES environment variables.

  The count value represents the number of failed authentication attempts allowed to take place before a temporary lock is applied to the username. The expires value represents, in seconds, how long the lock will be applied for.

  Note that, for security reasons, when this feature is enabled it will be applied to any submitted username, regardless of whether the user exists.

  authentication_lock:
    count: 5
      expires: 300
  

• The (optional) enable_add_repositories key specifies if repositories can be added via the application interface by either administrative users or standard users. In the absence of this key, the default is True. When enabled, this property also suppresses the availability of the Watch Repository toggle associated with any repository entries displayed in the Artifact Analysis view.

  Note that in the absence of one or all of the properties, the default is also True. Thus, this key, and a child key corresponding to an account type (that is itself explicitly set to False) must be set for the feature to be disabled for that account.

  enable_add_repositories:
    admin: True
    standard: True
  

• The (optional) ldap_timeout and ldap_connect_timeout keys respectively specify the time (in milliseconds) the LDAP client should let operations stay alive before timing out, and the time (in milliseconds) the LDAP client should wait before timing out on TCP connections. Each value must be a whole number greater than 0.

  When these values are unset (or set incorrectly) the app will fall back to using a default value of 6000 milliseconds. The same default is used when the keys are not enabled.

  These value can be overridden by using the ANCHORE_LDAP_AUTH_TIMEOUT and ANCHORE_LDAP_AUTH_CONNECT_TIMEOUT environment variables.

  ldap_timeout: 6000
  ldap_connect_timeout: 6000
  

• The (optional) custom_message key allows you to provide a message that will be displayed on the application login page below the Username and Password fields. The key value must be an object that contains:

  • A title key, whose string value provides a title for the message—which can be up to 100 characters
  • A message key, whose string value is the message itself—which can be up to 500 characters

  custom_message:
    title:
      "Title goes here..."
    message:
      "Message goes here..."
  

  Note: Both title and message values must be present and contain at least 1 character for the message box to be displayed. If either value exceeds the character limit, the string will be truncated with an ellipsis.

• The (optional) log_level key allows you to set the descriptive detail of the application log output. The key value must be a string selected from the following priority-ordered list:

  • error
  • warn
  • info
  • http
  • debug

  Once set, each level will automatically include the output for any levels above it—for example, info will include the log output for details at the warn and error details, whereas error will only show error output.

  This value can be overridden by using the ANCHORE_LOG_LEVEL environment variable. When no level is set, either within this configuration file or by the environment variable, a default level of http is used.

  log_level: 'http'
  

• The (optional) enrich_inventory_view key allows you to set whether the Kubernetes feature should aggregate and include compliance and vulnerability data from the reports service. Setting this key to be False can increase performance on high-volume systems.

  This value can be overridden by using the ANCHORE_ENRICH_INVENTORY_VIEW environment variable. When no flag is set, either within this configuration file or by the environment variable, a default setting of True is used.

  enrich_inventory_view: True
  

• The (optional) enable_prometheus_metrics key enables exporting monitoring metrics to Prometheus. The metrics are made available on the /metrics endpoint.

  This value can be overridden by using the ANCHORE_ENABLE_METRICS environment variable. When no flag is set, either within this configuration file or by the environment variable, a default setting of False is used.

  enable_prometheus_metrics: False
  

NOTE: The latest default UI configuration file can always be extracted from the Enterprise UI container to review the latest options, environment overrides and descriptions of each option using the following process:

# docker login
# docker pull docker.io/anchore/enterprise-ui:latest
# docker create --name aui docker.io/anchore/enterprise-ui:latest
# docker cp aui:/config/config-ui.yaml /tmp/my-config-ui.yaml
# docker rm aui
# cat /tmp/my-config-ui.yaml
...
...

24 - User Credential Storage

Overview

All user information is stored in the Anchore DB. The credentials can be stored as plaintext or in a hashed form using the Argon2 hashing algorithm.

Hashed passwords are much more secure, but can be more computationally expensive to compare. Hashed passwords cannot be used for internal service communication since they cannot be reversed. Anchore provides a token based authentication mechanism as well (a simplified Password-Grant flow of Oauth2) to mitigate the performance issue, but it requires that all Anchore services be deployed with a shared secret in the configuration or a public/private keypair common to all services.

Passwords

The configuration of how passwords are stored is set in the user_authentication section of the config.yaml file and must be consistent across all components of an Anchore Enterprise deployment. Mismatch in this configuration between components of the system will result in the system not being able to communicate internally.

user_authentication:
  hashed_passwords: true|false

For all new deployments using the Enterprise Helm chart, hashed_passwords is defaulted to true.

All helm upgrades will carry forward the previous hashed_passwords setting.

NOTE: When the configuration is set to enable hashed_passwords, it must also be configured to use OAuth. When OAuth is not configured in the system, Anchore must be able to use HTTP Basic authentication between internal services and thus requires credentials that can be read.

Bearer Tokens/OAuth2

If Anchore is configured to support bearer tokens, the tokens are generated and returned to the user but never persisted in the database. Users must still have password credentials, however. Password persistence and protection configuration still applies as in the Password section above.

Configuring Hashed Passwords and OAuth

NOTE: Password storage configuration must be done at the time of deployment, it cannot be modified at runtime or after installation with an existing DB since it will invalidate all existing credentials, including internal system credentials and the system will not be functional. You must choose the mechanism at system deployment time.

Set in config.yaml for all components of the deployment:

Option 1: Use a shared secret for signing/verifying oauth tokens

user_authentication:
  oauth:
    enabled: true
  hashed_passwords: true
keys:
  secret: mysecretvalue

Option 2: Use a public/private key pair, delivered as pem files on the filesystem of the containers anchore runs in:

user_authentication:
  oauth:
    enabled: true
  hashed_passwords: true
keys:
  private_key_path: <path to private key pem file>
  public_key_path: <path to public key pem file>

Using environment variables with the config.yaml bundled into the Anchore provided anchore/enterprise image is also an option. NOTE: These are only valid when using the config.yaml provided in the image due to that file referencing them explicitly as replacement values.

ANCHORE_AUTH_SECRET = the string to use as a secret
ANCHORE_AUTH_PUBKEY = path to public key file
ANCHORE_AUTH_PRIVKEY = path to the private key file
ANCHORE_OAUTH_ENABLED = boolean to enable/disable oauth support
ANCHORE_OAUTH_TOKEN_EXPIRATION = the number of seconds a token should be valid (default is 3600 seconds)
ANCHORE_OAUTH_REFRESH_TOKEN_EXPIRATION = the number of second a refresh token is valid (default is 86400 seconds)
ANCHORE_AUTH_ENABLE_HASHED_PASSWORDS = boolean to enable/disable hashed password storage in the anchore db instead of clear text

25 - Vulnerability Scanner

The Anchore Enterprise vulnerability scanner scans applications for known vulnerabilities. The scanner updates vulnerability feed data on a configurable basis, then uses a stored software bill of materials (SBOM) to quickly scan for vulnerabilities in applications. By relying on local feeds as well as pre-computed SBOMs, the scanner is able to quickly scan a large number of applications many times a day.

While the SBOM will not change for cataloged applications, new vulnerabilities are constantly being found and reported. Anytime new vulnerabilities are added to public data sets, the Anchore Enterprise feed service will download and update the local data. Because the vulnerability data is always changing, it is important to continuously scan applications to alert on new findings in a way that does not create unnecessary load on the analyzers.

Unified Vulnerability Feed and Feed Sync Behavior

The vulnerability scanner uses a single “vulnerabilities” feed as a result of aggregating multiple source feeds combined with multiple source feeds.

The single “vulnerabilities” feed is created in the on-premises feed service from the same sources and databases that the vulnerability scanner uses. There is a driver in the feed service that aggregates the feed data together into a single atomic database and makes it available to the policy engine, including timestamps and digests for verification. The policy engine synchronizes the entire vulnerability database in a single atomic operation.

Visible changes to the operator:

1. All groups within the “vulnerabilities” feed should show the same update timestamp.
2. Support for enabling and disabling specific groups within the “vulnerabilities” feed for the vulnerability scanner is not supported and will return errors.

Vulnerability Scanner Configuration and Vulnerability Providers

The vulnerability scanner is configured as a vulnerability provider because it includes both the scanning engine and feed sync behaviors. A provider must be explicitly defined in the configuration file. If left unspecified, then the policy engine service will raise an error and fail to start. Anchore-provided deployment templates such as the Helm chart or and Docker Compose file in the Quickstart set deployments to the vulnerability provider.

The following YAML snippet is from the config.yaml used by the policy engine service and shows the “vulnerabilities” configuration key and the values to configure the provider.

Next-Gen Provider (“grype”):

    policy_engine:
        enabled: true
        ...
        vulnerabilities:
          provider: "grype"
          sync:
            enabled: true
            ssl_verify: true
            connection_timeout_seconds: 3
            read_timeout_seconds: 60
            data:
              grypedb:
                enabled: true
                url: https://<On-Premises Feed Service Host>:<port>/v1/databases/grype # This is typically set up for you in deployment templates
              ...

Using the Vulnerability Scanner

This is the default setting for Anchore-provided templates for deployments. To use it, you do not have to do anything if you are deploying from the Helm chart or Docker Compose file in the Quickstart.

Upgrading an Existing Deployment: Helm

We recommend upgrading previous versions of Anchore Enterprise to the latest version first without changing the vulnerability scanner, and use the new version of the vulnerability scanner in a development environment. Then update the configuration in the production deployment.

26 - Windows Container Scanning

Anchore can analyze and provide vulnerability matches for Microsoft Windows images. Anchore downloads, unpacks, and analyzes the Microsoft Windows image contents similar to Linux-based images, providing OS information as well as discovered application packages like npms, gems, python, NuGet, and java archives.

Vulnerabilities for Microsoft Windows images are matched against the detected operating system version and KBs installed in the image. These are matched using data from the Microsoft Security Research Center (MSRC) data API.

Requirements

Analyzing Microsoft Windows images is supported out-of-the-box with no configuration changes - but to get vulnerability results, your deployment must meet the following criteria:

1. Deploy an on-premises feed service
2. Have the microsoft vulnerability provider enabled in the feed service

Configuring Microsoft Provider

In the feed service configuration, enable the msrc driver.

services:
  feeds:
    ...
    drivers:
      ...
      msrc:
        enabled: true

Supported Windows Base Image Versions

The following are the MSRC Product IDs that Anchore can detect and provide vulnerability information for. These provide the basis for the main variants of the base Windows containers: Windows, ServerCore, NanoSerer, and IoTCore

Windows Operating System Packages

Just as Linux images are scanned for packages such as RPMs, DPKG, and APK, Windows images are scanned for the installed components and Knowledge Base patches (KBs). When listing operating system content on a Microsoft Windows image, the results returned are KB identifiers that are numeric. Both the name and version will be identical and are the KB IDs.