Verifying Feeds

Anchore Enterprise runs a feed service which downloads vulnerability data from a number of configurable sources.

This data is stored on disk and is processed into a holistic vulnerability dataset. Once built and compiled, the dataset is stored in a postgres database and served to other Anchore Enterprise application services via an API endpoint.

The API endpoint is periodically queried by the policy service to fetch the latest dataset. If a newer vulnerability dataset is available, the policy service will download and propagate the new dataset across all instances/pods. This updated data is used to generate vulnerability analysis results.

The accuracy of vulnerability analysis is determined by the ability for your Anchore Enterprise deployment to download, store and distribute this feed data. Ensuring the health of this process is critical to the operation of the platform.

Performing a basic health check

Run $ anchorectl feed list as admin and ensure that:

  • The last sync date shown is recent and that the feed has enabled set to true.
  • Review the feed list to see if you have the required feed sources enabled.
    • Missing an expected source? Review the configuration and operational sections on this page

Run $ anchorectl feed sync as admin which will:

  • Queue an update to fetch and propagate feed data across internal services.
    • Otherwise, this runs on a regular schedule.
    • Note: It can take several hours to download, build and distribute the dataset.

You can also visually check the health in the ‘System’ section of the UI when logged in as admin.

Anchore Enterprise relies on multiple sources in order to build a high resolution picture of vulnerability data. It is generally recommended that all vendor sources are enabled.

Direct Mode

We highly recommend that you enable the GitHub GHSA feed for high quality vulnerability data

  • This can often be misconfigured due to the nature of the nested feeds property in helm values file, due to the feeds service being a dependancy and independent chart.
  • You must also ensure you set a valid GitHub Token, see below for a complete Helm values example.
feeds:
  anchoreConfig:
    feeds:
      drivers:
        github:
          enabled: true
          # The GitHub feeds driver requires a GitHub developer personal access token with no permission scopes selected.
          # See https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/creating-a-personal-access-token
          token: your-github-token

Direct & Proxy Mode

Enable the applicable feeds within Anchore e.g. if you use Ubuntu base images, enable the vendor specific feed in this case: Ubuntu. See Helm values example below

feeds:
  anchoreConfig:
    feeds:
      drivers:
        PROVIDER: true

For an air-gapped environment your high side feed service should be configured with api_only = true. This will ensure the feed service will only serve built databases stored locally.

Finally, operate your feeds database on a separate server to the anchore server. This ensures separation of concerns, ease of backup and less performance impacts. Where possible use a managed postgres service like RDS.

Operational & Configuration checks

Check that the feed pod/container has enough disk space:

Storage

  • Ensure free disk space is (>10gb) for the /workspace directory where feed data is stored on the feeds pod / container.
    • Exec onto the pod / container and check disk usage with $ df -h
  • You can specify an external volume mount for the /workspace directory over using the default node / host disk.
    • If you choose to map an external volume, we recommend this to be with a local or fast disk and not a slower storage like NFS.

Compute

  • Ensure your feed pod / container has the required compute, memory and disk resources as covered in requirements.
  • Ensure your feeds DB server or pod/container has enough resources. We recommend 1 CPU and 2 GB RAM minimum.

Network

  • Ensure your feed pod / container has network connectivity to feed source(s) by exec’ing into the container and then:
    • Verifying each domain is contactable by running a connectivity test on each source. e.g. $ curl https://secdb.alpinelinux.org.
      • If you are using Proxy Mode, you will want to test $ curl https://enterprise.vunnel.feed.anchore.io/.
      • If you are using Direct Mode, you will want to test all applicable vulnerability sources on the domains listed here.
    • If you have a network proxy deployed, you might need to configure your feed service to utilize it:
      • Depending on proxy, you might need to also use a custom cert in your Anchore deployment as per instructions here.
        • You can test, if this is required/working ok by running $ curl -v https://secdb.alpinelinux.org and verifying the output returns “SSL certificate verify ok.”
  • Ensure your policy pod / container has network connectivity to your local feed pod / container
    • Run e.g. curl http://anchore-feeds:8448/v2/databases/grypedb OR curl $ANCHORE_GRYPE_DB_URL returns a success response to confirm connectivity.

Configuration

  • Check your policy pod / container configuration by exec’ing into the container and then:
    • Output the configuration $ cat '/config/config.yaml' and check policy_engine.vulnerabilities.sync.data.grypedb.url property is set.
      • Ensure the set value e.g. http://anchore-feeds:8448/v2/databases/grypedb OR $ echo $ANCHORE_GRYPE_DB_URL points to your local desired feed service.
        • Query the set value with curl and review the response, which should be a list of one or more grypedb vulnerability records.
        • Update your Helm values or Compose Env variables to resolve.
        • PS: This can also point to our OSS feeds, if no value is set. The OSS feeds offer less data than the Enterprise feeds.
  • Check that your desired configuration has been correctly applied for both the feeds and policy services:
    • In each pod / container run $ cat '/config/config.yaml' or /scripts/anchore-config for Helm deployments.
      • Review the policy_engine and feeds: sections in the respective configuration output and ensure it matches your desired configuration and adjust if required.

Operational

  • Patience - For both Direct and Proxy Modes, the feed data can take several hours to download, build and distribute an updated vulnerability dataset.
  • Check the status of a feeds sync “run” on the feeds pod / container by:
    • Reviewing the feed logs for any errors. You can enable DEBUG logging for increased verbosity.
    • Querying the feeds tasks api to return a metadata about the current and past feeds “runs”. For the current run you need to find the highest task id that is a parent id.
      • Helm - $ curl ${ANCHORE_FEEDS_EXTERNAL_URL}tasks and for the latest “run” results use $ curl ${ANCHORE_FEEDS_EXTERNAL_URL}tasks/<parent-id-or-task-id>.
      • Compose - $ curl http://feeds:8448/v2/tasks and for the latest “run” results use $ curl http://feeds:8448/v2/tasks/<parent-id-or-task-id>.

NOTE:

  • On occasion a feed source might be experiencing downtime or a maintenance window. In this scenario, the feeds service will continue to operate for the remaining providers and will retry on the next run to fetch the latest data.
Last modified September 16, 2024