Skip to main content
Preview. This part of the console is still being built. What you see here describes the intended behavior and can change before general availability.

Plan an outpost

Most outpost problems are sizing problems that show up a week later as a scan refused for capacity. The outpost does not guess: it adds up the memory limits of the containers a scan will run and refuses what the host cannot hold. Ten minutes with this page before you pick a host means you know in advance which scans it will accept.

Prerequisites​

  • A Dwaar role that holds outposts:create to create the enrollment token, such as Security admin. Anyone with outposts:read, such as Auditor, can follow the steps and watch outposts, but not enroll one. API keys cannot create or revoke outposts.
  • A host and the rights to change its network rules, or an AWS account where you can create a CloudFormation stack with IAM resources.
  • Root on the host. The outpost drives the host's Docker to run each scan's containers, and that is root equivalent whichever way you install it.

Host requirements​

RequirementWhat the outpost needs
Operating systemLinux. The outpost is a static Linux binary and drives Docker Engine through its socket.
Architecturearm64 or x86_64. The outpost image holds both, and Docker picks the one for the host. The analyzer images must exist for the same architecture.
DockerDocker Engine 20.10 or later. Compose v2 if you deploy with Compose.
CPUAt least 2. The largest analyzer container asks for 2 CPUs, and a host with fewer refuses every scan that has one. The CPUs also decide how many scans run at once: see below.
MemorySet by the scans you send. See below.
DiskAbout 16 GB for the analyzer images, plus room for the scans that run at once. See below.
NetworkOutbound HTTPS only. No inbound port, load balancer or public address.

Give the outpost a host of its own. It runs untrusted analyzers against code you did not write, so the fewer other things the host holds, the less a bad day can reach. See Security model.

How an outpost decides it has room​

Discovery and connection tests are light. They run inside the outpost's own process, four at a time by default, and do not count against the host's capacity.

Scans are the heavy part. An outpost runs at most one scan per 2 CPUs at once (--job-cpu, default 2), whatever Scan concurrency says: a 2 vCPU instance runs one scan at a time, an 8 CPU host four. It asks Dwaar for another scan only while its CPUs hold one more, and its free memory and disk one more scan of the size --job-memory-mb and --job-disk-mb assume.

Each scan is a set of containers, and Dwaar's scanner catalogue gives every analyzer a memory limit and a CPU limit. Some analyzers wait for others: most wait for the checkout, and a few wait for the software bill of materials one of them writes. So not every container of a scan runs at once. Before an outpost starts a scan, it works out the scan's peak: the largest total of memory limits over the containers that can run at the same time. Then it checks:

CheckRefused whenWhat you see
CPUOne container asks for more CPUs than the host has.InsufficientCapacity, at once.
MemoryThe scan's peak is more than the memory the host offers.InsufficientCapacity, at once, with both figures.
DiskNothing else runs and the scan's disk estimate is more than the scan root's free space.InsufficientCapacity, at once.
Room nowThe scan fits the host, but not beside the scans already running.Nothing fails. The scan waits on the outpost until a running one ends, and the outpost takes no other scan meanwhile.

CPU limits are shared ceilings, so their sum is not checked: a scan on a host with fewer CPUs than its containers ask for in total runs, more slowly.

The memory the host offers is:

offered = memory the kernel reports × 0.9 × memory overcommit

The kernel reports a little less than the memory you bought. One 32 GB host reported 31,679 MB, and one 64 GiB AWS instance 63,514 MB. Run outpost status on your host to see what it offers at the settings that apply, or read Free now in the Outposts tab once it is online.

SettingWhereDefaultWhat it means
Memory overcommitThe console: the Deploy outpost drawer, then the outpost's pane1.0Multiplies the memory offered. 1.1 offers 99% of what the kernel reports. From 1.0 to 1.3 in steps of 0.05, then 1.5, 1.75 and 2.0.
Scan concurrencyThe console, as above2Scan concurrency, at most, from 1 to 16. Fewer run when the host's CPUs allow fewer, or when the next would not fit.
--job-cpu, --job-memory-mb, --job-disk-mbFlags of outpost run2 CPUs, 4096 MB, 4096 MBWhat a scan is assumed to need before its plan is known. They decide how many scans the outpost asks for: at most its CPUs ÷ --job-cpu at once. The disk figure is also the disk each scan is admitted on.

Both console settings are the outpost's own. Dwaar's limit on how many of your scans run at once counts only scans on Dwaar, never an outpost's. The --memory-overcommit and --max-jobs flags still exist, as overrides on one host: see Capacity settings.

Size for the limits, not for the use​

Limits are ceilings that cover the worst repository an analyzer could meet, and the outpost admits on the ceilings so that one heavy scan cannot starve another. Most scans use far less. One full repository scan observed on an outpost used about 4 GiB at its highest, against a peak of 30 GiB by its limits. The number that decides whether a scan starts is the 30.

Here are the peaks of Dwaar's current scanner catalogue with every category on. They change when the catalogue changes, so the arithmetic below matters more than the figures.

ScanContainersMemory by all limitsPeak memoryHost memory reported, for the default 1
Repository, full scan16: the checkout, 14 analyzers, and the server one of them talks to35,840 MB30,720 MB34,134 MB or more
Container image, one platform9: the checkout and 8 analyzers15,872 MB12,800 MB14,223 MB or more
Container image, two platforms16: the image analyzers run once per platform30,208 MB25,088 MB27,876 MB or more
AWS account posture check12,048 MB2,048 MB2,276 MB or more
GitHub organization posture check1512 MB512 MB569 MB or more

Work out a peak yourself​

The peak of a repository scan, from the catalogue's limits:

The checkout runs first and alone: 1,024 MB

Then everything that waits only for the checkout runs at once:
trivy 4,096 + opengrep 4,096 + checkov 4,096 + modelaudit 4,096
+ sonarqube 3,072 + gitleaks 2,048 + skill-scanner 2,048
+ zizmor 1,024 + cbomkit-theia 1,024 + pii-scanner 1,024 + context 512 = 27,136 MB

Three analyzers wait for trivy's bill of materials:
osv 1,024 + scancode 3,072 + provenance 512 = 4,608 MB
They cannot run beside trivy, but they can run beside all the others.
4,608 is more than trivy's 4,096, so the peak swaps trivy for them:
27,136 - 4,096 + 4,608 = 27,648 MB

The SonarQube server runs as long as the scan: 3,072 MB

Peak 30,720 MB
Host memory reported for the default 1: 30,720 / 0.9 = 34,134 MB
Overcommit a 32 GB host needs: 30,720 / (31,679 × 0.9) = 1.08, so 1.1

To redo it for your connection, take the categories you leave on in Scan Center, Scanners, find their analyzers' limits, and repeat the three steps: what runs after the checkout, which ones wait for another and whether swapping them in is larger, and what runs for the whole scan. Turning a category off removes its analyzer from the sum. A scan whose analyzers' results for that same version are already cached runs only the rest, so a rescan of an unchanged version can be far smaller than the first scan.

What overcommit trades​

A Memory overcommit above 1.0 admits scans whose limits add up to more than the host offers, betting that they will not all use their limits at once. The bet usually wins, because limits are generous. When it loses, the kernel stops a container that runs out of memory, that analyzer fails, and the scan reports it as failed. The other analyzers and the host are not affected. Overcommit trades the certainty that every analyzer has its full limit for running larger scans on a smaller host. Write the decision down, so the person who meets a failed analyzer later knows why.

Three sizes the outpost will accept​

These are what the outpost's own checks accept with every category on, worked from the figures above. They are not measurements of speed.

SmallStandardLarge
Host4 CPUs, 16 GiB, 100 GB disk8 CPUs, 32 GiB, 100 GB disk8 CPUs, 64 GiB, 100 GB disk
ExampleThe AWS ECS default, t4g.xlarge or t3.xlargeA spare VMr6g.2xlarge on arm64
Offered, default 1Under 14,746 MB28,511 MB, for 31,679 MB reported57,163 MB, for 63,514 MB reported
Discovery, tests, posture checksYesYesYes
Image scan, one platformOne at a timeTwo at onceTwo at once
Image scan, two platformsRefusedOne at a timeTwo at once
Repository full scanRefused. It would need an overcommit above 2.Refused at 1. One at a time at 1.1.One at a time, beside an image scan. Two at once at 1.1.
Best forAWS posture, discovery and single platform image scansOne team's repositories and imagesSeveral teams, or many repositories

On the large host, two repository scans need 61,440 MB; at 1.1 it offers 62,878 MB. Raise Scan concurrency only when the memory and the CPUs are there: the outpost never runs more scans than fit, nor more than one per 2 CPUs, so a higher setting on a small host changes nothing.

You do not have to work these out by hand: Ask Dwaar, below, does the same arithmetic for the host and the work you describe.

Ask Dwaar​

If you would rather describe the host and the work than do the arithmetic, choose Ask Dwaar under Capacity in the Deploy outpost drawer, or in an outpost's pane. It is there when Dwaar AI is on and your role holds outposts:create, such as Security admin, and in a pane also outposts:update. Asking also needs ai:use, which Security admin holds by default.

You describeFields
The deviceCPUs, memory and disk, the architecture, and whether it is a host of its own or what else runs there. On AWS ECS, an instance type you intend to use, or leave it empty for Dwaar to recommend one. In an outpost's pane, its last heartbeat supplies the host and how it was deployed.
The workHow many repositories and their typical size, how many container images and with how many platforms, AWS accounts, GitHub organizations, scans a day, how long scans may wait, and anything else.

Leave out what you do not know, but give the CPUs: they limit how many scans run at once as much as the memory does. It opens a Dwaar AI panel beside the drawer for what you describe. Choose Recommend and Dwaar answers with a Scan concurrency, a Memory overcommit, how many scans of each kind fit at those values, counting the host's CPUs and memory, what is worth knowing, and why. On AWS ECS it may name an instance type and architecture, and it may suggest a disk per scan to set with --job-disk-mb.

What Ask Dwaar can and cannot decide:

  • It decides nothing on its own. Use these values fills the settings into the drawer or the pane and closes the panel, and nothing changes until you create the token or choose Save capacity.
  • Scan concurrency is the console's. The console works it out from the host and the workload. When that differs from the AI's suggestion, the first point under Worth knowing says so. Without the host's CPUs it keeps the AI's guess and asks you for them.
  • The counts are the console's. However many scans Dwaar says fit, the console recomputes each count from the values and the catalogue figures, and the answer says when it set aside parts that did not hold up.
  • Values stay within the settings' ranges. Anything outside them falls back to the default.
  • Instance types come only from a known list, and only for AWS ECS with the architecture you asked for.
  • Dwaar AI can be wrong. Read the warnings and reasons before you use the values. The outpost still admits every scan on its real plan.

What you type is used for that one answer and is not stored. The audit log records that you asked and the values Dwaar recommended.

Disk​

WhatHow much
Analyzer imagesAbout 16 GB for the full set, kept on the host. The outpost pulls the new images in the background when Dwaar's scanner catalogue changes, so a scan rarely waits for a pull.
Each running scanIts checkout or image layers, plus the analyzers' output. Admitted on --job-disk-mb, 4096 MB by default, and deleted when the scan ends.
The outpost's own stateA few kilobytes in /var/lib/outpost.

Plan on 100 GB for a host that runs two scans at once: that is the default disk of the AWS ECS deployment. A large monorepo or a large image can use more than the 4096 MB estimate. Raise --job-disk-mb to what your largest checkout needs, and the disk with it.

Outbound connections to allow​

The outpost only makes outbound connections, and only over HTTPS, or plain HTTP to a provider whose URL you gave as http://. Allow these from the outpost's host, directly or through your proxy:

DestinationPortWhy
Your Dwaar console, the address you open in the browser443Enrollment, heartbeats, jobs, credentials, progress and results.
ghcr.io and pkg-containers.githubusercontent.com443The outpost's own image, if you run it as a container or copy the binary out of it.
Dwaar's container registry and results storage, on Amazon Web Services under amazonaws.com443The analyzer images, with a short lived pull token the console hands out, and the upload of each scan's results. Dwaar support gives you the exact hosts for your console.
The providers of the connections you bindTheirsYour Git server, registry or cloud API: the reason the outpost exists.
The hosts some analyzers look things up on443See the next table.

Analyzers never reach the network directly. Each scan's containers sit on a Docker network with no route out, and the few that need the network go through a proxy the outpost runs for that scan, which lets each one reach only the hosts its task needs. The outpost then makes the connection from the host, through your own proxy if you set one. So these are connections from the outpost's host too:

AnalyzerHosts it may reach
The checkoutYour Git server for a repository. For an image, its registry and the hosts that registry sends a client to for tokens and layers, such as auth.docker.io and production.cloudfront.docker.com for Docker Hub.
Image signatures (cosign)The image's registry and its token and layer hosts.
Base imagesThe image's registry, and the public registries it compares against: Docker Hub, public.ecr.aws, gcr.io, cgr.dev, quay.io and registry.access.redhat.com, with their layer hosts.
Package provenanceregistry.npmjs.org, pypi.org and api.github.com.
GitHub organization postureapi.github.com, or your GitHub Enterprise Server.
AWS account postureAWS's API endpoints under amazonaws.com and api.aws.

Every other analyzer runs with no network at all. A request for a host that is not on a task's list is refused and logged as outpost.egress_denied, so if your firewall refuses something, the outpost's log usually names it. See Security model.

Proxies and TLS inspection​

Set HTTPS_PROXY and NO_PROXY for the outpost and they apply to all of the above, including what the analyzers reach through the outpost. Docker pulls the images on its own, so on a host with no direct route out, configure Docker Engine's proxy as well. The AWS ECS stack's HttpsProxy parameter does both for you.

If the proxy inspects TLS, give the outpost its CA when you enroll, with --ca-file: the outpost then trusts it besides the system's roots. Two exceptions: Docker's pulls use the host's trust store, and AWS calls use the proxy from the environment but the system's trust store, not --ca-file. Add the CA to the host's trust store, or let that traffic bypass inspection.

Try this next​

Open Scan Center, Scanners, and note which categories are on for repositories and images. Work out the peak for that set with the arithmetic above, divide by 0.9, and compare it with the memory of the host you have in mind. If the answer is "only with overcommit", decide now whether you will, and write it next to the host's name.