Diagnostics — health#

The Health tab shows whether connection tracking is coping. The router keeps a table of the connections it handles; when that table is full, new connections fail while existing ones keep working.

The tab has four tiles, the Connection tracking panel with the verdict, the Connection tracking settings panel, and the Per-processor counters table. It refreshes every 5 seconds.

The four tiles#

Until there are two readings to compare, the second line reads Since boot.

A tile is coloured only when the verdict explains the rise: Evicted connections is amber when connections were evicted in the window, Dropped packets is amber when packets were dropped and the kernel reported a full table, and Refused connections is red when connections were refused because a hash chain was full. Refusals and drops from insert races (below) leave the tiles plain.

When the agent cannot read the table's size, Connections counts the IPv4 connections the Sessions page lists, and its second line reads Tracked now.

The verdict#

The Connection tracking panel judges the last ten minutes, not the counters since the router started. A badge beside the title repeats the verdict.

Signs of pressure#

Any one of these in the window makes the verdict Not healthy:

• Connections were evicted before they expired, to make room.
• The table reached 80% of its limit at any reading in the window, whether or not anything was dropped.
• The kernel reported a full table: it dropped a packet because the table was full and nothing could be evicted. The kernel limits how often it reports this, so the count is a minimum.
• New connections were refused because the table's hash size is too small for the number of connections.

For the first three, the panel reads "Raise the table size below, or look on the Sessions page for a device with an unusual number of connections." For the last, it reads "Raise the hash size below." A larger table does not help a hash that is too small.

Insert races#

Two packets of one new connection can arrive on two processors at the same moment, and both try to add the same connection to the table. The router resolves most of these itself. When it cannot, it refuses one of the two and drops its packet, and the device sends it again. This happens whatever the table's size, and a larger table does not prevent it.

The panel reports it as information, not as a fault. For example:

In the last 10 minutes, 1 new connection was refused and 1 packet dropped, and the router resolved 200 more itself. No connections were evicted, the table peaked at 0.4% of its limit (958 of 262,144), and the kernel reported no full table.

Where the readings come from#

On the router, the agent reads the kernel's counters and the table's size every 10 seconds and watches the kernel log, whether or not the page is open. When the agent restarts, the window starts again.

When the agent does not run on the router and manages it over the network, each refresh of the page is a reading, so the window starts when the page is opened. The agent cannot read the table's size or the kernel log from there, so refusals and drops with no other sign of pressure are reported as Dropped, cause unknown, with the notes "The table's size is unknown: this agent is not running on the router, so it cannot read it." and "The router's kernel log is not read: this agent is not running on the router."

In Advanced mode, Kernel counters under the verdict lists each counter by the kernel's own name, with its total and its rise over the window.

Connection tracking settings#

The panel shows the three settings the verdict points at:

To change them:

1. Press Edit. The Connection tracking settings side panel opens.
2. Enter a Table size, in connections, from 1 to 50,000,000. Leave it empty for the default, 262,144.
3. Enter a Hash size, in buckets, from 1,024 to 50,000,000. Leave it empty for the default, 65,536.
4. Switch Connection accounting on to count bytes and packets for each connection, for the Sessions page.
5. Press Save changes, then Apply changes in the bar at the bottom of the page.

A value that is not a whole number in range is refused under its field, and Save changes stays disabled until it is corrected. Clearing a field returns that setting to the router's default; the pending change reads "Reset the connection table size to its default". Editing needs the operator role and a licence.

Connection accounting counts only connections opened after it is turned on. It is the same setting as Turn on connection accounting on the Sessions page. The router keeps accounting on only through this setting: a kernel tunable set for the same purpose is reset at boot and whenever connection tracking, the firewall or NAT is changed (Kernel tunables).

Per-processor counters#

The table lists the counters of each processor since the router started. Every column sorts.

A counter the router's kernel does not keep shows a dash.

What this tab does not do#

• It does not list connections. They are on the Sessions page.
• It keeps no history. The verdict covers the last ten minutes, and the tiles and the table count since the router started. To keep figures over time, collect the agent's metrics (Telemetry).

For automation#

GET /api/conntrack/stats (any role) returns the per-processor counters and the verdict: its state, the signs of pressure, the window, each counter's total and rise, the table's size and the kernel log. The settings are changed through the pending changes, like any other.

See also#

• Diagnostics — the other five tabs.
• Sessions — the connections, by device.
• Overview — the connection count and the router's memory, and this tab's verdict among what needs attention when the table is under pressure.
• Telemetry — keeping any of this.