GoAdmio

iptop

Report the busiest IPs by request count and block floods and bot networks

On this page: Overview · Quick start · Syntax · Options · Output · Examples · Troubleshooting · Details · See also


Overview

/go/iptop ranks the IP addresses that sent the most requests to the server in a chosen time window. It can also block the ones that look like bots or floods. It:

No setup is needed. GoAdmio's default cron jobs already run iptop block every hour, so high-traffic IPs and bots are blocked automatically.


Quick start

# Top IPs in the last 24 hours. Nothing is blocked.
/go/iptop

# Show the last hour and block offenders (what cron runs)
/go/iptop block

Syntax

/go/iptop [block | robot | underattack | hour | day | week | <seconds>]
Form Behavior
/go/iptop Report for the last 24 hours. Nothing is blocked.
/go/iptop hour, day, week Report for that period. Nothing is blocked.
/go/iptop <seconds> Report for that many seconds. Nothing is blocked.
/go/iptop block Last hour. Blocks high-traffic IPs and known bot networks.
/go/iptop robot Last 15 minutes, with a quarter of the normal threshold.
/go/iptop underattack Only runs when the server is overloaded. Blocks the single busiest IP.

The command has no -- options. Only the first argument is read.

An unrecognized argument is treated like no argument. There is no "unknown parameter" error. A typo such as /go/iptop blok runs the 24-hour report and blocks nothing.


Options

Modes

Argument Window Rows shown Blocks? Threshold
(none or unrecognized) 24 hours 25 No —
<seconds> (digits only) That many seconds 25 No —
hour 1 hour 25 No —
day 24 hours 25 No —
week 7 days See note No —
block 1 hour 50 Yes 2500 (default)
robot 15 minutes 50 Yes A quarter of the limit (625 by default)
underattack 15 minutes 1 Yes 100, only if server load is over 100%

Configuration

Item Required Default Notes
/var/go/etc/iptop (file) No 2500 A whole number: the request count at which an IP is blocked. See Threshold.
/etc/white (file) No — IPs to leave out of the count. See Whitelist.

Output

Outcome What you get
Report A table of IPs with visits, country, city, and company.
Blocking mode, IPs blocked The same table, then an email and a WhatsApp message listing the blocked IPs.
Blocking mode, nothing blocked The table only. No message is sent.
Blocking mode, another instance running Instance (<name>) already running. and exit code 1.
underattack, server load not over 100% No output and exit code 1.

Report

The columns are aligned with column. This sample is illustrative.

IP            VISITS  COUNTRY  CITY     COMPANY
--            ------  -------  ----     -------
203.0.113.7   4120    US       Chicago  Example Hosting LLC
198.51.100.22 310     DE       Berlin   Example Telecom

The server's own IPv4 address, its IPv6 address, and 127.0.0.1 are never listed. Whitelisted IPs are not counted, so they don't appear either.

Notification

Sent only in blocking modes, and only if at least one IP was blocked in that run.

Channel Subject or title
Email, through sendmail Blocking of High Traffic and Non-Human Bot IPs (<N> IPs)
WhatsApp, through sendwhatsapp *Blocking of High Traffic and Non-Human Bot IPs (<N> IPs)*

The body explains what happened, then lists one line per blocked IP. The number of minutes is the window used for that run.

- Robot 203.0.113.7 (3 Visits / 60 Minutes) - Country: US, Company: Example Hosting LLC.
- High Traffic 198.51.100.22 (4120 Visits / 60 Minutes) - Country: DE, Company: Example Telecom.

[!WARNING] Blocking is automatic and unattended. The bot-network list blocks an IP after a single request, and it matches broad names such as Microsoft, Comcast Cable Communications, and Beijing. A real visitor on one of those networks can be blocked. Read the notification emails, and use unblock for any false positive.


Examples

# Who visited most in the last 5 minutes?
/go/iptop 300
# Who visited most this week?
/go/iptop week
# Report and block, as cron does every hour
/go/iptop block
# Block at 5000 requests per hour instead of 2500
echo 5000 | sudo tee /var/go/etc/iptop
# Go back to the default threshold
sudo rm /var/go/etc/iptop
# Read the history of runs and their arguments
sudo tail /var/log/go/iptop.log

Troubleshooting

Symptom Likely cause Fix
Instance (<name>) already running. A blocking run started less than 60 seconds ago. The lock is shared with other commands. Wait, then run again. A lock older than 60 seconds is ignored.
Nothing was blocked Ran in a report mode, no IP reached the threshold, or the IP was exempt. Run /go/iptop block and check the Threshold and exemptions.
underattack prints nothing and exits Server load isn't over 100% of its CPU threads. Expected. The mode only acts under overload.
An IP over the threshold wasn't blocked A file /var/log/go/block/<IP>.* already exists, or the company is exempt. See Already-blocked IPs and exempt visitors.
A legitimate visitor was blocked They crossed the threshold, or their network is on the bot-network list. Run unblock for that IP. Raise the threshold if they were over the limit.
A whitelisted IPv6 address is still counted Whitelist entries without a dot are skipped. See Whitelist.
A whitelist entry covers more IPs than expected Entries match as a prefix. 10.0.0.1 also matches 10.0.0.10. Use a trailing dot or a full address you can live with.
Bots from a listed network weren't blocked The country from whois didn't match the geo lookup. See Bot networks.
No notification arrived No IP was blocked in that run, or email and WhatsApp aren't configured. Check the table for blocks, then test sendmail and sendwhatsapp.
Country or city is blank in cron output Cron runs use the whois record, which may lack those fields. Run the command in a terminal for fuller lookups.
The threshold change had no effect The file contains something other than digits. Put only a whole number in /var/go/etc/iptop. Anything else falls back to 2500.

Details

Threshold

Condition Limit used
/var/go/etc/iptop doesn't exist 2500
It exists and contains only digits That number
It exists but isn't a whole number 2500
Mode is robot The above, divided by 4 (whole number)
Mode is underattack 100, ignoring the file

An IP is blocked when its request count is greater than or equal to the limit. The count is for the window of the mode, so block means requests per hour and robot means requests per 15 minutes.

What counts as a visit

Check Rule
Time window Judged by the timestamp in each log line.
Matching Every request counts, whatever the response code.
Visitor IP The forwarded client IP if the log has one. Otherwise the connecting IP.
Counting Case-insensitive per IP, highest first.
Rows kept Per the Modes table.

Whitelist

If /etc/white exists, each line is read before counting.

Rule Detail
* The first * in the line is removed.
Entries with a dot Every request from an IP that starts with the entry is removed from the count.
Entries without a dot Skipped. IPv6 addresses are therefore not honored here.
Dots in the entry Match any character, because the entry is used as a pattern.

Because matching is by prefix, 10.0.0.1 also covers 10.0.0.10 and 10.0.0.100.

Bot networks

In blocking modes, a visitor whose company name contains any of these (case-insensitive) is treated as a bot network and blocked after one request:

Alibaba Microsoft UAB Host Baltic
Bunny Communications LLC Censys DigitalOcean
Arelion IPv4 address block Ace Data Center
ACEVILLE PTE Packethub Comcast Cable Communications
Amazon Data Services SOFTGAURD POWERTRONICS Beijing
Zhejiang CNC Group Arisk Communications
Intelligence Hosting Anthropic

Two more conditions apply before the block happens:

The block is recorded with the reason robot.

Exempt visitors

A company that is not on the bot-network list is never blocked for high traffic if its name contains:

Text Case-sensitive
google No
bing Yes (lowercase)
cloudflare Yes (lowercase)

Exempt visitors still appear in the table. Because the bing and cloudflare checks are case-sensitive, a name written as Cloudflare, Inc. would not match them.

The bot-network list is checked first. A company on that list is blocked after one request even if its name also contains an exempt word. High-traffic blocks are recorded with the reason ip.

Already-blocked IPs

An IP is skipped if any file named /var/log/go/block/<IP>.* exists. Such a file is the record left by block. An IP that already has one is not blocked again and is not included in the notification.

underattack

Step Detail
Load check Server load is the 1-minute load average as a percentage of CPU threads.
Not overloaded Exits with code 1 and no output.
Overloaded Looks at the last 15 minutes and keeps only the single busiest IP.
Block rule Blocked at 100 or more requests, or after one request if on the bot-network list.

How robot and underattack are scheduled isn't defined by this command. Only block is mentioned as a default cron job.

Week row count

The week mode doesn't set a row limit, so the ranking falls back to the default of the head command, which is likely 10 rows. The other report modes show 25.

Lookups

Data Terminal run Cron or other non-terminal run
Company whois record whois record
Country Geo lookup country: field of the whois record
City Geo lookup city: field of the whois record

Locking

Only block, robot, and underattack take a lock, so report-only runs can overlap freely.

Item Value
Lock file /usr/local/goadmio/lock/block.lock
Contents The name of the command holding the lock
Considered stale after 60 seconds without an update
Refreshed Once per IP processed, so a long run stays locked
Released On exit, including Ctrl+C

404top uses the same lock file. If iptop block and 404top block overlap, the second one exits with Instance (<name>) already running.

What it does

In short: take the lock if blocking, pick the window, count visitors, look each one up, block offenders, print the table, then notify.

flowchart TD
    A["iptop [mode]"] --> B{"block, robot, or underattack?"}
    B -- "Yes" --> L{"Lock under 60 s old?"}
    L -- "Yes" --> X1["Instance already running, exit 1"]
    L -- "No" --> C["Take the lock"]
    B -- "No" --> D
    C --> D["Pick window, rows, and threshold from the mode"]
    D --> U{"underattack and load 100% or less?"}
    U -- "Yes" --> X2["Exit 1, no output"]
    U -- "No" --> E["Count visits per IP, skipping whitelisted IPs"]
    E --> F["For each IP: skip own and local IPs, look up whois"]
    F --> G{"Company on the bot-network list?"}
    G -- "Yes" --> H{"Blocking mode, not blocked yet, country matches?"}
    H -- "Yes" --> BR["/go/block IP robot"]
    G -- "No" --> I{"Blocking mode, not exempt, at or over limit, not blocked yet?"}
    I -- "Yes" --> BI["/go/block IP ip"]
    H -- "No" --> J["Add row to the table"]
    I -- "No" --> J
    BR --> J
    BI --> J
    J --> K["Print table"]
    K --> M{"Any IPs blocked?"}
    M -- "Yes" --> N["Send email and WhatsApp"]
    M -- "No" --> O["Delete temp file and release the lock"]
    N --> O
  1. Appends the date and arguments to /var/log/go/iptop.log.
  2. Loads shared functions from /go/function and the access log path.
  3. In block, robot, or underattack mode, exits with an error if block.lock is under 60 seconds old. Otherwise writes the lock.
  4. Works out the number of CPU threads and the server load as a percentage.
  5. Reads the hostname and finds the server's public IPv4 and IPv6 addresses.
  6. Picks the time window and row count from the first argument. In underattack mode, exits with code 1 unless load is over 100%.
  7. Reads the threshold from /var/go/etc/iptop, falling back to 2500, then adjusts it for robot and underattack.
  8. Writes the visitor IP of every log line in the window to /usr/local/goadmio/tmp/iptop-<timestamp>.db.
  9. Removes whitelisted IPs from that file, using /etc/white.
  10. Counts the visits per IP and keeps the top rows.
  11. Skips the server's own IPs and 127.0.0.1, then looks up whois, company, country, and city for each remaining IP.
  12. In blocking modes, blocks bot-network IPs with reason robot. Blocks other non-exempt IPs at or over the limit with reason ip. Both skip IPs that already have a record in /var/log/go/block/.
  13. Adds each IP to the output table.
  14. Prints the table. If any IP was blocked, sends the email and the WhatsApp message.
  15. Deletes the temp file and the lock file.

If the command is stopped with Ctrl+C, the lock is released but the temp file in /usr/local/goadmio/tmp/ may remain.


See also

Command Description
404top Report IPs causing the most 404 errors and block scanners and probes
block Block an IP address on a GoAdmio server
unblock Unblock an IP address on a GoAdmio server
white Add an IP to the whitelist
bottop Block bad bots from visiting your server
bots Track all bot visits to your server (good and bad bots)
ddos Track and block DDoS attack IPs in GoAdmio
posttop Track and check IPs making POST requests
usertop Check recent visitors per domain
cron Set up main cron jobs in GoAdmio