GoAdmio
GoAdmio Documentation

Bot Blocker

From the GoAdmio knowledge base.

Bot Blocker

This page explains the Bot Blocker feature in GoAdmio. It blocks traffic from bad bots at the web server level and lets you exclude specific domains from the blocklist — so a domain you trust (or that legitimately needs a blocked bot) is not affected.

Location: Admin Panel → Security → Bot Blocker
Location (User Panel): Security → Bot Blocker (if enabled for the account)


Overview

The Bot Blocker is the enforcement layer for bot blocking. It applies the rules defined in Bot Manager (the curated list of 382 known bots) and blocks matching requests at the web server — typically returning 403 Forbidden.

Bot Blocker adds two important capabilities beyond the raw Bot Manager list:

  1. Domain exclusions — exclude specific domains from the blocklist, so they are not affected even if the bot is globally blocked
  2. Live management — add, remove, and review blocks with immediate effect
Use Bot Blocker to:
  • Enforce the Bot Manager list across the server
  • Exclude domains that legitimately need a blocked bot (e.g., a customer's SEO tool, a payment gateway's crawler, a monitoring service)
  • Override the server-wide block for specific customers
  • Troubleshoot false positives without disabling the entire blocklist
> Note: Bot Blocker blocks by bot name / User-Agent. It is different from Bot Manager (the list) and from IP Blocker (which blocks by IP). Use all three together for full coverage.

Bot Blocker vs. Bot Manager

| Feature | Bot Manager | Bot Blocker |
|---|---|---|
| Role | Defines the list of bots to block | Enforces the list at the web server |
| Scope | Server-wide | Server-wide, with per-domain exclusions |
| Controls | Toggle each bot ✔ / ✖ | Exclude domains; review enforcement |
| Where to change a bot's status | Bot Manager | — |
| Where to exclude a domain | — | Bot Blocker |

> Tip: Use Bot Manager to decide which bots to block. Use Bot Blocker to decide which domains are exempt from the block.


What the Page Shows

| Column / Field | Description |
|---|---|
| Domain | The domain being reviewed for exclusion |
| Excluded | Whether the domain is excluded from the bot blocklist |
| Bot(s) Excluded | Which bots (if any) are excluded for that domain |
| Status | Active (blocked) or Excluded |
| Actions | Add or remove the exclusion |
| Search | Filter the list by domain |

> Tip: The Search box is the fastest way to find a domain when the list is long.


How to Exclude a Domain

Excluding a domain from all bot blocks

  1. Log in to the GoAdmio Admin Panel.
  2. Navigate to Security → Bot Blocker.
  3. Use Search to find the domain.
  4. Set Excluded to Yes (or toggle the exclusion).
  5. Click Save.
Requests from blocked bots to that domain are now allowed through. Every other domain continues to block them.

Excluding a domain from a specific bot only

Some versions support per-bot exclusions — e.g., allow AhrefsBot only for example.com:

  1. Open Security → Bot Blocker.
  2. Select the domain.
  3. Select the bot (e.g., AhrefsBot).
  4. Add the exclusion.
  5. Save.
> Tip: Per-bot exclusions are the safest option — they allow only the needed bot for the domain, without disabling the entire blocklist for that customer.

How to Re-Enable Blocking for a Domain

  1. Open Security → Bot Blocker.
  2. Locate the domain (Search if needed).
  3. Set Excluded to No (or remove the per-bot exclusion).
  4. Save.
Blocking resumes immediately for that domain.

> Tip: If you only need a temporary exemption (e.g., a customer is running an SEO audit), enable the exclusion, then re-enable blocking after the audit — or set a duration if your version supports it.


Common Use Cases

| Scenario | What to exclude |
|---|---|
| A customer runs a Semrush / Ahrefs audit on their site | Exclude that domain from AhrefsBot / SemrushBot |
| A payment gateway's fraud-check crawler is blocked | Exclude the affected domain from that bot |
| An uptime monitor (Pingdom, UptimeRobot) is blocked | Exclude the monitored domains, or add the monitor bot to Bot Manager's allow list |
| A search engine that is globally blocked by mistake | Allow the bot in Bot Manager, or exclude all domains |
| A monitoring bot used only by one customer | Exclude that domain from the bot |
| A staging site needs a blocked bot for testing | Exclude the staging domain |

> Warning: Excluding a domain disables bot blocking for that domain only. The rest of the server remains protected. Never exclude a domain without a specific reason — the excluded domain becomes an easier target for scrapers and scanners.


How It Works

When a request arrives:

  1. The web server reads the User-Agent header.
  2. It checks the Bot Manager list for the bot's name.
  3. If the bot is blocked in Bot Manager:
- Check Bot Blocker for the target domain - If the domain is excluded (globally or for that bot) → the request is allowed - Otherwise → the request is blocked (403 Forbidden)
  1. If the bot is allowed in Bot Manager → the request proceeds normally.
> Tip: Exclusions are checked per request, so they take effect immediately after saving.

Verifying an Exclusion

Test with curl

From a machine, simulate a blocked bot hitting the domain:

```bash

Simulate a blocked bot against the excluded domain — should return 200


curl -A "AhrefsBot/7.0" -I https://excluded-domain.com/

Simulate the same bot against a non-excluded domain — should return 403

curl -A "AhrefsBot/7.0" -I https://other-domain.com/
↑