GoAdmio

contact

Collect sender names and addresses from the mail folders of one or all domains into the contact database

[!NOTE] The index describes this command as "Manage GoAdmio account contact details". The script documented here does not manage an account's own contact details. It reads the email in the mailboxes of a domain and stores the senders in /var/lib/sqlite/contact.db. Either update the index description to match this page, or check that this is the script you meant to document.

On this page: Overview · Quick start · Syntax · What is collected · Output · Examples · Troubleshooting · Details · See also


Overview

/go/contact builds a list of contacts from the mail that customers have received. For each mailbox of a domain it reads the messages, takes the sender's name and address, and saves them in the contact database together with the recipient's address.

[!WARNING] The command reads the headers of customers' email and stores names and email addresses. This is personal data. Before you use it, make sure that your hosting terms and the privacy law that applies to you allow it, and that your customers have been told.

I haven't seen which part of GoAdmio reads contact.db, so this page doesn't say what the contacts are used for.


Quick start

# One domain
/go/contact example.com

# Every domain with a mail folder
/go/contact all

Syntax

/go/contact [domain|all]
Form Behavior
/go/contact or /go/contact all Runs the command for every folder in /home whose name contains a dot and that has a mail folder.
/go/contact <domain> Collects the contacts of that domain's mailboxes.

The command re-runs itself with sudo if it isn't started as root. It has no other options.


What is collected

Item Detail
Which mail The cur folder of each mailbox, which is the inbox's messages that are already marked as seen. The new folder (unread mail), the sent folder, and other subfolders are not read.
How many Up to 1,000 messages per mailbox in one run.
From each message The sender name and address from a From: Name <address> line, and the part of the To: line after the first <.
Saved in Contact Domain, Name, EmailFrom, EmailTo, and TimeStamp. The time stamp is the time of the run, not the date of the message.

A message is skipped when:

The words are matched exactly as written, so Reply with a capital letter is not skipped.


Output

Outcome What you get
all <domain> Contacts... for each domain, then exit code 1.
One domain Nothing.

Logging

Every run is appended to /var/log/go/contact.log.

Marker files

For every message it looks at, the command creates a small file named after the MD5 hash of the message's file name, /usr/local/goadmio/db/<hash>.db. These files mark messages as processed. They sit in the same folder as GoAdmio's own database files, and there is one for every message ever processed.


Examples

# Collect the contacts of one domain
/go/contact example.com
# Collect for every domain
/go/contact all
# Look at what was saved
sqlite3 /var/lib/sqlite/contact.db "SELECT Name, EmailFrom, EmailTo FROM Contact WHERE Domain='example.com' LIMIT 20"

Troubleshooting

Symptom Likely cause Fix
New mail never adds new contacts The messages are read oldest first, and the loop stops at the first message that is already marked. After the first run, the oldest message is always marked, so the loop stops at once. See Notes for maintainers.
Senders with non-Latin names are missing Names that contain ? or = are skipped, which includes encoded names. Expected with this script.
Senders are missing The mail is unread (in new), is in another folder, or the From: line has no angle brackets. Expected.
EmailTo contains a comma or part of another name The To: line has several recipients, and the text after the first < is saved. Expected.
The /usr/local/goadmio/db folder holds a huge number of small files One marker file is created for every message. See the next row.
I want to process the mail again The markers say it was already processed. List the markers first with the find command below, and delete them only if the list contains nothing else.
No such file or directory for the mail folder The domain has no mail folder. The loop then runs over the files of the current folder. Pass a domain that has one.
The command is slow Every message needs several processes. Run it at a quiet time.

To list the marker files (check the result before you delete anything):

find /usr/local/goadmio/db -regextype posix-extended -regex '.*/[0-9a-f]{32}\.db' -size -2c -print

Replace -print with -delete only if the list contains nothing but marker files.


Details

What it does

  1. Re-runs itself with sudo if not root, and logs the call.
  2. Without a domain or with all: for each folder in /home whose name contains a dot and that has a mail folder, prints <domain> Contacts... and runs /go/contact <domain>. Then exits with code 1.
  3. For a domain: goes through each mailbox folder in /home/<domain>/mail, and through the files in its cur folder, in alphabetical order. For each file:
    1. reads the first From: and To: lines that end with >, removes the labels, the >, and quote characters, and splits them at < into a name and an address
    2. works out an MD5 hash of the file name
    3. stops reading this mailbox if a marker file with that hash exists
    4. otherwise creates the marker file
    5. stops after 1,000 messages
    6. saves the contact if it passes the filters above (INSERT OR IGNORE into Contact)
flowchart TD
    A["contact domain"] --> B["For each mailbox, read cur folder in alphabetical order"]
    B --> C{"Marker file exists?"}
    C -- "Yes" --> D["Stop this mailbox"]
    C -- "No" --> E["Create marker, check filters"]
    E --> F{"Passes filters?"}
    F -- "Yes" --> G["Insert contact"]
    F -- "No" --> H["Next message"]
    G --> H
    H --> I{"More than 1000 messages?"}
    I -- "Yes" --> D
    I -- "No" --> B

Notes for maintainers

Issue Why it matters Suggested fix
Messages are read oldest first (file names start with the delivery time), but the loop stops at the first one already seen After the first run, new mail is never processed, unless the oldest messages are deleted. The comment # Newest suggests newest first was intended. Read newest first, or don't stop, and track a position per mailbox.
One marker file per message in /usr/local/goadmio/db, next to the real database files A very large number of files, a heavy folder for serversync to copy, and a risk of deleting real files by mistake. Store the processed state in contact.db, with a unique key.
The whole message, including the body, is searched with grep Quoted and forwarded From: lines in the body are also found. Read only the header (up to the first empty line), and handle folded lines and addresses without <>.
Names with ? or = are skipped All encoded names are lost, including most non-Latin ones. Decode the names instead of dropping them.
Only the first To: line is used, and the text after the first < is saved Several recipients give a dirty EmailTo, and a To: without angle brackets gives an empty one. Save the mailbox owner (<folder>@<domain>) as the recipient.
Only the inbox's cur folder is read Unread mail, sent mail, and subfolders are ignored. Decide what should be included.
Customers' mail headers are processed without any opt-out Privacy and legal risk. Add an opt-out, a limited retention, and a description of the purpose.
The file-name hash changes when the message flags change The Maildir flags after :2, are part of the name, so a message can be processed again. Hash only the part before the flags.
The filters are exact, case-sensitive substrings Reply passes while reply is skipped, and names such as "Outlook Team" are dropped. Match case-insensitively, and use a list that can be changed.
cd /home/<domain>/mail is not checked If it fails, the loop runs over the files of the current folder. Stop if the folder doesn't exist.
TimeStamp is the time of the run The date of the message is lost. Save the date from the Date: header.
MySQLRootPassword, latest, ReceiverName, and Directory are set but not used Dead code, and a secret is loaded for no reason. Remove them.
The domain is not validated, and all exits with 1 Odd names are used in paths, and success looks like failure. Validate against ^[a-z0-9.-]+$, and use exit 0.

See also

Command Description
collectmail Collect all email addresses from all MySQL/MariaDB databases
mailtop Check the most active email accounts in GoAdmio
mail Create an email address in GoAdmio via the command line
restoremail Recreate mail accounts from folders on disk
copymail Copy mailboxes from one domain to another
serversync Copy a whole GoAdmio server to another server over SSH: accounts, files, databases, SSL certificates, and settings