GoAdmio

bulk

Run a command on many remote GoAdmio servers at once over SSH, and see all the results together

On this page: Overview · Quick start · Syntax · Options · Output · Examples · Troubleshooting · Details · See also


Overview

/go/bulk keeps a list of your servers and runs one command on all of them at the same time. It:

It works on servers, not on hosted domains. To run something for every domain on one server, use the command's own all option, for example bot.


Quick start

# Add two servers to the list
/go/bulk add server1.example.com
/go/bulk add server2.example.com

# Run a command on both
/go/bulk /go/restart

Syntax

/go/bulk add <server>
/go/bulk delete <server>
/go/bulk delete all
/go/bulk list
/go/bulk <command> [arguments]
/go/bulk

Options

Form What it does
/go/bulk add <server> Adds a server to the list. The name must contain a dot.
/go/bulk delete <server> Removes a server from the list. The name must contain a dot.
/go/bulk delete all Deletes the whole server list.
/go/bulk list Shows the server list.
/go/bulk <command> [arguments] Runs the command on every server in the list.
/go/bulk Opens an interactive prompt, bulk>. Each line you type is run on all servers. Type exit or quit to leave.

add, delete, and list are reserved words. You cannot send a remote command that starts with them in these forms.

[!WARNING] add and delete without a dot in the second argument, and list when no list exists, are not recognized as list commands. Depending on the form, they are sent to the servers as a remote command. Always use full server names.

Requirements

Requirement Detail
Server list At least one server in /var/go/etc/bulk.
SSH Root login with a password must be allowed on each server, on port 22.
Password All servers must accept the same root password, the one stored encrypted in /var/go/etc/go.key.
Tools sshpass, openssl, and ifconfig on this server.

Output

For every server, the command saves the output in /var/log/go/bulk/<server>.log. The first line of each file is the server name. These files are deleted at the start of every run, so they only hold the last run.

How the results are shown depends on the number of servers:

Servers Display
1 to 5 Side by side in colored columns, one column per server. Only the first 7 lines of each log are shown, and long lines are cut to fit the column.
More than 5 Each server's full log, one after another, with a line between them.

Read the log files for the complete output of a run with 5 servers or fewer.

If the list is empty or has no valid names, the command prints No Servers. Each run also adds a line to /var/log/go/bulk.log.


Examples

# Add and list servers
/go/bulk add server1.example.com
/go/bulk list
# Run a command on all servers
/go/bulk /go/update
# Remove one server, then clear the list
/go/bulk delete server1.example.com
/go/bulk delete all
# Interactive mode
/go/bulk
bulk> /go/restart
bulk> exit
# Read the full output of the last run for one server
cat /var/log/go/bulk/server1.example.com.log

Troubleshooting

Symptom Likely cause Fix
Nothing happens and nothing is printed No server list exists, so the command exits. Add a server with /go/bulk add <server>.
No Servers The list has no names containing a dot. Add full server names.
sed: can't read /var/go/etc/bulk on the first add The list file does not exist yet. Ignore it. The file is created by the same command.
Permission denied in a server's log The root password does not match, or root password login is off on that server. Check the remote root password, and use permitrootlogin.
Connection fails on one server The SSH port is not 22 there, or the server is down. Bulk always uses port 22. See port.
Every server fails with a password error The saved password cannot be decrypted. It is tied to this server's network card address. Check that the network card did not change.
Output is cut after 7 lines Side-by-side view with 5 servers or fewer. Read the log files in /var/log/go/bulk/.
Columns do not display or the output is garbled The terminal width variable COLUMNS is not available to the script. Run export COLUMNS first, or use more than 5 servers for the stacked view.
command not found in a server's log The command is not in the remote PATH. Use the full path, for example /go/restart.

Details

Server list

The list is a plain text file, one server per line. add removes any existing line that starts with the same text before adding the new one. delete <server> removes every line that starts with the text you give.

Running a command

  1. Logs the run and deletes the old log files in /var/log/go/bulk/.
  2. Decrypts the root password from /var/go/etc/go.key.
  3. For every name in the list, connects with sshpass and ssh as root in the background, and runs your command.
  4. Waits for all connections to finish.
  5. Shows the results.

The command is passed to the remote shell as typed. In interactive mode, each line is first evaluated by the shell on this server, so quote special characters.

Security

[!WARNING]

  • The root password is given to sshpass on the command line, so it can be seen in the process list on this server while the command runs.
  • SSH host key checking is turned off and known hosts are not saved. The command does not verify the identity of the remote servers.
  • A single root password for all servers means one leaked password exposes all of them.

Known issues


See also

Command Description
permitrootlogin Allow root SSH login from a specific IP
port Change the default SSH port (22) to another port
compare Compare multiple GoAdmio servers based on visits and resource usage
uptime Monitor multiple server IPs for uptime/downtime
serversync Back up and migrate a GoAdmio server with all configurations
update Update GoAdmio to the latest version via the command line