GoAdmio

s3

Mount an S3-compatible bucket (such as one created on MinIO) at /s3 with s3fs, save its credentials in /etc/passwd-s3fs, and add an /etc/fstab entry so it mounts again at boot

On this page: Overview · Quick start · Syntax · Options · Output · Examples · Troubleshooting · Details · See also


Overview

/go/s3 makes an object-storage bucket look like an ordinary folder, /s3, on the server. Files you copy into /s3 are uploaded to the bucket, and files in the bucket appear in /s3. It uses s3fs, a FUSE file system. The command:

When /s3 exists, create also makes a matching folder for each new account.

[!WARNING] The secret key is written to the log in plain text, because the whole command line is logged to /var/log/go/s3.log. It also appears in your shell history and, briefly, in the process list. The mount is readable and writable by every user on the server. See Security.


Quick start

# Mount a bucket from your own MinIO or other S3-compatible service
/go/s3 -b mybucket -k ACCESSKEY -s SECRETKEY -e s3.example.com

Syntax

/go/s3 -b <bucket> -k <access key> -s <secret key> -e <endpoint>
Form Behavior
/go/s3 -b ... -k ... -s ... -e ... Saves the credentials, mounts the bucket if /s3 doesn't exist yet, and updates /etc/fstab.
/go/s3 with no options Prints the usage line and exits with status 1.

The command needs s3fs to be installed. It does not install it. Run the command as root.


Options

Option Required Notes
-b <bucket> Yes The bucket name.
-k <access key> Yes The access key ID.
-s <secret key> Yes The secret access key.
-e <endpoint> Yes The host name of the service, without https://. A port is allowed, for example s3.example.com:9000. The connection always uses HTTPS.

The key and secret are only saved when you give both. When you leave them out, the existing /etc/passwd-s3fs is used. The /etc/fstab entry is only written when you give -b and an endpoint that contains a dot.


Output

Outcome What you get
No options Usage: s3 [-b] Bucket Name [-k] Access Key ID [-s] Secret Access Key [-e] End Point and exit status 1.
An unknown option The same usage line, and the command keeps running.
Mounted Nothing.
Mount failed The error message from s3fs, if it prints one. The command doesn't check for it.

Notes on the output:


Examples

# Mount a bucket
/go/s3 -b mybucket -k ACCESSKEY -s SECRETKEY -e s3.example.com
# Check that it is mounted and readable
df -h /s3
mount | grep s3fs
ls /s3
# Mount it again from /etc/fstab, for example after a failed attempt
mount /s3
# Unmount it and stop it from mounting at boot
umount /s3
sed -i "/s3 fuse.s3fs/d" /etc/fstab

If you set up the bucket with GoAdmio's own object storage, create the bucket and an access key first with bucket, and secure the endpoint with miniossl.


Troubleshooting

Symptom Likely cause Fix
s3fs: command not found s3fs isn't installed. Install it (s3fs-fuse on AlmaLinux, Rocky, CentOS, Red Hat, and Fedora, s3fs on Debian and Ubuntu), then run mount /s3.
Running /go/s3 again changed nothing The mount is only attempted when /s3 doesn't exist. Unmount, then run mount /s3 to use the new /etc/fstab line.
/s3 is empty and no bucket is mounted The first mount failed, or the server rebooted and the /etc/fstab line is missing. Check grep s3fs /etc/fstab, then run mount /s3 and read the error.
No /etc/fstab line was added The endpoint has no dot (for example localhost), or -b was not given. Use a full host name or IP address, or add the line yourself.
Transport endpoint is not connected The s3fs process died and left a stale mount. Run umount -l /s3, then mount /s3.
Access denied, signature, or bucket does not exist errors A wrong key, secret, bucket name, or endpoint. Re-run /go/s3 with the correct values, then unmount and mount again.
New accounts' folders are created on the local disk /s3 exists but the bucket isn't mounted. Mount the bucket, and move any folders created meanwhile.
Files show as owned by user 1000 The mount options force uid=1000 and gid=1000. Edit the /etc/fstab line if another owner is needed.
/tmp is filling up The s3fs cache is stored in /tmp/s3. Delete the cache, or remove use_cache from the /etc/fstab line.
The secret key is in /var/log/go/s3.log The command logs its whole command line. Create a new key, delete the old one in the storage service, and remove the line from the log.

Details

What happens on each run

flowchart TD
    A["s3 -b -k -s -e"] --> B["Log the call, including all arguments"]
    B --> C["Read the options"]
    C --> D{"All four options empty?"}
    D -- "Yes" --> X["Print usage, exit 1"]
    D -- "No" --> E{"Key and secret given?"}
    E -- "Yes" --> F["Write /etc/passwd-s3fs, mode 600"]
    E -- "No" --> G
    F --> G{"/s3 exists?"}
    G -- "No" --> H["Create /s3 and run s3fs to mount the bucket"]
    G -- "Yes" --> I
    H --> I{"Bucket given and endpoint has a dot?"}
    I -- "Yes" --> J["Replace the s3fs line in /etc/fstab"]
    I -- "No" --> K["Done"]
    J --> K

The mount options

The same options are used for the first mount and for /etc/fstab. /etc/fstab also has _netdev, which tells the system to wait for the network.

Option Effect
url=https://<endpoint> The storage service to connect to.
passwd_file=/etc/passwd-s3fs Where the access key and secret key are read from.
use_path_request_style Puts the bucket name in the URL path instead of the host name. MinIO and many other services need this.
allow_other Lets users other than root use the mount.
umask=0000 Gives everyone read and write access to every file.
uid=1000, gid=1000 Shows every file as owned by user and group 1000.
enable_noobj_cache Remembers that a file doesn't exist, to speed up repeated lookups.
use_cache=/tmp/s3 Keeps a local copy of recently used files in /tmp/s3.

The command has no setting for a region, an HTTP (non-HTTPS) endpoint, or other s3fs options.

The /etc/fstab entry

Every line containing s3 fuse.s3fs is deleted, and one new line is added at the end, so running the command again replaces the old entry. /etc/fstab is not backed up first, and the line doesn't use nofail.

Security

Risk Detail
Secret key in the log The log has every argument, including -s, in plain text.
Secret key on the command line Visible in your shell history and, briefly, in the process list.
Open access With allow_other and umask=0000, any user or hosted account on the server can read, change, and delete every object in the bucket.
Credentials file /etc/passwd-s3fs is written first and set to mode 600 right after. It is readable only by root.

Good uses

s3fs suits backups, archives, and media files. It is slow for many small files and not suitable for databases or sites' PHP code.


See also

Command Description
minio Install MinIO on a GoAdmio server
miniossl Install an SSL certificate for MinIO
bucket Create a MinIO bucket, make it public or private, or create an S3 access key
storage Auto-mount the /storage folder on boot using rclone
maria Incremental MariaDB backup to S3 or via Rclone
create Create a GoAdmio account, and a matching folder in /s3 when it exists