Build a simple and useful file sharing service

Published: 13 Nov 2022 | Last edited: 15 Sep 2023

Why is file sharing over the internet so difficult? You might know the situation where you have a great image you want to share but there’s some file limit that is stopping you. What do you do now? Maybe you have to compress the file, but now it doesn’t look as good. Maybe you have to use another platform entirely where that file limit doesn’t exist, like email, but this is just annoying for the recipient. Maybe you have to upload it to the cloud, but now the recipient has to open a slow web page.

In real life it’s easy, you can just put it on an USB stick and carry it to another computer, but for some reason the common operation of sharing a file from your computer online can be quite a hassle, even though it is so fundamental to the functioning of the internet.

Luckily there are many services which can be selfhosted that try to fix this issue, like for example PsiTransfer or transfer.sh. I tested a lot of these types of services, and these two were my favourites. Still, there are some problems with them. They both need a browser frontend to work and both need extra services running in the background all the time, next to the webserver itself. They also both have extra features which I don’t need. I just want the ability to share some files. I looked for alternatives that were simpler, but couldn’t find them. This is why I built my own setup. In this blog post I will go through the steps of installing it.

🌟 Introducing: transfer! 🌟

No extra background services; based on a simple web server and rsync™ technology; and best of all, doesn’t have a complicated frontend!

All files are first stored in one directory on your computer and these files are then synced to a server which exposes them to the internet. Sharing files is as easy as running the script with the file or directory name as a parameter and sending the link to your recipient. The shared files are protected from unwanted viewers with a random string before its file name, just like transfer.sh does. Deleting files is done by deleting the file in the directory on your computer and running the script without parameters, which syncs the changed directory to the server.

Normally when you share multiple files, they get compressed into one zip file. It is also possible to share multiple files and show them in a directory. Simply move multiple files into your desired directory, sync the files to the server and share the link of the directory. This is done using Nginx’s autoindex.

The setup consists of two main components: the web server and the client script.

Installing the service: server-side

In the following instructions, replace example.com with your own domain name and admin with the user that you use for SSH.

Set up the directory to store shared files

First, add the user to the www-data group:

# usermod -aG www-data admin

Make the directory /var/www/transfer.example.com/ on your server and give the directory the proper ownership and permissions with

# chown admin:www-data /var/www/transfer.example.com/

to make sure your user can properly write to it.

Set up the web server

Now we will set up the web server. For this I chose Nginx, but others should also be possible. Make sure you have an SSL certificate set up for transfer.example.com and www.transfer.example.com, because a file MITM at your recipient’s end would not be nice.

In the /etc/nginx/sites-available/transfer.example.com file on your server place the following content:

server {
    root /var/www/transfer.example.com;

    server_name transfer.example.com www.transfer.example.com;

    location ~ /.+ {
        autoindex on;
    }

    autoindex off;

    listen [::]:443 ssl; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/transfer.example.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/transfer.example.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
    if ($host = www.transfer.example.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    if ($host = transfer.example.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    listen 80;
    listen [::]:80;

    server_name transfer.example.com www.transfer.example.com;

    return 404; # managed by Certbot
}

After this activate the site with

# ln -s /etc/nginx/sites-available/transfer.example.com /etc/nginx/sites-enabled/

and restart the Nginx service.

Installing the service: client-side

Now we will set up the client side. First make sure to download the script from my github and place it in your PATH. The next step is to set the required environment variables.

The required environment variables are as follows:

export TRANSFER_URL="https://transfer.example.com/"
export TRANSFER_SRCDIR="$HOME/Public/"
export TRANSFER_RSYNC_CREDS="admin@example.com:/var/www/transfer.example.com/"

Make sure these are sourced on startup (place them in ~/.bashrc or the configuration file of whatever shell you use). Replace their values with your own.

TRANSFER_SRCDIR is the directory in which files that are to be shared are located on your computer.

TRANSFER_RSYNC_CREDS are the SSH credentials to log into your server and the directory where the files will be served from on the server.

Now source your shell configuration file or reboot to reload the environment variables.

After this, everything should work. To test it, try to share a random file by executing the following command:

$ transfer file

See if this returns a link and if this link is accessible. If it works, congratulations! I hope it will be useful for you.