Setting up a web server with nginx and docker compose

Motivation

When I set up my personal website some time ago, it was served directly by apache installed on my host VPS. Recently, I wanted to wrap this functionality in containers, in order to add a layer of control to the web server and decouple its configuration from my host. It was also a good occasion to clean up this web server configuration and store it in a repo with some documentation to make it easier to get back to later.

Setting up nginx and Docker Compose

Getting started with the nginx image

I wanted to use nginx and Docker Compose.

I started by getting my hands on the nginx Docker image to host simple static content. I ran the command below, which is more or less the first instruction in the image documentation. I just defined a fixed version for the nginx image by adding a tag. I usually do this for reproducibility reasons and to manually control updates (because omitting a tag is equivalent to using “latest”, which can yield a different version each time the image is pulled). I also exposed container ports 80 and 443, default ports for HTTP and HTTPS traffic.

docker run --name nginx-1 \
  -d \
  -p 80:80 \
  -p 443:443 \
  -v /home/tvoirand/webserver:/usr/share/nginx/html:ro \
  nginx:1.28-alpine
# http://<my-ip>:80 displays "hello world" message

In this example I’m only mentioning one website, but in fact I was serving several of them on this VPS. So I needed to configure nginx to route requests to each domain name towards their corresponding content. And we’ll see later that we’ll need to tweak this configuration again to handle HTTPS. I first fetched the nginx image default configuration file with:

docker run --rm --entrypoint=cat nginx:1.28-alpine \
  /etc/nginx/nginx.conf > /home/tvoirand/webserver/nginx.conf

I added a virtual host (server block) inside the HTTP block, to define how requests to my domain name are processed. This virtual host simply specifies the path to the static files within the nginx container. Let’s assume they are in /usr/share/nginx, we’ll see right after how to configure the container to access these files.

(Omitting the beginning of the file for brevity)
... 
http {
    server {
        listen 80;
        server_name thibautvoirand.com www.thibautvoirand.com;
        location / {
            root /usr/share/nginx/thibautvoirand;
        }
    }
}

I then ran the container with options to bind mount the configuration and the static content. Both mounts can be mapped to any location on the host.

docker run --name nginx-1 \
  -d \
  -p 80:80 \
  -p 443:443 \
  -v /home/tvoirand/webserver/nginx.conf:/etc/nginx/nginx.conf:ro \
  -v /home/tvoirand/thibautvoiranddotcom:/usr/share/nginx/thibautvoirand:ro \
  nginx:1.28-alpine

When reaching http://thibautvoirand.com in the browser, I could see my website, defined by the static files located in /home/tvoirand/thibautvoiranddotcom.

Polishing the configuration

Next, I improved this configuration by moving the virtual host definition in a separate file. Since I was serving several websites, having separate configuration files helped keeping things tidy. This is allowed by the include /etc/nginx/conf.d/*.conf instruction present in nginx’s default configuration. Each file stored in the conf.d directory is read as if it was included in the http block of the main nginx configuration. So the contents of the new configuration file /home/tvoirand/webserver/conf.d/thibautvoirand.conf were:

server {
    listen 80;
    server_name thibautvoirand.com www.thibautvoirand.com;
    location / {
        root /usr/share/nginx/thibautvoirand;
    }
}

After removing the server blocks in the main nginx.conf file, it was back in its default state. I could therefore avoid mounting this file in the docker run command. I added the new conf.d directory bind mount instead:

docker run --name nginx-1 \  
  -d \
  -p 80:80 \
  -p 443:443 \
  -v /home/tvoirand/webserver/conf.d:/etc/nginx/conf.d:ro \
  -v /home/tvoirand/thibautvoiranddotcom:/usr/share/nginx/thibautvoirand:ro \
  nginx:1.28-alpine

Now that I was happy with this command, I converted it to a compose.yaml file to be used by Docker Compose. This centralizes the web server definition, allowing to avoid typing everything in the terminal and to track changes in version control. Docker Compose also automatically sets up a user-defined bridge network for the services defined in a single compose.yaml file, isolating them from other containers potentially present on the host.

version: "3"

services:
  nginx:
    image: nginx:1.28-alpine
    ports:
      - 80:80
      - 443:443
    restart: always
    volumes:
      - /home/tvoirand/webserver/conf.d:/etc/nginx/conf.d:ro
      - /home/tvoirand/thibautvoiranddotcom:/usr/share/nginx/thibautvoirand:ro

The container is run with docker compose up -d, and stopped with docker compose down.

And that was it. I now had a nicely containerized web server to serve my website.

However, even though I exposed port 443 of the container to allow HTTPS, I also needed to add SSL certificates to actually enable HTTPS for the website. I used certbot for that.

Setting up SSL certificates and HTTPS traffic

Certbot flow step-by-step

I added a certbot service to the compose file, along with two bind mounts also shared with the nginx service:

version: "3"

services:
  nginx:
    image: nginx:1.28-alpine
    ports:
      - 80:80
      - 443:443
    restart: always
    volumes:
      - /home/tvoirand/webserver/conf.d:/etc/nginx/conf.d:ro
      - /home/tvoirand/thibautvoiranddotcom:/usr/share/nginx/thibautvoirand:ro
      - /home/tvoirand/webserver/certbot/www/:/var/www/certbot/:ro
      - /home/tvoirand/webserver/certbot/conf/:/etc/nginx/ssl/:ro

  certbot:
    image: certbot/certbot:v4.1.1
    volumes:
      - /home/tvoirand/webserver/certbot/www/:/var/www/certbot/:rw
      - /home/tvoirand/webserver/certbot/conf/:/etc/letsencrypt/:rw
    depends_on:
      - nginx

I updated the nginx virtual host configuration file to make it serve the acme-challenge files used by certbot to create the certificates:

server {
    listen 80;
    listen [::]:80;
    
    server_name thibautvoirand.com www.thibautvoirand.com;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }
    
    location / {
        root /usr/share/nginx/thibautvoirand;
    }
}

I tested the certificate creation with a dry run (because certbot limits requests available per day):

docker compose run --rm certbot \
  certonly \
  --webroot \
  --webroot-path /var/www/certbot/ \
  --dry-run \
  -d thibautvoirand.com

The command being successful, I ran it again without the --dry-run flag to actually create the certificates.

I then updated the nginx configuration to add a new nginx virtual host that would handle HTTPS traffic:

...
server {
    listen 443 ssl default_server;
    listen [::]:443 ssl default_server;
    http2 on;
    server_tokens off;

    server_name thibautvoirand.com www.thibautvoirand.com;

    ssl_certificate /etc/nginx/ssl/live/thibautvoirand.com/fullchain.pem;
    ssl_certificate_key /etc/nginx/ssl/live/thibautvoirand.com/privkey.pem;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }
    
    location / {
        root /usr/share/nginx/thibautvoirand;
    }
}

• [::]: means that we listen on IPv6 in addition to IPv4
• default_server means that requests that don’t fit any of the server_name filters will end up on this server
• http2 on allows to benefit from the latest HTTP protocol and improved performance
• server_tokens off removes nginx version from error messages, for security reasons

I then restarted nginx with docker compose restart nginx. And there I had HTTPS enabled for my website.

However, this procedure - consisting in 1/ adding the HTTP server block, 2/ obtaining the certificates, and 3/ adding the HTTPS server block - could be a bit of a pain if we were to reproduce this many times. In particular, I didn’t want to have to modify the nginx configuration files each time I wanted to deploy the server. But if I had used the final nginx configuration right away, nginx would have failed and complained that the files defined in the ssl_certificate and ssl_certificate_key instructions didn’t exist. And to create the certificates, certbot needs nginx running and serving its acme-challenge files.

Workaround to avoid editing the nginx configuration when deploying

A solution to this consists in creating dummy certificates before starting nginx, to make sure it starts and serves the acme-challenge files used by certbot. These dummy files then need to be cleared, after nginx has started and before requesting the real certificate. Otherwise certbot will see that certificates already exist and won’t dare overwrite them. So the workaround consists in:

1. Creating dummy certificates

   sudo mkdir -p certbot/conf/live/thibautvoirand.com
   sudo openssl req \
     -x509 \
     -noenc \
     -newkey rsa:2048 \
     -days 1 \
     -keyout certbot/conf/live/thibautvoirand.com/privkey.pem \
     -out certbot/conf/live/thibautvoirand.com/fullchain.pem \
     -subj "/CN=thibautvoirand.com"
   

2. Starting nginx

   docker compose up -d nginx
   

3. Removing the dummy SSL certificates

   sudo rm -rf certbot/conf/live/thibautvoirand.com*
   sudo rm -rf certbot/conf/archive/thibautvoirand.com*
   sudo rm -rf certbot/conf/renewal/thibautvoirand.com*
   

4. Requesting the real certificates (use the --dry-run flag first to make sure everything is correct, then the real command)

   docker compose run --rm certbot \
     certonly \
     --webroot \
     --webroot-path /var/www/certbot/ \
     -d thibautvoirand.com
   

5. Restarting nginx

   docker compose restart nginx
   

With this procedure, I was able to store my nginx configuration file as “single truth” in control version, without having to modify it. It involved quite many commands, but these can probably be automatized.

To complete the SSL certificates handling, I added a weekly cronjob to renew them (certificates are due to renewal 30 days before they expire):

52 2 * * 3 docker compose -f /home/tvoirand/webserver/compose.yaml run certbot renew
57 2 * * 3 docker compose -f /home/tvoirand/webserver/compose.yaml restart nginx

Wrap-up

And with this final touch, I was happy with my web server setup !

The web server was configured by a few files: compose.yaml, conf.d/thibautvoirand.conf, plus another .conf file for each additional website I needed to host. I could store these files wherever I wanted on my host. They could remain unchanged when deploying on any other host, as long as I used the same paths to the webserver config and website contents on the host. This allowed me to store and track them in version control.

With this setup, I could deploy the web server on a new host with my eyes closed by running only a few copy-pasted commands:

• Cloning the web server repo (and the websites repo themselves)
• Going through the cerbot flow
• Setting up the cronjob for SSL certificates renewal