Let’s Encrypt wildcard certificate with Certbot

Getting Started

Let’s Encrypt is a free, automated, and open certificate authority (CA), run for the public’s benefit. They provide free SSL certificates so every website can run with HTTPS/TLS. Since mid-March 2018 1,  Let’s Encrypt issues wildcard SSL certificates. They can be used for every subdomain so it’s not necessary to request everytime a new certificate when a subdomain is created. This makes it even easier to provide HTTPS.

Using Certbot

The recommended way to requests Let’s Encrypt’s SSL Certificates is with Certbot 2. Certbot supports obtaining and installing certificates. It’s important to set up an automatic renewal too because Let’s Encrypt only provides certificates which are valid for 90 days 3. The recommended time spawn to renew a certificate are 60 days.

Requirements

It’s the simplest to install and use Certbot as root. Theoretically it’s possible to run Certbot without root privileges 4 5. But this process needs more effort. Then, a better choice would be to use an ACME client which doesn’t rely on root rights 6 7. Such a client isn’t covered in this article. Certbot currently requires Python 2.7 or 3.4+ running on a UNIX-like operating system.

Installing Certbot

RHEL/CentOS 7

For RHEL/CentOS 7, Certbot is packaged in EPEL. You can install the EPEL repository with the following  command:

sudo yum install epel-release

If the package epel-release isn’t available, you can install it manually. Here’s a guide explaining how.

After you enable EPEL, you can run the following command:

sudo yum install certbot

Ubuntu 14.04, 16.04, 17.10

For Ubuntu, a PPA is provided for Certbot. To install, run:

sudo apt-get update
sudo apt-get install software-properties-common
sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install python-certbot-apache

If your distribution isn’t covered in this article, you can go on Certbots site and select your web server and distribution, to get a detailed install instruction.

Please make sure you got Certbot 0.22.0 or higher else wildcard certificates aren’t supported. You can check this by running

 certbot --version

If your version is below 0.22.0 then follow “Install Certbot from GitHub” paragraph below.

From GitHub

If Certbot 0.22.0 or higher isn’t available in your package manager repositories you can install Certbot directly from GitHub so you get the most current version independent of what’s available. Run the following commands:

cd /opt/
sudo git clone https://github.com/certbot/certbot.git
cd certbot
sudo ./certbot-auto

It may ask you to install additional dependencies. Cancel with ‘c’, because you want to request a wildcard certificate.

This guide uses Certbot from a package manager. So note that you need to replace certbot with /opt/certbot/certbot-auto in all statements.

 Installing the certificate

If you installed Certbot with your package manager, run:

sudo certbot certonly --manual --preferred-challenges dns --server https://acme-v02.api.letsencrypt.org/directory

If you installed Certbot from GitHub, run:

sudo /opt/certbot/certbot-auto certonly --manual --preferred-challenges dns --server https://acme-v02.api.letsencrypt.org/directory
  • certonly Only issues a certificate. Doesn’t configure apache or ngnix
  • --manual Allows to perform the domain validation either manually or through a shell script
  • --preferred-challenges Defines the ACME challenge type (eg. http, dns). To issue wildcard certificates, the challenge must be dns. The explicit argument prevents an unnecessary http challenge.
  • --server Defines the CA server. The default is the acme-v01 server but for wildcard certificates, the v02 is needed.
[[email protected] ~]# sudo certbot certonly --manual --preferred-challenges dns --server https://acme-v02.api.letsencrypt.org/directory
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Please enter in your domain name(s) (comma and/or space separated) (Enter ‘c’
to cancel): example.org *.example.org

Here replace example.org and *.example.org with your domain. When you would like to have also a wildcard certificate for one of your subdomains, simply add *.apps.example.org for apps as subdomain.

Obtaining a new certificate
Performing the following challenges:
dns-01 challenge for example.org
dns-01 challenge for *.example.org
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
NOTE: The IP of this machine will be publicly logged as having requested this
certificate. If you’re running certbot in manual mode on a machine that is not
your server, please ensure you’re okay with that.
Are you OK with your IP being logged?
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
(Y)es/(N)o: y

Here you need to agree with typing y. Now you will be asked to add a TXT records to your domain:

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Please deploy a DNS TXT record under the name
_acme-challenge.example.org with the following value:
vQZQ64KbF_709AtRDMaaQv52HeqFOJstV5HQok_jjWQ
Before continuing, verify the record is deployed.
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Press Enter to Continue
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Please deploy a DNS TXT record under the name
_acme-challenge.example.org with the following value:
1hBkvFz3Xjl6qa_xHdAqCss4A36Uy2gmnNC-zjCTaSo
Before continuing, verify the record is deployed.
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -s

You’re asked to add these records at your DNS provider. Most of the time your DNS provider is your domain registar so the place where you registered your domain.

If you don’t know how to set up a TXT record at your DNS provider, here’s a list of articles of some common providers:

If your provider isn’t listed, simply google “<DNS Provider Name> add TXT record”. Remeber to only enter _acme-challenge as host in your DNS TXT record and not the entire domain name. When you successfully added your DNS TXT records, it should look like this

It may take up to 30 minutes until the DNS Record is active. When you lose your SSH connection to your server or accidentally cancel, don’t worry. Simply rerun the command stated above. Certbot remembers the DNS challenge, so you don’t need to change or add a TXT record again.

To verify that your DNS records are online, you can use Dig from Google Toolbox or simply use dig, if installed, yourself:

dig +noall +answer -t txt _acme-challenge.example.org
_acme-challenge.example.org. 900 IN TXT “vQZQ64KbF_709AtRDMaaQv52HeqFOJstV5HQok_jjWQ”
_acme-challenge.example.org. 900 IN TXT “1hBkvFz3Xjl6qa_xHdAqCss4A36Uy2gmnNC-zjCTaSo”

When you verified that your DNS TXT records are available, you can continue with pressing enter.

Press Enter to Continue
Waiting for verification…
Cleaning up challenges
IMPORTANT NOTES:
-- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/nikio.io/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/nikio.io/privkey.pem
Your cert will expire on 2018-11-01. To obtain a new or tweaked
version of this certificate in the future, simply run certbot
again. To non-interactively renew *all* of your certificates, run
“certbot renew”
-- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let’s Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le

When everything worked out, you should now have received your wildcard certificate. Sweet! If not, add a comment with your issue.

Installed certificate files

When everything went successfully, Certbot provides the certificate files here:

/etc/letsencrypt/live/<domain.tld>/

  • cert.pem is your public key
  • chain.pem is the chain of Let’s Encrypt CA certificate (DST Root CA x3 -> Let’s Encrypt Authority X3)
  • fullchain.pem is the combination of cert.pem and chain.pem, so it contains the chain DST Root CA x3 -> Let’s Encrypt Authority X3 -> yourdomain.tld
  • privkey.pem is your private key. Never share your private key with others. The private key is necessary to decrypt the encrypted data sent by clients.

Configuring Apache2 with your certificate

Installing mod_ssl

Ensure you have installed the SSL module in apache2. If not, install it for RHEL/CentOS with:

 sudo yum install mod_ssl

Ubuntu:

 sudo a2enmod ssl

Global configuration

To enable the certificate in apache2, open the ssl.conf with your favorite console editor (eg. nano).

For CentOS/RHEL:

sudo nano /etc/httpd/conf.d/ssl.conf

For Ubuntu:

 sudo nano /etc/apache2/sites-available/default-ssl.conf

Put the following lines at the end of the file:

SSLCertificateFile /etc/letsencrypt/live/example.org/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.org/privkey.pem
SSLCACertificateFile /etc/letsencrypt/live/example.org/chain.pem

VirtualHost configuration

Now define a VirtualHost 8 configuration for your domain.

<VirtualHost *:443>
SSLEngine On
ServerName example.org
ServerAlias *.example.org
</VirtualHost>

Now restart your apache2 instance

RHEL/CentOS 7

sudo systemctl restart httpd

Ubuntu

 sudo service apache2 restart

Configuring http to https redirect

So your domain can only be used with https, set up the following VirtualHost

<VirtualHost *:80>
RewriteEngine On
RewriteRule ^(.*)$ https://%{HTTP_HOST}$1 [R=301,L]
</VirtualHost>

This redirects every request with http:// to the corresponding https:// URL.

Automated renewal of your certificate

With the current setup, you need to renew your wildcard certificate all 3 months manually. As this is really annoying an automated way is preferred.

For automated renewal of your certificate, it’s the easiest to use a Certbot plugin for your DNS provider, if available. Here’s the list of providers with such a plugin. In this article, we don’t cover the usage of a DNS provider plugin and using acme-dns instead which is an independent solution and doesn’t require support for ACME challenges from your DNS provider.

About acme-dns

acme-dns is a small DNS server with the only purpose to handle ACME DNS challenges9. This is a great way to automate your challenges without the support of your DNS provider.

Installation of acme-dns

To install acme-dns you need Go 10.

For CentOS/RHEL:

sudo yum install golang

For Ubuntu:

sudo apt-get install golang-go

Make sure you install Go 1.9 or higher. You can verify this by typing go version in the console.

go version
go version go1.9.4 linux/amd64

Now download acme-dns with the following command:

go get github.com/joohoi/acme-dns/…

This downloads acme-dns in the directory/home/<user>/go/bin/acme-dns Now copy or move the binary in a more appropriate place:

sudo cp /home/<user>/go/bin/acme-dns /home/usr/local/bin/acme-dns

Don’t forget to replace <user> with the user used to download acme-dns.

Configuration of acme-dns

The acme-dns executable needs a folder for its SQLite database and a configuration file placed in /etc/acme-dns/. So let’s create the two needed folder as root:

sudo mkdir /etc/acme-dns/ /var/lib/acme-dns

Now let’s get the configuration file template from GitHub and placed it in the newly created folder:

 sudo wget https://raw.githubusercontent.com/joohoi/acme-dns/master/config.cfg -O /etc/acme-dns/config.cfg

You need to execute wget with sudo because the folder is only writeable as root.  Now open the downloaded config.cfg file with your favorite console editor, also as root:

sudo nano /etc/acme-dns/config.cfg

In the example configuration file, the domain example.org is used. Modify the first section and replace example.org with your domain name.

# domain name to serve the requests off of
domain = "auth.example.org"
# zone name server
nsname = "ns1.auth.example.org"
# admin email address, where @ is substituted with .
nsadmin = "admin.example.org"

In the next section you need to modify the predefined DNS records:

# predefined records served in addition to the TXT
records = [
    # default A
    "auth.example.org. A 192.168.1.100",
    # A
    "ns1.auth.example.org. A 192.168.1.100",
    "ns2.auth.example.org. A 192.168.1.100",
    # NS
    "auth.example.org. NS ns1.auth.example.org.",
    "auth.example.org. NS ns2.auth.example.org.",
]

Again, replace example.org with your domain name. Replace 192.168.1.100 with the public IP address of your host. Remove the two lines with ns2.auth.example.org as we only need one nameserver entry.

acme-dns needs two ports. Port 53 for DNS and port 80 for its REST service. You shouldn’t change the port for DNS but you can change the HTTP port. This is necessary, when a web server is already running, like apache2 or nginx. So let’s change the HTTP port to 8081.

# listen port, eg. 443 for default HTTPS
port = "8081"

You can let the other values unchanged. You find more information about the possible configurations at acme-dns GitHub repository11

Systemd service to run acme-dns

So that acme-dns can run as a service on your host it’s convenient to create a systemd service which takes care about starting/stopping acme-dns.

Create the user acme-dns with the following command:

sudo adduser --system --base-dir /var/lib/acme-dns acme-dns

Change the owner of /var/lib/acme-dns so the freshly created user has the permissions needed.

chown acme-dns:acme-dns /var/lib/acme-dns

If you copied the acme-dns binary to another directory remember to change the path accordingly.

Now create and open the acme-dns.service12 file in /etc/systemd/system/:

sudo nano /etc/systemd/system/acme-dns.service

and put the following content into the file:

[Unit]
Description=Limited DNS server with RESTful HTTP API to handle ACME DNS challenges easily and securely
After=network.target

[Service]
User=acme-dns
Group=acme-dns
Type=simple
AmbientCapabilities=CAP_NET_BIND_SERVICE
Restart=on-failure
ExecStart=/usr/local/bin/acme-dns

[Install]
WantedBy=multi-user.target

Make sure the ExecStart value matches the place where you copied the acme-dns binary at in the “Installation of acme-dns” section.

Save and close the service file.

To activate the service we need to reload the systemd manager configuration:

 sudo systemctl daemon-reload

Now you can start acme-dns:

sudo systemctl start acme-dns

To check if acme-dns started successfully type:

sudo systemctl status acme-dns
acme-dns.service -- acme-dns Service
Loaded: loaded (/etc/systemd/system/acme-dns.service; disabled; vendor preset: disabled)
Active: active (running) since Mon 2018-08-06 00:05:25 CEST; 13min ago
[…]

and check if it’s Active: active (running)

Now enable the acme-dns service so it starts after a reboot:

sudo systemctl enable acme-dns

Firewall rules for acme-dns

So that acme-dns can answer DNS requests from the Let’s Encrypt servers you need to open port 53 in your firewall.

For firewalld:

sudo firewall-cmd --add-port=53/udp --permanent
sudo firewall-cmd --reload

For iptables13:

sudo iptables-A INPUT -p udp -m udp --dport 53 -j ACCEPT
sudo iptables -A OUTPUT -p udp -m udp --sport 53 -j ACCEPT

If you want to make the acme-dns REST API public available too, you need to open up the configured HTTP port too. This is not necessarily recommended as this service doesn’t need to be public if you run acme-dns on the same machine as Certbot.

For firewalld:

sudo firewall-cmd --add-port=8081/tcp --permanent
sudo firewall-cmd --reload

For iptables:

iptables -A INPUT -p tcp -m tcp --dport 8081 -j ACCEPT

DNS records

To make acme-dns able to receive DNS requests, we need to add two DNS records on your domains regular DNS provider:

  • NS record for auth.example.org pointing to ns1.auth.example.org.
  • A record for ns1.auth.example.org pointing to the public IP address of your host.

Replace example.org with your domain name. Depending on your DNS provider it may take up to 30 minutes until these records are active.

Testing acme-dns

You followed a lot of steps to make acme-dns working. Let’s test if everything’s working correctly.

To test if the required DNS NS record is set correctly you can use Dig from Google Toolbox or simply use dig. Don’t forget to replace example.org with your domain.

dig +noall +answer auth.example.org
auth.example.org. 2366 IN A 192.168.1.100

Verify that auth.example.orgis pointing to the public IP address of your host.

dig +noall +answer -t NS auth.example.org
auth.example.org. 3598 IN NS ns1.auth.example.org.

Verify that auth.example.orgis pointing to ns1.auth.example.org.

Run the following command to get register via the acme-dns REST API. We access the API directly via localhost so no firewall rule is needed.

curl -X POST http://localhost:8081/register
{“username”:”e0ad1a12-57eb-4916-8ac7-e115e44b018b”,”password”:”rrJSQj1-D4tgEw_vUegB_J28XhBkG31sr-6DKo0i”,”fulldomain”:”e8c5bb21-3e02-4e51-a99f-284d8a631a4a.auth.example.org”,”subdomain”:”e8c5bb21-3e02-4e51-a99f-284d8a631a4a”,”allowfrom”:[]}

If everything works correctly your getting a JSON with acme-dns user credentials. You don’t need to save these as it’s a test only.

Setup acme-dns hook for Certbot

To automate the ACME DNS challenge with acme-dns you need a hook for Certbot. There’re two available: One written in Python14 and one in Go15. We proceed using the Python version.

Download the latest hook script from GitHub into the /etc/letsencrypt/ directory and give it the necessary permissions:

sudo wget -O /etc/letsencrypt/acme-dns-auth.py https://raw.githubusercontent.com/joohoi/acme-dns-certbot-joohoi/master/acme-dns-auth.py
sudo chmod 0700 /etc/letsencrypt/acme-dns-auth.py

Now you need to change the predefined URL in acme-dns-auth.py:

sudo nano /etc/letsencrypt/acme-dns-auth.py
# URL to acme-dns instance
ACMEDNS_URL = "http://localhost:8081"

You can define http://localhost:8081 here if Certbot is running on the same machine as your acme-dns service. If you defined a different port in the configuration section, change it accordingly. Save and close the file.

Running Certbot with acme-dns-hook.py

Verify that you deleted all your __acme-challenge.example.org TXT DNS records to prevent unwanted failures.

Now to finally test Certbot with acme-dns and its hook run:

sudo certbot certonly --server https://acme-v02.api.letsencrypt.org/directory --preferred-challenges dns --manual --manual-auth-hook /etc/letsencrypt/acme-dn
s-auth.py --debug-challenges
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Please enter in your domain name(s) (comma and/or space separated) (Enter ‘c’
to cancel): example.org *.example.org

Replace example.org and *.example.org with your domain name and press enter.

You have an existing certificate that has exactly the same domains or certificate name you requested and isn’t close to expiry.
(ref: /etc/letsencrypt/renewal/example.org.conf)
What would you like to do?
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
1: Keep the existing certificate for now
2: Renew & replace the cert (limit ~5 per 7 days)
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Select the appropriate number [1-2] then [enter] (press ‘c’ to cancel): 2

As we already issued a certificate in the beginning but want to replace it to test the process with acme-dns, hit 2 and press enter. Note that you only can issue 5 certificates per week.

Renewing an existing certificate
Performing the following challenges:
dns-01 challenge for nikio.io
dns-01 challenge for nikio.io
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
NOTE: The IP of this machine will be publicly logged as having requested this
certificate. If you’re running certbot in manual mode on a machine that is not
your server, please ensure you’re okay with that.
Are you OK with your IP being logged?
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
(Y)es/(N)o: y
Output from acme-dns-auth.py:
Please add the following CNAME record to your main DNS zone:
_acme-challenge.example.org CNAME 23c2f5c5-e92e-4979-a6c2-5154b5ebb99f.auth.example.org.
Waiting for verification…
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Challenges loaded. Press continue to submit to CA. Pass “-v” for more info about
challenges.
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Press Enter to Continue

Now your asked to add a CNAME DNS record for__acme-challenge.example.org pointing to the URL printed to the console. Here it’s 23c2f5c5-e92e-4979-a6c2-5154b5ebb99f.auth.example.org, but make sure you’re using the URL in your console output.

As a reminder: Verify that you deleted all your __acme-challenge.example.org TXT DNS records to prevent unwanted failures.

After you added the required CNAME DNS record, verify in a separate terminal that it’s already active:

  dig +noall +answer __acme-challenge.example.org
__acme-challenge.example.org. 263 IN CNAME fd3e11e8-8af4-42dd-b70d-1cc6d9c4482c.auth.example.org.

It may take some time until the record is active. After you verified that the CNAME record is in place, you can continue the certbot process with enter.

Cleaning up challenges
IMPORTANT NOTES:
-- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/example.org/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/example.org/privkey.pem
Your cert will expire on 2018-11-03. To obtain a new or tweaked
version of this certificate in the future, simply run certbot
again. To non-interactively renew *all* of your certificates, run
“certbot renew”
-- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let’s Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le

If everything worked correctly you should see the message above. If not, check again if you followed all steps correctly. If it’s still not working, add a comment below describing your problem.

Setup cronjob for automated renewal

Finally, we can add a cronjob to run certbot renew daily. This command only renews the certificate if it close to expiration.

Run crontab as root:

sudo crontab -e

And add the following text to run cerbot renew every day at 2:30 in the morning.

30 2 * * * /usr/bin/certbot renew >> /var/log/le-renew.log

If you’re using the Let’s Encrypt certificate with ngnix or apache2, you can add a post-hook

30 2 * * * /usr/bin/certbot renew --post-hook "systemctl reload nginx" >> /var/log/le-renew.log

Replace systemctl reload nginx with the command needed to restart your web server. If something didn’t work as expected, you can check the output in the /var/log/le-renew.log file with tail -f or less.

Congratulations! You’ve finally set up the automated renewal of you wildcard certificate. If you found this guide helpful, have some feedback for me or if something didn’t work as expected, please drop a comment below.

  1. https://community.letsencrypt.org/t/acme-v2-and-wildcard-certificate-support-is-live/55579
  2. https://certbot.eff.org/
  3. https://letsencrypt.org/2015/11/09/why-90-days.html
  4. https://certbot.eff.org/docs/install.html#system-requirements
  5. https://certbot.eff.org/faq/#does-certbot-require-root-administrator-privileges
  6. https://github.com/diafygi/letsencrypt-nosudo
  7. https://github.com/zenhack/simp_le
  8. https://httpd.apache.org/docs/2.4/vhosts/examples.html
  9. https://www.eff.org/de/deeplinks/2018/02/technical-deep-dive-securing-automation-acme-dns-challenge-validation
  10. https://golang.org/
  11. https://github.com/joohoi/acme-dns#configuration
  12. Thanks gabe565 https://github.com/joohoi/acme-dns/blob/master/acme-dns.service
  13. https://serverfault.com/a/508682
  14. https://github.com/joohoi/acme-dns
  15. https://github.com/koesie10/acme-dns-certbot-hook

This Post Has 3 Comments

    1. Yes for sure. While entring the domain name(s), you can submit any domain which DNS records you can control.

  1. Loved the post keep it up!

Leave a Reply

Close Menu