HTTPS - how to make fine adjustments in Apache, nginx and robots.txt when installing a free Let's Encrypt SSL certificate on cloud Linux

Increasing security requirements have led HTTPS to become the default connection protocol for websites. This created several new problems for developers and system administrators, such as the need to redirect a domain from HTTP to HTTPS. In addition to this, in order to avoid duplication of content in search engines, it is necessary to redirect from www to non-www (or vice versa).

First, let's look at how to install Let's Encrypt certificate on cloud Ubuntu and from HTTP to HTTPS how to redirect in nginx.

Installing a free Let's Encrypt certificate on Cloud Linux

The first thing you should start with is to check the health of your domain on one of the many services that analyze your DNS records and Mail Server for errors.

For example, check google.com on the free service uptime.com

Errors in DNS Records

As you can see in the screenshot, there is a mismatch of DNS records. If your domain has similar errors, check the correct DNS on the hosting.

Errors in setting up Mail Server

If your Mail Server shows errors, you should check to see if your emails go to spam?

Despite the inconsistencies in the DNS records, google.com uses the https protocol and has a green lock.

If you check the certificate on ssllabs.com, then the connection class is "A", which is quite satisfactory.

Certificate Verification on ssllabs

As we can see, errors in the domain settings do not prevent you from installing an ssl certificate, but still check your domain.

Now let's look at how to make https for a site using the example of connecting an SSL certificate Let's Encrypt on cloud Linux ("Digitalocean - Serverpilot" bundle).

You can choose another cloud service or install Linux on Virtualbox on your computer for free to study the folder structure and train to execute commands on the command line.

Conditionally, we assume that you have already installed through Serverpilot, for example, Wordpress and connected the domain to DNS Digitalocean.

First we need to install Let's Encrypt and switch to its directory.

$ sudo git clone https://github.com/letsencrypt/letsencrypt $ cd letsencrypt 

Then we stop nginx-sp for a moment, because Serverpilot connects it to port 80, which we need to create our certificates. Otherwise, the installer will cause us an error.

 $ sudo service nginx-sp stop 

When port 80 is free, send a command that will allow encryption for your domain.

 $ sudo ./letsencrypt-auto certonly --standalone -d yourdomain.com -d www.yourdomain.com 

You can add all subdomains through -d subdomain.yourdomain.com.

If the creation of the certificate was successful, you will receive a response on the command line stating that your newly created certificate can be found in / etc / letsencrypt / live / $ domain. Where $ domain is your domain name, the correct full path will be shown on the command line. Now you can start the nginx-sp service again using the command:

 $ sudo service nginx-sp start 

Copy the full path where the new certificates were saved, it is useful for configuring HTTPS in nginx.

Now we switch to /etc/nginx-sp/vhosts.d, where Serverpilot stores the configuration files for the applications ("Apps") that you created on the Serverpilot control panel.

 $ sudo su $ cd /etc/nginx-sp/vhosts.d 

In the Serverpilot control panel, select the Apps section and find the name of the application to which we want to add an SSL certificate. In most cases, this is the domain name to the first point.

Next, we need to create an SSL configuration file in the /etc/nginx-sp/vhosts.d directory. We do this by typing:

 $ sudo nano yourappname.ssl.conf 

Remember to replace yourappname with the name of the application you just recorded.

Configuring SSL Certificate Configurations on the Nginx Server

The first question that interests us is redirecting from HTTP to HTTPS how to do it in nginx?

 server { server_name yourdomain.com www.yourdomain.com; listen 80; return 301 https://yourdomain.com$request_uri; } 

Then we need to make the SSL configuration, you can copy and paste the configurations presented below or select the ones suitable for you in the "Mozilla SSL Configuration Generator" section at: mozilla.imtqy.com/server-side-tls/ssl-config-generator/

 server { server_name yourdomain.com www.yourdomain.com; listen 80; return 301 https://yourdomain.com$request_uri; } server { listen 443 ssl http2; listen [::]:443 ssl http2; server_name yourdomain.com www.yourdomain.com; ssl on; ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem; #SSL Optimization ssl_session_timeout 1d; ssl_session_cache shared:SSL:20m; ssl_session_tickets off; # modern configuration ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_prefer_server_ciphers on; ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK'; # OCSP stapling ssl_stapling on; ssl_stapling_verify on; ssl_ecdh_curve secp384r1; add_header Strict-Transport-Security "max-age=31536000"; #ssl_ciphers EECDH:+AES256:-3DES:RSA+AES:RSA+3DES:!NULL:!RC4; # verify chain of trust of OCSP response ssl_trusted_certificate /etc/letsencrypt/live/yourdomain/chain.pem; #root directory and logfiles root /srv/users/serverpilot/apps/yourappname/public; access_log /srv/users/serverpilot/log/yourappname/yourappname_nginx.access.log main; error_log /srv/users/serverpilot/log/yourappname/yourappname_nginx.error.log; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-SSL on; proxy_set_header X-Forwarded-Proto $scheme; include /etc/nginx-sp/vhosts.d/yourappname.d/*.nonssl_conf; include /etc/nginx-sp/vhosts.d/yourappname.d/*.conf; } 

Change all "yourdomain" and "yourappname" to your data.

When done with the changes, press Ctrl + X to save the file. If you get an error that the file could not be saved, most likely you forgot to write sudo before entering the nano command.

Now restart nginx-sp for the changes to take effect:

 $ sudo service nginx-sp restart 

If you did everything right, check your domain at ssllabs.com

You should get class "A +".

Getting Class A Plus for SSL Certificate

Remember to renew the certificate every 90 days with the command:

 $ cd /home/user/letsencrypt $ sudo service nginx-sp stop $ sudo -H ./letsencrypt-auto certonly --standalone -d yourdomain.com -d www.yourdomain.com $ sudo service nginx-sp start 

Three months pass unnoticed, so many are interested in maintaining a permanent connection via the HTTPS protocol, how to update the SSL certificate automatically?

Just add the command for crontab to the end of the file:

 $ sudo crontab -e @monthly /home/ubuntu/letsencrypt/letsencrypt-auto certonly --renew-by-default --webroot -w /srv/users/serverpilot/apps/app_name/public -d domain.tld -d www.domain.tld 

Now your domain should be loaded with both http and https protocol.

But this is not all, it remains to figure out how to redirect from http to https in order to avoid duplication of content in search engines and not lose in traffic?

Redirect Apache from www to non-www (or vice versa) and HTTP to HTTPS

Redirecting from www to non-www and http to https:

 RewriteEngine On RewriteCond %{HTTPS} off [OR] RewriteCond %{HTTP_HOST} ^www\. [NC] RewriteCond %{HTTP_HOST} ^(?:www\.)?(.+)$ [NC] RewriteRule ^ https://%1%{REQUEST_URI} [L,NE,R=301] 

Redirect from non-www to www and http to https:

 RewriteEngine On RewriteCond %{HTTPS} off [OR] RewriteCond %{HTTP_HOST} !^www\. [NC] RewriteCond %{HTTP_HOST} ^(?:www\.)?(.+)$ [NC] RewriteRule ^ https://www.%1%{REQUEST_URI} [L,NE,R=301] 

How to redirect to https in Apache?

To configure redirection, select one of the above entries either in the Apache configuration file, if you have access to it, or .htaccess in the root directory of your site.

Where is the Apache configuration file located?

For Debian and Ubuntu, directory names containing the word "apache" are common; for Mandriva / Fedora, look for directories with the word "httpd". Finding them is not difficult, one of these paths contains the file you are looking for:

  • / etc / apache;
  • / etc / apache2;
  • / etc / http;
  • / etc / httpd2.

The names of the main configuration files:

  • httpd.conf;
  • httpd2.conf;

or

  • apache.con;
  • apache2.conf.

How it works?

Turn on the Apache conversion engine:

 RewriteEngine On 

We determine the need to redirect the request and the conditions of redirection:

 RewriteCond %{HTTPS} off [OR] RewriteCond %{HTTP_HOST} !^www\. [NC] 

Since conditions are connected via [OR], if either of these two conditions returns true, Apache will execute the rewrite rule (redirection).

The first condition determines whether there are HTTPS in the URL, the second is the presence of www in the URL.

Note that the pattern is a regular expression, so you must escape the dot in "www \." (not "www.").

The following line extracts www from the rest of the host name:

 RewriteCond %{HTTP_HOST} ^(?:www\.)?(.+)$ [NC] 

RewriteRule - redirection framework:

 RewriteRule ^ https://www.%1%{REQUEST_URI} [L,NE,R=301] 

Using this line, we tell Apache to redirect any request to a new URL, consisting of:

  • https: // www;
  • %1 : link to the non-www part of the host;
  • %{REQUEST_URI} : Request URI, without host name.

Three additional flags mean:

  • NE - do not execute special characters.
  • R = 301 - to use the HTTP 301 redirect status.
  • L - stop processing other rules and redirect immediately.

If you know the host name, you can improve the rule by entering the URL:

 RewriteEngine On RewriteCond %{HTTPS} off [OR] RewriteCond %{HTTP_HOST} ^www\. [NC] RewriteRule ^ https://example.com%{REQUEST_URI} [L,NE,R=301] 

Also specify the directive "Host: https://mydomain.com" in the robots.txt file.

Set the tag: "<link rel =" canonical "href =" https: // ... "/>" in the "<head>" section.

This simple configuration for redirecting www and non-www HTTPS requests to the canonical domain of the site will help get rid of duplicate content in search engines and increase the security of the connection.

Change all internal links from HTTP to HTTPS

It remains to change all internal links from http to https.

If this is a Wordpress site, change the settings of the admin panel http://mydomain.com to https://mydomain.com.

Change all http to https,

It is also necessary to find all the addresses of images and scripts and change http to https, or to relative links of the form src = "// mydomain.com/image.jpg". If you do everything correctly, a "green lock" will appear, and the connection to your site will be via https.

Improving client-server connection security

Due to the lack of protection of the http protocol, the https protocol was developed in 1994, which uses the SSL / TLS cryptographic system to encrypt data and establishes a secure connection using a secret one-time key. An SSL certificate is used to authenticate the connection, after which the exchange of data packets begins.

We figured out a simple example of obtaining an ssl certificate and a redirect to https, how to make the connection more secure, but this is not enough if you are working on a complex, serious project. For maximum protection of your site, it is necessary to pass a multi-step verification with the help of experienced qualified specialists. If you do not have the opportunity to do this, you can use the online vulnerability check services, for example: pentest-tools.com/website-vulnerability-scanning/web-server-scanner. This is better than nothing, but nevertheless, on many sites you can find a lot of vulnerabilities even with the help of free services.


All Articles