Security in DNS, TLSA records now available in our control panel to support DANE

February 11th, 2020 by

The Internet is better when it’s secure. Finally, thanks to Let’s Encrypt it’s possible to automatically get SSL certificates free of charge and as a result the Internet is dramatically more secure than it used to be. If you’ve used our DNS API you may have discovered that you can verify Let’s Encrypt SSL certificate requests using DNS records, including issuing wildcard certificates.

We support secure DNS (DNSSEC) which prevents DNS records from being forged, making the process of authenticating your SSL certificate through DNS records far more secure than the email-based authentication that was typically used for certificates issued by commercial certificate authorities. We have implemented support for CAA records which uses DNS to restrict the certificate authorities that can issue your certificates. This is most useful if the DNS is trustworthy which, again, requires DNSSEC.

However, there seems to be an opportunity here to improve things further. Rather than relying on a 3rd party certificate authority to confirm that you have control of your own DNS, why can’t you just publish your certificate in DNS directly? If you can trust DNS this would seem to be an obvious improvement, and with DNSSEC, DNS becomes trustworthy. We’ve now added support for the additional record type TLSA which allows exactly that, as part of DNS-Based Authentication of Named Entities (DANE).

Adding a TLSA record through our control panel.

DANE is a flexible mechanism that can be used to add an additional layer of security to certificates issued by a 3rd party authority, or to enable the use of self-signed certificates.

Unfortunately at the moment few clients support TLSA so for the majority of interactions you’re still going to rely on the certificate authority to verify the certificate. But implementations exist for both Exim and Postfix. Step by step, email is becoming more secure.

Two Factor Auth – TOTP now available

January 27th, 2020 by

Good security practice requires two different factors.

We’ve just rolled out a much requested feature to our control panel: Timed One Time Passwords or TOTP.

TOTP is a type of 2FA. If these acronyms are making sense to you, head over to the control panel and set up TOTP.

If not, read-on…

What is 2FA?

You’ll probably have noticed an increasing number of websites that you use encouraging or requiring you to enable “two factor authentication” or 2FA.

2FA refers to requiring two separate things to confirm your identity: something you know (your password) and something you have (e.g. your phone).

2FA protects against some of the most common ways in which accounts get compromised:

  • Username/password re-use. Despite advice not to do so, plenty of people re-use passwords across lots of different sites. Every now and again, sites get compromised, and databases of usernames and passwords become available on the shadier parts of the internet. These credentials will then be tried against other sites, looking for places that they’ve been re-used.
  • Email account compromise. If your email account is compromised, it’s very easy for an attacker to gain access to your other accounts, as it’s almost always possible to reset your password by sending an email.
  • Key-logging. If your computer is compromised, or you use an untrusted shared computer, key-logging malware may be installed that logs your password as you type it to log into your account.

2FA protects against all of these. It’s no longer sufficient to know the username and password to login, and you can’t reset your password just by having access to the email account. 2FA uses “one time passcodes” which means that whilst they can be captured by a key-logger, they’re of no value as they’ve already been used.

TOTP, SMS and Recovery Codes

We now support three different methods to provide the second factor: SMS, TOTP and recovery codes. With a Timed One Time Password your phone uses a secret key and the current time to generate a unique six digit code. The code is only valid for a short period, and can only be used once. The code proves that you have access to the secret key in the phone, but does not require you to send the secret key or any part of it to us.

With SMS we send you a time-limited, one-time code via a text message. Your phone collects this and you can type it in during login to prove that you’re holding your phone.

Recovery codes are intended to be a fall back should you lose access to your primary 2FA method. These are a set of one time codes that you can store securely (e.g. on paper, in a safe) and use each of them for a single login as required.

TOTP has a number of advantages over SMS. Firstly, it’s entirely offline on your phone so that if you’re somewhere with no phone signal you can still log in. Secondly, it doesn’t rely on trusting the mobile phone network; anyone with access to the phone network could intercept your SMS code or arrange for it to be delivered to another device. Similarly you may have things like message sharing enabled which means that your passcode is delivered to multiple devices.

Setting up TOTP

TOTP is very easy to setup. You’ll need an app on your phone. You can use Google Authenticator, but we prefer the open source FreeOTP. Once installed, go to the two-factor auth page in our control panel and hit the big green “Enable TOTP” button.

You’ll be shown a QR code which you can scan into the app on your phone, and you can then start generating codes. You need to enter a code to confirm that it’s set up correctly, and you can then choose to require 2FA whenever you log into your account.

Whilst you’re there, you should take the chance to print off some recovery codes.

Updates to sympl to continue to support Let’s Encrypt

October 25th, 2019 by

Before you 3D print the keys from the photo, you should know they are no longer in use.

We’ve now updated Sympl to support the new ACME v2 protocol for long term Let’s Encrypt support.

Let’s Encrypt is changing the protocol for obtaining and renewing certificates from ACME v1, to ACME v2 and the version 1 protocol is now end-of-life. In the next few days (1st November) this means that new accounts will no longer be able to be registered which will prevent new sites obtaining SSL certificates. Final end of life occurs in 2021 when certificate renewals will start to generate errors and then fail entirely.

Symbiosis is now end of life, as Sympl is an actively developed fork we’d recommend any Symbiosis users migrate to Sympl. We’d also recommend our managed hosting as a good place to run your Sympl server.

Multiple Mythic Beasts staff members contributed to this update.

Let’s Encrypt support for older Debian

October 9th, 2019 by
seure cat

This cat is secure, but not dehydrated. (Credit Lizzie Charlton, @LizzieCharlton

Debian Jessie and Debian Stretch include dehydrated, a useful command line tool for managing Let’s Encrypt certificates. We use it fairly extensively for managing certificates throughout our servers and with our managed customers. Unfortunately due to a change in capitalisation at Let’s Encrypt, the standard copy of dehydrated shipped with Debian Jessie and Debian Stretch is no longer compatible. As there’s no package in backports, we’ve spun our own packages of a newer version of dehydrated which is available on our mirror server.

If you use the older version you’ll see an error like


{
"type": "urn:acme:error:badNonce",
"detail": "JWS has no anti-replay nonce",
"status": 400
}

or


{
“type”: “urn:ietf:params:acme:error:malformed”,
“detail”: “Malformed account ID in KeyID header URL: “https://acme-v02.api.letsencrypt.org/acme/acct/””,
“status”: 400
}

The fix is very simple, you just need to install our dehydrated packages. This is very easy to do.

First add our signing keys


wget -O - -q https://mirror.mythic-beasts.com/mythic/support@mythic-beasts.com.gpg.key | apt-key add -

Then the correct repository based on your version of Debian

echo deb http://packages.mythic-beasts.com/mythic/ jessie main >/etc/apt/sources.list.d/packages.mythic-beasts.com.list

or

echo deb http://packages.mythic-beasts.com/mythic/ stretch main >/etc/apt/sources.list.d/packages.mythic-beasts.com.list

then

apt-get update
apt-get install --only-upgrade dehydrated
dehydrated -c

and your copy of dehydrated will be updated to 0.6 and your certificates can be created as normal.

Let’s Encrypt wildcard certificates

February 15th, 2019 by

Wildcard… sounds a bit like wildcat… cat pics!
Photo by Peter Trimming, CC BY 2.0

We’ve just made some changes to our plugin for dehydrated in order to better support Let’s Encrypt wildcard certificates.

Unlike normal certificates, which can be obtained using a web-based challenge, Let’s Encrypt’s wildcard certificates require a DNS-based challenge. In other words, you need to prove that you can control the DNS for the domain for which you are requesting a wildcard certificate.

Mythic Beasts provides a simple API for controlling DNS, which makes it possible to automate the process of responding to these challenges, and we provide a plugin for the popular dehydrated client that does just this.

We’ve just deployed a minor change which means that it’s now possible to obtain a single certificate for a domain, and a wildcard under that domain.

Access to our DNS API is included with all domain registrations. For more information, please see our instructions on using DNS-based challenges wih Let’s Encrypt. Please note that in order to obtain wildcard certificates you need to be using dehydrated version 0.6.0 or later.

Let’s Encrypt, Dehydrated, Curl and redirects

March 15th, 2018 by

We use Let’s Encrypt for SSL certificates, and our preferred client for obtaining certificates is the simple but effective dehydrated shell script, not least because it’s packaged for Debian.

On Sunday, we started getting some alerts relating to a failure to automatically re-issue Let’s Encrypt certificates. A quick bit of digging yielded this error:

+ Creating fullchain.pem…
  + ERROR: An error occurred while sending get-request to http://cert.int-x3.letsencrypt.org/ (Status 301)

Let’s Encrypt have started including an HTTP redirect as part of the certificate issue process and dehydrated doesn’t pass the necessary option to curl to follow the redirect. This can be fixed by patching dehydrated (and a packaged fix for Debian Stretch is now available via Debian backports), but it can also be solved with a simple config change:

echo 'CURL_OPTS="-L"' > /etc/dehydrated/conf.d/curl.sh

Naturally, customers of our managed hosting services and customers using the free HTTPS option on our hosting accounts need not worry about this issue. Our managed hosting includes monitoring all HTTPS websites for certificates nearing expiry, so we become aware of any issues well before your users do.

Chrome to brand non-HTTPS sites as “insecure” – time to click the button

February 12th, 2018 by

As reported by The Register, sites which do not use HTTPS will soon be actively labelled as “insecure” by the Chrome browser. HTTPS is the secure form of HTTP that makes the little green padlock appear in browsers.

Ultimately, sites which use HTTP are going to be labelled like this:

Example of HTTP site labelled as "not secure"

Not subtle, eh?

The Reg article suggests that initial changes will be deployed July 2018, and will be a little more subtle, but with Chrome having 55-60% market share, it really is time to switch your website to HTTPS.

Fortunately, if you’re hosted with Mythic Beasts this is really easy.  All of our hosting accounts include free SSL (aka TLS) certificates (provided by Let’s Encrypt), and you can enable HTTPS hosting by just clicking a button in the control panel.  Here’s how:

Enabling HTTPS for your Mythic Beasts-hosted website

First, log in to our customer control panel, click on “Hosting and shell accounts”, and click through to the hosting account for your site.  Now find your site in the list, and click on “web settings”:

If you have both a “www” prefixed and bare version, as above, you’ll want to do both. 

On the web settings page, scroll down to the “security” section:

Screen shot of security settingsYou almost certainly want the third option: this will enable HTTPS hosting, and ensure that users see the secure version of the site by default.  (Once you’re happy that your HTTPS site is working exactly as you want it, you could consider switching to the fourth option).

Click, hit “save changes”:

Screenshot of "changes saved" messageWe’ve got plans to make this faster, but for the moment, you’ll need to wait a few minutes.  We’ll go and obtain a certificate for your site, and once installed update your site so that it redirects to the HTTPS by default.

Screenshot of HTTPS location bar

Bingo!

If you haven’t got a working HTTPS site within 10 minutes, email us – we’re here to help.

Any gotchas?

The instructions here will only work if the HTTP version of your site is hosted by Mythic Beasts.   If you’re configuring a new site with Mythic Beasts, make sure that you can access your site via HTTP before enabling HTTPS.

If you’re transferring a site to us that is already using HTTPS, please see our transfer in instructions for how to do this with an interruption to service.

Managed hosting

We’ve been deploying HTTPS as the default for customers of our managed services for some time. We’re going to be doing an audit of all managed sites to warn customers of this upcoming change, but in the meantime, if you’re a managed customer with an http site, just email us and we’ll sort it out.

Meltdown and Spectre

January 17th, 2018 by

A rack of Pi 3s… possibly the only cloud computers immune to Spectre?

There’s been a lot of activity in the news regarding two new security issues called Meltdown and Spectre.

The security issues are newsworthy because they’re different to any security issues we’ve seen before. They’re not an issue in software, but in your computer itself. As a result the vulnerabilities cross multiple operating systems – Windows/Linux/OSX and multiple devices – Laptop/Desktop/Server/Phone, and they’re also a lot harder to fix.  Meltdown affects only Intel processors.  Spectre also affects AMD, Power, RISC and some ARM CPUs.

If you’d like to know how the vulnerabilities work, Eben Upton wrote up a clear explanation for Raspberry Pi – the only common functional computer that isn’t affected.

At present the fixes for Meltdown are effective but can cause significant slowdowns. Fixes for Spectre are incomplete and we have had reports that they can cause instability in Haswell and Broadwell families of Intel CPUs (which we own). Spectre is difficult and slow to exploit because it relies on reading memory one bit at a time.  At 1500bytes/second a full memory dump of one of our virtual server hosts (256GB RAM) would take around six years to complete.

Impact

Both issues allow information leakage so that lower priority processes on a server can read secret data from higher priority processes on the same CPU.  Any computer that accepts instructions from an untrustworthy source is at risk.  We’ve reviewed the impact across all of our services, and have applied or will be applying patches as required.  The impact on live hosting platforms is as follows:

Shared hosting servers

Our web hosting and shell account hosting platforms may have untrustworthy users on them. These servers have already been fully patched against Meltdown and fixes for Spectre will be applied as they become available.

Virtual server hosts

Our Virtual Server Cloud uses KVM with hardware virtualisation which is not vulnerable to Meltdown. Spectre patches are being worked on for the kernel which require new microcode for the CPU. KVM will also need to be updated to fully patch. When these updates are available and have been demonstrated to be stable we will be applying them to our host servers.

This will require a restart of our VM hosts and all guest VMs. Customers will be notified in advance of requiring a restart and each of our datacentres will be restarted at a different time to minimise disruption to customers with split site services.

Virtual server guests

Whilst the use of KVM with hardware virtualisation ensures that Meltdown cannot be used to break the isolation between virtual server guests, virtual servers themselves are potentially vulnerable to both Meltdown and Spectre.   Customers should ensure that their servers are patched and rebooted if they have untrusted users or execute untrusted code.

Dedicated servers

Dedicated servers are at no significant risk unless you allow untrusted third parties to upload and execute code. If that’s the case managed customers can contact support@mythic-beasts.com and we’ll apply the Meltdown and Spectre fixes and reboot as a mutually convenient time.

Raspberry Pi 3 servers

As mentioned above, the Raspberry Pi is not affected and no action is needed.

All other systems

We have reviewed the risk to all other systems and are applying patches as required.  This has included patching, as a high priority, all staff desktops and laptops; websites are allowed to execute javascript which can be used to execute a successful Meltdown attack.

Education, and the teacher becomes the student.

October 6th, 2017 by

Learn more about XSS with Google

For a long time we’ve sponsored Gwiddle, a project that outgrew its hosting on Microsoft Azure, providing free hosting accounts for students. They’ve now become a fully fledged charity, The Gwiddle Foundation, and we’ve had to upgrade the servers we donated to accommodate their ever expanding user base.

Part of their security team is the very talented Aaron Esau (15), who recently applied his penetration testing skills to our website and picked up a difficult to exploit bug.

On our page that allows you to search for domain names, our code embedded the search terms in the results page without appropriately escaping the content. This is a classic cross site scripting bug. Exploiting this bug was far from trivial, as the search term had to be short and from a restricted character set.

Aaron managed to craft an exploit using an ingeniously short payload to extract a session cookie and has posted a full write-up of the vulnerability and exploit.

If you had recently logged into our control panel, not logged out, and then visited a malicious page with this exploit, then the attacker could steal a cookie which would, in theory, give the attacker access to your control panel pages. However, we practise defence in depth, and our cookies are tied to an IP address so simply stealing the session cookie doesn’t give you access unless you also share a source IP address. This is an example where NAT and IPv4 is less secure than having IPv6.

Once Aaron brought the bug to our attention we swiftly fixed the page, thanked him for notifying us and sent him an Amazon voucher to thank him for his time and responsible disclosure.

We should emphasise that we do not believe that anyone has ever attempted to exploit this bug, and that the IP restrictions on session cookies mean that the consequences were fully mitigated.

Nonetheless, it’s embarrassing for us to have such a stupid bug in our code and we’ve been investigating how it occurred. It seems that the reason it crept in is because the domain ordering pages use a different form framework from everything else. Most of our pages have HTML generated by a template, and wherever dynamic data is included, it’s run through a filter to escape any HTML characters. The domain ordering pages use a different approach with much of the HTML being generated by a form module which we then include verbatim into our output. Obviously the HTML in this data mustn’t be escaped, as it would break the form; the form module is responsible for escaping any user input. Unfortunately, there are some other parts of the page which don’t come from the form module, and so do need to be escaped. It’s not very clear from the template code which is which, leading to the bug of not escaping some fields.

CAA records

September 1st, 2017 by

A handful of the hundreds of different organisations, all of whom must be trustworthy.

Everybody knows that SSL is a good idea. It secures communications. At the heart of SSL is a list of certificate authorities. These are organisations that the confirm the identity of the SSL certificate. For example, if GeoTrust says that Raspberry Pi is Raspberry Pi we know that we’re talking to the right site and our communications aren’t being sniffed.

However, the list of certificate authorities is large and growing and as it stands, you’ve got to trust all of them to only issue certificates to the right people. Of course, through incompetence or malice, they can make mistakes.

CAA records are a relatively new mechanism that aims to stop this happening, making it harder to impersonate secure organisations, execute bank robberies and steal peoples’ identities.



CAA records enable you to list in your domain’s DNS the certificate authorities that are allowed to issue certificates for your domain. So, Google has a record stating that only Google and Symantec are allowed to issue certificates for google.com. If someone manages to persuade Comodo they are Google and should be issued a google.com certificate, Comodo will be obliged to reject the request based on the CAA records.

Of course, in order to be of any use, you need to be able to trust the DNS records. Fortunately, these days we have DNSSEC (dns security).

How does it work?

A typical CAA record looks something like this:

example.com. IN CAA 3600 0 issue "letsencrypt.org"

This states that only Let’s Encrypt may issue certificates for example.com or its subdomains, such as www.example.com.

Going through each part in turn:

  • example.com – the name of the hostname to which the record apply. In our DNS interface, you can use a hostname of “@” to refer to your domain.
  • IN CAA – the record type.
  • 3600 – the “time to live” (TTL). The amount of time, in seconds, for which this record may be cached.
  • 0 – any CAA flags
  • issue– the type of property defined by this record (see below)
  • "letsencrypt.org" – the value of the property

At present, there are three defined property types:

  • issue – specifies which authorities may issue certificates of any type for this hostname
  • issuewild – specifies which authorities may issue wildcard certificates for this hostname
  • iodef – provides a URL for authorities to contact in the event of an attempt to issue an unauthorised certificate

CAA records can be added using the new section at the bottom of the DNS management page in our control panel:

The @ in the first field denotes a record that applies to the domain itself.

At Mythic Beasts, we’re a bit skeptical about the value of CAA records. In order to protect against the incompetence of CAs, they rely on CAs competently checking the CAA records before issuing certificates. That said, they do provide a straightforward check that CAs can build into their automated processes to detect and reject unauthorised requests, so publishing CAA records will raise the bar somewhat for anyone looking to fraudulently obtain a certificate for your domain.