Explain To Us: an explanation

September 17th, 2014 by

This post is written by Ben Howe who’s with us for his gap year. He took some holiday to go to Young Rewired State and he’s written up what he did.

During the summer I participated in the Young Rewired State Festival of Code, where I wrote a website called “Explain To Us” – designed to make the list of bills currently before parliament less confusing. For example, the “summary” listed on the website is the “long title” from the legal text, as opposed to being clearer more detailed. Explain To Us is a platform which allows members of parliament to upload a short video “pitch” to explain what their bill is about and why it should be given debating time in parliament.

Young Rewired State is an independent global network of young people aged 18 and under who have taught themselves to code. They bring together like-minded peers at events around the world where they use open data to make websites, apps and other solutions to real world challenges. The Festival of Code is an annual celebration of programming held over a week in multiple centres, culminating in a huge showcase weekend where everyone presents their solutions (this year in Plymouth, hosted by Plymouth University).

A Python script interprets the RSS feed for all bills before parliament. It then visits the <link> for each bill, scraping the sponsor information (as this is not included in the RSS feed). HTML pages are then generated, which are served by nginx. In future, I’m planning to automatically email members  of parliament (using contact details from TheyWorkForYou) when they submit a new bill asking them to upload a video.

After my initial presentation, I was nominated for “Best In Show” – although sadly I was knocked out by “YouDraw” (who later won Best In Show). Having gained valuable feedback from presenting to the Speaker’s Commission on Digital Democracy and Tracy Green (head of online services at parliament), I’m now intending on building a second beta with more functionality.

If you’d like to work for Mythic Beasts in your gap year, you need to demonstrate how you can use Google to find our jobs page and work the rest out from there.

SSL certificates: SHA-1 deprecation

September 16th, 2014 by

We’ve been asked a few times recently about the announcement from the developers of the  Chrome browser concerning SHA-1 deprecation. This post gives some background, and answers the most common questions. If you’re in a hurry: don’t panic! Mythic Beasts has got you covered.

What’s it all about? For an SSL certificate to be trusted by browsers around the world, it needs to be digitally signed by a well-known and trusted body, called a Certification Authority or CA. (For the SSL certificates that we sell, the CA is usually GeoTrust.) When a certificate is issued, the CA takes the details of your certificate (most notably the address of your website, and the public half of your crypto key) and digitally signs them. When a user browses to your site, they receive your signed certificate and they check the CA’s signature of it to be certain that they are talking securely to the right site.

Except… the CA doesn’t actually sign all the data in the certificate. It first passes it through a cryptographic hash function, which securely reduces the data to a small fingerprint, and then it signs that. Currently, the hash function most commonly used in SSL certificates is SHA-1. This is going to change over the next couple of years.

What’s wrong with SHA-1? Cryptographic hash functions become weaker over time, as Moore’s law makes computers ever faster, and cryptologists discover flaws in the algorithm. A significant flaw in SHA-1 was first published back in 2005, and it is now believed that it may be feasible to find a SHA-1 collision by 2018 or sooner.

Is there an alternative? Yes! SHA-2 (also known as SHA-256) is already implemented in all major browsers. However, because of the nature of the problem, every SSL certificate needs to be upgraded, so we can tell browsers to stop accepting SHA-1 signed certificates. Note that SHA-1 has not yet been “broken”, and while weakened, it is still strong enough for the time being. However, for a smooth transition to SHA-2, we need to start now.

What’s the Google announcement? Although this has generated a lot of discussion, it doesn’t actually say very much that’s new. Microsoft announced in November 2013 that they will not accept SHA-1 signed certificates after 2016. The Chrome developers have recently confirmed that they will do the same, and have filled in some details for a hopefully smooth transition.

Rather than having a single cut-off date, Chrome will be gradually ratcheting up the level of warnings. SHA-1 certificates with an expiry date in 2017 or later will start to receive the “yellow triangle” warning in the browser address later this year. Of course, the connection is still encrypted, but this is a clear indication to users, and hopefully the site administrators too, that something is amiss.

yellow-triangle

By the middle of next year, if those sites are still running with a SHA-1 certificate that lasts into 2017, the warning will be upgraded to a red cross, making it crystal clear to all concerned that action is needed.

red-cross

For SHA-1 certificates that have an expiry date in 2016, the situation is not so serious. Sites with these certificates will start to receive yellow triangle warnings next year, but the warning won’t be escalated beyond that. In both cases, only the security icon will be changed; there will be no click-through warnings. This move, which is also supported by Mozilla and Opera, is forcing the hand of the CAs, and much misinformation has been spread about it.

Is my certificate OK? Yes. Almost all* certificates issued by Mythic Beasts have only a 1 year expiry, so your current certificate will not solicit any warnings from Chrome, or any other browser. *Very occasionally we do issue certificates for longer than 1 year; we’ve checked the issue dates for all these, and are in contact with all affected customers to reissue their certificates.

And then…? We already have a process in place for issuing SHA-2 certificates and renewals. It’s currently a bit more fiddly than we’d like (in fact, it involves obtaining a SHA-1 certificate and then reissuing it as SHA-2!), but we will be trying to make this slicker over the next few weeks. In any case, after October 2014, we will only be issuing SHA-2 certificates.

bzip2

September 15th, 2014 by

bzip2 is one of the great unix tools. It compresses and uncompresses data, and it does it very well. We’ve been using it within Mythic Beasts for years and it’s operated absolutely flawlessly.

We’re happy to report that we’re now hosting the main distribution site for bzip2

Get your site ready for IPv6: a step-by-step guide

September 15th, 2014 by

We were asked on Twitter to add a page describing exactly how to get 10/10 on our IPv6 domain readiness checker, so here it is.

1. Add an IPv6 address to your web server

The first step is to get your web server listening on an IPv6 address, as well as an IPv4 address. How you achieve this will depend on how your web server is managed. If you’re on a shared hosting account, you’ll be dependent on your hosting provider. If you run your own server, you’ll need to obtain an IPv6 address from your hosting provider (assuming they support IPv6), configure your server to use it and then ensure that your web server (e.g. Apache is listening on this address).

If you’re using a standard Mythic Beasts hosting account, this step is already done for you. If you’re using a sphinx shell account, please contact support.

2. Add an AAAA record for your website

AAAA records are the IPv6 equivalent of A records, which resolve hostnames to IP addresses.  In order for users to find your website over IPv6, you will need to add an AAAA record for www.yourdomain.com pointing to the IPv6 address configured above.  You can check that this is in place using the dig command:

$ dig +short A www.mythic-beasts.com
93.93.131.39

$ dig +short AAAA www.mythic-beasts.com
2a00:1098:0:86:1000::15

It’s possible that your existing “www” record will be a CNAME for another hostname, in which case you should add the AAAA record to that hostname, rather than the “www” record.

If you’re using a Mythic Beasts hosting account, then provided that you are using the standard template for your account in the DNS configuration for your domain, then this step will already be done for you.

Our health checker will skip this test if your domain doesn’t have an A record for “www”.

3. Add an AAAA record for your bare domain

Most websites are configured to work if the user omits the “www” prefix from the name, for example http://mythic-beasts.com

In order for this to work, you will need an A record for your domain name itself, and to be IPv6-enabled, you’ll also need a corresponding AAAA record.

Once again, our checker will skip this test if the bare domain doesn’t have an A record.

4. Ensure your DNS servers have IPv6 addresses

The steps above make it possible to access your website over IPv6, but unless your DNS servers are accessible over IPv6, users (or more specifically, their DNS resolvers) will still need to use IPv4 in order to find your site in the first place. To avoid this, you need to ensure that your nameservers have IPv6 addresses.

You can find the nameservers for your domain using “whois”, and you can check whether the servers have IPv6 addresses using dig, as before:

$ whois mythic-beasts.com
[ ... ]
Name Server: NS0.BEASTS.ORG
Name Server: NS3.MYTHIC-BEASTS.COM
Name Server: NS2.MYTHIC-BEASTS.COM
Name Server: NS0.MYTHIC-BEASTS.COM
Name Server: NS1.MYTHIC-BEASTS.COM
[ ... ]
$ dig +short AAAA ns1.mythic-beasts.com
2600:3c00::f03c:91ff:fe96:beac

If your nameservers do not have IPv6 addresses, then unless you run your own nameservers, you’ll either need to persuade your hosting provider to enable IPv6, or switch your DNS provider to a different provider.

Mythic Beasts customers should be using ns1 and ns2.mythic-beasts.com, and both of these nameservers have IPv6 addresses, and IPv6 glue (see below).

For a full pass, our health checker requires that at least two of your servers have IPv6 addresses.

5. Add IPv6 glue for your nameservers, if necessary

In order to find the address for your website, a DNS resolver will first need to find the address of your nameservers. If your nameservers are in your own domain, this creates a bootstrapping problem. For example, in order to find the address for ns1.mythic-beasts.com, you need to ask the nameservers for mythic-beasts.com, which includes ns1.mythic-beasts.com. The solution to this is a glue record, a record containing the address of your nameserver which is held by the nameserver for the next zone up. In this case, the next zone up is .com, so the .com nameservers would contain glue records for the ns*.mythic-beasts.com nameservers.

If a nameserver has an IPv6 address, then any glue records for it should also contain that IPv6 address.

Checking for glue records is a little bit involved. The quickest way to do it is to use “dig +trace” to find a nameserver for the next zone up:

$ dig +trace ns1.mythic-beasts.com
[...]
com.      172800  IN  NS  a.gtld-servers.net.
com.      172800  IN  NS  b.gtld-servers.net.
com.      172800  IN  NS  c.gtld-servers.net.
[...]

We can now ask any of those servers for the NS records for our domain. Any glue records that exist will be returned in the “additional” section of the response:

$ $ dig NS mythic-beasts.com @a.gtld-servers.net.
[...]
;; ADDITIONAL SECTION:
ns1.mythic-beasts.com.  172800  IN  AAAA  2600:3c00::f03c:91ff:fe96:beac
ns1.mythic-beasts.com.  172800  IN  A 69.56.173.190
ns2.mythic-beasts.com.  172800  IN  AAAA  2a00:1098:0:80:1000::10
ns2.mythic-beasts.com.  172800  IN  A 93.93.128.67
[...]

If your servers are missing glue records, you will need to get your domain registrar to add them.

It’s worth noting that even if you don’t directly require glue because your nameservers are in a different zone, at some point along the chain there will be a nameserver that does require glue.

For a full pass, our glue checker requires at least two nameservers to be discoverable by a single-stack IPv6 resolver at every step of the chain of delegation.

6. Add IPv6 addresses for your incoming mail servers

In order to receive mail over IPv6, at least some of the mail servers listed in the MX records for your domain must have IPv6 addresses. You can find the mail servers for your domain using dig:

$ dig +short MX mythic-beasts.com
10 mx1.mythic-beasts.com.
10 mx2.mythic-beasts.com.

You can then check that these servers have IPv6 address by using dig to resolve an AAAA record, as before.

Once again, if you’re using a Mythic Beasts hosting account using the standard DNS template for your account type, this will be done for you.

In order to pass this test, at least one of the servers listed in your MX records must have an IPv6 address.

7. Add reverse DNS for your mail servers’ IPv6 address

It is generally advisable to have working reverse DNS for any addresses from which you send outgoing mail. In the case of IPv6, this becomes pretty much essential, as one of the biggest mail providers in the world, Google, will reject mail over IPv6 unless the sending server has working reverse DNS for its IPv6 address.

Unless you run your own mail servers, adding support for IPv6 will be down to your mail provider.

Unfortunately, there is no reliable way to obtain the outgoing mail servers that are used for a particular domain, so instead our health check makes a bold assumption that your outgoing servers are the same as the incoming servers listed in your MX records, and checks those. This assumption is certainly not true of all domains, which is why a failure of this test is only treated as a warning.

8. Check your SPF records

SPF (Sender Policy Framework) is a mechanism for publicly listing your outgoing mail servers, so that receivers can detect spoofed email sent from other servers. If you enable your outgoing mail servers to start sending mail over IPv6, and you have an existing SPF record, it is important that you make sure that it includes the IPv6 addresses for your mailservers.

There are various ways of doing this. If your incoming and outgoing mail servers are the same, then you can use the “mx” mechanism in your SPF record. This means that any hosts listed in the MX records for your domains will be regarded as a legitimate source of mail for your domain, and this will automatically include any IPv6 addresses (assuming you’ve done step 6).

If you list IPv4 addresses or address ranges in your SPF record explicitly, then you will need to add corresponding IPv6 addresses for those servers.

The rules applied by our health checker aren’t entirely trivial, as it’s not uncommon for legitimate third party servers to be included in a domain’s SPF record, and there’s no way of pairing up IPv6 addresses with their IPv4 counter parts. Our health checker applies some very broad rules: if you use the “mx” mechanism, then the checker requires at least one IPv6 address for a server listed in the relevant MX records. If there are any explicit “ip4″ addresses or ranges specified in the record, then the health checker expects to find at least one explicit “ip6″ mechanism.

If your domain does not list an SPF record then this test will pass automatically, as this effectively defaults to “accept from all”.

These rules aren’t watertight, but have proven to be quite effective in identifying mail sources that either haven’t been enabled by IPv6, or which have but haven’t been added to the SPF record.

Feedback

Hopefully this gives some insight into exactly what our domain tester is looking for, and how to comply with it. Customers who are having trouble getting 10/10 should email support. We’re also always very happy to hear any feedback on the health check, particularly suggestions for additional tests that we should be running.

New improved IPv6 site tester

September 5th, 2014 by

We’ve just given our IPv6 Health Check a significant overhaul.

IPv6 is coming, and there’s a lot more to being ready for it than adding an AAAA record and enabling IPv6 on your web server. Google are often cited as an example of an IPv6-enabled website, but the truth is, in an IPv6-only world, nobody would ever find Google because none of their nameservers have IPv6 addresses:

Screen Shot 2014-09-05 at 10.02.00

 

Our IPv6 Health Check aims to reach the parts that other testers don’t reach to find out if your domain is really ready for IPv6.  Do your mail servers have IPv6 addresses?  Do they have reverse DNS for those addresses?  Do your SPF records include IPv6 addresses?

The latest version of our health check includes an experimental IPv6 nameserver delegation and glue test that checks that all nameservers the delegation chain between the root DNS servers and your zone have IPv6 addresses, and sufficient glue to allow you to find them in an IPv6-only world. This test has already uncovered some interesting anomalies which we’ll dig into further in a future post.

Ice Bucket

September 1st, 2014 by

Thanks to Jonathan Wright who runs a very big website, for a nomination.

I’ve nominated Matt Smith, Rob McQueen and Neil McGovern.

Thanks to Ben Howe, our gap year student who’s adequately demonstrated to his colleagues the definition of a career limiting move by dunking a bucked of ice over his boss, The Haymakers for kindly providing the location for the company meeting, the chilled water and the ice, and the rest of my Mythic Beasts colleagues for filming and laughing.

Depending on nowhere by peering with everyone everywhere

August 29th, 2014 by

We’ve been adding some more peering sessions to improve our network redundancy. We already had direct peering with every significant UK ISP, we’ve now enhanced this so that one peering session terminates at one of the Telehouse sites, and the second terminates at one of the Telecity or Equinix sites. Each peering session is on a different London Internet Exchange (LINX) network which are physically separate from each other, and where possible we’ve preferred peering sessions that remain within a single building.

We have equal capacity on both networks at LINX, so unlike many ISPs with a single peering port or unequal capacity, in the event of a severe failure (e.g. a whole network or data centre) we just automatically migrate our traffic to our other peering link, rather than falling back to burst bandwidth with our transit providers. We feel that’s a risky strategy because failures are likely to be correlated, lots of ISPs will fall back to transit all at the same time in a badly planned and uncoordinated fashion which could cause a huge traffic spike upstream.

We light our own fibre ring around our core Docklands data centres, and have full transit and peering at both of our core POPs, with dual routers in each, and can offer full or partial transit at any of our data centres.

512k routes

August 13th, 2014 by

Some ISPs have started to report that their IPv4 routing tables now exceed 512k individual routes. At present we’re only seeing 502k routes but we’re nearly there.

Now for us this isn’t going to make any difference, our routers can all easily handle the routing table of this size, and the full IPv6 routing table of 30k routes all at the same time. However, it may start to affect things within other ISPs that we connect to. The likely things that we’ll see happening are,

  • Some ISPs will just drop some IPv4 routes, or cease processing updates, which means gradually odd bits of the internet will cease to work.
  • Some ISPs may fall into a software routing mode reducing in reduced performance.
  • Some ISPs will rely on filtering to reduce the routing table in size by aggregating routes together.
  • Some ISPs will alter their configuration for Cisco CAT6500 routers and disable IPv6 in order to increase the memory available in their routers for IPv4.

So watch out for oddities, and expect them to occur more and more frequently as the growth in the routing table gradually reveals which ISPs are running into trouble having not planned ahead.

Now accepting paypal

August 13th, 2014 by

Mythic Beasts have added paypal functionality to our billing system. You can now pay by credit or debit card, paypal, direct debit, BACS transfer or even cheque. Just don’t post us an envelope full of used fivers – save those for the sorts of services where you don’t get a VAT invoice.

HTTPS: the new default?

August 8th, 2014 by

Although SSL for websites (HTTPS) has been commonplace for e-commerce sites for years, the vast majority of “ordinary” websites still use standard HTTP. In recent months, two things have happened which look set to change that:

Whilst the importance of the second of these probably needs no further explanation, the relevance of the first may not be obvious.

Until now, one of the barriers to widespread adoption of SSL over HTTP is that, unlike non-SSL websites, each site requires its own IP address, and IP (or at least, IPv4) addresses are in short supply. This is because the HTTP request which specifies which website is being requested is only done after the SSL certificate has been presented, so if you have multiple sites on a single IP address, there is no way for the server to know which certificate to present.

A solution to this problem has existed for some years in the form of Server Name Indication (SNI). SNI is an extension to the SSL protocol, or more accurately its successor, the TLS protocol, which allows the site name to be included as part of the TLS negotiation so that the server can present the correct certificate.

Unfortunately, one widely-used platform had no support for SNI: Windows XP. With the ending of support for Windows XP, adopting SNI suddenly becomes a much more acceptable proposition.

Cheaper HTTPS hosting

The practical benefit of this is that hosting providers such as ourselves can offer much cheaper hosting of HTTPS sites, and that’s exactly what we’re doing. Buy one of our SSL Certificates and we’ll add an SNI-based HTTPS service to your Hosting Account at no extra charge.