IPv6: Name and Shame

After a discussion on IRC tonight, it didn’t take long to come up with a short list of key technologically-involved companies who have a complete or near complete lack of public IPv6 deployment progress.  This list is mostly focused on content providers, or those who others rely on to provide their content (short of ISPs):

  • Canonical (Ubuntu’s corporate overlords):no IPv6 peers up, much less prefixes announced.  As such, no IPv6 available for the default repositories, sites, wikis, bug trackers and many other systems involved in supporting Ubuntu.  Here’s a bug report against Ubuntu and a RT ticket aimed at the infrastructure folk.  Neither has received a proper reply in years.
  • Sourceforge: (and github et al) While a couple download mirrors have IPv6 enabled, the site itself has nothing – despite hosting thousands of forward-looking FOSS projects of all sizes.  A fair share of those are tools specifically for IPv6 undoubtedly, but hosted somewhere off the IPv6 Internet
  • Slashdot: (and others controlled by its corporate overlords) Similar story here – Slashdot has had more than its fair share of IPv6-related stories over the years, and stories of IPv4 runout.  However Slashdot does not have any visible IPv6-related deployment progress despite years of people asking in comments to IPv6-related stories “Why doesn’t /. have IPv6 yet?”
  • kernel.org: along with the OSU OSL, not sure what else needs to be said here.  They’re probably one of the few major, major software mirrors out there who don’t support IPv6
  • Microsoft: Be it microsoft.com (which had IPv6 for W6L but removed it shortly thereafter), Microsoft Azure, or the lack of IPv6 on Windows-supporting infrastructure such as Windows Update content servers; MS has a surprisingly small amount of IPv6 success on their own infrastructure, despite an operating system which supports IPv6 extremely well.  Particularly MS Windows Azure, their cloud platform, it is amazing that such a massive cloud infrastructure could be deployed without IPv6 from day one
  • Amazon: Same goes for Amazon AWS – no public information, roadmap, or statements about the intent to really make progress on deploying IPv6.  Additionally, I’m sure a nontrivial number of compute instances used by organizations such as IPv6-connected colleges could be run IPv6-only and help conserve IPv4 number resources that I’m sure Amazon is chewing through (both public addresses and RFC1918 space which I suspect they are using many times over).  Yes, the ELB offers limited IPv6 support for services which can be published through it, but that was unveiled quite some time ago with zero progress (that I’m aware of – feel free to correct me) since then.  This includes newer services like Route 53 and Glacier which have come out since Amazon had support on the ELB (indicating that someone is working on pushing an IPv6 agenda).  These brand new services launched without IPv6.  Lastly, there are companies such as ours whose entire usage of AWS would be over IPv6 (largely off-site backups and some monitoring) if it was available for services such as S3.
  • CPanel:  A bit different than the others listed – but CPanel is a building foundation (love it or hate it) for probably millions of sites worldwide.  The hosts of these sites cannot offer IPv6 support to their customers until CPanel itself offers IPv6 support.  Think about what the IPv6-ready content world would look like if CPanel wasn’t such a blocking factor for so many hosts.
  • Tumblr, WordPress.com, et al: These companies host an impressively large number of sites for  individuals of all popularities and organizations of all shapes and sizes.  Though they serve a different purpose than CPanel, they have a similarity in that they advertise simplicity of management, ease of publishing, and are a blocking factor for thousands upon thousands of sites.  (Disclaimer: WordPress’s software works flawlessly with IPv6 when hosted elsewhere; hat tip: blogspot is IPv6-enabled since Google enabled it for W6L)
  • Network Solutions: Network Solutions has been behind the ball on v6 for years.  In their DNS management interface you cannot view or edit any AAAA records for hosted zones, let alone try and register DNS glue or inquire about IPv6 for any of their other solutions.  These requests take an e-mail to their support address and can take days or weeks to see progress on.  It probably goes without saying that none of their infrastructure or services (such as their DNS servers) have IPv6 deployed (contrast with GoDaddy who seems to be making decent progress).  A quick search of the NANOG archives turns up some other anecdotal evidence.  Also see that they have no IPv6 prefixes announced or peers up in BGP.
  • Sendgrid et al: Sendgrid has confirmed for me that they have no IPv6 deployment and do not plan on deploying it until “more receivers move to IPv6″.  This includes a lack of access to their APIs, as they too have no IPv6 peering up and no IPv6 prefixes from ARIN (based on a whois search).  This is significant because companies such as Sendgrid strive to be a replacement for conventional datacenter infrastructure but fail to be the lowest common denominator between what customers may want or need.  That is, if someone has an IPv6-enabled mail infrastructure in the datacenter, moving to a hosted 3rd party solution like this requires a step backward.
More generally:
  • Many Others who publish articles/stories (not as much for syndicated stories, but original content) regarding the urgency for IPv6 deployment yet run their own sites without IPv6 accessibility.  They should be practicing what they preach and getting IPv6 deployed on their infrastructure.  In the case that they are hosted/managed by a company which does not support IPv6, they should vote with their wallets and lead by example for others.
  • Game Developers: I wrote a devlog post for King Arthur’s Gold earlier this year regarding IPv6.  The associated forum post has some additional follow-ups with more of my thoughts on the matter. Games stand to feel major hurt as CGN and IPv4 instability/unreliability increase and general quality decreases over the coming years.  In addition to problems with the general IP-literal-centric nature of gaming, I also discuss some issues with the commonplace behavior of IP address bans in many games, and how with CGN this is going to have unexpected consequences.

 

Have suggestions for others?  Leave them in the comments and they’ll gladly be added to the list above.

Have some free time?  Call/write/tweet the companies above early and often, asking for IPv6.  These companies (and plenty of others) need to realize that it is in demand explicitly from many customers, countless more who are passively and quietly waiting, and many many more who implicitly would benefit from its deployment.  Call your ISPs, e-mail support for your hosts, tweet Sprint, AT&T et al today!

These companies are ultimately no different from countless others who do not have IPv6 deployed and no public plans to – except these companies for the most part provide critical services and resources relied upon by countless individuals and organizations worldwide.  As such, you would think they should be amongst the first to have an extremely mature IPv6 deployment and be leading the pack.

 

As a follow-up to some comments on Reddit, I am adding some extra explanation here as to why it is time to name, shame, and generally pressure companies who are behind on IPv6 deployment.  Carrier Grade Nat (CGN) is increasing in popularity amongst mobile providers, ISPs of various sizes (though not some of the major ISPs in the US yet) and campus networks such as universities.  The more traffic remains on or is lit up on IPv4, the more traffic must flow through these bottlenecks.  Forcing traffic through a stateful chokepoint like this is inherently hard to scale properly.  The same goes for operation of an IPv6-only network like T-Mobile’s which uses NAT64 for reaching v4-only content.  NAT64 and CGN are transition mechanisms which need to see a decrease in IPv4 traffic over time in order to help bridge the gap.  Network operators have incentive to push content providers to get v6 deployed, as native IPv6 will forever be cheaper to maintain than operating/scaling CGN devices as the IPv4 content flow through them continues to increase.  We saw this a few months ago with one university’s open letter to Pandora asking them to please get IPv6 deployed, presumably since that is a non-trivial amount of IPv4 traffic flowing through their NAT boxes – additionally it maintains at least one long-running HTTP connection/state to Pandora’s content servers.  Both of these are expensive resources to offer to students when operating a mandatory, stateful chokepoint.

A number of people ask “why should content be made available on IPv6 when there are not really any users on IPv6-only yet?”.  The answer is that when content is available through CGN and NAT64 instead of native IPv6, this bottleneck inherently introduces an extra point of potential failure, congestion and complication.  Additionally traffic may need to go out of its way to reach the CGN outbound and more importantly inbound through a network to reach the user, adding path latency that would not be there if the traffic was able to utilize the networks’ native peering on IPv6 (or IPv4 in the case of a user who did not sit behind the NAT).  Users share IPv4 addresses in these scenarios, which will cause significant problems with bans from sites prone to comment spam (one zombie computer behind a CGN and suddenly the IPV4 /32 or /24 of NAT pool space is suddenly on a RBL and blocked), sites like mega/whatever-upload.com which have daily IPv4 “free download limits”, game bans, IRC/proxy/etc bans, and so on.  These problems are real and are being seen by users behind CGN today.

One last example is systems like the Playstation 3, which does not work properly behind NAT which alters the source port of UDP flows.  By definition of CGN you have many, many users behind a certain IP address.  There is no way you can ensure that each PS3 owner’s traffic will not have its source port changed in translation.  While they may be able to share a source port as long as they are connecting to different game hosts/servers, you and your neighbor may no longer be able to call each other up and join the same game.  This problem is two-fold, as the Playstation 3 has no IPv6 support (not since its launch nor in any subsequent updates).  This means that game developers on the platform haven’t even had the opportunity to start approaching the problem which is late/rushed IPv6 adoption.

The fact is that offering native IPv6 alongside CGN/NAT64 (granted native IPv6 is implied in the latter case) avoids *all* of these problems, along with others involved in IPv4 address depletion and IPv4 routing instability and unreachability as the IPv4 full table continues to grow.  These problems (shared IPv4 address reputation and accounting [in the case of free services], IPv4 quality issues, application breakage through NAT) all boil down to user experience issues.  These users are users of the content/site as well as the ISP, so this arguably puts twice the pressure on the content providers.  Users will complain to their ISP when IPv4-only content (regardless of whether the ISP offers IPv6 or not) is broken, bans them, etc.  The users will also turn to the content provider to voice their displeasure.  Then the ISP will also directly or indirectly have incentive to add pressure on the content providers.

Lastly, and this one is kind of a common sense answer to “why should pressure be exerted on those who have not deployed IPv6 on content yet? It’s not like there are any appreciable IPv6-only userbases?” is this: IPv6-only userbases  *can not* exist until sufficient content is on IPv6.  It’s as simple as that.  There used to be a chicken-and-egg issue where there wasn’t enough demand for IPv6 from content providers, and not enough IPv6-accessible content so that eyeball networks didn’t have demand/reason as well.  This is no longer the case, as 5 of the Alexa top 10 sites and countless others have enabled IPv6 in the past year.  Networks with CGN or soon to have CGN as life support for their IPv4 userbase have all of the reasons above to push for content on IPv6.  This puts the vast majority of responsibility for IPv6 deployment on content providers because they are now the blocking factor.

A few footnotes:

  • This post is almost exclusively talking about the reasoning and pressure for IPv6 deployment on content/content providers.  This post intentionally overlooks/ignores reasoning for deployment on user/eyeball networks though does allude to this not being a “blocking” matter any more.
  • There are plenty of companies who have been pushing IPv6 on the content side in addition to deploying IPv6 themselves.  In no particular order: Google, Hurricane Electric, Netflix, Facebook, Yahoo!, and plenty of others of all sizes.

6 comments

  1. Biggest blocking point to IPv6 adoption on the infrastructure side: corporate mergers. AT&T + the baby bells: probably set ADSL2 deployment back several years; still behind on IPv6. Paetec-Windstream: should’ve been able to add IPv6 a year before they merged. I’m sure there have been others.

  2. Guest says:

    >Game Developers
    WoW has a very complex infrastructure but they have native IPv6 for the realm servers and it works flawless, showing that it can be done. But it’s also very rare.

  3. Paul says:

    kernel.org used to be dual-stack, but they turned off IPv6 after the security breach in 2011.

  4. Will Dean says:

    I’d add Spotify to that list. Not only do they not have IPv6; they use IPv4 literals (breaking NAT64).

  5. Ejes says:

    Probably the list of well supported ipv6 sites would be shorter.

  6. Mike Crow says:

    Hey blog admin check this out! http://is.gd/o8hSvE

Leave a Reply

Your email address will not be published. Required fields are marked *