The Great IPv6 Experiment

• IPv6 History & Primer
• IPv4 Exhaustion Report
• IPv6 Tunnel Brokers
  • SixXS
  • XS26
  • Hurricane Electric
  • Earthlink R&D
• Your.Org Corporate Site

---

This page is describing the IPv6 experiment itself, and is primarily intended for networking researchers and software professionals to learn about and discuss the experiment. If you're here for the free content, it's not here! We're not ready for the world to know about this experiment yet, so don't go submitting this to Slashdot or Digg until the actual site is up.

About the Project

The great chicken or the egg dilemma. IPv6 has had operating system and router support for years. But, content providers don't want to deploy it because there aren't enough potential viewers to make it worth the effort. There are concerns about compatibility and breaking IPv4 accessibility just by turning IPv6 on. ISPs don't want to provide IPv6 to end users until there is a killer app on IPv6 that will create demand for end users to actually want IPv6. There hasn't been any reason for end users to want IPv6 - nobody's dumb enough to put desirable content on IPv6 that isn't accessible on IPv4. Until now.

We're taking over 100 gigabytes of the most popular "adult entertainment" videos from one of the largest subscription websites on the internet, and giving away access to anyone who can connect to it via IPv6. No advertising, no subscriptions, no registration. If you access the site via IPv4, you get a primer on IPv6, instructions on how to set up IPv6 through your ISP, a list of ISPs that support IPv6 natively, and a discussion forum to share tips and troubleshooting. If you access the site via IPv6 you get instant access to "the goods".

For those uninterested in the "adult entertainment side of things, we're making available tons of completely "family friendly" videos from several TV shows and websites. The site is exactly the same whichever content you choose, but those choosing not to see the adult content go onto a completely different domain and won't see it mentioned at all.

What exactly is the experiment?

We're taking some highly valuable content (tens of thousands of people are paying $30/month to access these videos), and making it available for free on IPv6. If you don't have IPv6 connectivity and try to access the site, you'll be redirected to an IPv4 page with detailed instructions on how to get IPv6.

Why are you doing this?

To find out the current state of IPv6 from a content provider's and end user's perspective. Specifically, we're going to analyze:

• How many people accessing the IPv4 page eventually get into the IPv6 page.
• How many already had IPv6 to start with.
• What kind of download speed delta there is between people downloading the IPv4 sample video trailers to the actual content on IPv6.
• Latency, number of hops, and packet loss delta between IPv4 and IPv6.
• What method people use to gain IPv6 connectivity. Tunnel broker? Native connection? Teredo? 6to4? etc.
• Breakdowns by country, OS version, ISP, etc.
• MTU size
• How many people have an IPv6 stack configured on their OS, but IPv6 connectivity is broken somehow.

What's the catch? REALLY, why are you doing this?

There's no catch. We (Your.Org) set up a full IPv6 infrastructure only to realize it was mostly unusable due to certain problems. Publishing AAAA records on several very large sites broke connectivity to a non-zero percentage of viewers. A few cases were tracked down to end-users enabling IPv6 without actually having IPv6 connectivity at all, and the OS/browser not realizing it. Other problems included far worse IPv6 performance than IPv4, IPv6 routing instability, not-quite-production-ready IPv6 software and tools, lower than normal MTUs with ICMP 'fragmentation needed' messages getting blocked by firewalls, etc. I posted about my experiences on NANOG, and got a ton of replies. Many helpful, many insisting that my problems were either imaginary or mistaken. Since experimenting on revenue-generating production networks is usually a bad thing, the idea of an experiment like this adds a bit of safety to the equation.

The companies and persons involved have no vested stake in the outcome or success of IPv6 either way. I have no emotional or financial investment in IPv6 succeeding or failing. But, I do believe that IPv6 is going to happen whether we're welcoming or avoiding it. I'd rather bring problems to light now, before the IPv4 exhaustion crunch hits us. The networking community as a whole still has plenty of time to resolve and improve the situation before the mad dash in a few years.

There is no plan to monetize or profit from this experiment in any way. The end-user sites will have no banners, paid advertisements, subscriptions or anything of the sort. The only exception is a contractual requirement that the content we're giving away be watermarked with a copyright notice including the name of the site/company that they came from. We do not expect that to generate any income for them, but is there only to identify the copyright holder and prevent other sites from "stealing" their content and relabeling it as theirs.

Do you really think that end users asking for "adult content" will make ISPs deploy IPv6 for them?

No, that isn't the goal at all, and I don't find that a likely outcome, no matter how many people are interested in this experiment.

What *IS* the end goal of the experiment?

There are lots of contradictory opinions on the readiness of IPv6 for the masses. Some argue that it's ready now, and it's just up to the (content providers|ISPs|hardware vendors|operating system vendors|etc) to take the first step. Others say it's not ready yet, and there are problems that still need to be resolved before IPv6 is pushed to the public.

It's important to remember that many sites are using IPv6 right now, many fall strongly towards the "highly technically inclined" viewership - those who have IPv6 turned on know it, and are capable of debugging problems themselves. What would the impact be if Google began publishing IPv6 addresses for www.google.com? What would break, and what can a content provider do about it?

In late 2005 we attempted publishing IPv6 addresses for a handful of very high traffic "adult industry" websites. For the most part, this is going to attract a vastly different cross-section of the Internet than most currently common IPv6 sites. Our own experience suggests that there is a non-zero percentage of non-technically inclined end users who have unknowingly or mistakenly turned on IPv6 on their desktops without having any kind of IPv6 connectivity at all. Some browser/OS combinations recover gracefully from this after a small timeout. Others don't, which caused a non-trivial number of tech support emails to the site asking why it was down. A tiny number of requests did start coming in over IPv6, and many of those nerdy enough to realize what was happening also emailed us saying that download speeds dropped significantly. After discovering these problems, we had to remove the IPv6 records immediately, as this was a potentially revenue-impacting problem.

To sell content providers on the idea of supporting IPv6, there needs to be some level of proof that it won't cause problems. If it truly doesn't cause access problems for IPv4 end users who configured their systems incorrectly, and the performance delta for IPv6 users isn't so bad that it ruins their experience on the IPv6 sites, then there isn't much excuse not to turn up IPv6 on the content side. If we can pool the collective intelligence of the IPv6 community during this experiment to conclusively say that we've done everything possible on the content end to make things run smoothly, and there were still problems, it gives the IPv6 world ammunition to go to the NSPs, OS and browser vendors, and network hardware companies with a list of what's still wrong that's outside our control to fix.

Isn't a site with the names you've selected only going to attract those already familiar with IPv6?

We can reasonably expect this experiment to get some attention in more mainstream media than IPv6 discussion groups. Despite my pleas to the contrary, people have already submitted this URL to digg.com, stumbleupon.com, slashdot.org, etc. I've had contact from several mainstream journalists who were interested in the story. It's safe to say we'll have attention brought to this project outside the normal circle of IPv6-interested parties.

Why "adult entertainment" content?

It's probably the highest demand, large bandwidth, globally desired content out there. I could easily put up FreeBSD ISOs on IPv6, but that's catering to an already "technologically astute" audience. The target here is Joe Average who wants to get free access to something his neighbor is paying $30/month to reach. The question is, can the average end user obtain IPv6 access now if they really really want to? What problems to they have in doing so?

Some claim that "adult entertainment" films are why VHS won out over Betamax. Sony refused to allow such content on Beta, giving an advantage to VHS for some early adopters. While I don't think our little experiment is going to turn the tide towards IPv6, it will be useful to see how strong of a motivation it can be.

One of our clients runs a network of very large adult oriented sites, who has agreed to "donate" several of their most popular videos from one of their sites for this experiment.

What about kids getting access to it?

The steps we'll take to prevent it go beyond what 99% of the online adult entertainment sites use. We'll add ICRA tags to all the pages, indicating the presence of adult content. This will automatically trigger most in-browser and external content filters, marking it inappropriate for some audiences. We'll also add the newer RTA label in use by some content filters, as well as submit our sites to all major software package vendors, such as NetNanny, CyberSitter, etc.

Preliminary testing shows that the vast majority of content filters don't care about layer 3 at all, and work equally well with IPv6 as IPv4.

Who is running this experiment?

This is being conducted by Your.org, Inc. We do content distribution, consulting, etc.

Can the IPv6 backbone handle a rush of traffic like this?

We certainly hope so. If any networks experience problems as a result of this experiment, we'll take whatever steps necessary, including blocking access to specific ASNs or prefixes. Or, ending the experiment completely if we're causing disruptions that can't be quickly cured. We don't want to affect "real" ipv6 use because of an experiment such as this.

I don't want you breaking my network, what do I need to do to block this entirely?

We'd greatly prefer that nobody blocks our experiment entirely - real world usage will be valuable for everyone, possibly even you. However, if you want to "opt-out" of this entirely, you can prevent your users from accessing our IPs by blackholing the networks set up for the experiment. Details will be published shortly.

Can your end handle this much traffic?

Again, we certainly hope so. Internal testing shows FreeBSD 7-CURRENT with lighttpd can handle 500+Mbps of IPv6 traffic on a dual Xeon. The goal is to set up a couple of servers at each of our POPs and be prepared for any amount of traffic that comes our way.

Couldn't a ton of new people all downloading huge files crush existing tunnel brokers?

We definitely want to avoid this. I've been in touch with the operators of a few tunnel brokers so far, and they seem reasonably certain they can handle anything. We're happy to peer with any tunnel broker operator or IPv6 ready ISP wherever we can though, to avoid any expenses or bottlenecks.

How can I help?

Right now, the main needs we have are:

• Someone who could sell/rent/loan us a couple Adaptive Services PICs for a pair of Juniper M series routers, so we can get better netflow statistics.
• All willing ISPs who do anything with IPv6 to peer with us. We're present in Equinix/Chicago, SARA in Amsterdam(AMS-IX connected), and Equinix/Tokyo. We'll peer with IPv4 as well.
• We wouldn't say no to a donation of IPv6 transit either, even if it's just a partial feed, just for the duration of the experiment. :)

When we're ready, we can really use some help in publicizing the experiment. Submitting it to your favorite news sites, whatever. But, don't do this yet! The content isn't up, the servers aren't ready, etc. This is being published now for the networking community to be aware of the experiment before it begins, but it's not ready for general consumption yet.

How can we peer with you?

See our peering page for more info. If possible, we'd like to peer over IPv4 as well as IPv6 for this experiment, since we're trying to actively measure the difference in performance between the two. The results will be more accurate if our peerings are equal on both protocols.

What are you going to do with the results?

At the conclusion of the experiment, a paper will be published listing all the results, perhaps coinciding with a presentation at a conference such as NANOG or ARIN. Completely anonymized logs may be made available to researchers on request.

I have a suggestion for data I want you to log!

Please tell us!

When will all this start?

We anticipate beginning this experiment in the February-March timeframe.

How will you capture the data?

The current measurement methods planned are:

• The IPv6 site will have an A record pointing to an IPv4 site with instructions and details about the project, to catch non-IPv6 users.
• All requests to either site will be logged on the web server, including the remote IP address, msec to download the file, browser and OS version, etc.
• A unique cookie will be sent to the browser on their first access to either site. This allows us to track the same user on both IPv4 and IPv6.
• The IPv4 site will have some sample content (about 10MB worth) that we can use as a baseline to determine average IPv4 download speeds.
• The IPv4 site will also have a small <img src> tag at the bottom. This image will be located on a hostname with both IPv4 and IPv6 DNS records. If a client loads the page, loads other images on the page, but is unable to load THAT image on the page, we can assume they have IPv6 configured but it's broken in some way.
• Netflow running on the routers, to capture as much information as netflow can see. (ASNs, etc)
• Periodic random packet dumps and OS socket tables, looking at MTUs, window sizes, packet loss rate, etc.

What data will you produce from those methods?

• Number of users who have IPv6 configured but are unable to load IPv6 content. How many people can't reach your site if you turn IPv6 on?
• Average IPv4 download speed compared to IPv6 download speed. The IPv6 infrastructure isn't as well connected as IPv4 - how much of an end-user visible impact does that make?
• If IPv6 is slower than IPv4, why? Latency? Packet loss? Substantially greater number of hops? Poor routing? We'll be building a user by user map of their IPv4 and IPv6 download speeds to compare individually.
• What method do people choose to get on IPv6 if they really want to? What problems do people have setting up a workstation to connect using Teredo or a tunnel broker? How many end users have native connectivity?
• What kind of responses are end-users getting if they request IPv6 from their providers? There will be a forum for participants in this experiment to discuss things, as well as a feedback form.
• While the plan is to deploy the videos as simple HTTP downloads, how many media players and browser plugins break when trying to play a stream over HTTP in IPv6?

Recent changes

5 January 2008Progress!

You now may sign up for a mailing list for more information or discussion about this experiment. Click here to sign up or peruse the archives.

Also added a bit above about opt-out instructions for networks.

18 December 2007Progress!

Thanks to a very generous offer of help, we've been given enough bandwidth to proceed with the experiment. Once we get some crossconnects and network upgrades completed, we'll move forward with the first phase!

4 September 2007Moving forward!

We're still alive! The project is still moving forward as planned, but we're a bit behind schedule. The community has been very generous in offering donations of bandwidth and equipment, but things are progressing in that area a little more slowly than we originally anticipated.

Since we've gotten way more media attention than was expected, we're planning for huge amounts of traffic. We strongly believe that if we don't at least have enough capacity to our upstreams ourselves, it will call into question the accuracy of any results we produce. So, we're waiting until we've got a few more bandwidth issues resolved. If you're able to help us with v4 or v6 bandwidth in Equinix/Chicago or Amsterdam, please let us know.

10 April 2007Clarifications

Added additional clarifications about the goals and methodologies of this experiment. Added notes to confirm that existing "content filtering" software will effectively block access to this experiment to those who don't want to see it.

8 April 2007Experiment site launched

First details of the experiment published.