A Peering Policy Survey

In 2009, William Norton, a.k.a. Dr. Peering, conducted a survey of 28 peering policies. A peering policy is a statement by one network of "inclination" to enter into a peering arrangement with other networks. Norton is one of many evangelists of Internet peering. His objective was to identify clauses that were commonly found in peering policies that could be used as models for prospective peers.

Traditionally, a settlement free peering arrangement was entered into between "peers," in other words, equal networks. Peering policies articulated the benchmarks used for determining if networks were "peers," that "peers" were roughly equal in size and exchanged roughly balanced traffic and therefore the increased value from interconnection would be mutually beneficial. In the nascent commercial Internet ecosystem of the 1990s, there were Tier 1 backbone networks, Tier 2 regional networks, Tier 3 access networks, end users, and edge providers. The network was hierarchical. Each participant in the ecosystem had a relatively discreet business plan. Thus "peers" were (somewhat) easy to identify.

The Internet ecosystem evolved tremendously in the last three decades. It is much more difficult to identify a "peer." The network is no longer hierarchical. Providers' business plans have expanded, offering access, regional, and backbone service. A provider may be a mix of hosting, cloud, CDN, and network services. Large access networks no longer are reliant upon third party backbone service and no longer pay for transit service; they charge content networks to access the access network's subscribers. Content networks are also no longer as reliant on third party backbone services, instead using CDN's to move content as close to end-users as possible. Instead of content moving up the Internet hierarchy until it reaches Tier 1 and an interconnection point, it is moved to the gateway of the access network where the CDN attempts to negotiate interconnection with the access network. This evolution has changed the inclination of networks to enter into settlement free peering arrangements.

This paper seeks to resurvey peering policies in light of the evolution of the Internet ecosystem. What has this evolution meant for the inclination of network providers to peer? Is this evolution reflected in peering policies? Is there a difference in peering policies between backbone service providers, large access providers, and content networks? Are there clauses that remain common across all policies? Can there be "peering" if there are no longer "peers"?

Methodology

The scope of the survey is peering policies for networks providing service in the domestic United States market. Peering policies were found using Google, reviewing CAIDA and DYN's list of top transit providers, and reviewing PeeringDB.

Peering policies were broken down into common clauses. The breakdown of those clauses sometimes did and sometimes did not track the breakdown used by the providers.

Providers could have multiple peering policies.  Some providers have distinct business units that have distinct inclinations.  For example, Verizon's CDN Edgecast has a different peering policy than Verizon broadband Internet.  Other companies have multiple policies due to mergers and acquisitions.  For example, Charter Communications acquired Time Warner Cable; both networks continue to have posted peering policies.  Verizon also has distinct posted peering policies for its acquired assets AOL, Yahoo!, and XO. Legacy peering policies of acquired networks were reviewed if available online even though it cannot be determined whether they remain in effect.

In total, 60 networks were surveyed. While the sample size of networks surveyed is relatively small, this sample represents the top backbone providers, BIAS providers, CDNs, and edge providers, representing a strong majority of network traffic. This methodology, however, biased the results towards the behavior of the largest providers versus small providers; markets have a long tail of hundreds of small providers who may produce different results.[1]

Providers were classified according to primary Internet business: edge provider, broadband Internet access service, content delivery network (both private and commercial grouped together), educational, and infrastructure (i.e., ARIN, ICANN, ISC). This proved difficult.  One of the premises of this survey is that providers no longer fit within discrete business plans where peers are easily discerned.  Providers have diversified their business plans, with incentives to move towards more profitable services.  Therefore, providers were classified according to their dominant characteristic at the point of interconnection.  For instance, Netflix is a video delivery service.  At the point of interconnection, Netflix could be appropriately characterized as a private CDN.  Many of the edge providers and content providers have constructed their own private CDNs and at the point of interconnection are attempting to deliver data to end users. It is a subjective classification but probably sufficient to observe market dynamics. 

Findings

Peering Policies

Sixty five percent of surveyed networks had posted peering policies. One hundred percent had PeeringDB entries (in other words, all networks without posted peering policies nonetheless had PeeringDB entries). [2] Two networks had no peering policy or PeeringDB peering information (they had PeeringDB entries but the peering fields were blank). It appears that for many networks PeeringDB has taken on the role of peering policies as a means of expressing peering inclination and establishing arrangements.

Fifty eight percent of backbone providers have peering policies compared to 79% of BIAS providers, 66% of CDNs, and 83% of edge providers. Sorting the survey for the largest backbone transit providers as identified on Dyn's Baker's Dozen 2016 produced a consistent result of 55% of backbone providers with posted peering policies. These numbers are a little muddled because a number of the BIAS providers are also backbone networks and reportedly charge paid peering, such as AT&T, Centurylink, Time Warner Cable (Charter), Comcast and Verizon; these networks were classified as BIAS providers and all of them had published peering policies.

These findings reflect that all participants in the Internet network ecosystem, not just backbones, are actively seeking peering arrangements. Curiously, backbone networks had the lowest percentage of posted peering policies and yet backbone interconnection conflicts were the original impetus for the calls for published peering policies. Then again, as peering policies are statements of inclination to peer, and backbone networks are inclined to sell transit, not peer, the lack of posted peering policy could be consistent with their lower interest in peering. [3] A backbone network does not enter an IXP with the anticipation of peering with the other firms at the IXP; the backbone enters the IXP to sell transit.
Peering Inclination

Generally, networks inclination to peer is divided into four: open, selective, restrictive, and no peering. "Open" is the most liberal inclination where a network is generally willing to peer with anyone who can meet basic requirements (such as meeting the network at an interconnection point). "Selective" is the similar to open, but the selective networks is looking for some basic requirements before they enter into such relationships (for example, minimum traffic utilization). "Restrictive" is the more traditional inclination where a provider is only willing to entering into peering with "peers," networks that meet a relatively high bar of criteria. These inclinations can be regionally specific. A network may have an open policy in one geographic regional and a selective policy in another.

According to PeeringDB,[4] 35% of networks surveyed had open peering policies, 43% had selective, and 18% has restrictive.[5] Combining "open" and "selective," 78% of networks are interested and inclined to enter into a peering arrangement. This is consistent with what one might anticipate that peering is a beneficial arrangement, but the largest group of networks has at least some criteria for when the expense and logistics justifies the arrangement.
Having a posted peering policy is associated with being more inclined to peer. The survey reveals that those with posted peering policies are 18 percentage points more inclined to peer that those without.
If we break the networks out by business plan, we see differences in inclinations. Sixty seven percent of backbone networks had restrictive peering policies (as noted above, backbones also had the lowest percentage of posted peering policies and have a business plan of selling transit, not peering).[6] The only other networks with restrictive peering policies were large BIAS providers who reportedly charge paid peering. No CDNs or edge providers had restrictive peering policies.

CDN's surveyed had 50% open policies and 50% selective policies. CDNs are in the business of delivering data; they get paid by edge providers for that service. Interconnecting with backbone and access networks facilitates the delivery of traffic and thus furthers their business model.

BIAS providers had 14% open policies, 64% selective, and 14% restrictive. BIAS providers are a mixed bag. BIAS providers that do not have significant backbones and are reliant on transit service to bring the full Internet to their customers have an incentive to avoid transit costs by entering into peering arrangements. Historically, access providers would have welcomed offers of settlement free peering as a means of defraying transit costs. However, large BIAS providers that charge access paid peering to CDNs have no incentive to agree to settlement free peering, increasing the settlement free capacity into their networks that edge providers could use instead of paid peering routes. AT&T, Centurylink, and Comcast have selective peering policies where Time Warner Cable (now Charter) and Verizon have restrictive peering policies.

A provider can have different peering policies based on the service it is providing. Verizon's general peering policy is restrictive whereas Verizon CDN Edgecast's peering policy is open.

Not surprising, educational and infrastructure networks generally have open peering policies. They are not in the business of selling network services; the goal is connectivity.

Words Per Policy

On average peering policies contained 679 words and 8 provisions. As would be expected, the open policies were shorter than the restrictive policies. Open policies had 451 words on average and 5 provisions; selective policies had 665 words and 9 provisions; and restrictive policies had 1183 words and 11 provisions. The shortest peering policy was Limelight's selective policy with 87 words. The longest policy was Time Warner Cable's (Charter) restrictive policy with 1701 words. The third longest peering policy was Charter's at 1404 words, even though it is a selective policy.

Green = Open. Yellow = Selective. Red = Restrictive.
Common Clauses

The most common peering policy clause was the requirement for a 24/7 Network Operations Center. Eighty two percent of surveyed peering policies required this. It appears to be the highest priority in interconnection relationship that partners are able to promptly respond to and resolve networking issues. The next highest clause was the "no abuse" provision (no gateway of last resort, no default routes, no transit traffic in a peering relationship, no static routes),[7] which, likewise, is an assurance that the partner will behave in terms of routing.

Clause	All Policies	BIAS	Backbone	CDN	All Policies Norton 2010	Percent Change 2017 versus Norton
24/7 NOC	82%	100%	86%	63%	89%	-7%
No Abuse	74%	91%	43%	63%	64%	+10%
Minimum Required interconnection points	67%	91%	71%	63%	46%	+21%
Consistent Routing Announcements	64%	82%	86%	50%	75%	-11%
Minimum Traffic	54%	91%	57%	38%	71%	-17%
Geographically Diverse Network	41%	73%	43%	38%	46%	-5%
Filter Routes	36%	73%	14%	0%	29%	+7%
Customers are not eligible to become peers	36%	45%	43%	0%	71%	-35%
Potato Routing	33%	45%	29%	38%	29%	+4%
PeeringDB	31%	9%	0%	63%	7%	+24%
Traffic Ratio	28%	64%	57%	0%	32%	-4%

Minimum Interconnection Points

Twenty-six policies had minimum required points of interconnection. Three of the highest minimums were for peering on a global basis and included international POPs. The average for the rest was a minimum of three points of interconnection. Some networks were willing to peer even if the networks meet at only one point of interconnection. Of the domestic networks, Charter had the highest minimum required points of interconnection with nine for content providers.

There were a number of networks that required that a potential partner interconnect at "all" points of interconnection where both networks are present.  These networks may be willing to interconnect with only a few POPs, but they want redundancy and Cold Potato delivery and thus want to interconnect anywhere where they both co-exist.  While one might expect this to be a clause of CDNs, it was found in one backbone's policy and three edge providers' policies.


Traffic Ratios

Twenty eight percent of peering policies contained balanced traffic ratio requirements.  These clauses were found in backbone and BIAS network policies.  All of the BIAS providers that are known to charge paid peering had balanced traffic ratios as a condition for interconnection. No CDN, edge provider, infrastructure network, or educational network included a traffic ratio clause.  The average requirement was 2:1 and the range was from 1.5:1 to 3:1.  Three networks simply indicated that a balance traffic ratio was required without specifying a number.
Interconnection Contract Required

In 2012, the OECD published a report that found that 99.5% of peering arrangements were without written contracts. This finding was on a global basis and was based on a per network measurement, not based on the amount of traffic carried by networks (in other words, the hypothesis is that larger networks, networks that carry more traffic, are more likely to require contracts, and represent a larger share of exchanged traffic).

According to PeeringDB, 26% of networks require contracts. None of the networks with open peering policies required contracts; 23% of selective networks require contracts; and 82% of restrictive networks require contracts. Consistently, none of the CDNs or edge providers require contracts where 75% of backbones require contracts. Of the top transit providers listed in Dyn's Baker's Dozen for 2016, every provider except Hurricane Electric requires a contract. Four of the BIAS providers known to charge access paid peering require a contract (TWC, acquired by Charter, indicates that a contract is not necessary).

Hot or Cold Potato Routing

Hot Potato routing is a traditional arrangement where traffic from the sending network is handed off to the receiving network at the closest interconnection point to the origin of the traffic. Hot Potato routing was common between peering Tier 1 backbones.

Cold Potato is the opposite, where the originating network carries the traffic as far as possible and hands it off as close to the receiving end user as possible. The sending network is responsible for long haul transport while the receiving network is responsible for local access transport. CDNs delivering traffic to access networks employ Cold Potato routing.

Thirty three percent of peering policies surveyed contained a Potato clause. Sixty nine percent called for Hot Potato routing; 31% called for Cold Potato routing. The Potato clauses did not fall out as one might expect; one might expect that BIAS providers want CDNs and backbones to carry heavy traffic as far as possible and deliver it as close to end users as possible in order to avoid network costs and improve end user experience. Yet sixty percent of BIAS providers that contained this clause called for Hot Potato routing; this reflects in part BIAS providers that own their own backbone networks. Charter, which owns the TWC backbone, called for Cold Potato routing in both of its peering policies. In the CDN group, Highwinds called for Hot Potato routing; the other two CDNs that had a Potato clause called for Cold Potato routing; most CDNs did not have a Potato clause (CDNs are in the business of quality delivery of content; one might expect them to have an incentive of avoiding congested backbones and improving performance by locating CDN servers as close to end users as possible). Only the backbone providers fell out as expected, with all networks that had this clause calling for Hot Potato routing.

Congestion

The recent interconnection disputes have left their mark on peering policies. Historically, peering disputes were between backbones (generally as backbones grew and tried to join the Tier 1 club) and lasted a few days (the value of network effect and providing service to customers exceeded the value of the dispute, creating an incentive for rapid resolution and getting back to business before customers switch to a backbone service that isn’t affected). Recent interconnection disputes have been between content networks and access networks and have been protracted.

Several peering policies indicate that partners must agree to avoid congestion (Fastly: "Maintain congestion free interconnection"; Yahoo!: "Maintain congestion free interconnection"; Blizzard: "maintain sufficient capacity to exchange traffic without congestion"; Microsoft: "sufficient capacity to exchange traffic without congestion", Frontier: "Peers must agree to resolve congestion issues"). Other networks include in the policy that capacity will be augmented at a specific benchmark. An industry norm is that capacity is augmented when utilization reaches 70%. Google and Microsoft include a clause that capacity will be augmented when utilization reaches 50%. Finally, Comcast includes a clause that partners will participate in capacity planning "and work towards timely augments as identified." Twenty three percent of networks surveyed that had peering policies included some clause to address congestion.

Divergence from Norton 2010 Survey

PeeringDB

A requirement that a potential interconnection partner use PeeringDB is the most significant increase from the Norton 2010 survey. Norton found that only 7% of policies required the use of PeeringDB. Norton observed "nLayer does not even suggest using it - surprising, since Richard Steenbergen from nLayer leads the peeringDB project." This survey found that 31% of peering policies require the use of PeeringDB (and 100% of surveyed networks have PeeringDB entries). The type of network requiring a PeeringDB entry showed interesting variation. Only 1 BIAS provider included this clause and no backbone providers. Conversely, 63% of CDNs and 60% of edge providers included this clause.

No Customers

The clause that had the most significant decrease and largest divergence from the Norton Survey was "no transit customers may become peers." Norton found that 71% of peering policies contained this clause while this survey found only 36%. It seems like this clause would only be of concern to those networks in the business of selling transit. This would be less of an issue for those providers with incentives to avoid transit costs or improve connectivity.

Observations: Peering Policies Dont Measure "Peers" Anymore

A "peering policy" is no longer a measuring stick by which "peers" are determined. By definition, "peering" is an interconnection arrangement between partners who exchange traffic from their own networks. Historically, "peering" was an arrangement between "peers" (equals). A "peer" was a network of roughly equal size that exchanged roughly balanced traffic. A "peering policy" articulated how a "peer" would be measured and determined.

There are no more peers.[8] Providers have diversified their business plans vertically and horizontally, and have merged and consolidated businesses. At the time of the Commercial Internet eXchange, Verizon, Comcast, AT&T, Charter (Time Warner Cable), and Centurylink did not have Internet backbones; now they are the top Internet network providers. Almost none of the original members of the Commercial Internet eXchange exist in their 1992 form. The big question of peering has evolved from an interconnection arrangement in the core between backbones to an interconnection arrangement on the edge between access networks and CDNs.

Since there are no longer "peers," a "peering policy" is no longer a mechanism for measuring "peers." Since there are not networks of "roughly equal size," the first of two necessary benchmarks, then the second benchmark "balanced traffic ratio" likewise no longer helps to measures equals. Instead, what was once a benchmark has become an interconnection condition. "Balanced traffic ratio" is now a condition that limits the amount of traffic that can be exchanged (in order for traffic exchange to be balanced, one network can send the other network no more traffic than it receives).

Peering continues to be an attractive interconnection arrangement. Smaller access networks that are reliant on transit have an incentive to avoid transit costs by entering into peering arrangements with each other, content providers, and edge providers. Content and edge providers have an incentive to directly interconnect with access networks, bypassing congested third-party transit providers. Large networks have an incentive to peer with other large networks, avoiding transit costs and providing quality routes for traffic. Direct interconnection through peering can result in efficiencies, quality delivery of traffic, and better management of traffic flows.

Peering policies reflect the evolution of the Internet ecosystem. Formerly a statement of peering inclination by backbone providers, now peering policies are posted by access networks, CDNs, and edge providers. The more peering furthers a business plan, the more inclined a network will be to enter into a peering arrangement. An access provider indicates that it will enter into a peering arrangement but only on the condition of cold potato routing and that traffic is delivered to specified interconnection points; a CDN indicates that, in order to further its business plan of high quality traffic delivery, it is willing to carry traffic as close end users as possible and openly enter into interconnection arrangements. Peering Policies have become statements of how providers will establish interconnection arrangements with non-equal providers.



Endnotes

[1] Contrast Aemen Lodhi, Natalie Larson, Amogh Dhamdhere, Constantine Dovrolis, kc claffy, Using PeeringDB to Understand the Peering Ecosystem, ACM SIGCOMM Computer Communication Review, 44(2), 20-27 (2014) (reviewing peering characteristics based on the full PeeringDB database.  When Lodhi et al.'s results were sorted for the largest providers, their results were consistent with the findings of this survey.).

[2] Compare Lodhi at 22 ("We find that 93% of the top-100, 80% of the top-200 and 74% of the top-300 ASes from [CAIDA's] AS-rank were present in the [August 2013 PeeringDB] dataset, including all known Tier-1 and major Tier-2 ASes").

[3] See Jayne Miller, IP Transit vs. Peering: How a Network of Networks is Built, Telegeography Blog (Sept. 20, 2016) ("Large networks that are the major providers of IP transit have no incentive to peer with potential customers.").

[4] I did not contrast self reported peering inclinations in PeeringDB with inclinations articulated in posted peering policies in order to confirm consistency.  However, the Using PeeringDB study did. Aemen Lodhi, Natalie Larson, Amogh Dhamdhere, Constantine Dovrolis, kc claffy, Using PeeringDB to Understand the Peering Ecosystem, ACM SIGCOMM Computer Communication Review, 44(2), 20-27, 23 (2014), http://www.sigcomm.org/sites/default/files/ccr/papers/2014/April/0000000-0000002.pdf ("we obtained the peering policy URLs of 50 networks in PeeringDB, and compared the policy seen on their URL with the policy mentioned in the PeeringDB record. In each case, the peering policy listed on PeeringDB (Open, Selective or Restrictive) matched the peering policy at that network’s policy URL.").

[5] Compare Aemen Lodhi, Natalie Larson, Amogh Dhamdhere, Constantine Dovrolis, kc claffy, Using PeeringDB to Understand the Peering Ecosystem, ACM SIGCOMM Computer Communication Review, 44(2), 20-27, 23 (2014), http://www.sigcomm.org/sites/default/files/ccr/papers/2014/April/0000000-0000002.pdf ("Of the 3392 ASes in the [August 2013 PeeringDB] dataset, 76% use Open peering, 21% use Selective, and 3% use Restrictive."). The Using PeeringDB study found that inclination to peer was inversely correlated to network size; larger networks were more likely to have restrictive peering policies.  Id. at 25.  This survey focused on the largest backbones, content providers, CDNs and BIAS providers and therefore the results reflect that methodology.

[6] Compare Aemen Lodhi, Natalie Larson, Amogh Dhamdhere, Constantine Dovrolis, kc claffy, Using PeeringDB to Understand the Peering Ecosystem, ACM SIGCOMM Computer Communication Review, 44(2), 20-27, 21 (2014), http://www.sigcomm.org/sites/default/files/ccr/papers/2014/April/0000000-0000002.pdf ("we find widespread adoption of Open peering among transit providers, which is counterintuitive given that transit providers prefer other ASes as their customers instead of peers. ").

[7] [William Norton, Don’t Abuse Peering, DrPeering] [Understanding Default Routes, IS-IS Feature Guide, Juniper Networks (May 20, 2010), ("A default route is the route that takes effect when no other route is available for an IP destination address.")] [Configuring Static Routes, CISCO Nexus 7000 Series NX-OS Unicast Routing Configuration Guide (June 14, 2017)] [Luc De Ghein, Specify a Next Hop Address for Static Routes, CISCO Technical Support, Sept. 2, 2014] [Configuring a Gateway of Last Resort Using IP Commands, CISCO Technical Support, Aug. 10, 2005 ("Default routes are used to direct packets addressed to networks not explicitly listed in the routing table. Default routes are invaluable in topologies where learning all the more specific networks is not desirable, as in case of stub networks, or not feasible due to limited system resources such as memory and processing power.")]

[8] There are Peering Fundamentalists who adhere to the tenant that "peering" is an arrangement between "peers."  An interconnection arrangement that is not between peers, therefore, cannot be "peering," regardless of the routes exchanged.  To them, the term "paid peering" amounts to heresy as it is implicitly a relationship between non-equals.  As a member of the Reformed Peering Denomination, I view such talk as semantics.  There is no platonic form to the word "peering."  It means just what we chose it to mean – neither more nor less.  It could be called "On-Net traffic exchange."  It could be called "Pineapples."  As long as we adhere to a term and a common understanding, we are good.

Peering Policies

A "Peering Policy" is "[t]he decision criteria that a provider applies in deciding with whom they will peer. " [NRIC Sec. 1.2.2] In the words of Bill Norton, a "Peering Policy" is "an articulation of peering inclination." [Norton, A Guide to Peering Contracts] [Norton Open Peering Policy] Originally, it was an articulation by a backbone network (the networks in the 1990s that peered) of whom it would perceive to be a "peer" or equal. It is like an amusement park sign that says, "you must be this tall to ride." Generally, to be a "peer," a network had to satisfy the settlement free peering proxy and be roughly equal in size and exchange roughly balanced traffic. A peering policy delineated how that would be measured.[Golding][BEREC p. 21 Dec. 6, 2012][NRIC FG 4]

A peering policy is not a peering contract; peering contracts are far more elaborate. It is not a legal "offer;" networks reserve the right to negotiate and to enter into a peering arrangement or not. [NRIC Sec. 1.2.2] [Norton, Peering Policies]

In the 1990s, as the nascent commercial Internet matured, Tier 1 backbones looked at smaller networks, concluded that they were not "peers," and migrated smaller networks to transit customer arrangements. [1997 Depeering] Smaller networks that found themselves with large transit bills complained. [Digital Handshake 2000] [First 706 Report, para. 105 (at the time, commenters unanimously opposed FCC intervention into peering and interconnection disputes with one exception; Bell Atlantic, now Verizon, recommended possible action by the FCC to lower barriers of entry to new entrants).] If they were not large enough to qualify for peering, then they wanted to know how large they had to be. They wanted to know what benchmarks they had to meet to avoid transit fees.

In 2001, the FCC's Network Reliability and Interoperability Council (NRIC, predecessor of CSRIC), led by Jim Crowe of Level 3, recommended that networks post peering policies on their websites. [NRIC Sec. 4.1] [NRIC Internet Peering Statement ("NRIC V encourages other Internet providers, and especially the large "backbone" Internet providers that comprise the core of the modern Internet, to consider, consistent with their business practices, publication of their criteria for peering.") Available on the Web Archive.] [GAO ("We were also told that peering policies should be made public.")] [FCC NRIC Encourages Publication of Peering Criteria to Promote Transparency (Oct. 30, 2001).] In the 2005 mergers (Verizon/WCOM, AT&T/SBC) and the 2007 merger (AT&T/Bell South), the parties agreed to post peering policies as a merger condition. [SBC / AT&T ¶ 133] [Verizon / MCI ¶ 134] [AT&T / Bell South, Appendix F]

Peering policies consist of two different types of clauses.[Compare Norton Survey (dividing the clauses into three groups: (1) operations-related Internet peering policy clauses; (2) Technical / Routing / Interconnection clauses; and (3) General Clauses).] First, provisions that determine whether a potential partner is a "peer" (roughly equal size with roughly balanced traffic), and, second, operational conditions regarding what is necessary to successfully interconnect. These provisions fall out as follows: 

Peer Criteria (roughly equal size / traffic)	Operational Conditions
Geographic reach Redundant network Presence at specified IXPs Minimum number of points of interconnection Minimum traffic capacity / utilization Balanced traffic ratio No customers as peers	24/7 NOC No Abuse Consistent Routing Announcements Filter routes Hot or Cold Potato Routing Resolving congestion / augmentation Use of IRR, PeeringDB

[Norton, Study of 28 Peering Policies] [Golding] [BEREC p. 21 2012] [NRIC Sec. 4.3 (examples include geographic coverage, proximity to exchange points, minimum capacity, symmetry of traffic exchange, minimum traffic loads, reliable network support, and reasonable address aggregation)] [Verizon (“The key common feature, however, is that these voluntary arrangements involve a mutual exchange of value of one form or another.”)] [Aemen Lodhi, Natalie Larson, Amogh Dhamdhere, Constantine Dovrolis, kc claffy, Using PeeringDB to Understand the Peering Ecosystem, ACM SIGCOMM Computer Communication Review, 44(2), 20-27, 21 (2014), ("Finally, we explore what historical snapshots of the PeeringDB database can tell us about the evolution of the Internet peering ecosystem.")]

In 2004, PeeringDB came on the scene. PeeringDB is a database created "by and for peering coordinators" that provides 

• a link to peering policy,
• peering inclination,
• whether interconnection at multiple locations is required,
• whether there is a balanced ratio requirement,
• whether a contract is required,
• peering contact information, and
• the peering facilities where the network is available for interconnection.

According to PeeringDB, "The purpose of this project is to facilitate the exchange of information related to peering. Specifically, what networks are peering, where they are peering, and if they are likely to peer with you." [Martin Levy, PeeringDB and why everyone should use it, presentation at African Peering and Interconnection Forum 2011, slide 7] In 2016, PeeringDB 2.0 was launched and PeeringDB was established as a non-profit organization. By the end of 2016, it listed 8194 peering networks, 2302 interconnection facilities, and 566 IXPs.  [Arnold Nipper, PeeringDB, presentation at CEE Peering Days 2017] [Terry Rodery, PeeringDB, presentation at NANOG 40 (2007)] [Aemen Lodhi, Natalie Larson, Amogh Dhamdhere, Constantine Dovrolis, kc claffy, Using PeeringDB to Understand the Peering Ecosystem, ACM SIGCOMM Computer Communication Review, 44(2), 20-27 (2014)]

📽 Peering Forum - IX Track - PeeringDB Update - Apricot 2017

Peering Forum - IX Track with PeeringDB 2.0 Update (video below is multiple presentations - link is directly to the PeeringDB presentation in the timeline)

📽 Geoff Huston - The Death of Transit and Beyond, APRICOT 2017

The Death of Transit and Beyond
(panel discussion of several presentations - link above is directly to Geoff's presentation in the timeline - video below is all four presentations and Geoff is the fourth)

📽 Peering 101 - Walt Wollny, Hurricane Electric, Apricot 2017

📽 PCH 2016 Survey of Interconnection Agreements :: NANOG 69

Origins of Settlement Free Peering

Internet culture stood as an antithesis to Bell telephone culture. In the early 1960s, Paul Baran, who was concerned about the vulnerability of U.S. communications, advocated that the Department of Defense migrate to a distributed packet-switched network that could survive failure and still get a “Go; No-Go” command to the field. Bell-trained DOD engineers dismissed packet-switched communications as an idea that could not possibly work. In 1971, Larry Roberts, having successfully demonstrated the viability of his packet-switched network experiment, attempted to give ARPANET to AT&T; AT&T was not interested. In the meantime, AT&T had refused to sell private lines to nascent data networks, refused to allow foreign devices (i.e., modems) to their network, and refused to interconnect their network with nascent rivals. Those seeking telecommunications services in order to build computer networks became frustrated, and serious question arose concerning whether AT&T’s telecommunications services would meet the needs of growing computer networks. 

Thus emerged a cultural rivalry between Nethead and Bellhead cultures. In the eyes of the Netheads, Bellheads were about perpetuating a century-old communications network monopoly that followed a command-and-control model of central operation which stunted innovation, while Netheads were all about innovation, building a network that fostered the cool things transpiring at the end. Netheads wanted nothing of the Bellhead model. [Greenstein 2015 at 38 ("most had an almost visceral dislike for" Ma Bell)] [Mayo (quoting Bob Taylor, "Working with AT&T would be like working with Cro-Magnon man. I asked them if they wanted to be early members so they could learn technology as we went along. They said no. I said, Well, why not? And they said, Because packet switching won’t work. They were adamant.").] 

In the 1980s, Netheads had to resolve how to interconnect networks. Netheads wanted to interconnect ARPANET with an NSF sponsored CSNET, but what should be the terms and who should pay whom? One model before them was the byzantine Bell accounting scheme which one engineer described as so complicated it was akin to drinking coffee one molecule at a time; a marvelous feat of engineering if not utterly pointless. [Moore 1999] [Jacobson 1999 (discussing how Internet culture is different than BellHead culture and the implications for the future of the internet)] [Kleinrock 1994 at 67 (bemoaning the additional cost and complexity of accounting necessary for usage-based pricing); [MacKie-Mason 1993 at 14 (describing the cost of phone company style billing and accounting as applied to the packet-switched Internet as “astronomical”).] Netheads' incentive was to grow network effect and continue innovation at the ends. To achieve that incentive, they wanted the lowest possible barriers to the expansion of the network. The solution was settlement-free interconnection, devoid of complicated Bell accounting. A paper by Lyman Chapman and Chris Owens describes how the arrangement came to be:

In modern terms, we would say that the customers of one ISP (ARPAnet) could not communicate with the customers of another ISP (CSNet), because no mechanism existed to reconcile the different Acceptable Use Policies of the two networks. This disconnect persisted as both sides assumed that any agreement to exchange traffic would necessarily involve the settlement of administrative, financial, contractual, and a host of other issues, the bureaucratic complexity of which daunted even the most fervent advocates of interconnection—until the CSNet managers came up with the idea that we now call “peering,” or interconnection without explicit accounting or settlement. A landmark agreement between NSF and ARPA allowed NSF grantees and affiliated industry research labs access to ARPAnet, as long as no commercial traffic flowed through ARPAnet.

[Chapin 2005 at 9] [Norton Chap. 8]

NSF followed CSNET's example for interconnection settlements; interconnections between NSFNET and regional networks was on a settlement-free basis.  [MacKie-Mason 1993 at 18 (“The full costs of NSFNET have been paid by NSF, IBM, MCI and the State of Michigan”).]

Commercial networks emerged in the early 1990s, but they could not exchange traffic through the academic NSFNET. On the one hand, Advanced Network Services(ANS), the contractor that operated the NSFNET, established a commercial backbone service and offered to sell interconnection to nascent commercial networks. ANS had all of the NSFNET clients as end-users including regional networks and academic networks; ANS as the largest network could leverage network effect. But the other commercial networks were not interested in paying ANS for the right to access end-users or helping ANS become the AT&T-monopoly of the Internet. Instead they established the Commercial Internet eXchange (CIX) and exchanged traffic on a settlement-free basis. For these early commercial networks, connectivity was paramount and growth of access services was king. [Brock, Economics of Interconnection at ii ("Commercial Internet service providers agreed that interchange of traffic among them was of mutual benefit and that each should accept traffic from the other without settlements payments or interconnection charges. The CIX members therefore agreed to exchange traffic on a "sender keep all" basis in which each provider charges it own customers for originating traffic and agrees to terminate traffic for other providers without charge.").] First UUNET, PSINET, and CERFNET joined CIX. Then Sprint joined. Soon most of the Internet could be reached through CIX. Connectivity grew network-effect which grew the value of the access service that these commercial networks were selling to end-users. CIX become the model of commercial interconnection, while ANS became isolated. [Greenstein 2015 at 81 ("Just a little less than a year later, CIX essentially had everyone except ANS. By the time Boucher held his hearing, ANS had become isolated, substantially eroding their negotiating leverage with others. By June 1992 ANS's settlement proposals no longer appeared viable. In a very public surrender of its strategy, it agreed to interconnect with the CIX on a seemingly short-term basis and retained the right to leave on a moment's notice.)] [Noam 2001 at 63 (""Soon the relative use by the commercial and nonprofit sectors kept shifting, and the power over interconnection moved to the former. By 1993, approximately 80 percent of all Internet sites could be accessed outside the NSFNET structure. CIX blocked ANS traffic from routing through the CIX router, thus depriving ANS users of connectivity to CIX members. Humbled, ANS joined CIX in 1994"")] [Srinagesh at 143 ("In October 1993, CIX, apparently without warning, blocked ANS traffic from transiting the CIX router. At this point, ANS (through its subsidiary CO-RE) joined the CIX and full connectivity was restored.")]. In 1994, ANS' assets were sold off to AOL. [History, Advanced Network Services (2004)] [Salus 1995 at 200]  

In the late 1990s, academic and government networks focused on research, development, and innovation, not commercial competition; they followed settlement-free peering as a simple accounting scheme for interconnection. Commercial backbone providers adopted settlement-free peering as a means of rapidly growing their business plans.

Access networks, which needed to provide full Internet service to their customers, interconnected with and paid transit to commercial backbone providers. Controversy swirled during the late 1990s as the commercial backbones matured their business plans, converting smaller networks that were dependant on the backbone networks' services from settlement-free peers into paying transit customers. Appeals for intervention reached the FCC, which declined to intercede, finding the Internet backbone market competitive.

Historic Evolution of Internet Interconnection

In the beginning was ARPANET. ARPANET was an end-to-end packet-switched network. ARPANET was The Network, thus interconnection was not a tremendous concern. 

ARPANET’s success begat ALOHANET, SATNET, PRNET and other packet switched networks. It was clear that ARPANET's network protocol would have to be revised in order to facilitate interconnection. In 1972, Vint Cerf and Bob Kahn released A Protocol for Packet Network Interconnection. The design objective of the Internet was to enable interconnection between otherwise incompatible networks, promoting the research and innovation occurring at the edges, and leveraging "network effect." IP made interconnection easy and coordination unnecessary. According to Bob Kahn,

The idea of the Internet was that you would have multiple networks all under autonomous control. By putting this box in the middle, which we eventually called a gateway, it would allow for the federation of arbitrary numbers of networks without the need for any change made to any particular network. So if BBN had one network and AT&T had another, it would be possible to just plug the two together with a [gateway] box in the middle, and they wouldn't have to do anything to make that work other than to agree to let their networks be plugged in.

[SEGALLER, NERDS 2.0.1: A BRIEF HISTORY OF THE INTERNET at 111]

The Internet reflected a culture where connectivity was paramount. With each interconnection of an additional network, the value of the Internet grew. [Carpenter, Architectural Principles of the Internet, IETF RFC 1958,  Sec. 2.1  ("the goal is connectivity, the tool is the Internet Protocol, and the intelligence is end to end rather than hidden in the network. The current exponential growth of the network seems to show that connectivity is its own reward.")] [THE INTERNET'S COMING OF AGE, COMPUTER SCIENCE AND TELECOMMUNICATIONS BOARD, NATIONAL RESEARCH COUNCIL 35 (2001) ("the value placed on connectivity as its own reward favors gateways and interconnections over restrictions on connectivity")] [REALIZING THE INFORMATION FUTURE: THE INTERNET AND BEYOND, COMPUTER SCIENCE AND TELECOMMUNICATIONS BOARD, NATIONAL RESEARCH COUNCIL 3 (1994) (setting forth vision for Open Data Networks, stating in first principle that network should "permit[] universal connectivity.")] [David Clark, A Cloudy Crystal Ball: Visions of the Future, Presentation at the IETF, Slide 4 (July 1992) ("Our best success was not computing, but hooking people together").]

In 1985, the National Science Foundation concluded that the Internet was good, and sought to expand its reach to the greater academic community. Instead of providing the entire end-to-end network, NSF elected to supply a crucial piece: the first dedicated nationwide Internet backbone. The NSFNET interconnected with regional networks, and the regional networks interconnected with local networks. In so doing, NSFNET offered long-distance transit as well as traffic exchange between networks.

Commercial networks concluded that the Internet was good. Commercial traffic, however, could not be exchanged across NSFNET. Therefore, the early commercial networks established the Commercial Internet eXchange (CIX) and exchanged traffic on a settlement-free basis. For these early commercial networks, connectivity was paramount and growth of access services was king. [Brock, Economics of Interconnection at ii ("Commercial Internet service providers agreed that interchange of traffic among them was of mutual benefit and that each should accept traffic from the other without settlements payments or interconnection charges. The CIX members therefore agreed to exchange traffic on a "sender keep all" basis in which each provider charges it own customers for originating traffic and agrees to terminate traffic for other providers without charge.").]

The US Government concluded that the Internet was good, and sought to make it available to all. NSF was tasked with privatizing the Internet. In order to be successful, NSF had to establish a means for emerging commercial networks to exchange traffic. Following the CIX model, NSF built four Internet exchange points, known as NAPs, in Washington, D.C., New York, Chicago, and San Jose. [National Science Foundation Solicitation 93-52, Solicitation for Network Access Point Manager, Routing Arbiter, Regional Network Providers, and Very High Speed Backbone Network Services Provider for NSFNET and NREN Program (May 6, 1993) (setting forth NSF's plan for privatizing NSFNET).] [GREENSTEIN, HOW THE INTERNET BECAME COMMERCIAL at 82 (NAPs were modeled on CIX and "helped the commercial Internet operate as a competitive market after the NSFNET shut down.").]

In 1995, NSF decommissioned the NSFNET and the commercial Internet was born in its image. There were Tier 1 backbone networks (WANs) that provided nationwide or global service, Tier 2 networks (MANs, Metro) that provided regional service, and Tier 3 networks (LANs, Local) that provided access service. Traffic from one end-user to another end-user would travel up the topology from Tier 3 networks to be exchanged at the Tier 1 level, and then travel back down. Within networks was robust capacity moving traffic. [See David Young, Why is Netflix Buffering? Dispelling the Congestion Myth, VERIZON PUBLIC POLICY BLOG (July 10, 2014) (diagramming Verizon network with backbone, metro and local networks)] Between networks was interconnection where capacity could be constrained. Interconnection capacity constituted an aggregation of all traffic from the different end-users, services, and firms to which a network provider offered service. It could become a traffic pinch-point. In order to avoid congestion, interconnection partners would have to cooperate. [See KLEINROCK, REALIZING THE INFORMATION FUTURE: THE INTERNET AND BEYOND at 183 ("Because the network is not implemented as one monolithic entity but is made of parts implemented by different operators, each of these entities must be separately concerned with achieving good loading of its links, avoiding congestion, and making a profit. The issue of sharing and congestion arises particularly at the point of connection between providers. At this point, the offered traffic represents the aggregation of many individual users, and thus cost-effective sharing of the link can be assumed. However, if congestion occurs, one must eventually push back on original sources. Options include pricing and technical controls on congestion; there may be other mechanisms for shaping consumer behavior as well.”)]

Early academic and government networks focused on research, development, and innovation, not commercial competition; they developed a simple accounting scheme for interconnection: settlement-free peering. Early commercial backbone providers adopted settlement-free peering as a means of rapidly growing their business plans. Access networks, which needed to provide full Internet service to their customers, interconnected with and paid transit to commercial backbone providers.

The Internet interconnection market would then undergo seismic evolution. By 2010, large access providers had reestablished a strong position in the network ecosystem, and leveraged network effect and interconnection to implement strategic goals. Broadband Internet access service providers were able to reverse the flow of interconnection settlement fees, going from transit customers to gatekeepers, charging paid peering for access to their end-users. The interconnection market had departed from the characteristics of the early commercial Internet, where backbone networks were king and the market was hyper-competitive, and returned to traditional consolidated network market economics.  [See generally WU, THE MASTER SWITCH: THE RISE AND FALL OF INFORMATION EMPIRES (discussing how communications markets go through cycles of innovation and disruption, competition, and then consolidation and market power).]

Statement from FCC Ch Tom Wheeler on the Comcast / TWC Merger

 STATEMENT FROM FCC CHAIRMAN TOM WHEELER ON THE COMCAST-TIME WARNER CABLE MERGER.   FCC Chairman Tom Wheeler issued the following statement today after Comcast announced its decision to abandon its $45 billion dollar bid to acquire Time Warner Cable.  STMT. OCHTW  https://apps.fcc.gov/edocs_public/attachmatch/DOC-333175A1.docx
https://apps.fcc.gov/edocs_public/attachmatch/DOC-333175A1.pdf

FCC Chairman Tom Wheeler issued the following statement today after Comcast announced its decision to abandon its $45 billion dollar bid to acquire Time Warner Cable. Comcast's announcement comes after the Federal Communications Commission staff informed the companies of their serious concerns that the merger risks outweighed the benefits to the public interest.

"Comcast and Time Warner Cable’s decision to end Comcast’s proposed acquisition of Time Warner Cable is in the best interests of consumers. The proposed transaction would have created a company with the most broadband and video subscribers in the nation alongside the ownership of significant programming interests.

 "Today, an online video market is emerging that offers new business models and greater consumer choice. The proposed merger would have posed an unacceptable risk to competition and innovation especially given the growing importance of high-speed broadband to online video and innovative new services.

I am proud of our close working relationship throughout the review process with the Antitrust Division of the Department of Justice. Our collaboration provided both agencies with a deeper understanding of the important issues of innovation and competition that the proposed transaction raised.”