The word GIX has been mentioned several times in the recent time.
The name is realy not what it should be, but the idea is to have THE Exchange,
sort of a grounding-point for a flat addressing architecture.
It is described as 'Level-2', to be able to handle all special cases, that
will arise when 'general infrastructure' and 'mission oriented' networks
are connected across the same medium.
I image the routing implemenation for the 'general infrastructure' networks to
be a handfull of route-servers run by the routing authoritys for each continent,
to avoid the n-squared peering problem, and to get stability.
In Ebone context, we can live with the 'Level-3' implemenation we have today
to solve the problem of getting maximal connectivity, from a user in one corner of
Europe to another user somwhere in the entire universe.
I envision that the Ebone open infrastructure, is a Level-3 extension from
this 'GIX' and that any European network/network-providor shoul have equal
access to the Ebone core.
We have the technology, the needed people but, as all networking in Europe is
about the network-POLITICS not the network-SERVICES it might not be possible
to have this pice of usefull infrastructure.
If we fail this time, please take your wathces and adjust them backwards five years!
I include the full 'GIX-proposal' for your information, remember, this documnet
is 'politicly' nothing, as it is a proposal from a small group people.
And do remember, time is critical now, the whole CIDR model is based on that
large chunks of networks, like continents are fully interconnected.
-Peter
------------------------------------------------------------------------
Proposal for Global Internet Connectivity
IEPG Working Document
Guy Almes, Peter Ford, and Peter Lothberg
12 June 1992
I. Introduction
A significant subset of the IEPG membership has advocated a
common forum for the coordination of world-wide research and non-
research networking. At the November 1991 meeting in Santa Fe, the
idea of a global internet exchange emerged in basic form. At the
March 1992 informal meeting, the idea was discussed in a day-long
meeting at which goals and basic concepts were shared and at which
a consensus began to emerge. At the June 1992 meeting in Tokyo, a
detailed proposal was discussed and refined. This paper documents
the ideas that may allow us to move the ideas from discussion to
plan to implemented reality. Key technical goals are to maximise
connectivity, promote appropriate routing, promote effective
sharing of resources such as links, and enable a variety of transit
options. Key non-technical goals are to ensure the participation
by all segments of the Internet community -- both the traditional
CCIRN constituency and others -- in working toward a high-quality
world-wide Internet.
The reader should consider that the substantive ideas of the
paper reflect discussions with several people. The helpful
suggestions of Tony Hain, Geoff Huston, Bernhard Stockman and
Claudio Topolcic are especially appreciated.
II. Goals
The proposal addresses the following technical and non-
technical goals:
<> Maximal Connectivity: Enhance access and connectivity of the
global Internet. Interconnectivity of networks should be
enhanced, including those networks which are not directly
funded by traditional CCIRN constituencies.
<> Cost-effective Transit: Make available to all our networks the
best available possibilities for worldwide transit. To the
extent possible, promote the sharing of global transit
resources and the sharing of costs.
<> High-quality Routing: Work toward routing management that
supports the connectivity and transit goals while being
technically optimal.
Each of these goals is expanded in turn.
In addressing maximal connectivity, we debated on whether
'universal' or 'maximal' was the right term. There was no
question, however, but that, in the words of one member "Connectiv-
ity is the key goal". There are several possible limitations on
connectivity. Some, such as connectivity limits imposed on a
particular community of interest, are unavoidable and even
desirable. Thus, a term such as ~selective connectivity~ might be
appropriate. Others, such as any limits imposed by usage con-
straints on intercontinental transit networks, are undesirable in
the long run. Transit networks and interconnection structures open
to both research, education, and other traffic will aid the broad
growth of the Internet. To summarise, we want to promote, enable,
and maximise universal connectivity among transit networks, so that
the only limits on universal connectivity are imposed by user
communities.
In addressing cost-effective transit, we noted that there is
a diversity of transit needs: some networks need universal
affordable transit, for example, while others need higher-perfor-
mance transit even if costs are not minimised. Similarly, there is
a diversity of transit possibilities: there are at least five
national providers in the US, for example, and we could contemplate
new transit structures. To summarise, we should work to harness
the variety of transit options to serve the diversity of transit
needs. At the same time, we should ensure that issues surrounding
transit do not limit interconnectivity, defeat our routing goals,
or adversely effect existing networks.
In addressing high-quality routing, we emphasised routing as
a means to supporting the connectivity and transit goals mentioned
above, but also noted technical symmetry, stability, and manage-
ability as explicit goals. Though it might go without saying, we
also agreed that we should use the best of currently available
routing techniques, while not requiring what does not exist. At
the same time, it is desirable to provide a platform for the early
deployment of advanced routing tools. To summarise, we should work
together to build worldwide routing and routing management in
support of maximal connectivity and cost-effective transit, while
using the best available techniques to support technical qualities
such as stability and symmetry.
Along with these goals, we also agreed on the following
qualities that must apply to our approaches to them:
-- Scalability: We understand that the size of the Internet,
measured either in traffic or number of connected networks, is
growing exponentially, and that our engineering and operations
must scale.
-- Manageability: We understand that the worldwide Internet must
be as well-managed as one under a single administration, and
yet recognise the autonomy of each constituent network.
-- Accountability: We understand that any successful worldwide
connectivity structure must be accountable.
-- Timeliness: We understand that the cost of staying with the
status quo, or the cost of delay, is very high. Maintaining
the integrity of the Internet requires prompt action.
In approaching solutions to these goals, we often had to deal with
tensions among competing goals and the limits imposed on our
technology.
III. Our Vision -- Technical
<briefly, a set of global interconnect points at which
cooperative advanced inter-AS routing is supported and at
which transit and regional networks meet. Universal
connectivity, good routing, and several transit options
are supported.>
At several places around the world, we envision global
interconnection points. Each such point would consist of a managed
facility with 24-by-7 coverage and excellent environmental support.
At each point there would be a high-speed broadcast LAN freely
available for all kinds of traffic. Each participant would be free
to, at its expense, bring a circuit to this facility and place a
router on the LAN. It would then be free to exchange routes and
traffic with (the routers of) other participants at that LAN. Each
participant would also pay for a pro rata share of the cost of the
floor space, environmental support, and administrative support
required for the interconnection points.
At this level of detail, we are simply describing the
engineering of the current US FIX structure. From a technical
point of view, we consider that it is the best approach among
current models. There are, however, several non-technical
requirements needed:
>> An Open Forum for Coordination. This forum would provide full
participation for non-US and for non-research-and-education
networks.
>> Very Broad Usage. A global interconnection point must be able
to pass research, education, and other traffic. Some connect-
ing networks may still have research/education AUPs, but the
interconnection structure should not.
In addition, we need solutions to several technical shortcomings of
the traditional FIX engineering structure:
>> N-squared Routing. All routing is done on the basis of pair-
wise peering among pairs of routers on the LAN. We consider
that, as the use of the interconnection points grows, this may
not scale.
>> Anarchic Routing. When a router on the LAN advertises a route
to a given destination network, its peer has no basis for
knowing whether that router is authorised by the destination
network to advertise it.
>> Destination-based Routing. If we were able to route across
the interconnection points on the basis of both source and
destination, then we could better use the variety of transit
possibilities, including federally funded ones, without
limiting connectivity due to policy issues. Our inability to
do so limits these choices, and also leads to undesirable
asymmetric routes.
In order to achieve our technical vision, each of these shortcom-
ings will need to be dealt with. In many cases, these problems
will be dealt with by means of improved routers of participants and
not by means of new technology in the interconnection points
themselves.
N-squared Routing. We anticipate that the number of routers
at the interconnection points and the number of network numbers
advertised by each router will both increase dramatically. In
order to cope with this, we will provide for the deployment and
management of a set of Route Servers on each of the interconnection
points. In simple terms, Route Servers use inter-AS routing
protocols such as BGP to learn and advertise routes, but do not
themselves participate in the forwarding of packets. Networks that
attach will have the option of using these Route Servers, of using
traditional pair-wise peering, or of using some combination. There
may, in fact, be multiple different Route Servers at a given
connection point used by different sets of participants. The
intention of Route Servers is not to impose policy, but to
implement the dissemination of routes in a manner that scales and
can be well managed.
Such Route Servers might also help solve some of the so-called
ROAD problems. For example, CIDR support in a Route Server could
help even in the period before CIDR is supported in the routers
used within a participating network. Similar examples could be
given with respect to Policy Routing and Source Demand Routing.
Route Servers will be a subject of Section V (Research
Efforts) below, and its deployment and management will be a subject
of the cooperative routing efforts discussed throughout the paper.
Anarchic Routing. One of the major successes of the NSFnet
Backbone Service has been the Policy Routing Database, a database
designed and administered by MERIT which records the ASes author-
ised to advertise certain IP network numbers to the Backbone. By
correctly and faithfully administering this database, MERIT
provides a communication link, as it were, between network managers
who set and agree on policies and the routers which implement
routing exchanges. For each IP network number, for example, the
database records the primary, secondary, and (sometimes) tertiary
ASes that can advertise that network number.
We anticipate a similar need at the global interconnection
points. Rice University's network number (128.42), for example,
could be authorised for the NSFnet Backbone Service as primary and
SprintLink as secondary (and used only for backup). A global
routing Registry would be designed and administered, and this
Registry would be instantiated at each interconnection point.
Registry management would be selected to ensure the neutrality of
the Registry with respect to possibly competing networks that
connect at the Interconnection Point. Tools could be written to
produce derivatives of the Registry for particular engineering
purposes, such as Cisco access lists for the control of route
filtering. If we can judge from experience on the NSFnet Backbone
Service, such a Registry would provide a framework in which
misunderstandings and conflicts in routing could be avoided and/or
resolved before they could cause undesired asymmetric routes or
other problems.
Destination-based Routing. There are currently many circum-
stances in which several transit networks must be traversed from
source to destination, in which several alternate paths are
possible, and in which constraints (such as AUPs) or preferences
(based on cost or performance issues) will influence the selection
of the most appropriate path. Further, not all of the constraints
and preferences can be cast in terms of traditional routing which
considers only the destination IP address. Examples of needed
improvements include:
-- TOS Support. Some traffic may be tagged to require low
latency (e.g., telnet) or high bandwidth (e.g., image trans-
fer) or low cost (e.g., email for high schools). To some
extent the TOS/QOS concept may support this, but operational
support for this on a worldwide basis is beyond current art.
-- Source-based Routing. Some traffic may be appropriate to
carry over special-purpose networks due to identification of
both source and destination. For example, ESnet might be
willing to serve as a transit network, but only for traffic
from one energy-community site to another.
Similarly, consideration of both source and destination
might be useful in avoiding asymmetric routes. For example,
in the case of traffic exchanges between a university and a
commercial site, destination-based routing often leads to the
use of research transit networks in one direction and commer-
cial transit networks in the other, even when the research
transit network can support the full exchange.
-- Flow Support. Some traffic may be part of a flow that
requires a form of bandwidth reservation supported by one
special transit network, and that transit might be configured
to support that flow. Support for global conferencing might
require this flow to be carried over general infrastructure
networks for part of the path and special transit networks for
other parts.
Routers in common use on the Internet do not support any of these
abilities, but experience with DARTnet and with wider experiments
with IDPR suggests that some of them could be deployed in the near
future.
An Open Forum for Coordination. We anticipate the need for
the full participation of engineers from all connecting networks in
the coordination of routing and management across the global
interconnection points. The global routing Registry would provide
one kind of communication path to allow for this coordination to be
natural and convenient in normal cases. The famous RIPE coffee
breaks offer another needed tool, but can only happen less
frequently. Advanced collaboration support tools, including
conferencing, shared blackboard, and email-based tools, could
provide some technical support tools for this effort.
The FEPG within the US agencies and RIPE within the European
IP community provide useful precedents. The key is effective and
full participation by networks in issues of concern to those
networks. This may require a formally incorporated organisation as
a structural umbrella.
Very Broad Usage. There will be no AUP constraints on the
interconnection points themselves. Further, by enabling effective
routing and by bringing several transit alternatives to the
interconnection points, we will enable participating networks to
cope responsibly with the AUP issues that remain.
IV. Our Vision -- Non-technical
<briefly, a structure to do collectively what must be
done collectively to accomplish the stated goals. This
collective action will probably require some organisa-
tional structure, and this structure should respect
autonomy and be accountable.>
Although the IEPG can manage the design of technical solutions
to our goals and although we perceive very broad agreement with the
goals and technical structures we propose, our proposal cannot
succeed without some cooperative structure for managing the global
interconnection points and setting the policies that govern it. At
a minimum, we need a small organisational structure that can do
this necessary management and governance.
The structure needs to secure the fullest possible participa-
tion of the Internet community. This must include networks from
North America, Europe, and the Pacific Rim. It must include
government agencies, university consortia, and for-profit and not-
for-profit companies. It must include networks that support
research, education, industrial, and mixed traffic.
The structure needs to be representative of and accountable to
all its participants. Thus, what policies the structure does set
must earn the kind of authority that comes from such accountabili-
ty.
The structure needs to address what must be done collectively
without itself becoming a network in competition with its partici-
pants. Thus, for example, the cooperative structure would not
provide connectivity directly to any user site. Similarly, the
cooperative structure would be careful not to favor on competing
participant over another.
The structure must avoid deciding issues best left to the
participants themselves. Thus, for example, the structure should
avoid deciding which networks should use which other networks for
transit, or how various pairs of networks should do settlements.
It is important to remember that the purpose of our efforts here is
to cooperatively harness the strengths of our various networks in
support of the Internet community -- not to replace our networks
with a homogeneous Internet.
Among the organisations that could help us find the right
organisational structure to meet these objectives are the CCIRN,
the Internet Society, EDUCOM, RARE, FARnet, and PACCOM.
Fortunately, there is near-universal recognition of the mutual
benefit involved in full connectivity.
V. Research Efforts Needed to Achieve the Vision
<briefly, we need specific research and development
efforts to improve on the routing techniques and network
management techniques available to us>
Several research and development efforts need to be conceived,
funded, executed, and deployed in order to achieve all the aspects
of our technical vision. Among them are:
>> Advanced Route Servers. Very urgent. While pair-wise routing
works today, scaling for the future will require successful
design and deployment of Route Servers. Initial experience
could take place using third-party BGP techniques with current
routers, and CIDR routers could become available in the near
future.
>> Source-based Routing. Early experience with source-based
inter-AS routing should be evolved into operational routers
that could pass packets among participant routers at the
interconnection points. These routers could allow us to avoid
asymmetric routes, but would have to be both sophisticated in
the policies supported and very fast to avoid becoming a
performance bottleneck.
>> TOS/QOS-based Routing. In order to support high-performance
applications on a worldwide basis, we will need to support
TOS/QOS at the interconnection points. This will need to be
supported in a way that interoperates with other emerging
approaches.
>> Flow-based Routing. As the concept of flows becomes better
defined, support for it at the interconnection points would
add significant value to the worldwide Internet. Routers that
recognise and expedite the forwarding of flows would be of
great utility at the interconnection points.
Some of these projects involve multi-year research and development.
Others, such as simpler versions of Route Servers and CIDR, may be
deployable within a year.
VI. Immediate Non-technical Agenda
<briefly, work with the Internet Society and the CCIRN to
set up an accountable cooperative structure to accomplish
the technical agenda. This should be in place by summer
1992 >
Under the leadership of such collective organisations as the
Internet Society and the CCIRN, we should immediately set up a
structure that can allow the first step at our technical agenda.
Efforts should be made to accomplish this by summer 1992.
We will consider several possibilities. One very specific
possibility, which we would view positively, follows. The CCIRN
and the Internet Society would charge a task force of the IEPG to
develop a more detailed design, mindful of the very similar agenda
before the US Interim Interagency NREN project. Considering the
urgency of providing an initial global interconnection point during
calendar 1992, and continuing this specific example, this task
force will attempt to liaise very closely with the NSF. Within
this strategy, we hope to leverage activities already planned at
FIX-E.
There are other possibilities that we will explore, but we
view this as the most hopeful.
VII. Immediate Technical Agenda
<briefly, set up one interconnect point using existing
routing and network management technology. It is likely
that the NSFnet Implementation Plan and the FEPG Interim
Interagency NREN Architecture, both of which are converg-
ing during summer 1992, will influence the resulting
design. In NSFnet/NREN terms, this section calls for a
NAP on the east coast of the United States.>
At one place on the east coast of the United States, we
envision a global interconnection point. This point would consist
of a managed facility with 24-by-7 coverage and excellent environ-
mental support. At this point there would be an FDDI ring freely
available for all kinds of traffic. Each participant will be free
to, at its own expense, bring a circuit to this facility and place
a router on the ring. It would then be free to exchange routes and
traffic with (the routers of) other participants at the ring.
N-squared routing would be addressed by the provisioning of
route servers. This would take some time to perfect, but it would
be taken on immediately both for experimentation with the route
server idea and to permit scaling of the rapidly growing route
tables we all keep.
Anarchic routing would be addressed by an interim Routing
Registry. One possible approach to this would be to adapt for our
use the registry the NSF will be setting up for the NSFnet Backbone
Service.
We would live with destination-based routing and forwarding,
but would immediately start some experiments with enhanced
forwarding. These experiments would not be allowed to compromise
the operational stability of the interconnection points.
The IEPG task force discussed in Section VII would meet
quarterly to work out problems and evolve the structure. Even
between meetings, this task force would serve an ombudsman role and
would oversee the interconnection point to ensure that the
interests of the international community are being well served. The
meetings should be coordinated with existing meetings such as RIPE,
IETF, and the Internet Society.
There would be no AUP constraints on the interconnection
points, themselves.
VIII. Evolution to the Vision
<briefly, the cooperative structure guides the immediate
technical structure toward the vision, making use of the
results of the stated research efforts>
Under the leadership of the CCIRN and the Internet Society,
the interconnection points and their management would evolve. This
will eventually lead to the need for us to procure the services
needed to support and manage multiple interconnection points. This
will require organisational maturity in order to manage such a
procurement, and in order to have an authority structure that
combines credibility, broad trust, and accountability to the
international community.
Similarly, we will need to ensure progress on the research
agenda required for high-quality interconnection points for the
future. This will pay off enormously when the time comes to
increase the number of interconnection points.
Similarly, after the initial single interconnection point is
established, a set of design tasks will be required prior to our
being able to field a second site. Examples of issues are the
coordination of the multiple instantiations of the Routing Registry
and the patterns by which the various options of transit among the
multiple sites are handled.
Later, we will also need to reconsider the technical basis of
the interconnection point as Internet technology evolves. During
the era of fractional T1 intercontinental links, our current choice
of an FDDI ring seems more than adequate. As more advanced
transmission media such as 150 Mb/s ATM with support for isochro-
nous traffic become deployed in the Internet, we will surely need
to reconsider this technical basis.
On all these areas of work, technical service, non-technical
management, and research management, we will need to ensure that
the broad needs of the international Internet community are being
addressed.
IX. Conclusion
<briefly, the Internet is saved and the IEPG wanders off
to the bar>
This paper has proposed a serious program for improving the
global interconnectivity of the Internet. The goals of maximal
connectivity subject to community-of-interest constraints, cost-
effective transit, and high-quality routing are addressed, and
attention to short- and long-term efforts, technical and non-
technical efforts, and research agenda is paid. There is solid
hope for success provided that these efforts are begun very soon.
