![]() ![]() | ||||||||||||||
|
![]() |
|
![]() |
![]() |
The controller keeps a complete view of the network topology so that it can compute routes. The topology is constructed on the basis of link-state updates generated by authenticated switches. Capabilities are created using the symmetric keys (to switches and hosts) established by the authentication service.
The controller will adapt the network when things go wrong (maliciously or otherwise). For example, if a switch floods the DC with control traffic (e.g. link-state updates), it will simply eliminate the switch from the network by instructing its immediate neighbor switches to drop all traffic from that switch. It will issue new capabilities so that ongoing communications can start using the new topology.
All packet forwarding is done by switches, which can be thought of as simplified Ethernet switches. Switches forward packets along the encrypted source route carried in each packet. They also send link-state updates to the DC so that it knows the network topology.
Note that, in a SANE network, IP continues to provide wide-area connectivity as well as a common framing format to support the use of unmodified end hosts. Yet within a SANE enterprise, IP addresses are not used for identification, location, nor routing.
The NSD maintains a hierarchy of directories and services; each directory and service has an access control list specifying which users or groups can view, access, and publish services, as well as who can modify the ACLs. This design is similar to that deployed in distributed file systems such as AFS [25].
As an example usage scenario, suppose martin wants to share his
MP3's with his friends aditya, mike, and tal in the
high performance networking group. He sets up a streaming audio
server on his machine bongo, which has a directory stanford.hpn.martin.friends with ACLs already set to allow his
friends to list and acquire services. He publishes his service by
adding the command
sane -publish stanford.martin.ambient:31337
to his audio server's startup script, and, correspondingly, adds the
command
sane -remove stanford.martin.ambient
to its shutdown script. When his streaming audio server comes on line, it publishes itself in the NSD as ambient. When tal accesses this service, he simply directs his MP3 player to the name stanford.martin.ambient The NSD resolves the name (similar to DNS), has the DC issue a capability, and returns this capability, which tal's host then uses to access the audio server on bongo.
There is nothing unusual about SANE's approach to access control. One could envision replacing or combining SANE's simple access control system with a more sophisticated trust-management system [15], in order to allow for delegation, for example. For most purposes, however, we believe that our current model provides a simple yet expressive method of controlling access to services.
All packets in a SANE network contain a SANE header located between the Ethernet and IP headers. In Figure 4, we show the packet types supported in SANE, as well as their intended use (further elaborated below).
![]() |
Communicating with the DC.
SANE establishes default connectivity to the DC by building a minimum
spanning tree (MST), with the DC as the root of the tree. This is done
using a standard distance vector approach nearly identical to that used
in Ethernet switches [1], with each switch sending HELLO messages to its neighbor, indicating its distance from the root.
The MST algorithm has the property that no switch learns the network
topology nor
is the topology reproducible from packet traces.
The spanning tree is only used to establish default routes for forwarding packets to the DC. We also need a mechanism for the DC to communicate back with switches so as to establish symmetric keys, required both for authentication and for generating and decoding capabilities. Note that the DC can initially only communicate with its immediate neighbors, since it does not know the full topology.
The DC first establishes shared keys with its direct neighbors, and it receives link-state updates from them. It then iteratively contacts switches at increasing distances (hop-counts), until it has established shared keys with all switches to obtain a map of the full topology.To establish shared keys, we opt for a simple key-exchange protocol from the IKE2 [28] suite. To contact a switch multiple hops away, the DC must first generate a capability given the topology information collected thus far. Once established, keys provide confidentiality, integrity, and replay defense for all subsequent traffic with the DC via an authenticator header, much like IPsec's ESP header.
All capability requests and link state updates--packets of type DC--are sent along the MST. As packets traverse the MST, the switches construct a request capabilityRequest capabilities are similar to network capabilities as discussed in [12,51] by generating an encrypted onion at each hop containing the previous and next hop, encrypted under the switch's own key. The DC uses the request capabilities to communicate back to each sender. Because these capabilities encode the path, the DC can use them to determine the location of misbehaving senders.
Point-to-Point Communication. Hosts communicate using
capabilities provided by the DC. This traffic is sent using FORWARD packets which carry the capability. On receipt of a packet,
switches first check that the capability is valid, that it has not
expired and that it has not been revoked (discussed later).
Before discussing how capabilities are constructed, we must differentiate between long-lived names and ephemeral connection identifiers. Names are known to the service directory for published services and their access control lists. Identifiers enable end hosts to demultiplex packets as belonging to either particular connections with other end hosts or to capability requests with the DC, much like transport-level port numbers in TCP or UDP. (They are denoted as client-ID and server-ID below.) So, much like in traditional networks à la DNS names and IP addresses, users use SANE names to identify end-points, while the network software and hardware uses connection identifiers to identify unique services.
The DC constructs capabilities using three pieces of information: the client's name and location (given in the capability request), the service's location (stored in the service directory), and the path between these two end-points (as calculated from the network topology and any service policies).
Each layer in the capability is calculated recursively, working backward from the receiver, using the shared key established between the DC and the corresponding switches.
CAPABILITY
(switch-name,
next-hop, prev-hop, CAPABILITY)
Where, denotes the encryption of message
under the
symmetric key
. Encryption is implemented using a block cipher
(such as AES) and a message authentication code (MAC) to provide both
confidentiality and integrity.
All capabilities have a globally unique ID Cap-ID for revocation, as well as an expiration time, on the order of a few minutes, after which a client must request a new capability. This requires that clocks are only loosely synchronized to within a few seconds. Expiration times may vary by service, user, host, etc.
The MAC computation for each layer includes the Cap-ID as well as the expiration time, so they cannot be tampered with by the sender or en-route. The initialization vector (IV) provided in the outer layer of capabilities is the encryption randomization value used for all layers. It prevents an eavesdropper from linking capabilities between the same two end-points.We use the same IV for all layers--as opposed to picking a new random IV for each layer--to reduce the capability's overall size. For standard modes of operation (such as CBC and counter-mode) reusing the IV in this manner does not impact security, since each layer uses a different symmetric key.
Revoking Access.
The DC can revoke a capability to immediately stop a misbehaving sender
for misusing a capability. A victim first sends a revocation request, which
consists of the final layer of the offending capability, to the DC. The DC
verifies that the requester is on the capability's path, and it returns a signed
packet of type REVOKE.
The requester then pushes the revocation request to the upstream switch from which the misbehaving capability was forwarded. The packet travels hop-by-hop on the reverse path of the offending capability. On-path switches verify the DC's digital signature, add the revoked Cap-ID to a local revocation list, and compare it with the Cap-ID of each incoming packet. If a match is found, the switch drops the incoming packet and forwards the revocation to the previous hop. Because such revocation packets are not on the data path, we believe that the overhead of signature verification is acceptable.
A revocation is only useful during the lifetime of its corresponding capability and therefore carries the same expiration time. Once a revocation expires, it is purged from the switch. We discuss protection against revocation state exhaustion in section 4.1.
Discussion thus far has assumed a clean-slate redesign of all components in the network. In this section, we describe how a SANE network can be used by unmodified end-hosts with the addition of two components: translation proxies for mapping IP events to SANE events and gateways to provide wide-area connectivity.
Translation Proxies.
These proxies are used as the first hop for all
unmodified end hosts. Their primary function is to translate between IP
naming events and SANE events. For example, they map DNS name queries
to DC service lookups and DC lookup replies to DNS replies. When the DC
returns a capability, the proxy will cache it and add it to the
appropriate outgoing packets from the host. Conversely, the proxy will
remove capabilities from packets sent to the host.
In addition to DNS, there are a number of service discovery protocols used in today's enterprise networks, such as SLP [44], DNS SD [4], and uPNP [6]. In order to be fully backwards-compatible, SANE translation proxies must be able to map all service lookups and requests to DC service queries and handle the responses.
Gateways.
Gateways provide similar functionality to perimeter NATs.
They are positioned on the perimeter of a SANE network and provide
connectivity to the wide area. For outgoing packets, they cache the
capability and generate a mapping from the IP packet header
(e.g., IP/port 4-tuple) to the associated capability. All incoming
packets are checked against this mapping and, if one exists, the
appropriate capability is appended and the packet is forwarded.
Broadcast.
Unfortunately, some discovery protocols, such as uPNP, perform service
discovery by broadcasting lookup requests to all hosts on the LAN.
Allowing this without intervention would be a violation of least
privilege. To safely support broadcast service discovery within SANE,
all packets sent to the link-layer broadcast address are forwarded to
the DC, which verifies that they strictly conform to the protocol spec.
The DC then reissues the request to all end hosts on the network,
collects the replies and returns the response to the sender. Putting
the DC on the path allows it to cache services for subsequent requests,
thus having the additional benefit of limiting the amount of broadcast
traffic. Designing SANE to efficiently support broadcast and multicast
remains part of our future work.
Service Publication.
Within SANE, services can be published
with the DC in any number of ways: translating existing service
publication events (as described above), via a command line tool,
offering a web interface, or in the case of IP, hooking into the bind call on the local host à la SOCKS [30].
Replicating the Domain Controller.The DC is logically centralized, but most likely physically replicated so as to be scalable and fault tolerant. Switches connect to multiple DCs through multiple spanning trees, one rooted at each DC. To do this, switches authenticate and send their neighbor lists to each DC separately. Topology consistency between DCs is not required as each DC grants routes independently. Hosts randomly choose a DC to send requests so as to distribute load.
Network level-policy, user declared access policy and the service directory must maintain consistency among multiple DCs. If the DCs all belong to the same enterprise-and hence trust each other-service advertisements and access control policy can be replicated between DCs using existing methods for ensuring distributed consistency. (We will consider the case where DCs do not trust each other in the next section.)
Recovering from Network Failure.
In SANE, it is the end host's responsibility to determine network
failure. This is because direct communication from switches to
end hosts violates least privilege and creates new avenues for DoS.
SANE-aware end hosts send periodic probes or keep-alive messages to
detect failures and request fresh capabilities.
When a link fails, a DC will be flooded with requests for new capabilities. We performed a feasibility study (in Section 6), to see if this would be a problem in practice, and found that even in the worst-case when all flows are affected, the requests would not overwhelm a single DC.
So that clients can adapt quickly, a DC may issue multiple (edge-disjoint, where possible) capabilities to clients. In the event of a link failure, a client simply uses another capability. This works well if the topology is rich enough for there to be edge-disjoint paths. Today's enterprise networks are not usually richly interconnected, in part because additional links and paths make security more complicated and easier to undermine. However, this is no longer true with SANE--each additional switch and link improves resilience. With just two or three alternate routes we can expect a high degree of fault tolerance [27]. With multiple paths, an end host can set aggressive time-outs to detect link failures (unlike in IP networks, where convergence times can be high).
This section discusses some additional considerations of a SANE network, including its support for middleboxes, mobility, and support for logging.
Middleboxes and Proxies.
In today's networks, proxies are usually placed at choke-points, to make
sure traffic will pass through them. With SANE, a proxy can be placed
anywhere; the DC can make sure the proxy is on the path between a client
and a server. This can lead to powerful application-level security
policies that far outreach port-level filtering.
At the very least, lightweight proxies can validate that communicating end-points are adhering to security policy. Proxies can also enforce service- or user-specific policies or perform transformations on a per-packet basis. These could be specified by the capability. Proxies might scan for viruses and apply vulnerability-specific filters, log application-level transactions, find information leaks, and shape traffic.
Mobility.
Client mobility within the LAN is transparent to servers, because the
service is unaware of (and so independent of) the underlying topology.
When a client changes its position--e.g., moves to a different wireless
access point--it refreshes its capabilities and passes new return
routes to the servers it is accessing. If a client moves locations, it
should revoke its current set of outstanding capabilities. Otherwise,
much like today, a new machine plugged into the same access point could
access traffic sent to the client after it has left.
Server mobility is handled in the same manner as adapting to link failures. If a server changes location, clients will detect that packets are not getting through and request a new set of capabilities. Once the server has updated its service in the directory, all (re)issued capabilities will contain the correct path.
Anti-mobility.
SANE also trivially anti-mobility. That is, SANE can prevent hosts and
switches from moving on the network by disallowing access if they do. As the
DC knows the exact location of all senders given request capabilities, it can
be configured to only service hosts if they are connected at particular
physical locations. This is useful for regulatory compliance, such as 911
restrictions on movement for VoIP-enabled devices. More generally, it allows a
strong ``lock-down'' of network entities to enforce strong policies in the
highest-security networks. For example, it can be used to disallow all
network access to rogue PCs.
Centralized Logging.
The DC, as the broker for all communications, is in an ideal position
for network-wide connection logging. This could be very useful for
forensics. Request routes protect against source spoofing on connection
setup, providing a path back to the connecting port in the network.
Further, compulsory authentication matches each connection request to an
actual user.
SANE attempts to degrade gracefully in the face of more sophisticated attacks. Next, we examine several major classes of attacks.
Flooding. As discussed in section 3.3, flooding attacks are handled through revocation. However, misbehaving switches or hosts may also attempt to attack the network's control path by flooding the DC with requests. Thus, we rate-limit requests for capabilities to the DC. If a switch or end host violates the rate limit, the DC tells its neighbors to disconnect it from the network.
Revocation state exhaustion. SANE switches must keep a list
of revoked capabilities. This list might fill, for example, if it is
maintained in a small CAM. An attacker could hoard capabilities, then
cause all of them to be revoked simultaneously. SANE uses two mechanisms
to protect against this attack: (1) If its revocation list fills, a
switch simply generates a new key; this invalidates all existing
capabilities that pass through it. It clears its revocation list, and
passes the new key to the DC. (2) The DC tracks the number
of revocations issued per sender. When this number crosses a predefined
threshold, the sender is removed from the service's ACLs.
If a switch uses a sender's capability to flood a receiver, thus eliciting a revocation, the sender can use a different capability (if it has one) to avoid the misbehaving switch. This occurs naturally because the client treats revocation--which results in an inability to get packets through--as a link failure, and it will try using a different capability instead. While well-behaved senders may have to use or request alternate capabilities, their performance degradation is only temporary, provided that there exists sufficient link redundancy to route around misbehaving switches. Therefore, using this approach, SANE networks can quickly converge to a state where attackers hold no valid capabilities and cannot obtain new ones.
By design, SANE switches have minimal functionality--much of which is likely to be placed in hardware--making remote compromise unlikely. Furthermore, each switch requires an authenticated public key, preventing rogue switches from joining the network. However, other avenues of attack, such as hardware tampering or supply-chain attacks, may allow an adversary to introduce a malicious switch. For completeness, therefore, we consider defenses against malicious switches attempting to sabotage network operation, even though the following attacks are feasible only in the most extreme threat environments.
Sabotaging MST Discovery.
By falsely advertising a smaller distance to the DC during MST
construction, a switch can cause additional DC traffic to be routed
through it. Nominally, this practice can create a path
inefficiency.
More seriously, a switch can attract traffic, then start dropping packets. This practice will result in degraded throughput, unless the drop rate increases to a point at which the misbehaving switch is declared failed and a new MST is constructed.
![]() |
In a more subtle attack, a malicious switch can selectively allow packets from its neighbors, yet drop all other traffic. An example of this attack is depicted in Figure 5: Node C only drops packets from node A. Thus, B does not change its forwarding path to the DC, as C appears to be functioning normal from its view. As a result, A cannot communicate with the DC, even though an alternate path exists through D. Note that this attack, at the MST discovery phase, precludes our normal solution for routing around failures--namely, using node-disjoint paths whenever possible--as node A has never registered with the DC in the first place.
From a high level, we can protect against this selective attack by hiding the identities of senders from switches en-route. Admittedly, it is unlikely that we can prevent all such information leakage through the various side-channels that naturally exist in a real system, e.g., due to careful packet inspection and flow analysis. Some methods to confound such attacks include (1) hiding easily recognizable sender-IDs from packet headers,Normally, DC packet headers contain a consistent sender-ID in cleartext, much like the IPSec ESP header. This sender-ID tells the DC which key to use to authenticate and decrypt the payload. We replace this static ID with an ephemeral nonce provided by the DC. Every DC response contains a new nonce to use as the sender-ID in the next message. (2) padding all response capabilities to the same length to hide path length, and (3) randomizing periodic messages to the DC to hide a node's scheduled timings.
Using these safeguards, if a switch drops almost all packets, its immediate neighbors will construct a new MST that excludes it. If it only occasionally drops packets, the rate of MST discovery is temporarily degraded, but downstream switches will eventually register with the DC.
Bad Link-State Advertisements.
Malicious switches can try to attract traffic by falsifying
connectivity information in link-state updates. A simple safeguard
against such attacks is for the DC to only add non-leaf edges to its
network map when both switches at either end have advertised the link.
This safeguard does not prevent colluding nodes from falsely advertising a link between themselves. Unfortunately, such collusion cannot be externally verified. Notice that such collusion can only result in a temporary denial-of-service attack when capabilities containing a false link are issued: When end hosts are unable to route over a false link, they immediately request a fresh capability. Additionally, the isolation properties of the network are still preserved.
Note that SANE's requirement for switches to initially authenticate themselves with the DC prevents Sybil attacks, normally associated with open identity-free networks [21].
Domain controllers are highly trusted entities in a SANE network. This can create a single point-of-failure from a security standpoint, since the compromise of any one DC yields total control to an attacker.
To prevent such a take-over, one can distribute trust among DCs using
threshold cryptography. While the full details are beyond the scope of
this paper, we sketch the basic approach.
We split the DCs' secret key across a few servers (say ), such
that two of them are needed to generate a capability. The sender then
communicates with
-out-of-
DCs to obtain the capability. Thus, an
attacker gains no additional access by compromising a single
DC.Implementing threshold cryptography for symmetric
encryption is done combinatorially [16]: Start from a
-out-of-
sharing (namely, encrypt a DC master secret under all
independent DC server keys) and then construct a
-out-of-
sharing
from it. To prevent a single malicious DC from revoking arbitrary
capabilities or, even worse, completely disconnecting a switch or
end host, the revocation mechanism (section 3.3) must also be
extended to use asymmetric threshold cryptography [20].
Given such replicated function, access control policy and service registration must be done independently with each DC by the end host, using standard approaches for consistency such as two-phase commit. When a new DC comes online or when a DC re-establishes communication after a network partition, it must have some means of re-syncing with the other DCs. This can be achieved via standard Byzantine agreement protocols [18].
All development was done in C++ using the Virtual Network System (VNS) [17]. VNS provides the ability to run processes within user-specified topologies, allowing us to test multiple varied and complex network topologies while interfacing with other hosts on the network. Working outside the kernel provided us with a flexible development, debug, and execution environment.
The network was in operational use within our group LAN--interconnecting seven physical hosts on 100 Mb Ethernet used daily as workstations--for one month. The only modification needed for workstations was to reduce the maximum transmission unit (MTU) size to 1300 bytes in order to provide room for SANE headers.
To support unmodified end hosts on our prototype network, we developed proxy elements which are positioned between hosts and the first hop switches. Our proxies use ARP cache poisoning to redirect all traffic from the end hosts. Capabilities for each flow are cached at the corresponding proxies, which insert them into packets from the end host and remove them from packets to the end host.
Our switch implementation supports automatic neighbor discovery, MST construction, link-state updates and packet forwarding. Switches exchange HELLO messages every 15 seconds with their neighbors. Whenever switches detects network failures, they reconfigure their MST and update the DC's network map.
The only dynamic state maintained on each switch is a hash table of capability revocations, containing the Cap-IDs and their associated expiration times.
We use OCB-AES [42] for capability construction and decryption with 128-bit keys. OCB provides both confidentiality and data integrity using a single pass over the data, while generating ciphertext that is exactly only 8 bytes longer than the input plaintext.
The DC consists of four separate modules: the authentication service, the network service directory, and the topology and capability construction service in the Protection Layer Controller. For authentication purposes, the DC was preconfigured with the public keys of all switches.
Capability construction.
For end-to-end path calculations when constructing capabilities, we use
a bidirectional search from both the source and destination. All
computed routes are cached at the DC to speed up subsequent capability
requests for the same pair of end hosts, although cached routes are
checked against the current topology to ensure freshness before re-use.
Capabilities use 8-bit IDs to denote the incoming and outgoing switch ports. Switch IDs are 32 bits and the service IDs are 16 bits. The innermost layer of the capability requires 24 bytes, while each additional layer uses 14 bytes. The longest path on our test topologies was 10 switches in length, resulting in a 164 byte header.
Service Directory. DNS queries for all unauthenticated users
on our network resolve to the DC's IP address, which hosts a simple
webserver. We provide a basic HTTP interface to the service directory.
Through a web browser, users can log in via a simple web-form and can
then browse the service directory or, with the appropriate
permissions, perform other operations (such as adding and deleting
services).
The directory service also provides an interface for managing users and groups. Non-administrative users are able to create their own groups and use them in access-control declarations.
To access a service, a client browses the directory tree for the desired service, each of which is listed as a link. If a service is selected, the directory server checks the user's permissions. If successful, the DC generates capabilities for the flows and sends them to the client (or more accurately, the client's SANE IP proxy). The web-server returns an HTTP redirect to the service's appropriate protocol and network address, e.g., ssh://192.168.1.1:22/. The client's browser can then launch the appropriate application if one such is registered; otherwise, the user must do so by hand.
As a concrete example, we describe the events for an ssh session initiated within our internal network. All participating end hosts have a translation proxy positioned between them and the rest of the network. Additionally, they are configured so that the DC acts as their default DNS server.
Until a user has logged in, the translation proxy returns the DC's IP address for all DNS queries and forwards all TCP packets sent to port 80 to the DC. Users opening a web-browser are therefore automatically forwarded to the DC so that they may log in. This is similar in feel to admission control systems employed by hotels and wireless access points. All packets forwarded to the DC are accompanied by a SANE header which is added by the translation proxy. Once a user has authenticated, the DC caches the user's location (derived from the SANE header of the authentication packets) and associates all subsequent packets from that location with the user.
Suppose a user ssh's from machine to machine
.
will issue a DNS
request for
. The translation proxy will intercept the DNS packet (after
forging an ARP reply) and translate the DNS requests to a capability request
for machine
. Because the the DNS name does not contain an indication of
the service, by convention we prepend the service name to the beginning of the
DNS request (e.g. ssh ssh.B.stanford.edu). The DC does the permission check
based on the capability (initially added by the translation proxy) and
the ACL of the requested service.
If the permission check is successful, the DC returns the capabilities
for the client and server, which are cached at the translation proxy.
The translation proxy then sends a DNS reply to with a unique
destination IP address
, which allows it to demultiplex subsequent
packets. Subsequently, when the translation proxy receives packets from
destined to
, it changes
to the destination's true IP address
(much like a NAT) and tags the packet with the appropriate SANE
capability.
Additionally, the translation proxy piggybacks the return capability
destined for the server's translation proxy on the first packet.
Return traffic from the server to the client is handled similarly.
|
Table 1 shows the performance of the DC (in capabilities per second) and switches (in Mb/s) for different capability packet sizes (i.e., varying average path lengths). All tests were done on a commodity 2.3GHz PC.
As we show in the next section, our naive implementation of the DC performs orders of magnitude better than is necessary to handle request traffic in a medium-sized enterprise.
The software switches are able to saturate the 100Mb/s network on which we tested them. For larger capability sizes, however, they were computationally-bound by decryption--99% of CPU time was spent on decryption alone--leading to poor throughput performance. This is largely due to the use of an unoptimized encryption library. In practice, SANE switches would be implemented in hardware. We note that modern switches, such as Cisco's catalyst 6K family, can perform MAC level encryption at 10Gb/s. We are in the process of re-implementing SANE switches in hardware.
One potential concern with SANE's design is the centralization of function at the Domain Controller. As we discuss in Section 3.5, the DC can easily be physically replicated. Here, we seek to understand the extent to which replication would be necessary for a medium-sized enterprise environment, basing on conclusions on traffic traces collected at the Lawrence Berkeley National Laboratory (LBL) [36].
The traces were collected over a 34-hour period in January 2005, and cover about 8,000 internal addresses. The trace's anonymization techniques [37] ensure that (1) there is an isomorphic mapping between hosts' real IP addresses and the published anonymized addresses, and (2) real port numbers are preserved, allowing us to identify the application-level protocols of many packets. The trace contains almost 47 million packets, which includes 20,849 DNS requests and 145,577 TCP connections.
![]() |
![]() |
![]() |
Figure 6 demonstrates the DNS request rate, TCP connection establishment rate, and the maximum number of concurrent TCP connections per second, respectively.
The DNS and TCP request rates provide an estimate for an expected rate of DC requests by end hosts in a SANE network. The DNS rate provides a lower-bound that takes client-side caching into effect, akin to SANE end hosts multiplexing multiple flows using a single capability, while the TCP rate provides an upper bound. Even for this upper bound, we found that the peak rate was fewer than 200 requests per second, which is 200 times lower than what our unoptimized DC implementation can handle (see Table 1).
Next, we look at what might happen upon a link failure, whereby all end hosts communicating over the failed link simultaneously contact the DC to establish a new capability. To understand this, we calculated the maximum concurrent number of TCP connections in the LBL network.To calculate the concurrent number of TCP connections, we tracked srcip:srcport:dstip:dstport tuples, where a connection is considered finished upon receiving the first FIN packet or if no traffic packets belonging to that tuple are seen for 15 minutes. There were only 143 cases of TCP packets that were sent after a connection was considered timed-out. We find that the dataset has a maximum of 1,111 concurrent connections, while the median is only 27 connections. Assuming the worst-case link failure--whereby all connections traverse the same network link which fails--our simple DC can still manage 40 times more requests.
Based on the above measurements, we estimate the bandwidth consumption of control traffic on a SANE network. In the worst case, assuming no link failure, 200 requests per second are sent to the DC. We assume all flows are long-lived, and that refreshes are sent every 10 minutes. With 1,111 concurrent connections in the worst case, capability refresh requests result in at most an additional 2 packets/s.This is a conservative upper bound: In our traces, the average flow length is 92s, implying that at most, 15% of the flows could have lengths greater than 10 minutes. Given header sizes in our prototype implementation and assuming the longest path on the network to be 10 hops, packets carrying the forward and return capabilities will be at most 0.4 KB in size, resulting in a maximum of 0.646 Mb/s of control traffic.
This analysis of an enterprise network demonstrates that only a few domain controllers are necessary to handle DC requests from tens of thousands of end hosts. In fact, DC replication is probably more relevant to ensure uninterrupted service in the face of potential DC failures.
The desire for a mechanism that supports ubiquitous enforcement, topology independence, centralized management, and meaningful end-point identifiers has lead to the development of distributed firewalls [14,26,2]. Distributed firewalls share much with SANE in their initial motivation but differ substantially in their trust and usage model. First, they require that some software be installed on the end host. This can be beneficial as it provides greater visibility into end host behavior, however, it comes at the cost of convenience. More importantly, for end hosts to perform enforcement, that end host must be trusted (or at least some part of it, e.g., the OS [26], a VMM [22], the NIC [31], or some small peripheral [40]). Furthermore, in a distributed firewall scenario, the network infrastructure itself receives no protection, i.e., the network is still ``on'' by default. This design affords no defense-in-depth if the end-point firewall is bypassed, as it leaves all other network elements (e.g., switches, middleboxes, and unprotected end hosts) exposed.
Weaver et al. [45] argue that existing configurations of coarse-grain network perimeters (e.g., NIDS and multiple firewalls) and end host protective mechanisms (e.g. anti-virus software) are ineffective against worms, both when employed individually or in combination. They advocate augmenting traditional coarse-grain perimeters with fine-grain protection mechanisms throughout the network, especially to detect and halt worm propagation.
Finally, commercial offerings from Consentry [3] introduce special-purpose bridges for enforcing access control policy. To our knowledge, these solutions require that the bridges be placed at a choke point in the network so that all traffic needing enforcement passes through them. In contrast, SANE permission checking is done at a central point only on connection setup, decoupling it from the data path. SANE's design both allows redundancy in the network without undermining network security policy and simplifies the forwarding elements.
Dealing with Routing Complexity.
Often misconfigured routers make firewalls simply irrelevant by routing
around them. The inability to reason about connectivity in complex
enterprise networks has fueled commercial offerings such as those of
Lumeta [5], to help administrators discover what connectivity
exists in their network.
In their 4D architecture, Rexford et al. [41,24] argue that the decentralized routing policy, access control, and management has resulted in complex routers and cumbersome, difficult-to-manage networks. Similar to SANE, they argue that routing (the control plane) should be separated from forwarding, resulting a very simple data path. Although 4D centralizes routing policy decisions, they retain the security model of today's networks. Routing (forwarding tables) and access controls (filtering rules) are still decoupled, disseminated to forwarding elements, and operate the basis of weakly-bound end-point identifiers (IP addresses). In our work, there is no need to disseminate forwarding tables or filters, as forwarding decisions are made a priori and encoded in source routes.
Predicate routing [43] attempts to unify security and routing by defining connectivity as a set of declarative statements from which routing tables and filters are generated. SANE differs, however, in that users are first-class objects--as opposed to end-point IDs or IP addresses in Predicate routing--and thus can be used in defining access controls.
Expanding the Link-layer.
Reducing a network from two layers of connectivity to one, where all
forwarding is done entirely at the link layer, has become a popular
method of simplifying medium-sized enterprise networks. However, large
Ethernet-only networks face significant problems with scalability,
stability, and fault tolerance, mainly due to their use of broadcast and
spanning-tree-based forwarding.
To address these concerns, several proposals have suggested replacing the MST-based forwarding at the link-layer with normal link-state routing [39,35]. Some, such as Myers et al. [35], advocate changing the Ethernet model to provide explicit host registration and discovery based on a directory service, instead of the traditional broadcast discovery service (ARP) and implicit MAC address learning. This provides better scalability and transparent link-layer mobility, and it eliminates the inefficiencies of broadcast. Similarly, SANE eliminates broadcast in favor of tighter traffic control through link-state updates and source routes. However, we eschew the use of persistent end host identifiers, instead associating each routable destination with the switch port from where it registered.
Capabilities for DDOS prevention.
Much recent work has focused on DoS remediation through network enforced
capabilities on the WAN [12,51,52].
These systems assumes no cooperation between network elements, nor do
they have a notion of centralized control. Instead, clients receive
capabilities from servers directly and vice versa. Capabilities are
constructed on-route by the initial capability requests. This offers a
very different policy model than SANE, as it is designed to meet
different needs (limiting wide area DoS) and relies on different
operating assumptions (no common administrative domain).
We set out to design a network that greatly limits the ability of an end host or switch to launch an effective attack, while still maintaining flexibility and ease of management. Drastic goals call for drastic measures, and we understand that our proposal--SANE--is an extreme approach. SANE is conservative in the sense that it gives the least possible privilege and knowledge to all parties, except to a trusted, central Domain Controller. We believe that this would be an acceptable practice in enterprises, where central control and restricted access are common.
Yet SANE remains practical: Our implementation shows that SANE could be deployed in current networks with only a few modifications, and it can easily scale to networks of tens of thousands of nodes.
Last changed: 21 June 2006 jel |