Gigabit Communications over Storage Area Networks

Aloke Guha
Storage Technology Corporation
guha@network.com; aloke_guha@stortek.com

The increasing use of distributed computing has resulted in higher density 
of access to data and a concomitant growth in the volume of storage. To 
avoid the limits of bus based server-storage connections, a storage area 
network (SAN) infrastructure has evolved to provide connectivity between 
storage devices and their application clients. One obvious difference 
between a SAN and a LAN is that SAN nodes are not peer devices but are 
either data/storage clients or storage resources. This distinction allows 
SANs to aggregate resources across the network and build scalable shared 
storage repositories [1]. 

The de facto SAN standard is based on the Fibre Channel (FC) protocol 
whose link bandwidth is 800 Mbs (100 MBs) with planned introduction of 
multiples of the base rate. Both switched and shared media (arbitrated 
loop) topologies are possible in FC. Besides supporting heterogeneous 
nodes, there are two key aspects in which SANs differ from 
internetworking: reliability and efficiency in large data movement. 
Storage networks are typically designed to transfer enterprise or mission-
critical data such as in backup operations. Unlike, most internetworking 
applications, backups impose hard real-time deadlines, so a SAN has to 
guarantee reliable data delivery. A FC-based SAN therefore does not rely 
on TCP but uses connection-oriented services and credit-based flow control 
to provide guarantees. 

SANs are often used to transfer large-block data as in backups that have 
to complete within a fixed time. This requires that SANs not only provide 
a high bandwidth link but also use efficient data link protocols. FC is 
therefore designed to work with large frame and very large burst sizes in 
both connection-oriented and connectionless modes. Besides moving bulk 
data, SANs are also evolving to provide file-level access [2]. In such a 
scenario, a client can directly read the file over a SAN from a storage 
device on the SAN, or alternately, make a file request to a file server 
that directs the connection and the transfer between the client and 
storage device. The primary advantages of using the SAN are eliminating 
the need for a single high-performance server and exploiting the available 
SAN bandwidth. 

Early empirical results on SAN performance are now being compiled. Results 
depend highly on the host bus adapter (HBA), the server I/O bandwidth, and 
the specific operations on the storage device. For example, sequential 
reads and writes to disk show higher performance than random operations. 
Similarly, as I/O block sizes increase, the client to disk throughput 
increases. Also, because of reduced dependence on the host processing, 
native FC performance has a significant advantage over IP over FC. A Sun 
10K with 2 FC HBAs starts saturating at around 240 Mbs while similar hosts 
with native FC can exhibit more than twice that rate. In native mode, we 
have measured data rates of over 920 Mbs between a host and RAID over the 
SAN. Clearly, the potential for moving very large amounts of data over a 
32-port switched SAN exists even today.

[1] A. Guha, "The Evolution to Intelligent Storage Area Networking," 
Infostor, August 1998, p. 24.
[2] M. O'Keefe, "Shared File Systems and Fibre Channel," Proc. 6th NASA 
Goddard Space Flight Center Conference on Mass Storage Systems and Tech., 
March 1998.