Scalable ethernet clos-switches

Abstract

Scalability of Cluster-Computers utilizing Gigabit-Ethernet as an interconnect is limited by the unavailability of scalable switches that provide full bisectional bandwidth. Clos' idea of connecting small crossbar-switches to a large, non-blocking crossbar - wide-spread in the field of high-performance networks - is not applicable in a straightforward manner to Ethernet fabrics. This paper presents techniques necessary to implement such large crossbar-switches based on available Gigabit-Ethernet technology. We point out the ability to build GigabitEthernet crossbar switches of up to 1152 ports providing full bisectional bandwidth. The cost of our configuration is at about €125 per port, with an observed latency of less than 10μsec. We were able to find a bidirectional point-to-point throughput of 210 MB/s using the ParaStation Cluster middle-ware[2]. © Springer-Verlag Berlin Heidelberg 2006.