Re: Switchless Opteron clusters

"David Kelly" <[EMAIL PROTECTED]> wrote in message news:<[EMAIL PROTECTED]>...
> I haven't seen a switchless architecture like this. Have you looked at the
> OctigaBay systems (www.octigabay.com) ? - They appear to be a switched
> architecture, directly connecting processors together, but I can't find
> sufficient technical detail to get a good grip on the pros and cons of the
> design.....

I talked to them at SC2003 in Phoenix.  They are connecting their CPU
boards together using a Mellanox InfiniBand switch chip.  They built
their own HT->IB interface for the Opterons to get to the IB fabric. 
I don't know if it's an FPGA or an ASIC.  However their HT->IB chip
doesn't use IB verbs, so they wrote their own MPI stack.  They claim
better latency numbers than solutions using InfiniBand HCAs (ie the
Mellanox HCA).  However they won't have the full gamut of protocols
that real HCAs support (you probably won't be able to use
Topspin/Infinicon ethernet or fibre channel gateways to get out of the
cluster).