Just to clarify, thanks to Peter who messaged me about iSCSI MPIO on the Netapp.  As per Peter, its not really needed because when a head in a clustered unit fails, the good head will take over the IP addresses for the failed head and will service subsequent requests. 

When we were setting up one of our Netapps we experienced really slow network performance.  Initial connections took about 5 seconds to initiate and after the connection was initiated, through-put was fine.  

After some packet captures we found out that our HP switch was putting the MAC address of the wrong gateway in the packet.  Netapp has an option called ip.fastpath.enable that tells the IP stack to respond using the source MAC address instead of the default gateway.  This allows the Netapp to circumvent bouncing around a subnet in case the packet came into the network on a link other then the default gateway.

If you experience this, first fix your switches!  If that's not an option, then submit the following command in a Telnet or SSH session to your Netapp filer:

options ip.fastpath.enable off

Cheers for now, more next week,

Chris