Discussion on:

Choosing iSCSI over fibre channel: Deciding factors

http://www.mcdata.com/products/hardware/iswitch/index.html
http://www.emc.com/products/systems/clariion/ax100/index.jsp
A good idea
I'd recommende IP Storage Router from McData (see first URL)
and EMC AX100i (second URL)
Slava

If the enterprise can live with bandwidths allowed by GigE and TCP/IP, ISCSI's moderate performance and low price are attractive. However, when the problem calls for SAN over 2 Gb fibre channel (FC) the selction is obvious. In our environment 2 Gb FC is barely adequate, even at 98% of channel rate, so we anxiously anticipate the common availability of 4 Gb FC. In our case, Speed is required for progress.

Do you know for a certainty that your bottleneck is the HBA? Assiming RAID5, sounds as though you are not exploiting all the bandwidth available to you. Are you running multiple paths to the same device? If so, are you using software that allows you to balance I/O across the HBAs to the LUNS? Perhaps you could add more HBAs unless your storage device is already maxed out. Depending on the manufacturer, if that was the case, simply updating the cache on the array would allow you more data transfer. Or, hate to say it, but you might just need another array to add to your SAN to 'spread the wealth' of your increasing storage requirements.

If you are having trouble coping with 2 GB FC then why not look at 10G iSCSI? There aren't a lot of options out there but the bandwidth is if you design correctly.

Yes 10Gbit ethernet is hitting the market. What most people miss however is the unbelievable difference in how two technologies that share a signalling protocol can go in such different directions. 10 Gbit ethernet does NOT provide a performance advantage over even 1 Gbit half-duplex Fibre Channel for the vast majority of block IO operations.

Fibre Channel of any speed is designed for large block IO operations. Ethernet is designed for small packet handling. Ignoring ALL other issues (and there are many) using ethernet for even moderately intensive disk IO operations puts enormous strain on your systems. Each 2-4K packet will require an interrupt and a scenic tour of your server to be transmitted. FC on the other hand will pump out up to 100MB+ in a single IO operation. Hence the CPU, RAM, etc. that you paid money for to run an application actually does that rather than managing network adapters and protocol stacks.

Don't get me started on the missed promise of TOE cards and how they typically make the situation worse (OSs typically still see the connection as an ethernet stack and pass the traffic through that stack prior to the card getting involved and duplicating the effort). RMA devices may help or just be another series of alphabet soup marketing to take your money.

Add in latency issues, application error handling of disk access issues, congstion, etc. and iSCSI doesn't seem quite as useful for anything over low use, high latency applications.

Putting any level of iSCSI solution in the path of a moderately active database is a recipe for disaster.

Know your applications, know their requirements and plan according to those, not the fluff in your trade rag.

As someone who's written some of what yoiu call fluff let me say:

Yes FC is deterministic and yes better for large block xfers but some of what you say is just wrong.

Adaptec and Qlogic'siSCSI HBAs use the same storport minidrivers as FC HBAs so data doesn't flow through OS stack.

My benchmarks have shown Equallogic's Peerstorage array running as fast or faster than FC solutions costing 3 times as much.

Do I want my biggest, baddest most disk I/O bound server on iSCSI. Not this year. Do I want my Exchange servers and 90% of the windows servers YES

HMarks,

I'd love to see any documentation on that. The last authoritative review on this I saw earlier this year showed that software iscsi was still outperforming hardware accelerated due to the OSs using both the IP stack and then passing to storage drivers (both linux and windows). Effectively running through both worlds.

The comments back were that this would continue to be a problem on windows until the chimney minidriver was released and a comperable linux solution developed.

And my understanding is that that project is now mired in licensing issues.

While I can see vastly improved performance if that architectural mess has been resolved. You are going to have to work hard convince me that iSCSI is going to win anything but heavily biased competitions.

iSCSI is cheaper and matches some workloads nicely. Anything mission critical, moderately used or even hints at a transactional databases are not in that list.

I don't really understand your comments. I really need to see any empirical assesments you have made.
Bandwidths allowed? What are you doing with the data? How do you push it to the node or application that requested it? TeraE?

Moderate performance? Please elucidate. Properly designed iSCSI SAN's with TOE or intelligent off-load blow FC away at a fraction of the capital investment and support costs.

Barely adequate? Consider distributing storage physically and logically closer to the requesting endpoint.

>>Speed is required for progress.
Well then, give 'em speed (effective throughput). Raw speed of nascent technologies coupled with questions of interoperability are a tough sell.

I also think that choosing iscsi over fc is a very good idea.
I personnaly have experience with de MDS9216i from cisco. Because most of the sites already use cisco switches it is easier to use because the command structure is the same.
I also recommend the EVA3000 or 5000 from HP Storage works because of the stability and the high mtbf
jan

... the next installation of the article. I've been interested in researching the differences between iSCSI and FC for myself. Hopefully your experiences will greatly reduce my own further research efforts. After all, there's so much to learn about everything and definitely not enough time.

Thanks.

I was recently given a fc unit from a friend of
mine who had sold his bussiness. I don't what
exactly he ran the thing with (he may just have
been selling them) but could forsee some major
problems for any organization with out a strong
unix department. After all it doesn't seem like
you have a "mainframe" backbone that would
utilize the fabric, which is where any benefit
would come from. I mean like I said the guy sold
his bussiness and I don't think he sold it poor.

After checking around, NetApp storage lets me the same storage for iSCSI or FC - no decision necessary.

For my network, iSCSI may be the perfect solution. We don't do a lot of app serving. Our big storage needs are for files, Exchange and SQL. Lefthand sells a nice setup. I'm sure that more solutions will start to appear.

iSCSI has been around for a few years now and I've been advising and setting it up at various customer sites. Technically I consider it to be superior to FC. Due to the ethernet technoolgy stepping into the 10Gb/s now, I think FC has had its day.
The main problem seems to be to the scepsis of people used to FC technology: "If it is 30% of the cost you'll get 30% of the speed"
In my experience the technolgies main advantage is its simple and straightforward approach. No need to learn FC and FC swtiches and the problems with redundant FC paths where you mistakenly see twice the capacity represented, just get on with designing the storage units and focus on that.

My advise:
Always use a good quality Gb/s switch with bonding capabilities and try to keep it separated from the normal non-storage network traffic. (You don't want network problems, such as MAC loops; broadcast storms etc, stopping your storage)
If you need more throughput, spend the money on extra disks, not on your FC.
If your network is starting to block, simply switch to using network boards having 4 ports (Intel Kingsport).
Keeps your SANs "grouped", don't go for a full enterprise unit thing at once, but rather opt for multiple smaller linked SAN's. It will improve performance and hardly impact granularity and manageability.
And if you cannot convince your peers: Use a SAN system that can accomodate both. Start with iSCSI and if it proves to slow you can make it into FC. (Although slowness usually stays as it inevitably is a storage design issue, hardly ever a transport issue).

I currently like to link multiple Intel Mt. Jefferson SANs (sell under various OEM names) especially as it uses cheaper SATA instead of SCSI for the drives. Combine it with their blades solutions. All the nice stuff; snapshots, SAN replication, virtualisation of multiple units etc. is in there.