| Author |
Message |
Al Dykes
Guest
|
 Posted:
Mon Feb 28, 2005 9:32 pm Post subject:
Re: 120 Hosts Running GigE at Wire Speed Minimum Cost |
|
|
In article <1126gtv5bra9lf3@news.supernews.com>,
T. Sean Weintz <strap@hanh-ct.org> wrote:
| Quote: | Randy Howard wrote:
You can achieve the same thing with multicast load
on IGMP switches, they'll work for a brief period
sending the stream only to subscribed ports, then
suddenly start flooding the traffic to all the ports.
I can vouch for GSM7312's (and gsm7324's) doing this. Their layer 2
stuff as well - gsm712, fsm726s, etc. Do NOT attempt to push out a GHOST
image over multicast from a gig host if you use these - the switch WILL
melt down.
|
What's the symptom ? Lots of dropped packets ? Lockup ?
(I'm assuming the references to smoke in this thread are
metaphorical.)
--
a d y k e s @ p a n i x . c o m
Don't blame me. I voted for Gore. |
|
| Back to top |
|
 |
Randy Howard
Guest
|
| Posted:
Mon Feb 28, 2005 10:21 pm Post subject:
Re: 120 Hosts Running GigE at Wire Speed Minimum Cost |
|
|
In article <1126gtv5bra9lf3@news.supernews.com>, strap@hanh-ct.org
says...
| Quote: | Randy Howard wrote:
You can achieve the same thing with multicast load
on IGMP switches, they'll work for a brief period
sending the stream only to subscribed ports, then
suddenly start flooding the traffic to all the ports.
I can vouch for GSM7312's (and gsm7324's) doing this. Their layer 2
stuff as well - gsm712, fsm726s, etc. Do NOT attempt to push out a GHOST
image over multicast from a gig host if you use these - the switch WILL
melt down.
|
MOST IGMP switches will do this. It's very hard to find one that
will not.
--
Randy Howard (2reply remove FOOBAR)
"Making it hard to do stupid things often makes it hard
to do smart ones too." -- Andrew Koenig |
|
| Back to top |
|
|
Randy Howard
Guest
|
| Posted:
Mon Feb 28, 2005 10:43 pm Post subject:
Re: 120 Hosts Running GigE at Wire Speed Minimum Cost |
|
|
In article <cvvh3o$nhh$1@panix5.panix.com>, adykes@panix.com says...
| Quote: | In article <1126gtv5bra9lf3@news.supernews.com>,
T. Sean Weintz <strap@hanh-ct.org> wrote:
Randy Howard wrote:
You can achieve the same thing with multicast load
on IGMP switches, they'll work for a brief period
sending the stream only to subscribed ports, then
suddenly start flooding the traffic to all the ports.
I can vouch for GSM7312's (and gsm7324's) doing this. Their layer 2
stuff as well - gsm712, fsm726s, etc. Do NOT attempt to push out a GHOST
image over multicast from a gig host if you use these - the switch WILL
melt down.
What's the symptom ? Lots of dropped packets ? Lockup ?
|
The symptom is mostly described above. Flooding all ports, combined
with management ports or interfaces becoming unusable during this
activity.
| Quote: | (I'm assuming the references to smoke in this thread are
metaphorical.)
|
Usually. Although, I have seen one switch literally cooked in
the sense that the normal green power LED to indicate happiness
switched to red and the switch shut down. Subsequent attempts to
revive it were useless. It literally died from overload.
--
Randy Howard (2reply remove FOOBAR)
"Making it hard to do stupid things often makes it hard
to do smart ones too." -- Andrew Koenig |
|
| Back to top |
|
|
jpd
Guest
|
| Posted:
Mon Feb 28, 2005 11:20 pm Post subject:
Re: 120 Hosts Running GigE at Wire Speed Minimum Cost |
|
|
Begin <MPG.1c8d0f2cb1d7b88f98a100@news.verizon.net>
On 2005-02-28, Randy Howard <randyhoward@FOOverizonBAR.net> wrote:
| Quote: | In article <1126gtv5bra9lf3@news.supernews.com>, strap@hanh-ct.org
[give a switch a decent load and...]
[...] - the switch WILL
melt down.
MOST IGMP switches will do this. It's very hard to find one that
will not.
|
Ok, now I'm intrigued. Which ones are known to _not_ do this? I can make
a couple of guesses (cisco, extreme, hp, alphabetical order), but I
haven't actually tried or anything. Thoughts? Experiences?
--
j p d (at) d s b (dot) t u d e l f t (dot) n l . |
|
| Back to top |
|
|
T. Sean Weintz
Guest
|
| Posted:
Tue Mar 01, 2005 3:00 am Post subject:
Re: 120 Hosts Running GigE at Wire Speed Minimum Cost |
|
|
jpd wrote:
| Quote: |
Ok, now I'm intrigued. Which ones are known to _not_ do this? I can make
a couple of guesses (cisco, extreme, hp, alphabetical order), but I
haven't actually tried or anything. Thoughts? Experiences?
|
I would be annoyed if a Nortel Passport 8600 did this. |
|
| Back to top |
|
|
Guest
|
| Posted:
Tue Mar 01, 2005 4:11 am Post subject:
Re: 120 Hosts Running GigE at Wire Speed Minimum Cost |
|
|
Randy Howard <randyhoward@FOOverizonBAR.net> wrote:
| Quote: | How do you "just cable 120 machines together" without switches ???
|
Machine 1 to machine 2 with a crossover cable(*)
[...]
machine 119 to machine 120 with a crossover cable.
Agree ahead of time that it's not a useful scenario, though they will
get "wire-speed". 8*)
(*) IIRC, a 1000BaseT crossover is a regular straight-thru cable, but
that's a detail. |
|
| Back to top |
|
|
Will
Guest
|
| Posted:
Tue Mar 01, 2005 6:04 am Post subject:
Re: 120 Hosts Running GigE at Wire Speed Minimum Cost |
|
|
Thank you for your excellent post, which included enough detail on the Cisco
product to tell me that (as usual) the truth is complex and we will probably
need to do the grunt work to detail out the requirements. While I
appreciate the need to do that, you will have to trust me that there are
many organizations who think they can just spend their way out of any design
problem, and they end up just not doing design.
I remember one situation where a company bought a mainframe upgrade for $1M
to speed up a key application. The application continued to be slow. I
did performance tuning to discover the bottleneck. What I found was that
their database vendor had a blocking queue on reports against the database.
Only one user at a time was allowed to run reports!! None of this was
documented. It was something they only admitted when we confronted them
with the data. No amount of additional CPU would have changed the
processing time.
It's difficult to believe that managers who control money in large companies
would spend $1M (or more) rather than spend $10K to just have someone think
and research to define what problem they really need to solve. But I see
exactly that all of the time. And I have given up on changing the world.
The world is largely run by people who act on gut instinct, and many of
those people get extremely offended when you point out that someone needs to
write requirements. They usually say something dismissive like "Well
that's what we have done!" or "We have excellent people and we know our
problem, now are you going to help me solve it?" It does no good to point
out that the chicken scratches on some chalkboard aren't requirements.
Some organizations fundamentally don't understand how to write requirements,
or how to analyze requirements.
If we could get a very large switch that really could do wire speed on 120
hosts at wire speed simultaneously cheaply, then you don't need to work out
the details of what the actual throughput would be on individual
workstations for purposes of the network design. To the extent that the
network scales to X usage on all ports simultaneously, you are covered by
network capacity if you actually use 25% of X. Intentional overdesign is
not a bad thing if it does not substantially change cost. When I see a 10
slot Extreme BlackDiamond switch (which claims 384 Gbps backplane speed)
selling for next to nothing, you at least wonder if overdesign might not
cost much. After reading your post, the first thought I have is that the
Extreme product probably has a dozen board-level bottlenecks that they do
not disclose. I would be interested in any feedback from anyone who has
tried to stress that product, particularly using its layer 3 capabilities.
As far as file servers go, the thought was that they would end up using
either PCI Express bus servers with quad gigE cards, or just increase the
number of servers using multiple single port cards. The number of server
ports they end up needing obviously does in turn affect the number of
network ports you need. And if you need to now cross connect two large
switches, how you do that without buying expensive new 10 Gbps technology
becomes a problem.
--
Will
"Walter Roberson" <roberson@ibd.nrc-cnrc.gc.ca> wrote in message
news:cvrq7e$bkj$1@canopus.cc.umanitoba.ca...
| Quote: | If your setup is such that there could be N simultaneous connections
to M servers, and N > M and you are asking us for a design in which
"the network itself is not a bottleneck", then you have an implicit
requirement that the server port must be able to operate at
somewhere between (ceiling(N/M) * 1 Gbps) and (N * 1 Gpbs), depending
on the traffic patterns. We have to know what that peak rate is
in order to advise you on the correct switch. Current off-the-shelf
technologies get you 1 Gbps interfaces on a wide range of
devices, 10 Gbps XENPAK interfaces on a much lesser range of devices;
2 Gbps interfaces are also available in some models -- but if that's
your spec then we need to know so that we rule out devices that
can't handle that load. |
|
|
| Back to top |
|
|
David Ross
Guest
|
| Posted:
Tue Mar 01, 2005 8:02 am Post subject:
Re: 120 Hosts Running GigE at Wire Speed Minimum Cost |
|
|
| Quote: | Most of the low-cost high port-count switches (24,
48) will not take kindly to you trying to run all
the ports wide open simultaneously. This has nothing
to do with the presence or absence of a published claim
to be a non-blocking switch.
|
What do you call "low-cost" or better yet what's the cost floor at which
you'd not expect to melt down? |
|
| Back to top |
|
|
Randy Howard
Guest
|
| Posted:
Wed Mar 02, 2005 3:00 pm Post subject:
Re: 120 Hosts Running GigE at Wire Speed Minimum Cost |
|
|
In article <38h5niF5n56k2U1@individual.net>,
read_the_sig@do.not.spam.it.invalid says...
| Quote: | Begin <MPG.1c8d0f2cb1d7b88f98a100@news.verizon.net
On 2005-02-28, Randy Howard <randyhoward@FOOverizonBAR.net> wrote:
In article <1126gtv5bra9lf3@news.supernews.com>, strap@hanh-ct.org
[give a switch a decent load and...]
[...] - the switch WILL
melt down.
MOST IGMP switches will do this. It's very hard to find one that
will not.
Ok, now I'm intrigued. Which ones are known to _not_ do this? I can make
a couple of guesses (cisco, extreme, hp, alphabetical order), but I
haven't actually tried or anything. Thoughts? Experiences?
|
It's mostly the "low cost" ones that smaller companies buy because
they afraid of all the zeros in the price of the serious vendors.
By and large, you can tell just by looking, because they all have the
same chassis, just different colored paint and logos attached. The
bulk of these switches (and increasingly some of the name brand
switches) are all implemented and built by a a few Taiwan companies.
Once you've found one with the problem, you can find several more
just by cracking the cases open and looking at the internals when
the chassis looks the same.
In general, anything made by Accton (and OEM'd to a bunch of vendors)
is not worth the time it takes to plug in the power cord. You just
never know what will happen with their stuff. If you've ever had
access to their stuff before it is gone RTS, you'll understand,
otherwise you wouldn't believe it possible. Even the stuff that
does get shipped is often riddled with bugs, but in areas that they
hope the majority of their customers will never see in practice.
Some of the stuff made by Delta is good, some is not. In general,
they're a notch above Accton most of the time.
Don't be fooled into thinking that the name brands you know of
build their own gear. Almost none of them do. Many don't even
stick with a consistent vendor from model to the next.
I've even seen some of the supposedly "good" gear, like HP Procurve
switches do flaky stuff, like not even handling WOL packets
correctly. In general, every switch (not just vendor) is different
and you have to look at each one individually to be sure.
The other aspect is that for the small businesses just using them
as modern "paper cup and string" links between PCs and to browse
the web, they usually will never notice the issues we're talking
about, so there is no real reason for them to buy the higher end
geear.
--
Randy Howard (2reply remove FOOBAR)
"Making it hard to do stupid things often makes it hard
to do smart ones too." -- Andrew Koenig |
|
| Back to top |
|
|
Randy Howard
Guest
|
| Posted:
Wed Mar 02, 2005 3:08 pm Post subject:
Re: 120 Hosts Running GigE at Wire Speed Minimum Cost |
|
|
In article <-LadnU8WJ_RamrnfRVn-rg@portbridge.com>, news02
@raleighthings.com says...
| Quote: | Most of the low-cost high port-count switches (24,
48) will not take kindly to you trying to run all
the ports wide open simultaneously. This has nothing
to do with the presence or absence of a published claim
to be a non-blocking switch.
What do you call "low-cost" or better yet what's the cost floor at which
you'd not expect to melt down?
|
I'm not sure you can simplify it that far. I've seen the same
EXACT switch sold from 2 or 3 different vendors, with price
differences greater than the sales price of the lowest one.
You can't go by just brand name, as the "brands" are really
nothing more than "chassis colors" in a lot of cases, with
the guts all coming from the same place.
The good news is if you buy one of the big, serious switches,
then you know you can at least get your salesman's attention
if/when you run into trouble. Unfortunately, there is often
a HUGE price difference between their stuff and 10 or 15
competing products at a much lower price point. Some of those
lower cost products are decent, some are not. You have to
look at each one.
So, what to do? If you plan on buying a dozen 24 port switches,
pick one you think does what you need at a decent price, buy
ONE of them from someplace with a reasonable return policy, and
put it through a trial by fire for a few weeks with everything
you can think to throw at it. If you do multicasting, then
send as much of that through it as you can. If you think you'll
be running the switch wide open, then do that. I've seen
switches reboot due to firmware bugs when performance counters
overflow internally. So send it long streams of data, for days
or weeks and make sure it doesn't roll over. If it makes it,
order 11 more. If not, return it and start over.
You cannot rely on third party testing and "certifications" to
guarantee they work. Many of those are technically only as
challenging as the vendor writing the check to the test house
so they can be listed on a website with some gold star next
to the model number. It's a joke, but not a funny one.
--
Randy Howard (2reply remove FOOBAR)
"Making it hard to do stupid things often makes it hard
to do smart ones too." -- Andrew Koenig |
|
| Back to top |
|
|
T. Sean Weintz
Guest
|
| Posted:
Wed Mar 02, 2005 10:08 pm Post subject:
Re: 120 Hosts Running GigE at Wire Speed Minimum Cost |
|
|
Randy Howard wrote:
| Quote: | So, what to do? If you plan on buying a dozen 24 port switches,
pick one you think does what you need at a decent price, buy
ONE of them from someplace with a reasonable return policy, and
put it through a trial by fire for a few weeks with everything
you can think to throw at it. If you do multicasting, then
send as much of that through it as you can. If you think you'll
be running the switch wide open, then do that. I've seen
switches reboot due to firmware bugs when performance counters
overflow internally. So send it long streams of data, for days
or weeks and make sure it doesn't roll over. If it makes it,
order 11 more. If not, return it and start over.
You cannot rely on third party testing and "certifications" to
guarantee they work. Many of those are technically only as
challenging as the vendor writing the check to the test house
so they can be listed on a website with some gold star next
to the model number. It's a joke, but not a funny one.
|
It really sucks that there is a general lack of any objective testing
and reviews -- in fact a dearth of ANY reviews, objective or not -- in
the network device world: managed switches, routers, enterprise
firewalls, etc.
Real life users of these products are often limited to one or two models
from one or two different vendors in their experience, so asking around
in the IT industry for opinions is of limited value.
Unfortunately, IMO, the advice you give above is the ONLY way to go for
this sort of thing. |
|
| Back to top |
|
|
Randy Howard
Guest
|
| Posted:
Thu Mar 03, 2005 3:49 am Post subject:
Re: 120 Hosts Running GigE at Wire Speed Minimum Cost |
|
|
In article <112bso2agu3qf06@news.supernews.com>, strap@hanh-ct.org
says...
| Quote: | Randy Howard wrote:
So, what to do? If you plan on buying a dozen 24 port switches,
pick one you think does what you need at a decent price, buy
ONE of them from someplace with a reasonable return policy, and
put it through a trial by fire for a few weeks with everything
you can think to throw at it. If you do multicasting, then
send as much of that through it as you can. If you think you'll
be running the switch wide open, then do that. I've seen
switches reboot due to firmware bugs when performance counters
overflow internally. So send it long streams of data, for days
or weeks and make sure it doesn't roll over. If it makes it,
order 11 more. If not, return it and start over.
You cannot rely on third party testing and "certifications" to
guarantee they work. Many of those are technically only as
challenging as the vendor writing the check to the test house
so they can be listed on a website with some gold star next
to the model number. It's a joke, but not a funny one.
It really sucks that there is a general lack of any objective testing
and reviews -- in fact a dearth of ANY reviews, objective or not -- in
the network device world: managed switches, routers, enterprise
firewalls, etc.
|
There are several reasons for this, IMHO. First of all, it is not
easy to conduct such a review. To /thoroughly/ exercise even a
fraction of the capabilities of a managed switch (and make sure that
they work for each and every port as expected) can take weeks to
complete, plus a lot of extra system hardware. Running wide-open
gige (and 10gige is even worse) on all ports requires not only
systems, but high-powered systems.
It also requires dedicated software tools to stress it (ttcp and
the like are a joke for type of work). All the good tools are
a) proprietary, b) expensive, c) both.
The typical trade rag magazine gets all their content from contacts
in the industry, they don't have the time, money, or capability
to perform this type of testing.
So, there are companies that do this type of work, but they
charge a lot of money for it. Some do it really well, some just
take the money and put out a pretty glossy full-color "book
report" telling the world that the product will power up and
the manufacturer can afford to hire them to publish it. Apart
from that, it's basically worthless to the consumer.
| Quote: | Real life users of these products are often limited to one or two models
from one or two different vendors in their experience, so asking around
in the IT industry for opinions is of limited value.
|
Correct.
| Quote: | Unfortunately, IMO, the advice you give above is the ONLY way to go
for this sort of thing.
|
About 6 months ago, I was involved with several others in some fairly
serious discussions about trying to start some sort of *legitimate*
version of what Consumer Reports ought to be. We were talking about
doing it all without accepting advertising, trying to kill every
product we got ahold of, and publishing solid, repeatable and
comparative performance data for switches, NICs, storage subsystems,
and high-end data center/enterprise class servers and workstation
equipment. Everyone would probably like to have access to that type
of information, and a library of results that could be directly
compared when making decisions, etc. The problem is, nobody wants
to pay for anything. Guess what? That type of hardware costs a
lot of money, even if you try and sell off a lot of it when you're
done with evaluations. We got hung up on trying to build a
business case that made sense without having to sell out, or
having watered down, useless information like Consumer Reports.
We figured that there was no way to get the vendors to send us
the hardware for eval, because as soon as we started publishing
the truth about their products, AND the performance differences
between them, nobody would send us anything anymore. Truth is
to be avoided at all costs. :-)
--
Randy Howard (2reply remove FOOBAR)
"Making it hard to do stupid things often makes it hard
to do smart ones too." -- Andrew Koenig |
|
| Back to top |
|
|
J. Clarke
Guest
|
| Posted:
Thu Mar 03, 2005 4:51 am Post subject:
Re: 120 Hosts Running GigE at Wire Speed Minimum Cost |
|
|
Randy Howard wrote:
| Quote: | In article <112bso2agu3qf06@news.supernews.com>, strap@hanh-ct.org
says...
Randy Howard wrote:
So, what to do? If you plan on buying a dozen 24 port switches,
pick one you think does what you need at a decent price, buy
ONE of them from someplace with a reasonable return policy, and
put it through a trial by fire for a few weeks with everything
you can think to throw at it. If you do multicasting, then
send as much of that through it as you can. If you think you'll
be running the switch wide open, then do that. I've seen
switches reboot due to firmware bugs when performance counters
overflow internally. So send it long streams of data, for days
or weeks and make sure it doesn't roll over. If it makes it,
order 11 more. If not, return it and start over.
You cannot rely on third party testing and "certifications" to
guarantee they work. Many of those are technically only as
challenging as the vendor writing the check to the test house
so they can be listed on a website with some gold star next
to the model number. It's a joke, but not a funny one.
It really sucks that there is a general lack of any objective testing
and reviews -- in fact a dearth of ANY reviews, objective or not -- in
the network device world: managed switches, routers, enterprise
firewalls, etc.
There are several reasons for this, IMHO. First of all, it is not
easy to conduct such a review. To /thoroughly/ exercise even a
fraction of the capabilities of a managed switch (and make sure that
they work for each and every port as expected) can take weeks to
complete, plus a lot of extra system hardware. Running wide-open
gige (and 10gige is even worse) on all ports requires not only
systems, but high-powered systems.
It also requires dedicated software tools to stress it (ttcp and
the like are a joke for type of work). All the good tools are
a) proprietary, b) expensive, c) both.
The typical trade rag magazine gets all their content from contacts
in the industry, they don't have the time, money, or capability
to perform this type of testing.
So, there are companies that do this type of work, but they
charge a lot of money for it. Some do it really well, some just
take the money and put out a pretty glossy full-color "book
report" telling the world that the product will power up and
the manufacturer can afford to hire them to publish it. Apart
from that, it's basically worthless to the consumer.
Real life users of these products are often limited to one or two models
from one or two different vendors in their experience, so asking around
in the IT industry for opinions is of limited value.
Correct.
Unfortunately, IMO, the advice you give above is the ONLY way to go
for this sort of thing.
About 6 months ago, I was involved with several others in some fairly
serious discussions about trying to start some sort of *legitimate*
version of what Consumer Reports ought to be. We were talking about
doing it all without accepting advertising, trying to kill every
product we got ahold of, and publishing solid, repeatable and
comparative performance data for switches, NICs, storage subsystems,
and high-end data center/enterprise class servers and workstation
equipment. Everyone would probably like to have access to that type
of information, and a library of results that could be directly
compared when making decisions, etc. The problem is, nobody wants
to pay for anything. Guess what? That type of hardware costs a
lot of money, even if you try and sell off a lot of it when you're
done with evaluations. We got hung up on trying to build a
business case that made sense without having to sell out, or
having watered down, useless information like Consumer Reports.
We figured that there was no way to get the vendors to send us
the hardware for eval, because as soon as we started publishing
the truth about their products, AND the performance differences
between them, nobody would send us anything anymore. Truth is
to be avoided at all costs. :-)
|
Just a comment but Consumer Reports as a matter of policy buys everything
they test through the normal purchasing channels, they don't rely on the
manufacturers sending them samples that they believe might be altered to
improve the test results.
--
--John
to email, dial "usenet" and validate
(was jclarke at eye bee em dot net) |
|
| Back to top |
|
|
David Ross
Guest
|
| Posted:
Fri Mar 04, 2005 2:48 am Post subject:
Re: 120 Hosts Running GigE at Wire Speed Minimum Cost |
|
|
| Quote: | Just a comment but Consumer Reports as a matter of policy buys everything
they test through the normal purchasing channels, they don't rely on the
manufacturers sending them samples that they believe might be altered to
improve the test results.
|
Which creates a separate problem in that they really don't get a
representative sample of the goods they buy. I'm sure they've discussed
this and decided this is best for them but if they get the 1 in 1000
that's way worse or way better than average, the review is useless.
And if they buy rev 2 because where they bought is the last part of the
country to get the rev 3 units, it's worse than useless.
CU does a good job if you know how to use the results. But they are far
from perfect. |
|
| Back to top |
|
|
David Ross
Guest
|
| Posted:
Fri Mar 04, 2005 2:52 am Post subject:
Re: 120 Hosts Running GigE at Wire Speed Minimum Cost |
|
|
| Quote: | So, what to do? If you plan on buying a dozen 24 port switches,
pick one you think does what you need at a decent price, buy
ONE of them from someplace with a reasonable return policy, and
put it through a trial by fire for a few weeks with everything
you can think to throw at it. If you do multicasting, then
send as much of that through it as you can. If you think you'll
be running the switch wide open, then do that. I've seen
switches reboot due to firmware bugs when performance counters
overflow internally. So send it long streams of data, for days
or weeks and make sure it doesn't roll over. If it makes it,
order 11 more. If not, return it and start over.
|
HP is having a 30 day return policy for up to 2 of some models of their
switches just now. I need 4 for one office but am buying 2 just to run
them through their paces before I jump in with both feet.
I'm looking at about $10k total. I can just imagine the sale FUD if I
was looking to spend $250K or $2.5M. Almost like the old mainframe days
when IBM and others (some still do) made you sign licensing agreements
where you would never make public any benchmarks you might do. |
|
| Back to top |
|
|
|
|
|
|
FAQ
Memberlist
Register
Profile
Log in to check your private messages
Log in