Network Slowdowns?

Bill Hacker wbh at conducive.org
Thu Oct 12 02:31:38 PDT 2006


Matthew Dillon wrote:

    Packet overhead for bulk data transfers is only around 3.4%.  The
    ethernet MAC is 12 bytes.  The TCP/IP header is 40 bytes (20 bytes
    for UDP/IP), plus additional bytes for options, and the data payload
    is usually around 1448 bytes.
    Ethernet framing overhead is harder to calculate.  10BaseT is easy,
    it's just manchester encoded at 20MHz -- so 8 bits per byte at 10MHz,
    basically.  I think there is a short preamble and a CRC tacked on.
    100BaseT encodes 4 bits into 5 over the wire (10 bits per byte), but
    the line frequency is actually 125MHz, not 100MHz, so one byte of
    data takes 10 bits @ 8ns/bit = 80ns == 12.5 MBytes/sec.  If you
    subtract the 3.4% packet overhead you get around 12 MBytes/sec.  I
    think there is also a premable and some other junk there that
    reduces the effective bandwidth to 11 MBytes/sec with 1500 MTU packets.
    GiGE also uses a line frequency of 125MHz, encoding 2 (or is it 3?)
    bits into each 8ns period.  But (and now I am looking at the Wikipedia
    entry for GigE) it transmits in both directions on all four cable pairs.
    So you get 125 x 2 x 4 = 1 GBit.  Each 8 bit byte is converted to 10 bits.
    It's all very confusing, actually.  GigE requires a lot of signal 
    processing and uses echo cancellation to work.

    CPU overhead is a different beast entirely.   SSH has a lot of encryption
    and decryption overhead... my transfers over localhost top out at
    21.8 MBytes/sec on my test box.  At 100BaseT speeds the packet rate
    is only around 6000-8000 packets per second and cpu overhead for packet
    processing is usually in the noise.  At GiGE rates, however, the packet
    rate is over 66,000 packets per second and cpu overhead can be
    significant even if you discount interrupt and device access overheads.
						-Matt

Stuff all well and true - as far as it goes, i.e in the lab, and under 
controlled conditions (no other daemons running, for starters).

Now .. put yourself in the shoes of a field-Engineer armed, in ascending order with:

- 'promiscuous' mode IP NIC settings:
 "Gee - there are IP packets eating up b/w that aren't mine at all..."
- protocol analyzer:
  "Hmm.. and ethernet traffic that isn't even IP protocol.."
- oscilloscope:
   Holy *shit* what fool routed the CAT5 next to the power cables for the 
elevator motors?

Used to laugh at the folks from our C&W 'sister company' on Ethernet frantically 
shouting across the office and phoning each other 'shut down your PC!' around 4 
in the afternoon as air-mass thunderstorms overtook Tyson's Corner, Virginia.

Our business unit kept right on working.

ARCnet/TCNS on coax basically ignore lightning strikes a mere 500 meters away. 
Ethernet doesn't have that luxury.

;-)

Bill






More information about the Users mailing list