cvs commit: src/sys/dev/netif/em if_em.c

[Sepherosa Ziehau]

sephe       2006/11/27 05:05:15 PST

DragonFly src repository

  Modified files:
    sys/dev/netif/em     if_em.c 
  Log:
  The Problem [1]:
  In em_process_receive_interrupts(), if em_get_buf() fails, which
  may be a common case since it calls m_getcl() with MB_DONTWAIT,
  then the rest of the RX ring are _not_ processed at all.  The RX
  ring may contain many, or even worse is full of, ready RX descriptors.
  RDT will not be updated too, in this case.  So hardware will take
  this situation as there is _no_ free space in RX ring, and this
  probably leads to following catastrophe:
  too many RX engine overruns -> RX engine stalls -> interrupts stall [2]
  
  The Fix:
  If em_get_buf() fails then
  - The current RX descriptor still will be reaped, but the frame,
    which belongs to the current RX descriptor, is not delivered to
    the upper layer.
  - RX ring processing keeps going as if there is no failure happens.
  
  Thank dillon@ to help diagnose the problem and give various hints [3]
  
  Thank Mike Tancsa <mike at xxxxxxxxxx> to provide enough information
  to locate the problem.
  
  Reviewed-by: dillon@
  Reported-by: Mike Tancsa <mike at xxxxxxxxxx>
  Tested-by: Mike Tancsa <mike at xxxxxxxxxx> (82571EB copper dual port)
             me (82540EM)
  
  #
  # [1] This problem unveiled itself when Mike tried to bench packet
  #     forwarding performance.  The information he had provided that
  #     is critical to locate the problem is:
  #     1) Both polling(4) and normal mode (interrupt based) do not work.
  #        This means that interrupt processing possibly not the root of
  #        the problem (pointed out by dillon@)
  #     2) One line in the output of sysctl hw.em0.debug_info=1:
  #        ...
  #        em0: Std mbuf cluster failed = 2
  #        ...
  #        This statistics is the value of adapter->mbuf_cluster_failed,
  #        which is updated only when m_getcl() in em_get_buf() fails
  #
  # [2] This is just a guess from the output of 'vmstat -i' provided by Mike
  #     Before his benching:
  #     interrupt                   total       rate
  #     ...
  #     em0                            28          0
  #     ...
  #     After his benching (i.e. em0 choked):
  #     interrupt                   total       rate
  #     ...
  #     em0                            62          0
  #     ...
  #     See, only 34 interrupts came o_O
  #
  # [3] There is still prossibility that RX engine gets confused by
  #     the driver.  It is described by dillon@:
  #     (I didn't find the URL in the archive, so it is pasted here)
  #     "...
  #      Clearly when that case occurs ALL the receive frames will be full.
  #      Lets look at a degenerate case:
  #
  #      [0 ...................... N-1]
  #      * RDH set to 0
  #      * RDT set to N-1
  #      * N frames come in  RDH is set to N-1 (??)
  #      * We process N frames
  #      * The frame at RING[N-1] is cleaned up
  #      * i = N
  #      * We set RDT to i-1 == N-1.  It's the same value it was set to before
  #        we processed all N frames.  The receive engine will think that the
  #        ring is still full when it is empty.
  #
  #      I Think what we need to do here is set RDT to N-2 (mod N of course) in
  #      the case where we have processed ALL N frames.  I'll bet the firmware
  #      is getting confused in the overrun case because we are setting RDT to
  #      the same value it was set at before.
  #      ..."
  #
  
  Revision  Changes    Path
  1.50      +8 -4      src/sys/dev/netif/em/if_em.c


http://www.dragonflybsd.org/cvsweb/src/sys/dev/netif/em/if_em.c.diff?r1=1.49&r2=1.50&f=u