Pipe woes (not DF specific I guess)

Matthew Dillon dillon at apollo.backplane.com
Thu Feb 5 09:49:39 PST 2004

    It's impossible to know how a program is going to be used.   e.g. I
    would never make grep line-oriented to a pipe by default.  I don't
    mind adding options ('-l' is typically, e.g. see tcpdump) to request
    line buffering, but we should never default to it when outputting
    to a pipe.

					Matthew Dillon 
					<dillon at xxxxxxxxxxxxx>

:On Thu, Feb 05, 2004 at 02:48:41PM +0100, Simon 'corecode' Schubert wrote:
:> Heyas,
:> While doing some shell-fu (with grep and tee) I once again noted that 
:> pipes don't really work the way I'd like them to work: Be line buffered 
:> (not fully buffered) when the final output is a terminal.
:> Joerg wrote an approach with nonblocking IO and polling, but actually 
:> I'd like to have some more generic thing, where not every involved 
:> program needs to be changed.
:There are aspects suboptimal and related to this topic:
:- reading from a pipe does not block, to support pipes the polling is not
:  necessary. This is not true e.g. for sockets. I'm not sure about FIFOs.
:- stdio does line buffering for stdout if the output goes to a terminal,
:  otherwise it is fully buffered
:- some tools are switching between stdio for stdin and read/write for all
:  other output
:IMO we should really discuss this. At least all the tools meant for
:streaming should do only a minimal amount of buffering. E.g. grep
:should only do line buffering, tee and cat no buffering at all.
:Other tools which might be reasonable called and used in pipes should
:be changed to add some fsync at strategic places. Alternatively an
:environmental flag could be added to force libc to use line buffering
:by default for all streams, instead of full buffering.
:> cheers
:>   simon
:> -- 
:> /"\   http://corecode.ath.cx/#donate
:> \ /
:>  \     ASCII Ribbon Campaign
:> / \  Against HTML Mail and News

More information about the Bugs mailing list