stdio.h and BUFSIZ

[Nelson H. F. Beebe]

A posting today on the OpenBSD list is snipped below, and doesn't
mention DragonFlyBSD or several other members of the BSD family.  I
just checked on DragonFlyBSD 6.4.0 and noticed that its stdio.h has
BUFSIZ set to 1024, a value that likely dates back to PDP-11 and VAX
days.

>> ...
>> On Fri, Mar 29, 2024 at 2:57benh <enh at google.com> wrote:
>> >
>> > (if anyone knows the equivalent freebsd list, please add them --- this
>> > code is the same between all three bsds.)
>> >
>> > Apple patched their copy of the FreeBSD fvwrite.c to change BUFSIZ to
>> > INT_MAX to massively improve the performance of large unbuffered
>> > writes[1], and patched the buffered case to use the largest multiple
>> > of the buffer size to massively improve the performance of large
>> > buffered writes:
>> >
>> > https://github.com/apple-oss-distributions/Libc/commit/c5a3293354e22262702a3add5b2dfc9bb0b93b85#diff-3b844a19cfb0aab1a23f7fbc457d3bce7453513730c489a72f66ff89d6557ff3
>> >
>> > i've tested similar changes to openbsd's fvwrite.c with great results.
>> >
>> > is it a _requirement_ that buffered writes _only_ happen in multiples
>> > of the buffer size? that seems unlikely to me (a) because of short
>> > writes and (b) musl just does a writev() of what's in the buffer and
>> > what it was just given in case of a larger-than-buffer write and (c)
>> > we already changed the openbsd fread() to read directly into the
>> > caller's buffer regardless of size, so the equivalent behavior on the
>> > _write_ side seems reasonable to me?
>> >
>> > ____
>> > 1. BUFSIZ is only 1024 bytes on the three BSDs and Android/iOS. glibc uses 8KiB.
>> ...

I know that DragonFlyBSD architects have done a lot of work to improve
multiprocessor performance and reduce lock stalls in the kernel.
Presumably BUFSIZ was one of the things addressed, but I'll be happy
to hear comments from list members about the effectivess of increasing
BUFSIZ: 4GB seems to me to be well beyond what might be needed, and I
would expect throughput to level out with even modest increases in
BUFSIZ on most operating systems.  Device drivers are also likely to
buffer I/O, perhaps with sizes that differ from the default BUFSIZ.

-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                                                          -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe at math.utah.edu  -
- 155 S 1400 E RM 233                       beebe at acm.org  beebe at computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------