phk malloc, was (Re: ptmalloc2)

[Dan Melomedman]

Maxim Sobolev wrote:
> Dan Melomedman wrote:
> >This begs the question whether we could have a feature to disable
> >overcommit on a sysadmins', or the software's demand.
> >
> >Doesn't it break a process that wants to preallocate _physical_ memory
> >on start-up, and reuse it, without allocating any further memory to
> >_guarantee_ it will not fail due to memory allocations over its lifetime?
> >If the OS is lying to such software that it has acquired physical memory, 
> >but
> >the OS will only try to allocate it when it _thinks_ the process needs it,
> >this process will still fail if the physical memory is unavailable at
> >that time. Shame!
> 
> We already have such feature: it's called mlock(2).
> 
> -Maxim

Great, but can it be used with malloc, or one would have to write
their own malloc? Am I missing a point? IMHO sysadmins running
software such as 'MessageWall' won't know what hit it when the mentioned
scenario occurs. Most won't even know what 'overcommit' is. Is it
mentioned in the FreeBSD Handbook? Furthermore, many programmers aren't
aware of these issues. I think there should be a big fat warning in the
man pages that the OS is lying to your software by default about
allocated memory, and carries reliability risks a coder must be aware
of.

AFAIK MessageWall uses malloc() to preallocate memory on its start and
create a buffer of the size of the maximum allowed message size times
the maximum allowed number of connections, etc. Will it crash on a BSD
when OS finally decided to give real memory to MessageWall's buffer,
but doesn't have it available?

If one writes code from scratch and wishes to use mlock(), will the
process be required to run as root? Is it allowed to drop privileges
after mlock succeeds, and keep the lock? Also from the manual page, at
least on FreeBSD, memory locks are not inherited over exec, but I am not
sure about the implications here. Finally, is mlock portable?

Disclaimer: I am not an OS internals guru, but I would greatly appreciate
if the gurus answer these questions.
Please don't respond with something like 'This is not a MessageWall
mailing list', etc. I hope I didn't offend anyone.