phk malloc, was (Re: ptmalloc2)

[Maxim Sobolev]

er.dragonflybsd.org> <20050227231003.GA17810 at xxxxxxxxxxxxxxxx>
In-Reply-To: <20050227231003.GA17810 at xxxxxxxxxxxxxxxx>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Lines: 37
NNTP-Posting-Host: 195.70.151.35
X-Trace: 1109550383 crater_reader.dragonflybsd.org 717 195.70.151.35
Xref: crater_reader.dragonflybsd.org dragonfly.kernel:7863

Dan Melomedman wrote:
> Gary Thorpe wrote:
> 
>>Basically, there is no free solution. If you want reliability in a 
>>system that uses over commit, you would have to manually set resource 
>>limits on basically ever process and make a static allocation of the 
>>resources you have and/or buy much more hardware than you need for the 
> 
> 
> All I'd like to see is a way to guarantee physical memory to special
> processes with an environment variable.  This would make everyone happy,
> either for reliability or performance reasons. By performance I mean the
> fact that sometimes you must have physical memory for fast access, and
> can't use swap. I don't want to delay DNS or LDAP responses because
> their data was in swap.
> 
> We would run single process, statically-alloced processes with the
> variable set, and not care if any of the other, nonimportant processes
> get killed due to overcommit and OOM situation.

You have sources with you, have not you? Go to the 
/usr/src/lib/libc/stdlib/malloc.c and add mlock() conditionalised on 
getenv() where appropriate. I don't think that it will require any more 
than dozen (probably even less) of changes in places that are easy to 
identify. Since malloc.c isn't something that gets changed often, and 
history in FreeBSD cvs repo confirms that, you won't have any problems 
with carrying your changes forward.

-Maxim

>>tasks most of the time (i.e. plan for the worst case). If you disable 
>>over commit, your machine will dynamically assign resources and should 
>>never start killing processes, but it may not allow 100% utilization 
>>(but I would bet that it is better than the first alternative). Both 
> 
> 
> I agree. Reliability is more important most of the time.