Thursday, September 26, 2013

starman workers use too much memory and do not release it to the operating system in time

if your large data is a string, this might help

if not, you can redefine the Net::Server::PreForkSimple's "done" method to mark the worker as "done" as soon as the request is finished: adjust


to your liking and stick this in your psgi file

package Net::Server::PreForkSimple;
use strict;
use warnings;
use BSD::Resource;
no warnings 'redefine';
my $memory_soft_cap = 148680;
sub done {
    my $self = shift;
    my $prop = $self->{'server'};
    $prop->{'done'} = shift if @_;
    return 1 if $prop->{'done'};
    return 1 if $prop->{'requests'} >= $prop->{'max_requests'};
    return 1 if $prop->{'SigHUPed'};
    if (! kill 0, $prop->{'ppid'}) {
        $self->log(3, "Parent process gone away. Shutting down");
        return 1;
# above this is the original code 
# from Net::Server::PreForkSimple::done 
 my $rus = getrusage();
    if ( $rus->{maxrss} >= $memory_soft_cap ) {
        $self->log(3, "Maximum memory exceeded, this child will not serve more requests");
        return 1 ;

    return 0;
# end customization


This is useful if only a few requests need a lot of memory and the workers are not recycled soon enough; if all the requests use large amounts of memory this is not the best solution, it will make starman to spawn a new worker for each request.