[Mulgara-dev] RMI Out of Memory Exception in Mulgara

[Paul Gearon]

On Thu, Jul 23, 2009 at 12:23 PM, Richard Cave<rcave at plos.org> wrote:
> Hi,
>
> We encountered a RMI Out of Memory exception today (exception and dump
> below).  Mulgara is set to use 1Gb memory:
>
> JAVA_OPTS="-Xmx1000m -XX:MaxPermSize=256m -Djava.rmi.dgc.leaseValue=30000"
>
> Do we need to increase the memory more?  Is the RMI heap space tied directly
> to Mulgara or can we increase memory for RMI separately?

Increasing memory more will certainly help the current issue, but at
the expense of performance.

Unfortunately for this situation, RMI is handled as a normal library,
meaning that it uses heap just like everything else. I note that the
Topaz code makes heavy use of ehcache, which is also using RMI. I
don't know for sure, but I'm suspicious that this library may be what
is tipping the balance here. We haven't seen this type of OOM in
systems without the Topaz resolvers, so this bears investigation.

In the short term, all I can suggest is increasing the heap and to
minimize the performance cost by putting as much RAM as possible into
that machine.

Longer term, we will need to look at the profile of Mulgara in
general, and Mulgara with the Topaz resolvers. It may also be
worthwhile moving Topaz to use the HTTP interfaces, as these are
cheaper and more standardized. I don't want to do myself out of a job,
but I think it would be a good thing for Topaz to start moving towards
the SPARQL standard. Even if this happens, the Mulgara implementation
of SPARQL is still going to be best for Topaz, due to some of the
specialized modules that Topaz uses (SPARQL does some of the same
functionality through things like FILTERs, but slower).

Regards,
Paul Gearon