[Mulgara-general] Mulgara size limits
Life is hard, and then you die
ronald at innovation.ch
Fri Jul 11 00:15:40 UTC 2008
On Wed, Jul 09, 2008 at 01:58:58PM -0700, Chuck Borromeo wrote:
> I pulled the latest version of Mulgara from svn and built it. I ran
> it using Java 1.6 (as per Ronald's suggestion). Now the files load.
> It seems to take a long time to load these files. It takes about 40
> minutes to load a 97 MB RDF/XML file. I'll try to put together a
> more formal measurement of the loading times.
Interesting! Which version exactly were you running before? It sounds
like the only major change was Paul's object-pool removal - if that's
the case, this would be a great confirmation of that work.
Cheers,
Ronald
> --- On Tue, 7/8/08, Alex Hall <alexhall at revelytix.com> wrote:
>
> > From: Alex Hall <alexhall at revelytix.com>
> > Subject: Re: [Mulgara-general] Mulgara size limits
> > To: cborromeo3 at yahoo.com, "Mulgara General" <mulgara-general at mulgara.org>
> > Date: Tuesday, July 8, 2008, 4:36 PM
> > Chuck Borromeo wrote:
> > > Hi Everyone,
> > > I just wanted to check to see if anyone has figured
> > out the issue I was encountering when I tried to load
> > RDF/XML files larger than 98 MB.
> >
> > I've been looking at this off and on since last week,
> > but with little
> > results to show. I can tell the offending class that is
> > filling the
> > heap, and I can tell where it is being allocated, but I
> > haven't been
> > able to track down where the references to the class are
> > not being
> > released. To add a degree of difficulty, I'm
> > developing on a 32-bit
> > Windows laptop so I can't duplicate the conditions
> > causing the
> > out-of-memory error.
> >
> > However, I do have a couple of suggestions for you to try:
> >
> > 1. Paul recently committed some changes that affect how
> > objects are
> > allocated. These were meant to simplify our code and make
> > it easier to
> > track object allocation, but there's an outside chance
> > they might
> > address your memory problem. Try building and running the
> > latest
> > revision from SVN (directions at the Mulgara Download page)
> > and see if
> > you still get out-of-memory errors.
> >
> > 2. I would also try running Mulgara with assertions
> > enabled. Do this
> > by adding the "-ea" flag to the Java VM
> > arguments. Scan the log file or
> > console for AssertionError stack traces, and let us know if
> > you find
> > anything.
> >
> > In both cases, I would also gather the heap profile, just
> > to be sure
> > that the AVLNode class is still the culprit.
> >
> > Beyond these two suggestions, I can't tell how the
> > AVLNode references
> > are not being released. The new Mulgara XA2 storage layer,
> > planned for
> > next year, will eliminate the use of the AVLNode class
> > altogether, so
> > this should not be an issue under the next-generation
> > architecture.
> > Given my limited time and resources, I just can't
> > justify putting much
> > more effort into trying to track this down when I'm
> > unable to reproduce
> > the problem myself. If either of the previous suggestions
> > produces new
> > results, I'll be happy to take a look at them, but
> > absent any new
> > developments I'm afraid there's not much more I can
> > do at the moment.
> >
> > Regards,
> > Alex
>
More information about the Mulgara-general
mailing list