[Mulgara-dev] Fix to MGR-2 in revision 106 - and musings on thefuture of Mulgara.

[David Moll]

>With that in mind, once MGR-19 is out of the way, there are a couple of
weeks worth of general clean-up I would >like to do.  After which
there's a product-of-sums performance bug which imposes serious
performance penalties 
>on views which needs work (another couple of weeks work there).  At
which point, does anyone have a spare 64-bit >machine with lots of disk
and a few G of memory which could be used to run some
performance/profiling tests?  
>Mulgara hasn't been through a profiling/optimisation phase for just
over 2 years now, and I'm hearing reports 
>that we have picked up a couple of bottlenecks that are impeding our
performance at the moment.  There is some 
>testing I can do on my own workstation here, but for the real
scalability testing we really need to be 
>loading/querying >1e8 quads on a 64-bit jvm - and that requires a
64-bit machine with a reasonable amount of 
>diskspace.

Right now I'm running Mulgara with the 64-bit JVM from the 1.5.0_09 JDK
on a 3.20 GHz Xeon under a 64 bit version of Win 2k3.  The machine also
has 4 GB of RAM, with a 140GB HD that has ~80GB free.  Most of the HD
space is being taken up by old /server1 directories that I have been
archiving for various reasons - mainly to have something to go back and
load up for failure testing.

I haven't rebuilt Mulgara with the most recent revisions yet, but what I
currently have running is working very well.  There have been some
warnings and exceptions thrown which I believe are a result of malformed
queries.  I have managed to cause the JVM to run out of heap space with
a particular series of badly formed queries.  At the moment Mulgara has
been running a set of selects/inserts/deletes repeatedly for the past
15-20 hours without crashing or throwing an exceptions.  

Mulgara was started with "java -jar mulgara-1.0.0.jar"

Task manager claims that java.exe is currently at 3.1GB of memory usage,
with a peak memory usage of 3.7 GB.  But I'm not entirely certain how
accurate the Windows Task Manager is.

I'm going to bring Mulgara down, update from Subversion and rebuild it
to start running these queries again.  Right now restoring from a 135 MB
.gz backup to an empty data store takes around 15 minutes.  The
uncompressed backup is ~670 MB, has ~18 million lines, and ~3 million
triples I believe.

~Dave