[Mulgara-dev] trouble with large models

[David Smith]

I'm currently running into trouble with a very large Mulgara model  (with string pool and nodes files larger than 4GB) under windows (server 2008 r2)

The application is currently running under Mulgara 2.1.8, but I am see similar problems with 2.1.13 (even after fixing a bug seeing below)

The main symptom is very slow performance... especially startup, but even after it has started up it will pause on occasion, with the same symptoms... using mapped files.
During startup, mulgara is issuing continuous "FlushBuffersFiles" calls ... going through all the xa.g_XXXX_tb  and xa11.sp_avl files.
I don't see this when using "smaller" models.
My guess is that I'm looking for another int vs long issue.

The system will run, but with a different level of degraded performance, if I set the IO to explicit.

If anyone has any suggestions on where to look in the code or what log4j categories to turn on, I would appreciate it! 
I can try and track it down from there.

Dave Smith
Viewpoint Systems, Inc.

==

I did find, and fix a bug in 2.1.13 related to large files... this bug manifested as an IO exception : trying to seek to a negative offset....

In src\jar\util\java\org\mulgara\util\io\LMappedBufferedFile.java,
 In truncate and mapFile
 lines  152, 212, 215, the second parameter to the fc.map call needs to be performed as a long operation
 my fix is to use a long for the pageSize local variable (lines 113, 185)
I'll submit more formally after I track down my issue above.

==


As an FYI:

Directory of D:\Mulgara\server1\xaStatementStore

07/20/2014  01:27 PM    <DIR>          .
07/20/2014  01:27 PM    <DIR>          ..
07/20/2014  01:27 PM         8,388,608 		xa.g
07/20/2014  01:27 PM                 0 		xa.g.lock
07/20/2014  01:27 PM        83,812,352 		xa.g_3012
07/20/2014  01:27 PM         8,388,608 		xa.g_3012_fl
07/20/2014  01:27 PM       134,217,728 		xa.g_3012_fl_ph
07/20/2014  01:27 PM     6,509,559,808 		xa.g_3012_tb
07/20/2014  01:27 PM         8,388,608		 xa.g_3012_tb_fl
07/20/2014  01:27 PM       134,217,728 		xa.g_3012_tb_fl_ph
07/20/2014  01:27 PM       150,855,680 		xa.g_3120
07/20/2014  01:27 PM         8,388,608 		xa.g_3120_fl
07/20/2014  01:27 PM       134,217,728	 	xa.g_3120_fl_ph
07/20/2014  01:27 PM    12,314,476,544 	xa.g_3120_tb
07/20/2014  01:27 PM         8,388,608 		xa.g_3120_tb_fl
07/20/2014  01:27 PM       134,217,728 		xa.g_3120_tb_fl_ph
07/20/2014  01:27 PM       142,475,264 		xa.g_3201
07/20/2014  01:27 PM         8,388,608 		xa.g_3201_fl
07/20/2014  01:27 PM       134,217,728		xa.g_3201_fl_ph
07/20/2014  01:27 PM    12,096,372,736 	xa.g_3201_tb
07/20/2014  01:27 PM         8,388,608 		xa.g_3201_tb_fl
07/20/2014  01:27 PM       134,217,728 		xa.g_3201_tb_fl_ph
              20 File(s) 32,161,579,008 bytes

 Directory of D:\Mulgara\server1\xaStringPool

07/20/2014  01:27 PM    <DIR>          .
07/20/2014  01:27 PM    <DIR>          ..
07/20/2014  01:27 PM         8,388,608 		xa11.sp
07/20/2014  01:27 PM                 0		xa11.sp.lock
07/20/2014  01:27 PM     4,760,084,480 		xa11.sp_avl
07/20/2014  01:27 PM         8,388,608		xa11.sp_avl_fl
07/20/2014  01:27 PM       268,435,456		xa11.sp_avl_fl_ph
07/16/2014  06:14 PM     3,623,195,416 		xa11.sp_nd
               6 File(s)  8,668,492,568 bytes

This E-Mail and any attachments thereto may contain confidential, privileged and ITAR controlled information. It is intended solely for the recipient(s) indicated. Any review, use, or distribution by anyone other than the intended recipient(s) is strictly prohibited. If you have received this E-Mail in error or are not the intended recipient, please notify the sender and delete all copies immediately.