[Mulgara-dev] Comments solicited

[Alex Hall]

Life is hard, and then you die wrote:
> [snip]
>>> The "further suggestions" are both ideas that I think would be
>>> nice, but certainly aren't essential.  Using mulgara-config.xml to
>>> locate external modules for inclusion in the jar file would make
>>> it trivial to re-package Mulgara with optional modules.  As it
>>> stands right now, you must hack the embedded-jar target to include
>>> any additional components that are not part of the default
>>> distribution.  Using a custom class-loader would allow anybody
>>> developing their own Resolver to deploy without having to
>>> re-package Mulgara.  I don't know if that's important to anybody
>>> right now, but it could be in the future.
>> I've never had a need to do hot deployment, and I've always been able
>> to include any components I want in Mulgara. I also hate chasing down
>> Jars (and other libs) when trying to make something run, so I prefer
>> doing monolithic deployment. But this may be important for anyone
>> creating a custom resolver or content handler.
>>
>> Ronald.... you have a custom resolver. Do you have an opinion on this?
> 
> I'm not sure I fully understand Alex's suggestion: are you suggesting
> putting a file-system path in the mulgara-config.xml and the custom
> classloader would look there for additional classes?

Sorry for the confusion.  The parsing of mulgara-config.xml to locate
dependencies and the use of a custom classloader are two separate ideas - they
are presented as separate bullet points in Paul's wiki post
[http://mulgara.org/trac/wiki/BuildArch].

The first thought, regarding mulgara-config.xml, is one that Paul and I kicked
around a little bit in an offline discussion that never got fully fleshed out.
The thinking was, at build time a task would locate all the component modules
referenced by the configuration and add them to the mulgara.jar archive.  As the
configuration is changed to reflect updated components (say, upgrading to the XA
1.1 store), no changes to the build script itself would be needed.  Right now,
any additional components that you want to include must be added to the
embedded-dist target in build.xml, and if you reference anything in
mulgara-config.xml that is not included in embedded-dist (e.g. the relational
resolver) then you'll get ClassNotFound exceptions.  IIRC, the general consensus
was that such a mechanism might be neat, but not entirely necessary.

> The way we do things right now is we repackage the mulgara.jar with a
> modified mulgara-config.xml, a modified log4j.xml, the resolver
> classes, and our additional resources. This works fine, except that
> maven (and ant) are really slow dealing with mulgara's 22MB jar (a
> couple minutes to unpack and repack).
> 
> If you want to allow folks to really use the unmodified mulgara.jar
> and still provide additional resolvers, then I think the solution
> would be to add another command-line switch to be able to provide an
> additional classpath in which to look for resolvers, together with an
> extra classloader instance (URLClassLoader will work fine, with the
> parent being the currently used classloader) like Alex suggests for
> the resolver loading. This should be trivial to do, btw.

I really like the idea of using a custom classloader to provide an additional
classpath.  This is the approach that I used in the past to develop custom
components.  It allows you to build and deploy Mulgara and your custom component
completely independently of each other, which I think is a good thing.  As
Ronald mentions, it is trivial to do, especially since we are already using a
custom classloader (the main-class attribute in the mulgara.jar manifest is
org.mulgara.util.Bootstrap, which extends URLClassLoader and invokes the
embedded main class specified by the Embedded-Main-Class manifest attribute).

Actually, now that I look more closely at Bootstrap.java, I see that its main
purpose appears to be to allow you to extend the classpath by specifying
additional jars with the "Embedded-Jar" manifest attribute.  All in all, though,
I think that using a command-line switch to provide an additional classpath is
much more useful than mucking with the mulgara.jar manifest.  Or, the approach I
took was to grab all jars from a "lib" directory in the same directory as the
Mulgara jar and add them to the classpath, so no command-line switch was
necessary (I hate having to enumerate individual jar files with -cp, but then I
guess that's what shell scripts are for).

> (why on earth 'java' doesn't support using both -cp and -jar together
> is beyond me, though - that would obviate the need for an additional
> classloader).

I couldn't agree more.

>>> More specific comments about the core/optional modules:
>>>
>>> 1. I think a good rule of thumb is, if a component is called out
>>> by name in mulgara-config.xml then it belongs in its own module.
>>> Granted you can't really run Mulgara without a store
>>> implementation, and since there's only one available at the moment
>>> I think it makes sense to relax that rule.
>> That makes sense. Which makes me think that many of the things being
>> called out at the moment are "unnecessarily configurable". I was
>> already about to merge Krule in as the system rules engine. I'm not
>> averse to configuring something that overrides, or supplements the
>> system modules, but I think the system modules can be hard coded.
>>
>> Does anyone have thoughts, agreement, or violent dissent on this?
> 
> Just interesting to note that recently somebody sorta said "why are
> the stores configurable since there's only one", and just a couple
> days ago the xa-1.1 store was added... So I guess I'd be a little
> hesitant to remove configurability unless it really provides a big
> benefit somewhere.

OK, I guess the store example was a bad one :-)  The point I was trying to make
is, like Paul says, too many things in mulgara-config are unnecessarily
configurable.  The classic example of this is the Database class.  On a related
note, why are the stringpool and nodepool factories configured independently?  I
suppose it might be possible to use the 1.0 stringpool with the 1.1 nodepool,
but why anybody would want to do that is beyond me.

> [snip]
>>> 4. The resolver-test component is a bit of a special case.  It
>>> shouldn't be packaged with the core distribution, but it exists
>>> entirely to test the Annotation mechanism which is an important
>>> part of the core module.  That brings up the issue of creating a
>>> separate source tree for test code, but I'm not sure I want to
>>> open that can of worms.
>> Actually, this is the default setup in Maven. You keep a parallel
>> directory structure under a root of "test" with the same packages in
>> it. This way you can build the main tree easily, and afterward you can
>> build the testing tree, with a dependency on the main distribution.
>> You're still in the same package, so testing is not impeded.
> 
> This is something I actually quite like (in general I quite like the
> default maven layout). This way you A) get clear separation between
> the tests and the functionality, but B) the tests can still access
> package-protected classes and methods if they need to.
> 
>> At the moment we mix it all in to the same directory. The way we keep
>> the testing classes out of the distribution is to have a filter on the
>> files that go into the Jar. Ant lets you filter out files with
>> expressions like exclude="**/*Test.java".
>>
>> I think I prefer the Maven way.
> 
> Agreed :-)

Don't get me wrong, I prefer this approach as well.  I'm certainly not
advocating the current setup.  I just recognize that pulling all the test
classes into a separate source tree could be a lot of work.

Regards,
Alex