Looking for help with Maven configuration

Part of the delay of the next 2.0 RC is that I want that release to include a 2.0 snapshot in the maven repository for people to try.  Unfortunately, I have determined that my maven skills are not what I need them to be to make this happen, and am hoping I can get some help.

If you look at the liquibase source from http://liquibase.jira.com/source/browse/CORE you’ll see that we have what is probably an uncommon source configuration–which is the root of a lot of my maven issues.

The main source is broken up into three major sub-modules:

  • liquibase-core
  • liquibase-core-jvm
  • liquibase-maven-plugin

each with its own pom.xml.  The general idea is that the liquibase-core and liquibase-core-jvm modules are compiled and combined into a single jar file that is released as liquibase-core.jar while the liquibase-maven-plugin module is released independently.  There is a liquibase-dist module that attempts to bind everything together in to liquibase-core, but I’m not sure if it is really doing it all correctly.

Prior to 2.0, liquibase was build using Ant and we do have a repository on sourceforge that is rsynced with the central maven repository.  The old process used an ant task to update a local copy of that maven repository and I would upload the new/changed files to the sourceforge site manually.

The main questions that I know of currently are:

  1. How do I get maven to create the “liquibase-core ” module to release to the maven repository?
  2. What is the best way to create and release it?  I would like the process to be based on builds from the build server, not based on building the modules locally
  3. What improvements should I make to the module structure?
  4. Is what I am trying to do too much for maven’s preferred way of doing things?
  5. What should I be doing to help OSGi support?

12 Comments

  1. Brendan:

    Of course there are several options, but there are two approaches I can think of:
    1) It’s not really the “Maven way” to combine two artifacts that you then expect Maven users to use as a singular dependency. Your liquibase-dist module, type pom, declaring two dependencies, is really all you need. All 3 projects (dist, core, core-jvm) would publish their artifacts and if I wanted to use liquibase as a dependency (including in liquibase-maven-plugin), I just declare a dependency on liquibase-dist.

    2) Otherwise, if you really want to deploy a single JAR for Liquibase, which is reasonable, you want to use the Maven Assembly plugin bound to the package phase. In the assembly descriptor you’ll build a ‘jar-with-dependencies’ and be sure to ‘attach’ the resulting artifact to the build so it will get deployed. You’ll probably have to play around to get the jar named what you want because normally an Assembly attached adds a classifier to the artifact, but that’s not that bad.

  2. Ron Piterman:

    This is really not the maven way, and I will not recommend it, but before: other than making “sense” – why do you want to merge both jars? and if so, why not merge the code base?

    I would leave it in two artifacts, after all, everything one needs is to specify liquibase-core-jvm as dependency.

    You could rename the artifacts to liquibase-api and liquibase-impl. That way it is sort of clear one needs the impl…

    Best Regards, Ron

  3. Nathan Voxland:

    The reason there are two separate modules is because I am currently experimenting with a .net version of liquibase as well which shares as much code as possible with the the java version. The liquibase-core module contains all the “if database type is X, use this SQL” logic and I am using ikvm to compile that down to a .dll. The liquibase-jvm module adds more java-specific logic (how to look stuff up with the java classloader, JDBC, etc.). So there is really no reason that any java usage of liquibase would ever use the liquibase-core module on it’s own because it requires classes from liquibase-jvm to even bootstrap. I would like to not confuse any users of liquibase by this breakdown which does not affect them by combing the resulting jars into a single “liquibase-core.jar” (poorly named due to the overloaded term) which would get pushed out to the maven repository.

    I do have the following config in my liquibase-dist/pom.xml file (see the svn browser)

    But even if I get one jar, is the fact that I don’t have a pom.xml that thinks of it as a single jar going to cause problems for my maven repository creation?

    Would it be better to keep the liquibase-core and liquibase-core-jvm sources separate, and have liquibase-core compile as need be, but simply configure the liquibase-jvm pom to use the source folders from both? Can I configure it to do that?

  4. Ron Piterman:

    Thought about breaking the source code into separate source folders:

    src/main/core
    src/main/java
    src/main/net

    and respectivley
    src/main/core-resources
    src/main/java-resources
    src/main/net-resources

    Configuring maven to use such a setup is not a problem. It breaks the “convension”, but will not introduce problems with tools and repositories…

    Many Greetings, Ron

  5. Ron Piterman:

    found this for you, didn’t test it, but its kind of what I imagined should work:

    http://blogs.deepal.org/2008/07/multiple-source-directories-with-maven2.html

    Greetings, R.

  6. Dave:

    Why not just get rid of Maven?

    To quote an old chinese proverb – “it really is an awful pile of poo”

  7. dhartford:

    If the exercise is to simply ‘merge to jars’ without merging the two codebases, then something like ‘uberjar’, which I think now has a new name: http://maven.apache.org/plugins/maven-shade-plugin/

    The assembly plugin is definately another approach, and is very common going from ANT to Maven as most ant-scripts do have a lot of custom-assembly.

    Although I do understand the hesitation to merge the codebases, it really comes down to that the logical separation should be *consistent* – either two code bases with two jars, or one code base with one jar. Trying to come up with fancy approaches just does not maintain well.

  8. hf:

    Personally I don’t think you want multiple source directories.

    For the liquibase-dist project, I recommend making use of the maven appassembler plugin ( http://mojo.codehaus.org/appassembler/appassembler-maven-plugin/ ). It will also generate the shell script that launches liquibase (which, last time I checked out liquibase trunk, had some errors which prevented execution on linux).

    In my opinion, nothing you are trying to do demands some of the more exotic solutions offered.

    The maven release plugin is the best way to perform releases. It does provide ways to perform releases non-interactively (for use by Bamboo or other CI engine). It will even take care of tagging/branching. I recommend creating a special Bamboo job dedicated to launching the release process.

    In general, it is best to not stray outside of maven convention. If you find yourself needing to, reconsider what you are trying to do. I’ve lorded over hundreds of maven builds, and as soon I gave up what I thought were necessary build idiosyncrasies, life became easier.

    I would be glad to offer up actual patches and contributions on this front — I’ve definitely a lot of experience in this area!

    ~hf

  9. Thomas:

    Hi,

    Regarding question no. 2, the build server would push the latest binaries to a snapshot repository, every time new code is committed. It is a fully non-interactive process.

    On the other hands, releasing (with mvn release:prepare + mvn release:perform) is a manual process. It combines tagging + deploying artifacts to a release repository + upgrading snapshot version in all POMs, in a “transactional” way, so to speak. Only a few people (like you) may release a version of the project, even if the code base is open. This is enforced by restricting uploading to the release repository.

    Thus you have 2 repositories: one for snapshots, with the latest version in development, and one for releases, with only those versions you decided to promote to a tag. The most important rule with releasing is letting Maven manage the tags in the POMs. Don’t bind parent and child POM with the ${version} variable, they must be all stated explicitely and may not be modified by hand. The release plugin takes care of incrementing them for you in a consistent way.

    Regarding the other questions, I agree with most people here: better stick with Maven conventions if you can. If necessary workarounds are always possible with Ant tasks embedded or by copying files around with the resources plugin, but it kills what Maven is made for in the first place.

    HTH :)
    Thomas

  10. Nathan Voxland:

    Thanks for the comments everyone, they have been very helpful.

    This is what I am planning to do:
    1. reconfigure the liqubiase-core-jvm pom to not depend on liquibase-core but rather to just include liquibase-core/src/main/java as an additional source directory. I think this will be more straightforward than dealing with the jar combining plugins.

    2. Look at the appassembler plugin for the liquibase-dist pom and see if that helps me.

    One issue I have with the automatic publishing of snapshots is that
    the build server we are running on now is a shared build server that
    I don’t have filesystem access to create a maven settings.xml file
    with the scp username/password, and even if I requested filesystem
    access I’m not sure if I would want to store that information on a
    server I’m not in charge of.

    I saw there was a repository:bundle-create plugin that creates a
    bundle file for inclusion in the central repository. Am I able to
    have the build server generate the the bundle and install it in my
    local repository that’s rsynced with the main maven repository? Or is
    the bundle creation only for posting one-off versions to the central
    repository directly?

  11. Fernando Padilla:

    I recommend that you go look to host your releases with Sonatype’s OSS hosted Nexus server: http://nexus.sonatype.org/oss-repository-hosting.html

    Through that you can find a mailing list with others using sonatype to host their projects, that might be the best resource to bounce ideas and get recommendations. :)

  12. Sandy:

    Hi Nathan,

    I have configured liquibase for our product db changes requirement, Its really nice.

    Also, I need to use the maven plugin for our CI build. I need your help in understanding the same, Can you please explain how it works. What I would like is once we do a basic install and use this plugin to update (goal:update) then it should check for the latest changes included, basically run the latest update.xml file. Or we need to always specify the file to be included (changeLogFile) because then doesnt we need to change pom.xml every time we include a new change.

    Please ask me if I am not clear or missing anything.

    Thanks,
    Sandy

Leave a comment