Liquibase

Just a reminder that we will be having an online meetup in a few hours: Noon US Central Time (GMT-5) using the “Chat” functionality of the LiquiBase community forum (http://liquibase.org/forum/chat/index.php). To log in, you will need to be registered on http://www.liquibase.org/forum/

Hope to see you there!

As part of re-evaluating the LiquiBase APIs as part of the 2.0 codebase, I started an experimental .Net implementation to see what changes would need to be incorporated into the regular codebase to produce a liquibase.dll with as new little code as possible.  I have the code to the point that I know it would work, and can see the direction it would need to go, but I will not have time to fully implement it for quite some time.  That is where you could come in…

Despite thinking about alternate languages, I ended up finding that ikvm is the best option for us.  I was able to split the existing java code into a “core” and “core-jvm” source directories.  The “core” code is/will be java code, but without any jdbc or xml libraries (or java-specific technologies like Ant, Maven, and Servlets).  The core-jvm source is all the remaining “java specific” code.

I then compiled the “core” java library into a dll and added it as a reference in a new “core-clr” source directory and visual studio solution.  I was able to begin implementing the liquibase abstraction interfaces using OleDbConnection-based classes.

What is the current state?  The liquibase code is divided between core, core-jvm, and core-clr, but the core-clr is far from complete.  Since we can use the core liquibase.dll, 90% of the liquibase code will be shared between the two projects, so bugfixes will be applicable to both, new features will be applicable to both, and the .net port will include all the cross-database support and refactorings that currently exist in liquibase.  What needs to be done is:

• Re-implement the connection based logic using OleDbConnection (or better .net interface?).  This includes translating the abstracted liquibase “execute this sql” calls as well as the database metadata reading logic.
• Re-implement the XML parser using .net libraries.  This has not been started, but the abstractions are already there in the liquibase code.
• Create any .Net specific integrations (IIS, NHibernate, MSBuild, Installer, etc.)

If you are interested in helping, please send me an email (nathan [at] voxland.net).  You do not need to be proficient in java to help, we can handle any changes to the java liquibase codebase to support you as needed.

The LiquiBase plugin contest is underway and there is still time to submit your extensions.    If you are looking for inspriation, there is now an “Ideas Page” on the Extension Portal to get you started.

If you are not planning on creating an extension, but have something that you would like to see created, please add it to the list as well.

Remember, the contest deadline is August 31st and the sooner you submit an entry, the more time you have for feedback to improve it before the contest ends.

Although it is currently just a skeleton of what it will be, the LiquiBase Extension Portal is now open at http://www.liquibase.org/extensions.

The goal of the extension portal is to provide:

• A place for database-specific or experimental functionality that has not/should not be incorporated into the main LiquiBase library
• A single location for end-users to find plug-ins of value to them
• An easy way for 3rd parties to submit, manage and maintain plug-ins by providing a documentation wiki, issue tracking, and source control
• Documentation on how to create your own extensions for public or private use

I was hoping to improve the sample extension pages (set-identity-insert and vacuum) to provide a better template, but I wanted to get something out before the online meetup tomorrow.  We will improve the content on the portal over the next week once real work settles down.

As always, let us know if you have any questions or comments on the portal.

Saw LiquiBase as a requirement on dice.com today.

It’s nice to see that it is becoming important enough to companies for them to list it as an important skill.

Either that or it is so difficult to use that companies do not think it is something that they can train new hires on in a reasonable amount of time….

@leopinheiro: I want to use Liquibase with Rails (like a plugin), any idea?

When I started LiquiBase, I quickly threw out the idea of creating a Java port of Rail’s Migrations. Over the years, a common question is “why doesn’t this work like Migrations?” It is nice to see at least one person in the Rails community that sees the light

Someone updated the the LiquiBase wiki documentation about  preconditions today.  It had been:

<preConditions>
<dbms type="oracle" />
<dbms type="mssql" />
</preConditions>
Will require running on Oracle AND MS-SQL, which will always be false, unless a huge and unexpected merger takes place.

and they changed it to:

<preConditions>
<dbms type="oracle" />
<dbms type="mysql" />
</preConditions>
Will require running on Oracle AND MySQL, which will always be false, unless a huge and unexpected merger takes place.

“just for the pure irony value.”

There are times when I have seen projects with a version 1.9 that means “the last 1.x release before 2.0″. In our case, “1.9″ simply means “the release after 1.8″

After a bit of discussion on the liquibase-user mailing list, we decided that we didn’t want to go to a 2.0 release at this point because there has not been any major changes and we have kept backwords compatibility.

Therefore, we are going to continue our current 1.x numbering strategy and continue to provide evolutionary improvements for the foreseeable future.  1.10, 1.11 etc. do look a bit confusing and like 1.1.0, 1.1.1 etc, but I think most LiquiBase users will be smart enough to figure it out

We have moved our blog to a new hosting provider and to WordPress from Blogger.

Please let us know if you have any issues or if any RSS feeds don’t transfer correctly.

There was a great post the other day on Coding Horror titled “Get Your Database Under Version Control“. It references a good series of posts by K. Scott Allen on database version control, but also makes the point that “When it comes to version control, the database is often a second or even third-class citizen.”

I have noticed this in the past as well, and have wondered why?

Lack of tools?
Subversion, CVS, RCS and others have been around for years, but when I came the the realization that the database must be version controlled I could find no tools that fit my need and so created LiquiBase. I think lack of tools is a symptom of the problem rather than a cause, however. Developers have never been a group to sit around waiting for a tool to solve a pressing need they see.

DBA Overlords?
In some organization changes to the database must go through a database change process managed by DBAs. The fact that the database changes are managed by an external group could create a “not our problem” situation where the developers depend on the DBAs to track changes. Again, I don’t see this as a reason because the majority of projects do not have such a process in place and so wouldn’t be depending on it.

Only Now A Big Problem?
The answer that makes most sense to me is that database versioning is a relatively new problem. Code changes need to be propagated to every developer on a team quickly and reliably and therefore automated tools like Subversion and CVS have been around for a long time and no one would ever consider a group project without them. Most developers would not even consider a solo project without them.

Databases, on the other hand, do not change as often and so manual and error-ridden processes have worked well enough in the past. As databases have become more and more central to projects of all types the old manual database update scripts are showing their limitations.

I think the final straw in the ad-hoc database management schemes has come from the growth of agile processes. Pre-agile, even smaller projects would often design the database up-front and changes to it could be managed in a piecemeal fashion throughout the project. As agile does away with as much of the up-front design as possible, the number of database changes introduced throughout a project increases dramatically and a way to quickly, reliably, and automatically apply changes becomes a necessity.

I see the existing database versioning tools like LiquiBase, DBDeploy, and ActiveRecord:Migration as the RCS of database versioning: they are a great start, but there is a lot of ground left to cover. Recent version of LiquiBase have added features such as database change rollback, database comparisons, DBDoc, and change contexts, but there is still a lot to do. That doesn’t mean, however, that you shouldn’t Get your databse under version control!