Database Change Management: The Blog
Archive for the ‘Uncategorized’ Category.
@leopinheiro: I want to use Liquibase with Rails (like a plugin), any idea?
When I started LiquiBase, I quickly threw out the idea of creating a Java port of Rail’s Migrations. Over the years, a common question is “why doesn’t this work like Migrations?” It is nice to see at least one person in the Rails community that sees the light 
Someone updated the the LiquiBase wiki documentation about preconditions today. It had been:
<preConditions>
<dbms type="oracle" />
<dbms type="mssql" />
</preConditions>
Will require running on Oracle AND MS-SQL, which will always be false, unless a huge and unexpected merger takes place.
and they changed it to:
<preConditions>
<dbms type="oracle" />
<dbms type="mysql" />
</preConditions>
Will require running on Oracle AND MySQL, which will always be false, unless a huge and unexpected merger takes place.
“just for the pure irony value.”
There are times when I have seen projects with a version 1.9 that means “the last 1.x release before 2.0″. In our case, “1.9″ simply means “the release after 1.8″
After a bit of discussion on the liquibase-user mailing list, we decided that we didn’t want to go to a 2.0 release at this point because there has not been any major changes and we have kept backwords compatibility.
Therefore, we are going to continue our current 1.x numbering strategy and continue to provide evolutionary improvements for the foreseeable future. 1.10, 1.11 etc. do look a bit confusing and like 1.1.0, 1.1.1 etc, but I think most LiquiBase users will be smart enough to figure it out
We have moved our blog to a new hosting provider and to WordPress from Blogger.
Please let us know if you have any issues or if any RSS feeds don’t transfer correctly.
There was a great post the other day on Coding Horror titled “Get Your Database Under Version Control“. It references a good series of posts by K. Scott Allen on database version control, but also makes the point that “When it comes to version control, the database is often a second or even third-class citizen.”
I have noticed this in the past as well, and have wondered why?
Lack of tools?
Subversion, CVS, RCS and others have been around for years, but when I came the the realization that the database must be version controlled I could find no tools that fit my need and so created LiquiBase. I think lack of tools is a symptom of the problem rather than a cause, however. Developers have never been a group to sit around waiting for a tool to solve a pressing need they see.
DBA Overlords?
In some organization changes to the database must go through a database change process managed by DBAs. The fact that the database changes are managed by an external group could create a “not our problem” situation where the developers depend on the DBAs to track changes. Again, I don’t see this as a reason because the majority of projects do not have such a process in place and so wouldn’t be depending on it.
Only Now A Big Problem?
The answer that makes most sense to me is that database versioning is a relatively new problem. Code changes need to be propagated to every developer on a team quickly and reliably and therefore automated tools like Subversion and CVS have been around for a long time and no one would ever consider a group project without them. Most developers would not even consider a solo project without them.
Databases, on the other hand, do not change as often and so manual and error-ridden processes have worked well enough in the past. As databases have become more and more central to projects of all types the old manual database update scripts are showing their limitations.
I think the final straw in the ad-hoc database management schemes has come from the growth of agile processes. Pre-agile, even smaller projects would often design the database up-front and changes to it could be managed in a piecemeal fashion throughout the project. As agile does away with as much of the up-front design as possible, the number of database changes introduced throughout a project increases dramatically and a way to quickly, reliably, and automatically apply changes becomes a necessity.
I see the existing database versioning tools like LiquiBase, DBDeploy, and ActiveRecord:Migration as the RCS of database versioning: they are a great start, but there is a lot of ground left to cover. Recent version of LiquiBase have added features such as database change rollback, database comparisons, DBDoc, and change contexts, but there is still a lot to do. That doesn’t mean, however, that you shouldn’t Get your databse under version control!
I would like to thank Yasuo Honda for all his hard work on the Japanese translation of the LiquiBase documentation. He was also a great help in determining the best wiki tool to use to support translations.
If you are interested in assisting with translating the LiquiBase documentation, feel free to edit the site wiki directly, or contact the documentation mailing list for more information.
You can view the Japanese translation by starting at http://www.liquibase.org/ja/home or by clicking the “ja” box on any page.
I submitted a JavaOne 2008 session into the call for papers. I went for a combination of LiquiBase usage and general database change management. Let’s hope I’m selected!
If you didn’t see, version 1.1.1 of LiquiBase was released the day after 1.1 because of problems reported by users with the new diff tool.
Now, we could simply say “it’s a new feature, you should expect bugs”, but they were pretty obvious ones that should have been caught by simply executing the diff tool against all our supported databases. A simple unit test or two that actually ran the diff tool exposed the bugs and they were easy to catch.
Why weren’t those tests wrote before the 1.1 release? It was because I was testing with a mock database connection which didn’t capture all the idiosyncrasies of different databases. The tests were there, and the coverage was there, but there were still bugs because of limitations in the (mock) abstraction layer.
The lesson to be learned is that when writing database access level code, never use a mock database connection.
Reg Developer in the UK recently posted an interview with Scott Ambler about database refactoring at http://www.regdeveloper.co.uk/2007/07/04/evolutionary_database_design/.
In the interview, he discusses questions developers and DBAs often have when first approaching evolutionary database design (EDD). The two main points he covers are:
While I agree with both these points, I worry about his focus on “transition window” database refactoring methods. There is certainly value in being able to provide these windows for databases that have many independent systems depending on them, but I think that for the vast majority of projects it makes database refactoring seem too difficult to even attempt.
I would propose that more focus needs to be placed on the simpler aspect of EDD: databases that are specific to a single system or set of related systems that are all managed and updated in a frequent, agile manner. These are the teams most likely to adopt an EDD process and there is still a lot of work to get them the tools they need to work efficiently.
In the interview, Scott says that “right now we’re at the beginning of the adoption curve” in regards to EDD and I agree with that. So–for now–let’s focus on the needs of the majority of those early adopters.
I see the tools and practices required to do EDD efficiently as a spectrum that range from a single-project database with no DBA, through a set of related projects with a part time DBA involved, to a database that supports a large set of independent systems that is under the constant care of a DBA. “Transition Window” methods of database refactoring work great for the high end of the spectrum and I don’t think it should be ignored because it is important to know that a process can scale.
For now, however, we need to start a the simple end and build a strong foundation of tools and techniques before working our way up the stack to the top. There is the start of the required tools in the form of DBUnit for unit testing and LiquiBase for managing refactorings, but there is still large holes including IDE support for refactorings, best practices and pattern catalogs for database testing, and more.
Managing, tracking, and applying database changes is difficult, especially in an agile database environment where there are many changes made throughout a project’s life cycle. Even with a tool like LiquiBase, It takes a lot of discipline to apply your database changes in a consistent and traceable manner.
For projects that need to deal with SOX-compliant releases the process is even more difficult because your release documentation needs to include not only how to update your database, but also how to roll back in case of a problem with the release.
It was to address this problem that we added automatic rollback support to LiquiBase. For each changeSet in your change log file, LiquiBase can (usually) generate rollback SQL. For changes that cannot be automatically undone (drop table, insert data, etc.), or if you want to override the default rollback method, you can specify a tag containing the correct SQL. This method of generating rollback commands works well because for most cases you don’t have to do anything, and when you do have to specify rollback SQL, it is stored along side the original change.
To control the generation of generating SQL for updating a database and for rolling it back, see the command line migrator documentation.