How advanced do you have to be to take advantage of your Agile Database Techniques and Refactoring Databases book? Can a beginner also benefit from these books?
Agile Database Techniques (http://www.ambysoft.com/books/agileDatabaseTechniques.html ) is written from the assumption that a wide range of IT professionals will read it. It contains three sections, one that covers the fundamentals of object orientation, one that covers the fundamentals of data techniques, and one that covers agile database techniques. Most data professionals will likely need the first section, most developers the second section, and everyone the third section.
Refactoring Databases (http://www.ambysoft.com/books/refactoringDatabases.html ) is a mid-level book. You should understand the fundamentals of databases and of programming to be able to take advantage of the book.
What is agile model-driven development and why should database professionals use it?
Agile Model Driven Development (AMDD) (http://www.agilemodeling.com/essays/amdd.htm) is, as the name implies, an agile approach to modeling. It describes how modeling is performed on agile project teams, at least the ones who are taking a more mature approach. Traditional data professionals often have unreasonable expectations as to how much data modeling needs to occur on project teams, and AMDD provides realistic advice for how to gain the benefits of modeling without the costs of needless documentation and bureaucracy. The Agile Modeling (AM) method in general provides good advice for modeling and documentation. See www.agilemodeling.com for details.
Are DBAs the least non-agile group of people and do you see a shift towards agile from this group?
We’re starting to see a shift, but it’s pretty slow. The data community in general really seems to struggle when it comes to modern development approaches, let alone agile. The “thought leaders” in that community really aren’t doing much in the way of leading right now, or if they are they’re going in a direction significantly different than what everyone else is doing. I have some thoughts on this at www.agiledata.org/essays/culturalImpedanceMismatch.html .
I have been doing agile and scrum for a while now and must say that it does make a difference. After we started using scrum our turnaround time has decreased by a lot. Is there a preferred agile methodology for database professionals?
The Agile Data method, www.agiledata.org, describes how data professionals can be effective members of agile software development teams. The philosophies and techniques of the AD method should be tailored into base processes such as OpenUP, XP, RUP, or Scrum. At the AD site there are articles which describe how development DBAs, operational data administrators, and enterprise architects can become more agile. Considering that agile software development approaches have been adopted by the majority of organizations, it behooves the data community to start moving in this direction.
How can agile help with data quality?
Data quality is a very serious challenge for most organizations. Traditional approaches to dealing with data quality have struggled in practice, for example TDWI claims that data quality is a $600 Billion annual problem for US firms. A lot of the problem with traditional approaches is that they are very risky and fragile in practice, requiring lots of up front investment, significant overhead, and long payback periods. It’s been a recipe for disaster. The Agile community has developed concrete technical strategies such as database regression testing (www.agiledata.org/essays/databaseTesting.html), database refactoring (www.agiledata.org/essays/databaseRefactoring.html), and continuous database integration which address data quality problems in a much less risky manner.
Can you describe how you would scale agile approaches?
This is the focus of my blog at http://www.ibm.com/developerworks/blogs/page/ambler . The secret is to recognize that there are several scaling factors - such as team size, distribution, legacy environments, and governance to name a few – which complicate matters. The next thing to realize is that you’ll need to adopt techniques, such as those described in the AM and AD methods as well as Rational Unified Process (RUP), to address these challenges. Third, recognize that you need better tooling for these complex situations. For example, at http://jazz.net you can download the beta of IBM Rational Team Concert which is an Eclipse-based collaborative development environment designed to improve team productivity by enabling real-time communication and collaboration across software delivery teams. We’ve also posted some webcasts and white papers at www.ibm.com/rational/agile/ that people should find useful.
I have read a while back in one of your agile columns that it takes some companies 3 months to change a column name or data type in a table. Why do you think this is the case?
It’s even worse than that, the DDJ 2006 Data Quality survey (http://www.ambysoft.com/surveys/dataQualitySeptember2006.html) found that some organizations believed that it was too risky to even contemplate renaming a column. Yikes. There are several reasons for these problems. First, it’s been an underlying assumption in the traditional data community that changing a production database schema is hard to do. When you make that assumption you’re no longer motivated to find ways around the problems. Yet, as Pramod Sadalage and I show in Refactoring Databases it’s trivial to do so in a relational database. That book, BTW, is written from the point of view of a large scale relational database that is accessed by 100 systems, written in 100 languages, running on 100 platforms, on 100 different release cycles, by 100 different teams. Second, some organizations have long, drawn out release processes which make it difficult to make production changes. There’re good reasons for that, usually, but it makes it hard to implement changes. Third, in many organizations there is a very poor working relationship between data professionals and developers, and that makes it difficult to implement changes as well. The DDJ 2006 Data Management survey (http://www.ambysoft.com/surveys/dataManagementAugust2006.html) found that most of the problems sat squarely on the shoulders of data professionals, although developers were a bit guilty too. We need to work together more effectively, which is what the Agile Data method is trying to help with.
Is there a 'safe' way to refactor databases? Are there any tools to help?
The Refactoring Databases book, http://www.ambysoft.com/books/refactoringDatabases.html ) is all about safety. If refactoring wasn’t safe you couldn’t do it in practice. There are several tools available, both commercial and open source, and I maintain a list at www.agiledata.org/essays/tools.html.
If someone could only read one chapter of your refactoring databases book; which chapter would this person get the most value out?
You really need to read the first five chapters as it describes the whole refactoring process, tooling support, and enterprise-level support. But if I had to pick one it would be Chapter 3 which describes the overall process of refactoring.
What is the agile attitude?
To work in a highly collaborative manner, focusing on developing high quality work and delivering it on a regular basis.
Also, when we discover that something is hard to do we try to do it more often and find ways to make it easy. I suspect that’s why the agile community has pushed data quality techniques much further than the traditional community.
Some developers don't think about performance and large data sets until it is too late, how big is the emphasis on performance when doing Agile?
Performance is one of many issues that should be addressed by both the development team and by the independent continuous testing team (http://www.ddj.com/dept/debug/196603549 ). The Agile Data method suggests that data professionals work side-by-side with developers, so hopefully DB-related performance problems won’t happen as often. It also promotes the idea that developers need training, education, and mentoring on data-related activities.
Why do you write technical books?
Because I’m not good at writing fiction. ;-)
Seriously though, to share practical techniques with others.
How many different databases do you work with on a regular basis?
It’s hard to say as it depends on the customers I’m currently working with and what I’m helping them with.
I have listen to the Scott Ambler on Agile (.Net Rocks, January 2007) podcast when it came out and highly recommend it. If you had to pick a favorite podcast you did from your podcast page which one would it be? (Besides the dotnetrocks one of course)
It will be linked to in mid-April, but I recently did a keynote on Scaling Agile Software Development at the Software Development conference. It shares a lot of ideas which people will find interesting and some stats from the recent agile adoption survey. The Agile and GDD keynote that I did at Software Development Best Practices 2007 is also pretty good.
Where can we see or hear Scott Ambler this year? Are there any conferences, seminars or webinars coming up where you will be present?
I’ll be at the Rational Software Developer Conference (RSDC) in Orlando in June http://www.ibm.com/software/rational/events/rsdc2008/ , the Agile 2008 Conference in Toronto in August http://agile2008.org/ , and several recorded webcasts will be published on IBM DeveloperWorks and linked to from www.ibm.com/rational/agile/. I’m also doing a collection of smaller conferences and user groups around the globe, which I announce on my mailing list (http://www.ambysoft.com/mailingList.html ).