Andrew,

Any chance you could use line breaks in your blog entries :)

-Jamie

I can kind of understand why people are of the mindset that designing databases is simple. You can use tools/wizards to do it, RDBMSs have self-tuning features to mitigate performance problems,

The perception is that a database contains tables which conain columns and nothing else. What can be difficult about those?

Sadly this perception hides the real truth - a truth which can be expressed in various ways. I like to say that database design is more an art form than a science. There are no rules and the more practice you have the better you are going to be at doing it.  The attitudes that you describe here Andrew infuriate me to the point of expletives that I'm not going to write on here.

Thank you for your continued petitioning on these matters. I shall keep pointing developers here and doubtless they will continue ignoring your sage advice.

-Jamie

I find this too. In the interest of creating the perfect C# object model, they will demand compromises in the DB. The funny thing is that 95% of the time, the performance problems are in the data tier, so you would think that would dictate how to get the code to respond.

I too am a former developer and try to keep my C# skills, well sharp. I am paid as a data architect and watch with fun sometimes as the developers spin and spin around their object model to make life so easy, when in the end their perfect models which are designed for maintainability are too beautiful to change and thus resistance always seem to form when something outside what they designed comes. I piss them off all the time with the following two mottos:

1. It is easy to add and change tables to meet the customers needs.

2. My data will outlast your code.

:)

Hi Andrew,

  I couldn't agree more.

- http://sqlblog.com/blogs/andy_leonard/archive/2007/07/12/database-professionals-an-enterprise-requirement.aspx

- http://sqlblog.com/blogs/andy_leonard/archive/2007/07/13/a-follow-up-to-database-professionals-an-enterprise-requirement.aspx

:{> Andy

Agreed here as well. Although I think some of it is being in the SQL Server world - I don't run into nearly the same attitude working in as a (close your eyes everyone) Oracle DBA [gasp!].

This is one of my biggest pet-peeves, and I'll continue to argue in defense of the need of a real database professional in a SQL Server environment. A big part of it is a need for real education for all into the complexities of an RDBMS (any).

The company I work in has had this mentality up until a very short time ago (measured in months not years).  One of the main contributors to the stance that "we don't need no stinking DBA" came from the small volumes of business that we had starting out.  As a result we have triggers on almost every table that take actions on other tables that have triggers...  We do not use check constraints or foreign keys at all because it was easier to put the logic in triggers (in my opinion a very easy pit for programmers to fall into).  There was also no thought given to maintenance of the database, so indexes were only rebuilt when they started causing problems, there was no SLA developed with maintenance windows in mind, and even configuration of the server has suffered.  After spending a substantial chunk of change to upgrade our disk system, we discovered that the server we upgraded from did not have AWE enabled.  

"If you wait until the database is hundreds of GB’s or even TB’s in size before you make the proper design changes to accommodate that it is often too late to do the right things"

This is where it has left us.  We have a pretty substantial code base supported by a small team, which means change is very expensive and often involves a rework of large portions of multiple applications.

I was never a proponent of my company hiring a DBA until I learned a little more about SQL Server and its complexity.  It has a very low barrier to entry if you are doing something relatively small over a short period; past that you need more of an expert.

I'm with James and Jamie on this. I don't think Oracle suffers this as badly as SQL Server.

I have this theory (with nothing to back it up) that this is based on Microsoft's own marketing and the "wizards" the build into SQL Server.  In my fantasy world, this is what's happening:

Microsoft markets SQL Server to be so easy that even your grandmother can use it. The wizards make it seem even more likely that this is the case. Your C# developer (with little SQL knowledge) is able to create databases easily, put the data in and get the data out with little trouble.

As time goes by, the database starts getting more and more traffic. They begin to suspect that SQL Server just can't scale and isn't enterprise-ready so they decide to move to Oracle.

When they move to Oracle they decide that they need PhDs (or at least people with a great deal of Oracle experience) to set up and configure the database. The new consultants go in and make them fix the problems that were causing performance issues in SQL Server but now it's in Oracle.

In the end, everybody is happy and 10 new people go out saying that SQL Server is fine for basic queries but you have to move up to Oracle if you want to scale. As SQL Server is still not taken seriously by these people, they still don't hire the right talent to do a proper database design...

One other comment I will make is that most DBA's don't do a very poor job of integrating with the developers. I make it a point to ensure that part of my job is bringing up the level of knowledge on SQL Server for each of the developers. Some of them have an I don't care attitude, but others are willing to learn the right way to do things. I do notice it takes a bit of extra work as you tend to have to prove your case with hard statistics or facts (e.g. querying one record out of a million rows verses ten rows is virtually no difference if properly indexed - you must show this beyond just saying it).

I honestly would be satisfied if everyone else is fully capable of dealing with SQL Server such that they no longer need me. Hording knowledge leads to bad networking and lack of upward mobility.

Today I gave two talks at New England Code Camp 8 . A fun experience as always, and for those of you

One thing to point out is there is obviously a large difference between allowing a developer to design a database and allowing a developer to write a few stored procedures and/or views.  In my experience as a developer I have mostly worked in environments where I might write a stored procedure or two but the DBA was always the one designing the database or at least reviewing the design.  I think the double standard comes out of ignorance which is not unusal among IT management in my experience.  

I am a developer with a small .net dev firm where we don't have the resources for a dedicated DBA.  As a result we have ended up with

   a) no single point of contact for db schema changes

   b) limited database documentation

   c) limited use of optimisation techniques

   d) ad-hoc maintenance of development databases (db reindexing and such).

But most of all the biggest problem has been led to some DATA REDUNDANCY, MISSING FOREIGN KEYS & IN-CONSISTENT NAMING CONVENTIONS... So in my opinion I agree completely with u DB ppl and any dev person is naive if they think that a DBA job is trivial

Great post, Andrew, and great follow-ups by Jamie and James.

I believe that for smaller applications, having a single person wearing multiple hats can make sense. A developer can design and build a database with moderate success, so long as it doesn't need to scale. A DBA can design and build an application so long as it doesnt need to scale (or be pretty, or highly usable). The tools exist that allow non-experts to do a decent job in arenas that are outside their core competencies.

James, I'd also like to suggest that Microsoft's training and certification is also partially to blame for the "perception of simplicity" that you've described. Yes, SQL Server can scale for the enterprise now. Yes, there are true experts who have the deep platform and architectural knowledge and skills to MAKE it scale. But the perception being perpetuated by the marketing folks is being reinforced by the training and certification materials that seem to focus only on the features available and not how to consistently apply best practices.

I'd like to close with an observation and a question for Andrew: I meet many developers who are interested in learning (at least a little) about database design. I meet very few DBAs who are interested in learning (even a little) about application development. Is this simply because I tend to play more on the dev side, or have you noticed the same thing too? Inquiring minds...