Being dumbed down? I hardly think so. SQL Server is becoming a much more complex system than it ever was. Sure Microsoft are improving the quality of their tools to make it easier to manage this complexity, but I'd hardly call it dumbing down. Admittedly Microsoft's marketing department (who are responsible for this slogan) have a lot to answer for. They have made many stupid statements over the last few years, especially in the development space.

Just as well that I think the developers who create these products have their head screwed on. :)

>>For example, the Entity Framework presentation at PASS included a slide that said, “Imagine a world with no stored procedures, no user-defined function, and no views” as a good thing.

To that I say Imagine a word where the developer does NOT have dbo permissions and every obect created/altered in the DB has to go through change control. That makes almost all of the ORM tools unusable (some of them do use procs)

We usually give exec permissions to procs and select permissions to tables to users, certainly no db_ddladmin role

For some good entertainment listen to The ORM Smackdown! It is between Ted Neward and Oren Eini (NHibernate) You can download it here

http://www.dotnetrocks.com/default.aspx?showNum=240

Denis

I think Dejan Sarka's comments hit the nail on the head. When I hear the phrase "Beyond Relational", my first thoughts are - "and that would be....". The entire idea of a database of any kind is that information is organized in some sort of way that it is related to *something*. Object-Relational - the relationships are more along the lines of the instance of an object (yes, that's over simplification). Multi-dimensional - that's uber-relational, with data described (related) by perhaps dozens of different attributes. GeoSpatial? That's relational basically keyed upon some sort of coordinate system. A filing cabinet? (yes, a filing cabinet) That's basically a single table that is a clustered index (the method of filing). It doesn't matter how you paint the picture, the work of Codd et alia still applies, just in slightly different or obfuscated ways.

It's unfortunate that much of what we work with is driven by marketing, but unfortunately that's life in the world of capitalism.

>But it’s one more way that the database professional is being dumbed down by

>Microsoft.

I wonder what other ways you think contribute to dumbing down. What MS means

by 'beyond relational' is 'forget relational already':) Looking at sql server

as if it was somehow an embodiment of relational theory is every bit

a form of dumbing down as some silly utterance by some poor nitwit at MS.

There never was and never will be any 'intent' by MS to offer a 'relational'

database. Sql servers only intent now is to be responsive to its biggest

customer, visual studio. And that team is as knowledgeable in relational

databases as the server team. Why does the community still view sql server

thru an imaginary lense?  Did you ever hear of somewhat pregnant?

If you open the dumbing down door be prepared to greet all those who comes thru:)

best,

http://www.beyondsql.blogspot.com

"I think this is the debate of our time - 'what role does the database and the database professional play in application design and development?'"

Paul, I wholeheartedly agree. I've read some great ideas about how to address The Great Divide (TGD) between developers and database professionals, and I have a few ideas of my own.

I think the core of the issues surrounding TGD is a general misunderstanding of the complexity involved. Were it a centralized, compartmentalized, easy-to-define-and-locate issue it would have been solved by now. I visualize the issue as an air-filled maleable cylinder that - while constantly in motion - when collapsed at one end expands at the other. It's being approached as "a problem" when it's really a problem about problems (a "meta-problem").

In my opinion, there is no solution per se - there are methodologies and design patterns that comprise a set of solutions for a set of problems. But the combinatorics invloved are formidable and tend to result in customized responses to each scenario.

What I have observed to work best is to have cross-disciplined individuals work together in small teams that function with trust and respect. I've personally had the most fun working in these types of environments.

:{> Andy

There is no reason to have a great divide.  Microsoft perpetuates divides by creating roles like Mort, Elvis, Einstein, Data Dude, etc.  Everyone is supposed to go off into their own little world and not interact with each other.  Alas, Steve is absolutely correct when he says that MS views SQL Server's biggest customer to be Visual Studio and its users.  I think that the product will continue to slowly get sucked into VS and eventually there will be only one product.  We'll see how that plays out over the next several years -- it's not like it will be an overnight thing -- and when we, as DB professionals, will be required to make a career choice: Stick with MS as developers, or jump the MS ship and go to greener DB pastures.

Adam,

  I see your point and cannot argue with your logic, but I draw a different conclusion: I see database developers being drawn into the VS world and database administrators remaining in their SQL Server world. Despite attempts by Microsoft and corporate IT departments, these roles remain vastly different.

  Most database developers have development experience already so I don't see this as a significant leap for them. And, I anticipate the tools for administration will remain administrator-oriented, like SSMS is now and Query Analyzer/EM was before.

  I think there will always be a need for both roles - and distinct tools for each. That's not to say database developers will never use SSMS or DBAs won't use Business Intelligence Development Studio.

:{> Andy

The fact that you need VS Pro edition to debug procs is also an insult

Hi Paul,

  I agree with your analysis about the difference between application and database development, but I believe moving all development roles into a common IDE is a good thing. On some level development is development, and I applaud the Microsoft's effort to consolidate the practice.

  I do not think the IDE teams got everything right on their first passes with BIDS and Team Edition for Database Professionals templates, but I am very impressed with the products - especially considering they are v1.0.

  What I've seen so far of the v2.0 products leads me to believe they (the Microsoft Dev Teams) are right on track.

  My complaint: VS2008 RTM should have been database/BIDS-project backwards compatible.

  Denis, why insulted?

:{> Andy

>>Denis, why insulted?

Andy with SQL 2000 you installed client tools and you could debug a proc. With 2005 you have to also install VS (Pro version if I am not mistaken so that means $$$$) Not everyone has a MSDN subscription. BTW the debugger is not that good anyway, I usually 'debug' by using PRINT

It would be nice if SQL also had pre processer directives like #if debug

Denis

Denis,

  That makes sense. I also use PRINT to debug.

:{> Andy

Paul: I agree that different types of development are different roles, but not that the same people can't be cross-functional.  I myself focus on the database, but also like to do some appdev stuff from time to time (mainly in the data access and business tiers -- I avoid UIs whenever possible).  Normalization applies to both data and OO modeling, although the two disciplines are somewhat different in how the world is viewed.

Regarding debugging: I don't use PRINT much anymore -- I've found that RAISERROR WITH NOWAIT is much nicer (in coordination with SELECTs).  The fact that you can't see the contents of a temp table makes the debugger totally useless in my eyes.  

BTW, I created TSQLMacro to help with some of the PRINT/RAISERROR/SELECT debugging scenarios:

http://datamanipulation.net/tsqlmacro/

... I should probably update it and transition it to use SQLCLR some point, but for now it's more or less stable and workable.

When I saw "Beyond Relational", my first thought was that it was one of the hierarchical/multi-valued databases.  IBM has the Unidata product that's an example of this.  It makes no sense to me at all.  Just because I could store one entity inside another doesn't mean it's a good idea.  Why don't we just store our stuff in one big XML file?  

There's a line-of-business app in our industry that one of our competitors uses.  Even though they are regarded as a "better place to work", I would never go there because of the technology.

Sorry for the rant on multi-valued db's -- it's just what comes to mind for me when I hear the phrase "beyond relational".

Denis/Andy,

RAISERROR WITH NOWAIT is the way to go.  PRINT (shivers) ?  

Raiserror with nowait gives you immediate responses and you can "real-time" see what is going on, whereas PRINT you have to wait til the proc finishes executing before you start seeing results.

Bart, I know about using raiserror, remember this teaser?

http://sqlblog.com/blogs/denis_gobo/archive/2007/11/09/3266.aspx

@Denis

I just read what u mentioned above and said u debug with PRINT :)

'Beyond relational' is a marketing phrase, and should left as such. Isn't it a bit silly to treat a loose marketing term as a precise technical term and criticize it as such? In today's DBMS market place, every major DBMS vendor is going beyond relational. Oracel and DB2 are certain doing so, and what Microsoft is not doing anything out of ordinary. I think we are making a mountain out of a mole hill.

Linchi: How are Oracle and DB2 going "beyond relational?"

> I think we are making a mountain out of a mole hill.

I think not. There is longer a great divide, a debate, an impedance mismatch. MS has issued their own Emancipation Proclamation. And as a result they no longer support the relation model.  

'A Call to Arms'

by Jim Gray, Microsoft

Mark Compton, Consultant

April 2005

http://www.acmqueue.org/modules.php?name=Content&pa=showpage&pid=293

This is an invitation to embrace a new model. It's just as much 'A Farewell to Arms', an emancipation from the relational model which they are leaving behind.

What does sql server look like in this new model?

'Interview with David Campbell'

General Manager of Strategy, Infrastructure and Architecture of Microsoft SQL Server.

May 14, 2007

http://www.sqlmag.com/article/articleid/96048/sql_server_blog_96048.html

''I believe the next major advance in Information Technology will come from addressing the gap between people and information.'

That gap is the logical model itself.

'The focus will move to the data itself rather than on the machinery used to manipulate it. We'll be less concerned with the plumbing and more concerned with data quality, data protection, and information production.'

'Most of the data services provided by SQL Server will be driven from a  common data model. Whether you're creating a report, building an  information cube, or integrating data from another system, you will be able to start from a common model of the key data entities such as  "customer", "order", or "prospect".'

'Finally, fewer and fewer people will miss, (or remember), the "open

databases" sp_configure option...'

The class replaces the table as the basic unit of work.  VS replaces QA/SSMS as the interface for application development. There is no concept of relational anything in this object world. Sql constructs are independent of application development.

The language of the relation model is replaced with the language of entities. There is no concept of a dba.

MS is no longer in the database wars as we know it. They are trading 3rd place in that world for 1st place in another. And they now have the freedom talk about this new world. It just sounds silly to those who have not left the old one.

Ironically some were hoping for a new sub-language to further AD. Perhaps the lesson here is to be careful of what you wish for.

I too was hoping they'd enter a new world but not the one they have chosen.

http://www.beyondsql.blogspot.com

The relational engines seem to have matured to the extent that you no longer can squeeze a lot juice out of them. I know this is a dangerous claim, and historically claims of this nature are often proven wrong because human innovations are more surprising than not by their very nature. But if you look at what the vendors are doing and what are being produced by the research community, the focus is shifting to things that are not relational.

You guys seem to be 'panicking' at the assumption that the support of non-relational features is being introduced at the expense of the relational engines. That doesn't necessarily have to be the case. Well, since many of you are consultants, this may just be opening a lot of consulting opportunities for you guys :-) But seriously, I don't see the possibility of the relational engines being replaced by any other database engines any time soon unless there is a sudden earth-shattering break-through that makes these other engines as efficient or more efficient in storing and retrieving data. For a long time to come, the relational engine and these other engines will probably co-exist and in some cases the relational engine will actually serve as the foundation for the other engines for some time.

> The language of the relation model is replaced with the language of entities.

Again, I'm not sure whether this is really the case. I see it as more complementary. What's the language of entities anyway? I'll be real worried when they invent one that is as elegant as SQL.

> There is no concept of a dba.

I'm not shedding any tears here. The traditional DBA function is dying (or fading) anyway, and should go the same way of the old roomful of number crunchers you see in those black-and-white photos from the industrial age.

> MS is no longer in the database wars as we know it. They are trading

> 3rd place in that world for 1st place in another. And they now have the

> freedom talk about this new world. It just sounds silly to those who have

> not left the old one.

This is a rather harsh characterization, but it may not be a bad strategy.

> I too was hoping they'd enter a new world but not the one they have chosen.

I don;t know exactly what new world you are talking about. But the new world they are entering isn't really that new, not exclusively new to Microsoft anyway. That gets us to the question of how Oracle and DB2 go beyond relational, as Adam asked.

For Oracle, its support for unstructured data--the equivalent of FILESTREAM--is call SecureFiles.  Personally, I think it's better implmentation than SQL Server FILESTREAM because SecureFiles is more seamless than FILESTREAM. Someone characterized it as papered over the database and files more smoothly. Whatever the characterization, it has deeper integration into the database (e.g. supported by more database native features). For spatial data, Oracle has a feature called Oracle Spatial. It's s eparately licensed option, but the spatial data types are considered native. Similarly, Oracle supports most of teh XML features (I'm not an XML guy so I have not done a feature-by-feature cross examination on all its XML support).

But if you talk to IBM, you'd notice that IBM is going nuts on its XML support. In fact, IBM claims that its DB2 9.5 is a dual-engine system, having a relational engine and a completely separate but integrated XML engine. Sitting in their seminars, sometimes I wonder if they are going overboard with their excitment about XML in DB2. But when you have guys like Don Chamberlin (co-inventor of SQL and co-designer of XQuery) on your staff, I guess you can afford to boast your XML prowess.

Linchi: I have yet to see a widespread commercially available example of a "relational" engine.  Sorry if this sounds like I'm channeling [Celko|Pascal|Steve Dassin], but from my point of view I'm annoyed by the fact that I'm seeing all of these claims about surpassing technology that has not yet even been achieved.  

As Dejan pointed out in his original post, XML, geographical, and "filestream" datatypes are not "beyond relational" at all.  The Relational Model fully supports such datatype extensions. So I for one am not panicing that these features are being added to SQL Server; on the contrary, I think they have their places and will be quite useful in certain scenarios.  But I feel that to use the term "beyond relational" to describe them is a slap in the face to people who actually care about the academic underpinnings of the databases we use.  Marketing term or not, Paul is 100% correct when he states that this is a dumbing down.

Let us think why do we see this "beyond relational" sales pitch in the first place. I think there is a sentiment brewing here in IT, a sentiment that "relational" means "outdated, inconvenient legacy monster".

Adam;

> I have yet to see a widespread commercially available example of

> a "relational" engine.

I'm not sure I understand. I take the core database engines of SQL Server, Oracle, and DB2 as the widespread commerically available examples of a relational engine. We can dispute the fine differences in the interpretations of what consists a relational model and what qualifies as its implementation. But if we go too far on that route, the discussion will degenerate into a that-depends-on-what-is-is semantic game, and won't be a very useful exercise.

> The Relational Model fully supports such datatype extensions.

I don't think I really agree, if we can agree that THE relational model is Codd's model. Okay, you can shred an XML document into a bunch of tables, and I take that as an example of the relational model supporting an XML datatype. No problem there! But one has to ask whether that's the best approach to supporting XML on a number of criteria--the same criteria one would use to evaluate an assembly language vs. a high-level language. So the fact that an assembly language can fully support a high-level language doesn't mean we shouldn't go beyond assembly. I'm not arguing that the relational model is as a low-level tool as an assembly language is.

Going beyond relational is really about a database system (in this case SQL Server) supporting constructs that are inherently or conceptually not 'relational' regardless whether they can or can't be represented relationally. Folks who don't like the 'beyond relational' seem to suggest that these database systems are somehow inherently relational or the relational data model has a special place in these database systems. I'd argue that's not the case. They might have been developed to support the relational model initially, but who says they should stop there. After all, a database system is there to faciliate the storage and retrieval of data. Data that can be easily represented relationally are just a subset of data people may want to use a database to store and retrieve. There is no reason a database system should be put into the straighjacket of a relational model. In fact, before the relational model came along, database systems supported the hierarchical model and the network model among others.

So if XML can be supported by a relational engine effectively and efficiently, fine! Go for it. If it can't be effectively and efficiently supported, perhaps the database system should be altered or expanded to support it directly without taking a detour via the relational engine.

On a somewhat metaphysical level, one may have to ask at what point when a relational model that is successfully extended to support constructs like XML. spatial data, and filestreams is so extended that it is no longer relational.

Then again, I'm not sure it's worth all this time discussing given that it's just a marketing phrase, and its discussions will probably end up opinion based eventually.

Linchi: There is nothing semantic, nor is there any room for interpretation about what is and is not relational.  The Relational Model is based on mathematics and is therefore quite strictly defined.  SQL-based DBMSs are simply not relational, and can not possibly be relational because the language itself violates the model.

Furthermore, XML values do not have to be shredded for the database to be considered "relational".  Chris Date covers similar topics, such as table-valued columns, quite well in "The Third Manifesto"; I don't recall specific coverage of XML, but I think the table-valued arguments could be extended in that direction.

> There is nothing semantic, nor is there any room for interpretation about

> what is and is not relational.

Apparently, that's not the case; otherwise, you won't see all these debates both in industry and in academia. Plus, as mentioned before, when the model is kept being extended, one has to wonder where is the line beyond which whether it is still the relational model as Codd proposed.

> SQL-based DBMSs are simply not relational, and can not possibly be

> relational because the language itself violates the model.

I guess discussions can be conducted at multiple levels or with different frames of reference. We are indeed running the danger of talking at different levels or with a different frame of reference, and talking past each other. The above statement may be true in some strictly academic sense. It may be fine and time well spent for academics to debate whether these commercial database engines or the SQL language they implement are relational or not. But I personally don't think it's a useful argument for the profession we are in because we have practical problems to solve and we need a common language to help with that work. If we don't want to call these database engines relational database engines (or the SQL languages relational) and we agree to call them something else (e.g. relational-minus), I'm fine with that. But last time I checked, I believe these database engines are >in general< called relational engines in the industry.

There is always a tention between academic purity and practical utility. SQL, like any language that is meant to have a real life, has to serve the latter while not trampling the former too much. I bet a language that doesn't violate the model at all won't be that useful practically. The reason we don't just strictly stickt to relational algebra but accept such monstrosities as procedural constructs in manipulating the tables and the bit-wise operators is a practical one.

By the way, let's just call these Oracle, DB2, SQL Server, MySQL, and Sybase engines SQL database management systems. Forget about 'beyond relational', and let's say 'beyond SQL'. But then that is not as sexy as 'beyond relational', and may still annoy the SQL folks who consider the spatial operators and XQuery nothing but an enrichment of SQL.

Okay, scratch that and stick to 'beyond relational'. At least, it stirs up some controversies and scores some marketing attention.

Linchi: "SQL DBMSs" is my preferred way to refer to DBMSs that use SQL. Unfortunately, our industry (or perhaps the people in the industry) is well known for ignoring, distorting, and/or trampeling the academic underpinnings... That doesn't mean that we all have to play along or agree :-)  Whether or not a truly relational DBMS and query language would be practical?  I think that it absolutely would.  Date's Tutorial D language, for example, is simple, easy to understand, and lets you do a lot of things in a much more straightforward manner than SQL.  Merely transitioning from bag to true set-based logic would automatically fix a lot of issues IMO.

As an aside, it's sad that companies feel the need to market these extensions as "beyond" anything.  It's like they're saying, "our current technology is totally flawed, but if you upgrade to the next version things will be much better!"  Is self-deprecation really a good marketing tactic?

> "our current technology is totally flawed, but if you upgrade to the next

> version things will be much better!"  Is self-deprecation really a good

> marketing tactic?

But they are in a bind. If you ask a vendor what's wrong with your product, they'll say none. But if instead you ask the vendor what's great about your next release, they'll give you plenty. It almost never fails that the marketing promos (maybe not really marketing) from the vendor are the best source to see what the vendor itself sees as the main problems of their current product.

> Merely transitioning from bag to true set-based logic would automatically

> fix a lot of issues IMO.

Adam, I'm actually curious about what you consider as the top one or two problems caused by these DBMS engines not being strictly relational. I typically don't see a knife that's too sharp as a problem even if that means some may cut themselves too easily. But I'm not sure if that's what you are referring to.

BTW, this Jim Gray talk is relevant to our discussions on 'beyond relational'.

Oops! Forgot to paste the link: http://research.microsoft.com/~gray/talks/ADBIS2004_DB%20Revolution.ppt

Hi Steve, if a stored procedure is a table, what's your opinion on using a stored procedure just for its side effect?

Wow! I knew my blog was a bit provocative; however, I did not expect such a long debate. To make my original point a bit clearer: Adam understands and interprets correctly what I meant. I have to say I have similar background as Adam. I consider myself a data-oriented developer, which means something similar to Adam – I deal with databases and data access layer, can talk about middle tier, but do not care about UI.

My main point is that the relational model is far from being obsolete. The relational model is a mathematical, background independent model. As a mathematical model it can be implemented in database, class or some other design. Background independent means the model by itself is not data type, language, key selection, physical implementation, or naming convention dependent. When something is based on science, on mathematics, and is background independent, it has much longer lifespan than best practices approaches.

Therefore, I do not see much sense in wording “beyond relational”. Similarly, I do not see much sense in very common discussions about natural and surrogate keys, or about physical implementation, or about language support. I do not feel “The Great Divide”. I like having CLR, XML, spatial data, filestream etc. inside SQL Server. Nevertheless, I do not think that because of this support SQL Server is “beyond relational”. Sooner or later any mathematical theory gets superseded; however, so far I have not heard about sound, strict, mathematical theory that I would consider as a successor of the relational model.

I would like to point out the David’s Campbell sentence Steve mentioned: “We'll be less concerned with the plumbing and more concerned with data quality, data protection, and information production.” Relational model, as a strict, logical model, is in my knowledge currently the model that enforces data quality the best of all models. Relational model is about constraints, starting with normalization, which guarantees that a table represents exactly one set.

I think it is important to distinguish between relational model and different implementations of the model. With proper understanding, you can design good applications; without this understanding, you depend on tools, luck, etc.

Supporting so-called “complex” data types is not what makes a system “beyond relational”. To throw a bone: try to understand why some types seem complex to us and some seem to be simple. Are scalar types simple? Well, what exactly is a scalar type?

Hello Linchi

On side effects. That's a kewl issue. I think the real question is how can a table, scalar, row or any system type that results from a procedure (returned) at the same time cause a side effect (change the state of the db)? That's not logically possible. So the answer is no result can at the same time issue a side effect. So the only way to do a side effect in a procedure is to trick the compiler, go out of the current scope do your thing and resume in the current scope to whatever end (type) is the result. So how do  you change scope? Dynamic code. In D4 the only way to use a procedure, regardless of whether it returns anything, is to trick the compiler, hide the code from the compiler until runtime, with dynamic code just like dynamic sql.  

I never gave an example of this. It opens alot of sql server issues too:) Why the separation of functions and procedures. Why can't a function do certain things.

I never got around to a comparison. But I'm sure it wouldn't make me any new friends:) Even more interesting is dynamic code/sql, which I have commented on.

Oh yeah, using an sp in d4 just for a side effect is perfectly fine. It's simply an executable statement, no select, no anything, just the name of the sp.

Hope I atleast come close to an answer:)

Wherever you see the word 'relational' just substitute 'fog'. As in fog of war:)

>But when you have guys like Don Chamberlin (co-inventor of SQL and co-designer

>of XQuery) on your staff, I guess you can afford to boast your XML prowess.

He is revered in the sql world and reviled in the relational one. He was a lead designer of System-R, the prototype of all sql database systems. Those guys

created a query language based on Codds description of basic relational

operators like projection, union and join. But they did NOT implement the

relational model Codd described. They just ripped out these constructs without

regard for their meaningfulness in the entire relational model. So what you have today is nothing like the relational model as it was envisioned. (IT successfully marginalizes the huge difference and those that point it out:) And now comes 'beyond relational'. What does this phrase really mean to MS? They are more than willing to tell us. Aside from Grays article/presentation, everyone should read the articles on this site, the 'Comega language':

http://research.microsoft.com/Comega/

Especially this article:

'Unifying Tables, Objects and Documents'

http://research.microsoft.com/users/schulte/Papers/UnifyingTablesObjectsAndDocuments(DPCOOL2003).pdf

Here you'll find history repeating itself. MS, just like IBM did with System-R, has extracted relational operators out of the relationl model and put them in an imperative object environment without any regard to relational theory. The great irony is that the extensions that MS added to net to realize tables and xml within net is the foundation for a true relational model! Streams, structs(structures) and the anonymous type represent tables and rows as variables. (I wish someone would tell me just how the anonymous type is being used. It's the basis for assignment and comparison in the relational model.) Had MS any idea of the friggin true relational model they would make a different kind of history. Talk about dumbing down. Talk about of only academic interest. Talk about relational fog (I should add that Alphora (Dataphor) recognized the ability of the object imperative environment to support the D relational language and implemented it. And it works:)

Here is what Anders Hejlsberg, MS VS guru, and now the head of database technology  has to say about the disconnect:

InfoWorld

Interview of Microsoft Distinguished Engineer Anders Hejlsberg

'Microsoft's Hejlsberg touts .Net, C-Omega technologies'

June 10, 2005

http://www.infoworld.com/article/05/06/10/HNhejlsberg_1.html

"So what we're looking at is really trying to much more deeply integrate the

capabilities of query languages and data into the C# programming language.

And I don't specifically mean SQL, and I emphatically don't mean just take SQL

and slap it into C# and have SQL in there. But rather try to understand what is

it expressively that you can do in SQL and add those same capabilities to C#."  

Anders Hejlsberg is microsofts version of Don Chamberlin at IBM. So what they have done is replace one flawed implementation of sql with another. And this is how they achieve efficiency in application development. Now that is unfriggin believable:) Well there's no free lunches. And I await to be enlightened on just how this environment will replace the concept of the logical model in solving business problems. I would say the real meaning of beyond relational is sideways.

> So what they have done is replace one flawed implementation of sql

> with another.

Steve; I have not read all the articles on your site, and I'm a bit lost on the big picture. So perhaps a few simple examples would help enlighten me, if at all possible. Can you give a few examples of the flaws on the top of your list and their damage?

I'm not claiming that SQL Server, Oracle, DB2, and so on are perfectly done. I take a more practical view and am not annoyed by serious compromises--that must be made to create a commercial system--along a multitude of dimensions, many of which have nothing to do with adademic theories.

I hope D4 will not go the way of Lisp, Prolog, and Smalltalk. Those languages have a much better implementation of the mathmatical theories than your C and C++.

Anyway, 'beyond relational', as I see it, is beyond relational from a system perspective. It's not an academic paper proposing a model that is beyond relational. I don't think anybody has claimed to have come up with a better or more unifying theory/model. Until such is the case, being annoyed by the phrase is just silly.

From a system perspective, grand design and implementation based on a strict interpretation of a rigorous model or theory often lead to failure or terrific success whose scope is limited to research lab prototypes, adademic journals, and/or university tenures. On the other hand systems that seem to be a bit loose, but incoporate many of these nice theories as sub or mini components seem to enjoy better life. So for instance, a system that nicely implements but only implements regular expressions proper can't compete with a system that integrates regular expression as a mini-language. Similarly, a tool that just does parser generation is not as useful as a general-purpose language that also implements a parser generator as a mini-language.

I'm not trying to downgrade the relational model to the scope of regular expressions or a parser generator. But the idea is the same in that we don't all look at the world through the relational model, and the world is inherently more complex than the relational model. You can have a beautiful and theoretically rigorous implementation of the relational model. But you can't force people of all sorts of cognitive styles or different utility preferences to think about everything relationally or solve all the database problems via the relational model. The other models may not be as theoretically tight as the relational model, and may even be reducable to the relational model one way or another. But if I can express/solve a problem in a way easily (relationaly or not), that's the way I'll solve it.

Linchi,

I do not think being annoyed with the phrase is silly. In my life, I've seen so many databases with awful data quality, and so few with good quality. I have seen so many developers who were impatiently waiting to forget about relational model, databases, etc., and deal with facade only. Expression like this might lead those developers to a false thought that their dreams came through and we will get even more sloppy applications than we have nowadays.

Dejan;

In that case, perhaps for their sake we should then protect these developers from the relational model and give them just the entity relationship model. What's wrong with the MS approach to the entity relationship is that they didn't factor in the benefit of having some beer to help the developers focus on data quality.

Linchi,

Agree with that. I really do not care that much about the name, as long as the data integrity and quality is maintained, and there is some beer left:-)

Hello Linchi, Dejan

I prefer scotch to beer:)

You guys are raising all kinds of issues. I can't keep up with them all.

>Can you give a few examples of the flaws on the top of your list and their >damage?

There is the association of relational to mathemetics (set theory). So people

criticize sql based on this point of view. Sql allows duplicates rows, doesn't

require a table to have a key, dependencies based on ordinal position, is a poorly designed language etc. etc. These things really are critical but the real problem is the prevailing idea that relational is just a question of mathemetics.If it's just mathemetics then allowing duplicate rows is perceived as 'close enough'. All the objections from the set theory point view are not perceived as compelling

enough to really question the validity of sql. IMO the real holes of sql have

nothing to do with mathemetics. Rather it's the foundation, the computer science

if you will, that set theory and relational algebra are embedded in. This point of view is unfortunately not prevailing in IT. What the hell do I mean by the computer science of the relational model? Well first, the set theory that relational depends on is not some special kind of set theory. There is only one set theory. In the same way there is only one computer science, there is no special kind of computer science. But sql has invented such a special computer science and this is the biggest flaw. What am I talking about?:) Consider this:

Here is a table variable:

DECLARE @MyTableVar table(

   EmpID int NOT NULL primary key,

   OldVacationHours int,

   NewVacationHours int,

   ModifiedDate datetime);

Here is a server table:

create MyTable

      EmpID int NOT NULL primary,

      OldVacationHours int,

      NewVacationHours int,

      ModifiedDate datetime);

Here's the key question. If @MyTableVar really is a variable then what is MyTable? In other words, @MyTableVar is to variable as MyTable is to ?????. If MyTable is persisted in the database what is it persisted as? What computer science term describes it? Well whatever the hell it is (a constant?) it certainly isn't a 'variable'. And if it isn't a variable end of ballgame, end of relational model. And what of @MyTableVar? Bol says 'A table variable behaves like a local variable.' and at the same time says 'Assignment operation between table variables is not supported.'. When is a door not a door?..when it's ajar:) Who the hell ever heard of a variable that doesn't support assignment? Who ever heard of a variable that doesn't support comparison? No one. Whatever @MyTableVar really is it sure as hell ain't a variable. In a relational db I should be able to assign the table @MyTableVar, all its rows, to MyTable.

MyTable=@MyTableVar

And I should be able to compare them.

if MyTable=@MyTableVar

then print 'All rows in MyTable are in @MyTableVar and all rows in @MyTableVar

           are in MyTable'

else print 'Nope they're not equal'

A relational db demands a table be a variable just like an integer variable. Sql simply does not support basic computer science for tables. Whatever a table is in sql it doesn't have a 'type' because computer science is computer science and a variable must be typed. The only way sql can recognize a table is by its name not its type. This is why sql doesn't support relational division and why dynamic sql must be used to so much. A table as a variable is a completely different animal than a table in sql. This is why the expressive power of a relational db is orders of magnitude greater than an sql db. Sql views and constraints are redefined

relationally. The Types in Dates work:

'Databases, Types and the Relational Model, The Third Manifesto'

is about the central importance of variables of a particular type (a table as one of many types) in a relational db. What a table as a variable means and its

significance. It is really a basic computer science book. Ripping out the athematics of relational theory (at least trying to copy it), ie. the syntax to join, union tables, without the computer science of relational has done all the damage. MS can't change sql server because they are caught in an insane computer science. The difference in computer science between sql and net is the impedance mismatch they're trying address. But I'm afraid they still don't get the idea of a table as a variable. This is different than a table as a class. So their doing the same IBM did forty years ago. The damage is the difference between a pickup game in a playground and organized sports. You can draw up plays in the dirt but they

don't quite work the same as those run in a stadium. We're still doing application

development in the playground. Sometimes it works, sometimes it doesn't but we're

not basing it on the science of any model. Sql is not a model of anything, it's an

invention all its own. Close enough is only for horeshoes:) And maybe my blog will

make more sense now:)

Hi Experts

what does this mean to normal DB Developers/DBAs like us? should we be concerned staying in the DB world long with the fear that we become obsolete one day? I hate to see that happen and shift the career choice to become a C# developer (not that I hate the lnaguage) but my preference and forte being DB world. I am sure there are folks like me outside who will be worried if the DB becomes obsolete because of this shift. May be its just I am paranoid . Not really sure how it will shift the paradigm of the relational world.

Thanks

Ankith

>should we be concerned staying in the DB world long with the fear that we become

>obsolete one day?

Although I'm not an expert I can understand we're your coming from. It would

be nice to get a clear and concise answer to where MS is going and what you

should do about it. But there is no Oracle when it comes to MS. There is no

one position paper, no one person that clearly lays out their five year plan

and what it means to you. The experts here have enormous importance and influence in the db community. But they also have an enormous investment. How far can they be reasonably expected to go without putting themselves in an awkward position should they take a position that is not currently in line with company thinking? In the end it's a question of connecting the dots. You get a dot here a dot there. You have to do your homework. Study what they say and write and study what they offer. Sql server pros shouldn't neglect what's going on in VS and it's impact. If you study the company and the various technologies enough you should be able to draw your own picture. Think of it as the MS X-files:)

Specific DBMS platform aside, no matter what a vendor does there will always be a 'DB world' as long as there is data to be managed, in my opinion. But I do believe that some of the traditional DBA functions are going to disappear, be absorbed into the underlying plumbing, or become so trivial that doing those functions alone would not consitute a viable profession. The traditional functions I'm referring to are such things as creating databases/tables/users, doing backup/restore, checking database health, managing space usage, running scripts to deploy changes, collecting/compiling routine database performance info, and so on. Most of these are so-called production support DBA functions.

To be more realistic, I don't believe these functions will totally disappear. But large companies will ensure that they become so routine that they can be done en mass (highly automated), can be given to the users themselves, or can be given to the much less experienced (therefore the much less paid). As a result, the number of such jobs will shrink (going off shore for instance) and the pay may become much less attractive.

'Microsoft SQL Server 2008 and Microsoft Data Platform Development'

http://www.microsoft.com/sql/techinfo/whitepapers/sql_2008_dp.mspx

Does anyone find it the least bit odd that an sql server technical article is all

about VS, LINQ and the entity framework? At the expense of the logical model

and the sql language.

> Does anyone find it the least bit odd that an sql server technical article is all

> about VS, LINQ and the entity framework?

Well, it's more a marketing article than a technical article.

What Linchi Said is right. LINQ is more a marketing article than a tech jargon. I have always an arguement with my .NET friend on what LINQ Can and Cannot do. Ofcourse it tell me what's wrong in a Query plan and if there is a "merry go round scan" or "Paramater sniffing" happening. It has limitations.

As you both steve and Linchi pointed out that as db professionals all of us outside should also pay attention to the VS framework and I feel SSIS is a good start to get into that world.

Thanks for your encouragement