Search results matching tags 'Database Design' and 'Opinion'

Timewarp: What Is a Relational Database?

KKline — Wed, 05 Sep 2012 19:23:00 GMT

Relational?!? Move On, Geezer!

Maybe you're thinking that relational databases management systems (RDBMSs), like Microsoft SQL Server and Oracle, are going the way of punched cards and rotary phones. After all, there's been a lot of hype these days in the IT media about the rise of so-called NoSQL (Not Only SQL) databases. Many new and upcoming CS and MIS graduates who like working with data might think that relational databases are, at best, soon-to-be legacy systems and, at worst, are a career dead-end.

Wrong!!!

It's true that all the cool-cat computing services (Amazon, Facebook, Google, Pinterest, etc) are indeed making heavy use of NoSQL technology. They're also making heavy use of traditional RDBMS'es too. In fact, some of the world's biggest users of SQL databases are hand-in-hand the biggest users of NoSQL databases. The reason for that is that both types of data platforms are exceeding good at specific types of data storage and data processing. They also have their own unique weaknesses too. Meaning, each platform has a sweet spot and a weak spot, and that none are a 100% panacea for all imaginable data processing scenarios. Take a look at this article by my friend and former colleague, Guy Harrison - 10 Things You Should Know About NoSQL Databases, for a good discussion on the pros and cons of NoSQL in comparison to SQL data platforms.

Timewarp! Let's Take a Look Back at Why Relational Databases Were Needed.

These days, relational database management systems (RDBMSs) like Microsoft SQL Server and Oracle are the primary engines of information systems everywhere, particularly for enterprise computing systems and web applications. Though RDBMSs are now common enough to trip over, it wasn’t always that way. Not too long ago, you would probably trip over hierarchical database systems, or network database systems, or flat-file systems (heck, that still happens in many government IT shops who still use COBOL). A quick-and-dirty definition for a relation database might be: a system whose users view data as a collection of tables related to each other through common data values.

Perhaps you are interested in more than a quick-and-dirty definition for the term relational database? Here goes. The whole basis for the relational model follows this train of thought: data is stored in tables, which are composed of rows and columns. Tables of independent data can be linked, or related, to one another if they each have columns of data that represent the same data value, called keys. This concept is so common as to seem trivial; however, it was not so long ago that achieving and programming a system capable of sustaining the relational model was considered a longshot with limited usefulness.

Relational data theory was first proposed by E.F. Codd in his 1970 paper to the ACM entitled “A Relational Model of Data for Large Shared Data Banks”. Soon after, Codd clarified his position in the 1974 paper to the Texas Conference on Computing Systems entitled “The Relational Approach to Data Base Management: An Overview”. It was in this paper that Codd proposed the now legendary 12 Principles of Relational Databases.

If a vendor’s database product didn’t meet Codd’s 12 item litmus tests, then it was not a member of the club. Note that the rules do not apply to applications development. Instead, these rules determine whether the database engine itself can be considered truly “relational”. These rules were constructed to support a data model that would ensure the ACID properties of transactions and also eliminate a variety of data manipulation anomalies that frequently occurred on non-relation database platforms (and still do occur on non-relational database platforms). (As an aside, the transactional paradigm was conceived by my hero, Gray, Jim in 1981 while at Tandem Computer and presented in the paper "The Transaction Concept: Virtues and Limitations").

Codd’s 12 Rules for a Truly Relational Database System

Are you curious about Codd’s 12 Principles of Relational Databases? Don’t be ashamed that you don’t know them by heart; few technology professionals do, and no one on the marketing staff of technology companies do. However, the few folks who do know these principles by heart treat them like religious doctrine, and would likely be mortified by their “lightweight” treatment here. Nevertheless, I'll give them to you in my own paraphrasing:

Information is represented logically in tables.
Data must be logically accessible by table, primary key, and column.
Null values must be uniformly treated as “missing information” not as empty strings, blanks, or zeros.
Metadata (data about the database) must be stored in the database just as regular data is.
A single language must be able to define data, views, integrity constraints, authorization, transactions, and data manipulation.
Views must show the updates of their base tables and vice versa.
A single operation must be able to retrieve, insert, update, or delete data.
Batch and end-user operations are logically separate from physical storage and access methods.
Batch and end-user operations can change the database schema without having to recreate it or applications built upon it.
Integrity constraints must be available and stored in the metadata, not in an application program.
The data manipulation language of the relational system should not care where or how the physical data is distributed and should not require alteration if the physical data is centralized or distributed.
Any row-processing done in the system must obey the same integrity rules and constraints that set-processing operations do.

If you know much about SQL, then you probably recognize immediately that SQL ended up fulfilling rules #5, #7, #11 and possibly more. Others of the rule are manifest in the system tables of a relational database, such as DMVs in Microsoft SQL Server and V$ and X$ views in Oracle.

Relational Rises

There is some debate about why relational database systems won out over hierarchical and network database systems back in the late 1980's and early 1990's, but a couple of reasons seem self-evident. First, the high-level language interface (SQL) is much simpler to learn and more intuitive than that mishmash of languages supporting non-relational databases. (In fact, the lack of something like SQL is a hindrance to adoption of many NoSQL database platforms). Second, relational databases provide efficient and intuitive data structures that easily accommodate ad-hoc queries and reporting. People just intuitively understand the value of storing data in tables. From phone books to hotel registries, relational databases (of the paper sort) are second nature to most people. Third, relational databases provide powerful integrity controls such as check constraints and referential integrity - thus providing higher quality data. And high quality data is near and dear to the heart of CFOs around the world.

In fact, the strength that relational databases demonstrate with data quality, consistency, and durability are the same reasons that they'll be with us - quite possibly - forever. So were NoSQL databases excel at storing data that is moderately important and requires eventual consistency, SQL database excel at storing data that is of paramount importance and requires immediate consistency. As long as we're exchanging money, there's a need for relational database technology and ACID transactions.

And, just my opinion here, but database administration is currently, and will continue for decades to be, an excellent career choice. Why? First, although databases are widespread, good databases are not. So there's always need for those who can tune, troubleshoot, and optimize what is currently in the marketplace. Second, just because database are widespread doesn't mean that they're everywhere they need to be. Some estimates gauge that only half of the enterprises that need SQL databases actually use SQL databases. Imagine if only half of the citizenry wore shoes, and of the half that wore shoes, only half of them wore both shoes and consistently tied them. It'd be a good time to be a maker of loafers! Well, that's where we're at today with relational databases.

So what do you think? Am I off the mark on the longevity of relational database? Do you think the sun has set on them? Will they be smashed, degraded, and humiliated by NoSQL database platforms? Or will they stand shoulder-to-shoulder with a variety of data platforms in the years to come?

How Do You SKU?

KKline — Wed, 30 Mar 2011 14:49:00 GMT

I’d like your opinion here.

Follow my logic here for a moment as I walk through a couple rhetorical questions. Have you ever had a friend developed an application entirely on SQL Server Developer Edition? (Not that YOU would ever do such a thing, but maybe you know someone who has. Right?) And has your friend’s IT department actually deployed said application only to discover that they’re only licensed for Standard Edition in their production environment? And then was your friend’s IT management team is horrified to learn that they’ve either got to go through the very expensive process of extracting all of the Enterprise and/or Datacenter Edition features for the production application in order to remain in compliance, upgrade to the more expensive SKU licenses, or risk a potential future audit?

I’m not saying that this has happened to any of us. We’re too smart for that, after all. But have you ever known anyone who’s had this experience?

Having worked with a lot of customers another commercial RDBMS platforms (which I’ll euphemistically call “SEER” from Redforest City and “IB4” from Upstate City), I can tell you that auditing is a fun and exciting way for those platform vendors to make a LOT of money. This is especially true because a production application, once successfully deployed, tends to be too valuable to disable or otherwise compromise because high-end features slipped in to the development cycle even though the production environment only a “standard edition” SKU in place. Ouch! Talk about being caught between a rock and a hard place. Now, keep in mind that this is a strategy used by SEER and not by Microsoft. But Microsoft could implement the same sort of licensing audits if they wanted to. (Please leave a comment here if you have ever been audited. I’d love to hear your experiences, at least as much as NDA’s allow).

So if you use SQL Server Developer Edition (DE), of any version, would you like to see a feature that enables you to run DE not in its default “full featured mode” but at another SKU level, such as good ol’ Standard Edition? I know I would.

If you’re on the same page as I am, there are a number of suggestions logged on Connect about this very feature! Make your voice heard! Check out:

https://connect.microsoft.com/SQLServer/feedback/details/496380/enable-sql-developer-edition-to-target-specific-sql-version

Of course, the more skeptical reader might say “Hey, that’s their tough luck. Developers should know the difference in the SKU licensing options and feature sets of whatever SKU they’re developing on compared to what they’ll deploy on.” And I wouldn’t fault you for saying so.

But I would go on to point out that much of Microsoft’s success in enterprise IT settings can be traced back to their very strong relationship with developers. And anything that Microsoft can do to empower developers to save time, money, and resources during the development phase of an IT project in turn energizes that relationship between developer and Microsoft.

It also makes the life of the DBA that much easier, because they don’t need to imply that those cowboys on the development team went off half-cocked again. So what’s your opinion? Should SQL Server Developer Edition include a feature that sets the SKU-level of the database engine?

Adventures in the Land of CloudDB/NoSQL/NoAcid

KKline — Fri, 18 Feb 2011 13:44:00 GMT

Cloud, Bunny, or CloudBunny?

Last year, some of my friends from Quest Software attended Hadoop World in New York. In 2009, I never would've guessed that Quest would be there with products, community initiatives, as a major sponsor and with presenters?

There were just under 1,000 attendees who weren’t the typical devheads and geekasaurs you'd normally see at very techie events like Code Camps, SQL Saturdays, Cloud Camps and or even other NoSQL events such as the Cassandra Summit. We're talkin' enterprise customers with active Hadoop projects underway.

Some observations from the show that may be of interest to you:

- Hadoop World was a trending topic on Twitter during its duration.

- Hadoop has "arrived" with an average cluster of 66 nodes weighing in at 114TB. (For the philosophers among us, how much does a terabyte weigh?) The most famous Hadoop cluster is FaceBook with a trifling 30PB in storage - that's petabytes. That's more written information than has ever been written by man, cumulatively, including the Advice on Men column from Cosmo Magazine. Unfortunately, that's only a few hundred thousand pictures of teenagers pursing their lips at themselves and holding a digital camera while standing in front of the bathroom mirror. They're expecting about 60PB by the end of 2011.

- HP was there, creating a lot of buzz, from a hardware perspective.

- Quest was there as the leading independent tool maker for cloud apps.

- Oracle OraOop got attendees pulse's racing, since many want a high speed, scalable connector between Oracle and Hadoop to fill a necessary gap. I'm not sure if there's something in place for SQL Server and I'm not currently aware of any high-speed connectors built in to SQL Server Integration Services.

Some other good coverage to check out about the show as well:

Why Should You Care?

All of this is very important because NoSQL in general and Hadoop in particular are picking up speed and momentum. Even if your organization isn't using NoSQL technology today, chances are very good that your CIO will be asking you for details on how and when it should be deployed. And if you don't think it should be deployed, the natural response of the CIO is "Why not?". So you'd better get your ducks in a row, Mr SQL Server DBA.

There are lots of great sites to get Hadoop information, but I invite you to take a gander at Jeremiah Peschka's (blog | twitter) blog for much NoSQL goodness. Start with Jeremiah's blog post here, and ignore all indications that you might be in a biker bar or a San Francisco tattoo parlor. That's just Jeremiah's style.

His Hadoop writings are here, though lately he's been writing a lot about RIAK - which sounds like a euphemism for vomiting, as in "Jeremiah spent a lot of time riaking after chugging that bottle of cough syrup."

Enjoy! -Kev

Twitter at kekline

My Last "Catch-Up" Post for 2010 Content

KKline — Fri, 31 Dec 2010 20:57:00 GMT

I did a lot of writing in 2010. Unfortunately, I didn't do a good job of keeping all of that writing equally distributed throughout all of the channels where I'm active.

So here are a few more posts from my blog, put on-line during the months of November and December 2010, that I didn't get posted here on SQLBlog.com:

1. It's Time to Upgrade!

So many of my customers and many of you, dear readers, are still on SQL Server 2005. Join Kevin Kline, SQL Server MVP and SQL Server Technology Strategist for Quest Software and Brent Ozar, SQL Server Domain Expert for Quest Software as they introduce the top ten features and capabilities in SQL Server 2008 that they find to be the most exciting and valuable.

2. Dealing with the Micromanaging Boss

This is probably my favorite professional development article of the year. Micromanagers make us feel untrusted and stymied by their constant need for tediously detailed and frequent updates, constant changes to minor details of our work, and overly developed attention to administrative details that really don’t matter in our daily job. But there’s hope! Get all of my career advice on dealing with micromanaging bosses here.

3. [Video] Troubleshooting Memory Pressure on SQL Server

This is a little bit of an older video. But it's still useful info if you're working with SQL Server 2005 or 2000. Check it out here.

4. [DBTA] What the Heck is Microsoft's Database Product Trajectory?

I was once asked what I thought Microsoft's overall product trajectory for SQL Server was, in light of Oracle's rather obvious trajectory of acquiring multiple application vendors who will, in turn, deploy more and more of their applications to the Oracle database platform. You can read all about my thoughts in my monthly column at Database Trends & Application magazine - [READ MORE]

5. Eight Characteristics of Excellent Leaders [Plays Well With Others]

In this installment of my professional development column called Plays Well With Others, I talk about what distinguishes truly great leaders. There's a lively discussion on the topic and I encourage you to take part. Read it here.

6. The Shape of Database Licensing Costs to Come [DBTA]

One fall semester many years ago, I was a university freshman. Actually, I was anything but "fresh." I was dumb enough to think that 8 a.m. was a wonderful time to attend Economics 101. After staying up until the wee hours most every night, the "dismal science" took on more than one meaning as I set my clock just early enough to get to class on time. Along with 30 other very naïve classmates, I staggered into class and did my bleary-eyed best to focus on the lessons at hand. There were lots of Greek compound words and lots of graphs.

I learned, for example, that the word economics derives from the Greek "oikonomikos," which means, approximately, "death by slidedecks" and, specifically, "house" (oikos) and "management" (mikos). I barely survived the experience and never took an 8 a.m. class again. Imagine my surprise, then, when a lesson I'd learned (and promptly forgotten) all those years ago jumped back into my consciousness late last year. - [READ MORE]

7. So You're the Boss Now... [Plays Well with Others]

If you have any ambition at all, you have probably wanted (and possibly gotten) a promotion over your current colleagues. If you've been there before, you know that once friendly relationships can get, well, weird. Read my tips and tricks for how to make the most of this situation in this professional development article.

8. The NoSQL Movement - Hype or Hope?

NoSQL is no fad. And you need to be in the "know", not necessarily in the "No". Gno? Pneu? Too many phonetically similar pronunciations! Read my thoughts on the NoSQL movement on one of Database Trends & Applications magazine's most popular articles of the year. - [READ MORE]

9. Effectiveness and Efficiency at Work [Video]

In this golden oldie (yes, two years constitutes "old"), I present my thoughts on how to be both effective and efficient on the job and in life in general. And if you didn't know that these are different concepts, then you need this video most muchly.

10. What's Your Data Management and Retention Policy? [DBTA]

If managing your corporate data for the long term isn't currently on your mind, it should be, and in several different ways: cost, performance, business continuity, and compliance. [READ MORE]

Follow me on Twitter!

Originally Posted on YoutTube November 12, 2008.