Search results matching tag 'Trends'

Timewarp: What Is a Relational Database?

KKline — Wed, 05 Sep 2012 19:23:00 GMT

Relational?!? Move On, Geezer!

Maybe you're thinking that relational databases management systems (RDBMSs), like Microsoft SQL Server and Oracle, are going the way of punched cards and rotary phones. After all, there's been a lot of hype these days in the IT media about the rise of so-called NoSQL (Not Only SQL) databases. Many new and upcoming CS and MIS graduates who like working with data might think that relational databases are, at best, soon-to-be legacy systems and, at worst, are a career dead-end.

Wrong!!!

It's true that all the cool-cat computing services (Amazon, Facebook, Google, Pinterest, etc) are indeed making heavy use of NoSQL technology. They're also making heavy use of traditional RDBMS'es too. In fact, some of the world's biggest users of SQL databases are hand-in-hand the biggest users of NoSQL databases. The reason for that is that both types of data platforms are exceeding good at specific types of data storage and data processing. They also have their own unique weaknesses too. Meaning, each platform has a sweet spot and a weak spot, and that none are a 100% panacea for all imaginable data processing scenarios. Take a look at this article by my friend and former colleague, Guy Harrison - 10 Things You Should Know About NoSQL Databases, for a good discussion on the pros and cons of NoSQL in comparison to SQL data platforms.

Timewarp! Let's Take a Look Back at Why Relational Databases Were Needed.

These days, relational database management systems (RDBMSs) like Microsoft SQL Server and Oracle are the primary engines of information systems everywhere, particularly for enterprise computing systems and web applications. Though RDBMSs are now common enough to trip over, it wasn’t always that way. Not too long ago, you would probably trip over hierarchical database systems, or network database systems, or flat-file systems (heck, that still happens in many government IT shops who still use COBOL). A quick-and-dirty definition for a relation database might be: a system whose users view data as a collection of tables related to each other through common data values.

Perhaps you are interested in more than a quick-and-dirty definition for the term relational database? Here goes. The whole basis for the relational model follows this train of thought: data is stored in tables, which are composed of rows and columns. Tables of independent data can be linked, or related, to one another if they each have columns of data that represent the same data value, called keys. This concept is so common as to seem trivial; however, it was not so long ago that achieving and programming a system capable of sustaining the relational model was considered a longshot with limited usefulness.

Relational data theory was first proposed by E.F. Codd in his 1970 paper to the ACM entitled “A Relational Model of Data for Large Shared Data Banks”. Soon after, Codd clarified his position in the 1974 paper to the Texas Conference on Computing Systems entitled “The Relational Approach to Data Base Management: An Overview”. It was in this paper that Codd proposed the now legendary 12 Principles of Relational Databases.

If a vendor’s database product didn’t meet Codd’s 12 item litmus tests, then it was not a member of the club. Note that the rules do not apply to applications development. Instead, these rules determine whether the database engine itself can be considered truly “relational”. These rules were constructed to support a data model that would ensure the ACID properties of transactions and also eliminate a variety of data manipulation anomalies that frequently occurred on non-relation database platforms (and still do occur on non-relational database platforms). (As an aside, the transactional paradigm was conceived by my hero, Gray, Jim in 1981 while at Tandem Computer and presented in the paper "The Transaction Concept: Virtues and Limitations").

Codd’s 12 Rules for a Truly Relational Database System

Are you curious about Codd’s 12 Principles of Relational Databases? Don’t be ashamed that you don’t know them by heart; few technology professionals do, and no one on the marketing staff of technology companies do. However, the few folks who do know these principles by heart treat them like religious doctrine, and would likely be mortified by their “lightweight” treatment here. Nevertheless, I'll give them to you in my own paraphrasing:

Information is represented logically in tables.
Data must be logically accessible by table, primary key, and column.
Null values must be uniformly treated as “missing information” not as empty strings, blanks, or zeros.
Metadata (data about the database) must be stored in the database just as regular data is.
A single language must be able to define data, views, integrity constraints, authorization, transactions, and data manipulation.
Views must show the updates of their base tables and vice versa.
A single operation must be able to retrieve, insert, update, or delete data.
Batch and end-user operations are logically separate from physical storage and access methods.
Batch and end-user operations can change the database schema without having to recreate it or applications built upon it.
Integrity constraints must be available and stored in the metadata, not in an application program.
The data manipulation language of the relational system should not care where or how the physical data is distributed and should not require alteration if the physical data is centralized or distributed.
Any row-processing done in the system must obey the same integrity rules and constraints that set-processing operations do.

If you know much about SQL, then you probably recognize immediately that SQL ended up fulfilling rules #5, #7, #11 and possibly more. Others of the rule are manifest in the system tables of a relational database, such as DMVs in Microsoft SQL Server and V$ and X$ views in Oracle.

Relational Rises

There is some debate about why relational database systems won out over hierarchical and network database systems back in the late 1980's and early 1990's, but a couple of reasons seem self-evident. First, the high-level language interface (SQL) is much simpler to learn and more intuitive than that mishmash of languages supporting non-relational databases. (In fact, the lack of something like SQL is a hindrance to adoption of many NoSQL database platforms). Second, relational databases provide efficient and intuitive data structures that easily accommodate ad-hoc queries and reporting. People just intuitively understand the value of storing data in tables. From phone books to hotel registries, relational databases (of the paper sort) are second nature to most people. Third, relational databases provide powerful integrity controls such as check constraints and referential integrity - thus providing higher quality data. And high quality data is near and dear to the heart of CFOs around the world.

In fact, the strength that relational databases demonstrate with data quality, consistency, and durability are the same reasons that they'll be with us - quite possibly - forever. So were NoSQL databases excel at storing data that is moderately important and requires eventual consistency, SQL database excel at storing data that is of paramount importance and requires immediate consistency. As long as we're exchanging money, there's a need for relational database technology and ACID transactions.

And, just my opinion here, but database administration is currently, and will continue for decades to be, an excellent career choice. Why? First, although databases are widespread, good databases are not. So there's always need for those who can tune, troubleshoot, and optimize what is currently in the marketplace. Second, just because database are widespread doesn't mean that they're everywhere they need to be. Some estimates gauge that only half of the enterprises that need SQL databases actually use SQL databases. Imagine if only half of the citizenry wore shoes, and of the half that wore shoes, only half of them wore both shoes and consistently tied them. It'd be a good time to be a maker of loafers! Well, that's where we're at today with relational databases.

So what do you think? Am I off the mark on the longevity of relational database? Do you think the sun has set on them? Will they be smashed, degraded, and humiliated by NoSQL database platforms? Or will they stand shoulder-to-shoulder with a variety of data platforms in the years to come?

New on "Database Trends & Applications"

KKline — Tue, 16 Aug 2011 19:37:00 GMT

In last month's column, "2012 Might Really Be the End of the World as We Know It," I described a number of major developments in the IT industry that are likely to disrupt the life of database professionals everywhere. I categorize those four disruptors - virtualization, cloud computing, solid state drives (SSD), and advanced multi-core CPUs - into two broad groups. I'm going to continue an analysis of these disruptive technologies in inverse order. Today, let's discuss SSDs.

[READ MORE ON DATABASE TRENDS & APPLICATIONS ]

Managing Complex DB Environments

KKline — Wed, 03 Aug 2011 15:05:00 GMT

Check out the new white paper “Key Methods for Managing Complex DB Environments” at Quest.com. Heterogeneous database management hassles used to be much less common than today. When I first started at Quest nearly ten years ago, I'd put forward that on 15-20% of my big customers regularly managed more than one major database platform. (That doesn't mean they didn't have more than one, rather secondary platforms were either unmanaged or considered entirely unimportant). Today, I'd put forward that 70-80% of my big customers support at least two major database platforms. Often, they support three or more.

This white paper addresses methods for successfully managing today’s complex heterogeneous database infrastructures. Topics discussed include: balancing key business metrics, understanding challenges to the DBA, and managing multiple database platforms (i.e. Oracle, SQL Server, DB2, Sybase) simultaneously.

Enjoy,

-Kev

P.S. Follow me on Twitter!

What I'm Reading, July 22 2011

KKline — Thu, 21 Jul 2011 14:43:00 GMT

I read too much, and that, my friends, is an entirely separate topic for a blog post. But I thought I'd share with you a little more about what I'm reading because sometimes, if I'm lucky, it might be something you'd enjoy too. So I'm going to start sharing what I'm reading at least once per week, partly so that I don't firehose too many reading links directly into your brain (where I to do it say once per month) and partly to solidify in my own mind the information that I'm reviewing. So here are a few good links for the seven days leading up to July 22, 2001:

Microsoft and Whitehouse partnership on BigData: BigData isn't a particularly new concept. But I was intrigued to learn that the National Science Foundation, Microsoft, and 13 other teams were partnering on developing better BigData analytics for lots of government data from activities such as healthcare, economic development, education, transportation, and the power grid. Cools stuff! Plus, Microsoft has developed a new tool called Project Daytona to better harness the power of the cloud, in general, and Windows Azure, specifically.
While we're on the topic of Federal IT in the Cloud be sure to read this linked article from ComputerWorld. Say what you will about our government, but putting government IT in the cloud and increasing both its transparency and availability will make a huge difference in how the Federal government will be able to service the public. We're talking as big a difference as corporations experienced between the "catalog on the web" experience of the 1990's to the Web2.0 experience of today.
If you're the social media type, give this article a read discussing the Power of Hashtags in Social Media.
The Register, of the UK, whose tagline is "Biting the hand that feeds IT" has a great article on a spat over database technologies between the IT sage Michael Stonebreaker and Google. It's a great read if for no other reason than to prove that databases are worth fighting over.
And if you think Microsoft is still towing the relational database barge without thinking about other technologies, you need to read up on Projects Dryad and Daytona.
Finally, I'm still getting lots of questions about when and where to limit SQL Server's Max Degrees of Parallelism. Be sure to read Microsoft's Recommendations and Guidelines for 'max degree of parallelism' configuration option here.

And just because so many of us in IT are closet or former musicians, there's Live Guitar Lessons with Steven Krenz, sponsored by my hometown boyz at Gibson Guitar. Got a favorite article or tool tip? Let me know! Enjoy,

-Kev

Follow me on Twitter.

Getting Ahead of the Curve – Big Data

KKline — Thu, 14 Jul 2011 14:41:00 GMT

I have to confess that I'm incredibly excited about BigData. I haven't been this excited about new innovations in IT since relational databases first appeared on the scene early in my career. But what is BigData? Back in those days, I can still feel the echos of adrenaline when I was hired to work on a NASA project that would involve over 100Mb of data. ONE HUNDRED MEGABYTES! Good grief, that was fantastically huge to us on the team. (That database was over 130Mb when I finally moved on to another project). And remember - PC software was installed using 640Kb floppy disks at the time. In fact, my Oracle v5 instance required shuffling through about a dozen floppy disks to get the thing installed on a 286 IBM PC. BigData today takes on an entirely meaning as database sizes scale into the petabytes. But the emphasis is still the same today as it was back in the 1980's - turning data into actionable information. However, with BigData, we can achieve amazing new insight from this data and mine for tidbits that would never have seen the light of day with smaller data sets. The two major themes to remember about big data are 1) the more data you have on a given domain, the more power you have, 2) the better the analysis you can perform on the data, the more power you have. In fact, theme 2 might be the most important thing to consider because lots of data is meaningless unless you can extract knowledge from it. And that's where better analytical techniques come into play. Here are some articles about Big Data that you might enjoy:

Let me know what you think. Best regards, -Kev

Follow me on Twitter at kekline

NOSQL- A Quick Overview

KKline — Fri, 03 Jun 2011 03:55:00 GMT

Several attendees at the SQLRally were asking about NoSQL ("Not Only SQL") and its benefits. This article gives a quick overview: HERE.

Also, two good resources have come out from my friend Guy Harrison. This piece on Cassandra, one of the most popular NoSQL databases, was published a while back in Database Trends and Applications Magazine. I also really liked these entries from Guy at TechRepublic and TechTarget. Guy also had the good fortune of having one of his articles published on GigaOm and then picked up in the New York Times Technology section.

The article, titled “Real World NoSQL: HBase at Trend Micro,” is the first in a five-part series Guy wrote, spotlighting NoSQL (non-relational) database deployments at five different companies. The other good learning experience for me was hearing about GigaOm for the first time. GigaOm, now on my reading list, is considered an influential and prestigious publication in the NoSQL realm.

Let me know what you think. Enjoy!

-Kev

~~~

Twitter at kekline

Adventures in the Land of CloudDB/NoSQL/NoAcid

KKline — Fri, 18 Feb 2011 13:44:00 GMT

Cloud, Bunny, or CloudBunny?

Last year, some of my friends from Quest Software attended Hadoop World in New York. In 2009, I never would've guessed that Quest would be there with products, community initiatives, as a major sponsor and with presenters?

There were just under 1,000 attendees who weren’t the typical devheads and geekasaurs you'd normally see at very techie events like Code Camps, SQL Saturdays, Cloud Camps and or even other NoSQL events such as the Cassandra Summit. We're talkin' enterprise customers with active Hadoop projects underway.

Some observations from the show that may be of interest to you:

- Hadoop World was a trending topic on Twitter during its duration.

- Hadoop has "arrived" with an average cluster of 66 nodes weighing in at 114TB. (For the philosophers among us, how much does a terabyte weigh?) The most famous Hadoop cluster is FaceBook with a trifling 30PB in storage - that's petabytes. That's more written information than has ever been written by man, cumulatively, including the Advice on Men column from Cosmo Magazine. Unfortunately, that's only a few hundred thousand pictures of teenagers pursing their lips at themselves and holding a digital camera while standing in front of the bathroom mirror. They're expecting about 60PB by the end of 2011.

- HP was there, creating a lot of buzz, from a hardware perspective.

- Quest was there as the leading independent tool maker for cloud apps.

- Oracle OraOop got attendees pulse's racing, since many want a high speed, scalable connector between Oracle and Hadoop to fill a necessary gap. I'm not sure if there's something in place for SQL Server and I'm not currently aware of any high-speed connectors built in to SQL Server Integration Services.

Some other good coverage to check out about the show as well:

Why Should You Care?

All of this is very important because NoSQL in general and Hadoop in particular are picking up speed and momentum. Even if your organization isn't using NoSQL technology today, chances are very good that your CIO will be asking you for details on how and when it should be deployed. And if you don't think it should be deployed, the natural response of the CIO is "Why not?". So you'd better get your ducks in a row, Mr SQL Server DBA.

There are lots of great sites to get Hadoop information, but I invite you to take a gander at Jeremiah Peschka's (blog | twitter) blog for much NoSQL goodness. Start with Jeremiah's blog post here, and ignore all indications that you might be in a biker bar or a San Francisco tattoo parlor. That's just Jeremiah's style.

His Hadoop writings are here, though lately he's been writing a lot about RIAK - which sounds like a euphemism for vomiting, as in "Jeremiah spent a lot of time riaking after chugging that bottle of cough syrup."

Enjoy! -Kev

Twitter at kekline

Cloud Evolving, SQL Server Responding

KKline — Wed, 02 Feb 2011 14:49:00 GMT

Brent Ozar (blog | twitter) and I did an interview with TechTarget’s Brendan Cournoyer at last summer's Tech-Ed, which as turned into a podcast titled “Cloud efforts advance, SQL Server evolves.” The podcast covers all the major trends at the conference (like BI), virtualization features in Quest’s products (like Spotlight), Brent’s new book and MCM certification, and more. Here’s a link to hear it, appearing on 6/11/10: http://searchsqlserver.techtarget.com/podcast/Cloud-efforts-advance-SQL-Server-evolves.

Enjoy!

-Kev

Twitter at kekline

My Last "Catch-Up" Post for 2010 Content

KKline — Fri, 31 Dec 2010 20:57:00 GMT

I did a lot of writing in 2010. Unfortunately, I didn't do a good job of keeping all of that writing equally distributed throughout all of the channels where I'm active.

So here are a few more posts from my blog, put on-line during the months of November and December 2010, that I didn't get posted here on SQLBlog.com:

1. It's Time to Upgrade!

So many of my customers and many of you, dear readers, are still on SQL Server 2005. Join Kevin Kline, SQL Server MVP and SQL Server Technology Strategist for Quest Software and Brent Ozar, SQL Server Domain Expert for Quest Software as they introduce the top ten features and capabilities in SQL Server 2008 that they find to be the most exciting and valuable.

2. Dealing with the Micromanaging Boss

This is probably my favorite professional development article of the year. Micromanagers make us feel untrusted and stymied by their constant need for tediously detailed and frequent updates, constant changes to minor details of our work, and overly developed attention to administrative details that really don’t matter in our daily job. But there’s hope! Get all of my career advice on dealing with micromanaging bosses here.

3. [Video] Troubleshooting Memory Pressure on SQL Server

This is a little bit of an older video. But it's still useful info if you're working with SQL Server 2005 or 2000. Check it out here.

4. [DBTA] What the Heck is Microsoft's Database Product Trajectory?

I was once asked what I thought Microsoft's overall product trajectory for SQL Server was, in light of Oracle's rather obvious trajectory of acquiring multiple application vendors who will, in turn, deploy more and more of their applications to the Oracle database platform. You can read all about my thoughts in my monthly column at Database Trends & Application magazine - [READ MORE]

5. Eight Characteristics of Excellent Leaders [Plays Well With Others]

In this installment of my professional development column called Plays Well With Others, I talk about what distinguishes truly great leaders. There's a lively discussion on the topic and I encourage you to take part. Read it here.

6. The Shape of Database Licensing Costs to Come [DBTA]

One fall semester many years ago, I was a university freshman. Actually, I was anything but "fresh." I was dumb enough to think that 8 a.m. was a wonderful time to attend Economics 101. After staying up until the wee hours most every night, the "dismal science" took on more than one meaning as I set my clock just early enough to get to class on time. Along with 30 other very naïve classmates, I staggered into class and did my bleary-eyed best to focus on the lessons at hand. There were lots of Greek compound words and lots of graphs.

I learned, for example, that the word economics derives from the Greek "oikonomikos," which means, approximately, "death by slidedecks" and, specifically, "house" (oikos) and "management" (mikos). I barely survived the experience and never took an 8 a.m. class again. Imagine my surprise, then, when a lesson I'd learned (and promptly forgotten) all those years ago jumped back into my consciousness late last year. - [READ MORE]

7. So You're the Boss Now... [Plays Well with Others]

If you have any ambition at all, you have probably wanted (and possibly gotten) a promotion over your current colleagues. If you've been there before, you know that once friendly relationships can get, well, weird. Read my tips and tricks for how to make the most of this situation in this professional development article.

8. The NoSQL Movement - Hype or Hope?

NoSQL is no fad. And you need to be in the "know", not necessarily in the "No". Gno? Pneu? Too many phonetically similar pronunciations! Read my thoughts on the NoSQL movement on one of Database Trends & Applications magazine's most popular articles of the year. - [READ MORE]

9. Effectiveness and Efficiency at Work [Video]

In this golden oldie (yes, two years constitutes "old"), I present my thoughts on how to be both effective and efficient on the job and in life in general. And if you didn't know that these are different concepts, then you need this video most muchly.

10. What's Your Data Management and Retention Policy? [DBTA]

If managing your corporate data for the long term isn't currently on your mind, it should be, and in several different ways: cost, performance, business continuity, and compliance. [READ MORE]

Follow me on Twitter!

Originally Posted on YoutTube November 12, 2008.

Google Wave is Dead! Long Live the Wave!

KKline — Tue, 10 Aug 2010 23:16:00 GMT

I Never Could Hang 10 (Minutes) on Google Wave

http://bits.blogs.nytimes.com/2010/08/04/google-kills-wave-its-collaboration-tool/?ref=technology

Even if you don't read the article, you can see from the URL that Google has decided to put an end to the collaboration experiment known as Wave. Wave will be available through the end of the year and most of its major components are now available as open source, should any devotees choose to continue developing the code base. However, Wave didn't reach the critical mass that Google was looking for and, without that critical mass of users, it wasn't seeing a lot of innovation or updates to the features or UI. I view Google's reach of 1M users as a "failure" with a bit of grin. How many other vendors out there would consider 1M users too few? Otoh, if they wanted really wide adoption, why in the world did they require a private invitation? Superior products are frequently hampered by inferior marketing and market delivery, this being a really good example.

Slide to the Rescue?

Great Idea Leads To Great Product Success, Right?

The Graveyard of Ideas

I also feel that Google Wave is a good example of a technological solution looking for a problem, as well as a product looking for a marketing message. When launching a product, it's crucial to have a crystal clear message to a well-defined audience. Any ambiguity in the message or muddling of the audience can spell doom. And, IMO, Google clearly missed the boat on both counts. Many of their demos were all about sharing photos. Uh, ever heard of Facebook, n'est pas? Then again, many later PR was about collaboration. Then how come we didn't get smokin' hot project management demos? A book that I recommend called The Innovator's Prescription (website is here) says it very well:

"The graveyard of failed products and services is populated by things that people *should* have wanted--if only they could have been convinced those things were good for them. The home-run products in the marketing hall of fame, in contrast, are concepts that helped people more affordably, effortlessly, swiftly, and effectively do what they already had been trying to get done." (Christensen, The Innovator's Prescription, p. 16)

I really like Christensen's point. So many people who build products focus on the "should" of a product, as in "this should make a lot of people happy", over and above providing an effortless aid to people's daily tasks. This leads me to a topic for another day, user-interface design. But enough writing for now. It's bed time.

So what are your thoughts? Do you think other factors contributed to Wave's decline?

Thanks!

-Kevin

Twitter @kekline

[caption id="" align="alignright" width="305" caption="I Never Could Hang 10 (Minutes) on Google Wave"][/caption]

While enduring an endless series of flight delays and disgruntled passengers in the Baltimore airport that was my own personal travel hell on the evening of Thursday, August 5th, I came across this interesting and important article: http://bits.blogs.nytimes.com/2010/08/04/google-kills-wave-its-collaboration-tool/?ref=technology Even if you don't read the article, you can see from the URL that Google has decided to put an end to the collaboration experiment known as Wave. Wave will be available through the end of the year and most of its major components are now available as open source, should any devotees choose to continue developing the code base. However, Wave didn't reach the critical mass that Google was looking for and, without that critical mass of users, it wasn't seeing a lot of innovation or updates to the features or UI. I view Google's reach of 1M users as a "failure" with a bit of grin. How many other vendors out there would consider 1M users too few? Otoh, if they wanted really wide adoption, why in the world did they require a private invitation? Superior products are frequently hampered by inferior marketing and market delivery, this being a really good example.

Slide to the Rescue?

[caption id="" align="alignleft" width="320" caption="More Fun than Google Slide?"][/caption] I also feel the need to point out that I have a lot of respect for Google giving the old heave-ho to a product that needs to go. Many companies cling to a great idea, funneling huge amounts of resources into what everyone else can see as a black hole. Failure, under vibrant and forward thinking leadership, is only success delayed. Read Google's take on the situation here. Certainly, this means we'll see Google pushing their new social media acquisition, Slide, much more as well. And, since many of the technological bits of Wave will live on, I'm sure we'll see Slide advance in interesting ways. Frankly, I found the general idea of Wave to be fascinating and powerful. But after spending quite a bit of time, like at least 20 minutes, tinkering around with it, I still had no idea how to do anything with it. I was so motivated to use it that I almost watched one of the videos that they'd posted to train you. But honestly, am I just ridiculously jaded or has the overall market for cloud-based apps moved the bar for ease-of-use that anything that takes more than 15 minutes to figure out is drama? I hate to say it, but I think the answer is a resounding "YES". By extension, I think that this is the main reason that email still trumps all other methods of collaboration. (Yes, that includes Microsoft SharePoint too for all you fanboys.) That is, email does not disrupt any existing workflows, it has a clean UI, it doesn't make you learn new ways of working, and it's so widespread that you're not hampered by a product that has a very limited user base.

Great Idea Leads To Great Product Success, Right?

[caption id="" align="alignright" width="341" caption="The Graveyard of Ideas"][/caption] I also feel that Google Wave is a good example of a technological solution looking for a problem, as well as a product looking for a marketing message. When launching a product, it's crucial to have a crystal clear message to a well-defined audience. Any ambiguity in the message or muddling of the audience can spell doom. And, IMO, Google clearly missed the boat on both counts. Many of their demos were all about sharing photos. Uh, ever heard of Facebook, n'est pas? Then again, many later PR was about collaboration. Then how come we didn't get smokin' hot project management demos? A book that I recommend called The Innovator's Prescription (website is here) says it very well: