|
|
|
|
-
A lot of my time lately has been spent getting ready for next week and I hope to see you there. First up SQL Rally. On Wednesday, I will be doing a pre-con session on Database Design. I have already blogged about it here, and I am definitely excited about it. There are still seats available, and I will be giving away a few goodies including my new design book (though it won’t yet be printed, I will get it to attendees), several of my, Tim Ford and Glenn Berry’s DMV books from RedGate, and who know what else I might have in my bag (Apress is sending me a lot of swag to give away during the conference too). Friday (last session of the conference), I will be presenting my new session entitled Sequences (What They Are and Are Not). This session is a great departure from my typical presentation. I typically do 95% slides, and sometimes 100% slides. In this presentation I currently have 17 slides, and that will only change to include a bit more contact information. I have a lot of code using sequences in multiple ways, including racing them against identities and different caching levels. After the rally ends, I will hop in my yellow Focus and make the following delightful journey:  Assuming traffic is kind to me and Bing’s time estimates are reasonably correct, a bit over 9 hours of driving and a night in a Hampton Inn to be named later and I should arrive at Birmingham’s SQL Saturday event. Sounds insane right? Well, yeah, it kinda does. But I wanted to make their event again this year, and it was just a couple of hours out of my way back to Nashville. You may also think it sound like a bit of vanity, and it isn’t really. Assuming traffic is favorable, my hope is that I can arrive for the 2:00 sessions by Chris Price on Data Quality (or Kevin Boles on Column Store Indexes), Sven Aelterman’s session on File Tables as well before my 4:30 session on Sequences. Obviously if traffic is bad it might turn out that I only make my own session, but even then I plan to hang out for the after event so it won’t be a total loss. In any case, hope to see you all in Dallas or Birmingham next week!
|
-
Very often, I find myself wanting to query system metadata for columns. Some of the metadata is fairly easy to deal with, like if you want to find column names, just simply querying sys.columns is very straightforward. But if you want to query data types, it gets stickier. There are type types listed, one that is the declared type, and another for the base data type, which if you are using CREATE TYPE to create alias data types. So I started working on the query based on INFORMATION SCHEMA values, because it is a lot easier to work with. In my design book, I used that version (and will link this version in erratta/book updates. But in the Books Online entry for the COLUMNS object (and others too), I discovered the following warning, a bit too late the change: “Do not use INFORMATION_SCHEMA views to determine the schema of an object. The only reliable way to find the schema of a object is to query the sys.objects catalog view.” So I rewrote it to use the catalog views. The query is fairly complex looking, but overall is pretty simple. I put comments through the code that explains what it does. I will note that I have tested the database using a case sensitive collation, but I haven’t tried it with a and CLR types. The query returns the following: - table_type – The type from sys.tables. Could be a user table, system table, user view, internal table
- schema_name – The name of the schema of the table that the column is in
- table_name – The name of the table
- column_name – The name of the column
- nullability – Whether that column is nullable, plus if the column is an identity column
- declared_datatype – The data type as declared. For alias types (like sysname), this will be the name the user specified.
- base_datatype – The data type as is implemented. For alias types this will be the base intrinsic type (like for sysname this will be nvarchar(128))
- primary_key_column – A bit value, indicating that the column participates in a primary key constraint
- column_id – The internal key of the column, which is used for ordering the columns
- default_value – If there is a default constraint on the column, this will be the text of the declaration
- column_check_constraint – If there is a check constraint that only deals with the one column, it will be considered a column check constraint. This will contain the text of the check constraint
- table_check_constraint_reference – A bit value that indicates that there is one or more table level check constraints that reference the column
The query text follows. It is pretty unwieldy, so instead of the derived table you might want to consider building it into a view or even a procedure. I keep a SSMS project with all of my metadata queries that I maintain over time to call in any place I need them, so I use use this one ad-hoc. You can download the query from my website’s downloadable packages page here: http://www.drsql.org/Documents/ColumnMetadataQuery.sql SELECT * FROM ( SELECT REPLACE(LOWER(objects.type_desc), '_', ' ') AS table_type, schemas.name AS schema_name, objects.name AS table_name, columns.name AS column_name, CASE WHEN columns.is_identity = 1 THEN 'IDENTITY NOT NULL' WHEN columns.is_nullable = 1 THEN 'NULL' ELSE 'NOT NULL' END AS nullability, --types that have a ascii character or binary length CASE WHEN columns.is_computed = 1 THEN 'Computed' WHEN types.name IN ( 'varchar', 'char', 'varbinary' ) THEN types.name + CASE WHEN columns.max_length = -1 THEN '(max)' ELSE '(' + CAST(columns.max_length AS VARCHAR(4)) + ')' END --types that have an unicode character type that requires length to be halved WHEN types.name IN ( 'nvarchar', 'nchar' ) THEN types.name + CASE WHEN columns.max_length = -1 THEN '(max)' ELSE '(' + CAST(columns.max_length / 2 AS VARCHAR(4)) + ')' END --types with a datetime precision WHEN types.name IN ( 'time', 'datetime2', 'datetimeoffset' ) THEN types.name + '(' + CAST(columns.scale AS VARCHAR(4)) + ')' --types with a precision/scale WHEN types.name IN ( 'numeric', 'decimal' ) THEN types.name + '(' + CAST(columns.precision AS VARCHAR(4)) + ',' + CAST(columns.scale AS VARCHAR(4)) + ')' --timestamp should be reported as rowversion WHEN types.name = 'timestamp' THEN 'rowversion' --and the rest. Note, float is declared with a bit length, but is --represented as either float or real in types ELSE types.name END AS declared_datatype, --types that have a ascii character or binary length CASE WHEN baseType.name IN ( 'varchar', 'char', 'varbinary' ) THEN baseType.name + CASE WHEN columns.max_length = -1 THEN '(max)' ELSE '(' + CAST(columns.max_length AS VARCHAR(4)) + ')' END --types that have an unicode character type that requires length to be halved WHEN baseType.name IN ( 'nvarchar', 'nchar' ) THEN baseType.name + CASE WHEN columns.max_length = -1 THEN '(max)' ELSE '(' + CAST(columns.max_length / 2 AS VARCHAR(4)) + ')' END --types with a datetime precision WHEN baseType.name IN ( 'time', 'datetime2', 'datetimeoffset' ) THEN baseType.name + '(' + CAST(columns.scale AS VARCHAR(4)) + ')' --types with a precision/scale WHEN baseType.name IN ( 'numeric', 'decimal' ) THEN baseType.name + '(' + CAST(columns.precision AS VARCHAR(4)) + ',' + CAST(columns.scale AS VARCHAR(4)) + ')' --timestamp should be reported as rowversion WHEN baseType.name = 'timestamp' THEN 'rowversion' --and the rest. Note, float is declared with a bit length, but is --represented as either float or real in types ELSE baseType.name END AS base_datatype, CASE WHEN EXISTS ( SELECT * FROM sys.key_constraints JOIN sys.indexes ON key_constraints.parent_object_id = indexes.object_id AND key_constraints.unique_index_id = indexes.index_id JOIN sys.index_columns ON index_columns.object_id = indexes.object_id AND index_columns.index_id = indexes.index_id WHERE key_constraints.type = 'PK' AND columns.column_id = index_columns.column_id AND columns.OBJECT_ID = index_columns.OBJECT_ID ) THEN 1 ELSE 0 END AS primary_key_column, columns.column_id, default_constraints.definition AS default_value, check_constraints.definition AS column_check_constraint, CASE WHEN EXISTS ( SELECT * FROM sys.check_constraints AS cc WHERE cc.parent_object_id = columns.OBJECT_ID AND cc.definition LIKE '%~[' + columns.name + '~]%' ESCAPE '~' AND cc.parent_column_id = 0 ) THEN 1 ELSE 0 END AS table_check_constraint_reference FROM sys.columns JOIN sys.types ON columns.user_type_id = types.user_type_id JOIN sys.types AS baseType ON columns.system_type_id = baseType.system_type_id AND baseType.user_type_id = baseType.system_type_id JOIN sys.objects JOIN sys.schemas ON schemas.schema_id = objects.schema_id ON objects.object_id = columns.OBJECT_ID LEFT OUTER JOIN sys.default_constraints ON default_constraints.parent_object_id = columns.object_id AND default_constraints.parent_column_id = columns.column_id LEFT OUTER JOIN sys.check_constraints ON check_constraints.parent_object_id = columns.object_id AND check_constraints.parent_column_id = columns.column_id ) AS rows WHERE table_type = 'user table' AND schema_name LIKE '%' AND table_name LIKE '%' AND column_name LIKE '%' AND nullability LIKE '%' AND base_datatype LIKE '%' AND declared_datatype LIKE '%' ORDER BY table_type, schema_name, table_name, column_id
|
-
On May 9, 2012, I will be presenting a pre-con session at the SQL Rally in Dallas, TX on relational database design. The fact is, database design is a topic that demands more than a simple one hour session to really do it right. So in my Relational Database Design Workshop, we will have seven times the amount of time in the typical session, giving us time to cover our topics in a bit more detail, look at a lot more designs/code, and even get some time to do some design as a group. Our topics will be: - Theory - More or less the foundational principals and processes that will presumably help you understand the "why" behind the rest of the material.
- General Data Modeling - The basic concepts behind database design, data modeling, graphically and semantically. Terminology and concepts will be covered to make sure that when I say a word, you know what I am meaning. The field of computer science is littered with confusing terminology that needs to be made clear.
- Normalization - Basically the process of making your database work well with the relational engine form both a performance and data integrity point of view.
- Physical Modeling - The actual process of creating a working database by choosing proper data types, protecting the data integrity, etc. We will discuss the steps, as well as take a look at a model that is implemented.
- Implementation Patterns- For the most part, the primary pattern that we use in a relational database is normalization. Database, tables and columns are rather easily mapped to requirements, and using normalization, we usually will arrive at a solution.However, there are certain types of solutions that crop up in common implementations. Examples include uniqueness, hierarchies, files, user-specified schema, data driven design, and more.
- Other Miscellaneous Advice - Time permitting, I have a set of slides that cover some basic performance/security/implementation related material.
A lot of the material/examples/demos come from my 2012 book (SQL Server 2012 Relational Database Design and Implementation) that will be shipping just before the Rally, so I am making arrangements to get copies of the book for most if not all of the attendees. So the day will introduce the material to you in a rapid format, and then you can take the material home and read it again (and my email is always available for further questions on the material as well.)
|
-
Finally… took longer than I had expected when I wrote this a while back, but I had to move my website and get DNS moved before I could post code… When I write code, I do my best to test that code in as many ways as necessary. One of the last types of tests that is necessary is concurrency testing. Concurrency testing is one of the most difficult types of testing because it takes running multiple processes simultaneously and making sure that you get the correct answers multiple times. This is really difficult when you need to make > 1 statement that takes only a second or two simultaneously with another (or even 10 other) connections. What I am trying to replicate is the WAITFOR statement with a DELAY, but multiple connections use the same delay. As I am writing my sequences presentation for SQL Rally, I wanted to test how values were allocated to each row based on different caching levels. I was looking at the clock, creating WAITFOR TIME statements, and copying the values into multiple windows as I have done so many times before. I was also thinking that I should start blogging again, and (like my sequence presentation) do something code oriented rather than design for a bit (I just finished my design book, so I am going to rest on that subject for a bit, well, other than the SQL Rally precon that I am subliminally trying to get you to sign up for before prices go up again). So envisioned a kind of multi-user WAITFOR statement that would say: Start all processes in 20 seconds, then start multiple connections. (As I write this, I am envisioning a kind of semaphore version of this that you could hold until ready to go, but more on that some other day.) I initially used some 2012 features like FORMAT, but I wanted to make sure this was useful to a more wide audience. The usage of the system is really simple, to have your connection wait for 20 seconds, you execute: EXEC Utility.WaitForSync$StartWait @WaitSeconds=20 To have multiple connections start at the same second, you execute the same statement on a different connection. The stored procedure calculates the time in 20 seconds, stores the value off and executes a WAITFOR TIME statement on each connection you have executed this statement on. After the time passes and the connection continues, you have to reset the connection or you will receive the following message (not an error, just a PRINT statement, since you may want the batch to complete): WaitForSync: An error occurred. Message: Too late to start another WAITFOR session. Last session DELAY time was 2012-03-20 17:55:03. Use Utility.WaitForSync$reset to start new sessions To reset the session, as the message says, just execute Utility.WaitForSync$reset. There are also procedures to view the time when the processes will continue. If you want to not do the action if the WAITFOR TIME statements have already completed before you start an operation, the procedure will return a –100. So you might use something like the following TRY…CATCH Block to control execution: BEGIN TRY DECLARE @RETVAL int EXEC @RETVAL = Utility.WaitForSync$StartWait @WaitSeconds=10 IF @RETVAL = -100 RAISERROR ('Time Elapsed',16,1) --Other code END TRY BEGIN CATCH PRINT ERROR_MESSAGE() END CATCH Which would return something like: WaitForSync: An error occurred. Message: Too late to start another WAITFOR session. Last session DELAY time was 2012-03-20 21:20:06. Use Utility.WaitForSync$reset to start new sessions Time Elapsed
You can download the code from my website’s downloadable code page. Of course, the most interesting thing about downloading code is learning how to write your own code that uses the same patterns and techniques, so let me cover the highlights. The table is pretty simple: CREATE TABLE Utility.WaitForSync ( WaitforDelayValue nchar(8) NOT NULL, StartTime datetime2(0) NOT NULL, --used to make sure you can do a waitfor that crosses a day boundry OnlyOneRow tinyint PRIMARY KEY DEFAULT (1) CHECK (OnlyOneRow = 1) ); It only allows a single row using the PK Constraint, a default, and a check constraint. A bit of a trick, but definitely workable trick to ensure a given cardinality in a table. The important two columns are one for the value that will be used in the WAITFOR statement, and the StartTime column is used to allow you to do your demos right around midnight (and yes, I put this in because when I was first using the procedure, it was taking a really long time to complete late one night.) The idea is, the first user will put a row in here at a given time, then every caller after that (and before the StartTime) will use the same time. After the StartTime, the process has already started. The simplest procedure is the reset procedure. It just deletes the row from the table (even if other users are still in progress, but this is just a utility for demos and testing). CREATE PROCEDURE Utility.WaitForSync$Reset AS ---------------------------------------------------------------------------------------------- -- Louis Davidson drsql.org -- -- Simple enough, just delete the row from the table ---------------------------------------------------------------------------------------------- DELETE FROM Utility.WaitForSync GO The StartWait procedure is a bit more complicated that that.. It takes a parameter of @WaitSeconds that lets the first caller to set the time. Usually 10 seconds works great for a few connections, but it depends on just how much you need to coordinate. The code is commented, and is really pretty simple. The basics are that it checks to see if a row exists and grabs the value, if it doesn’t, then it creates a new row. ALTER PROCEDURE Utility.WaitForSync$StartWait ( @WaitSeconds int --minimum amount of time to wait. The WAITFOR statement --always starts as minute changes.. ) AS ------------------------------------------------------------------------------------------------ ---- Louis Davidson drsql.org ---- ---- Either starts a new wait for session or uses the existing one ------------------------------------------------------------------------------------------------ SET NOCOUNT ON BEGIN TRY DECLARE @StartTime datetime2(0), @WaitforDelayValue NCHAR(8), @ProcStartTime datetime2(0) = SYSDATETIME(); --Get the row from the table where we hold the sync value. SELECT @StartTime = StartTime, @WaitforDelayValue = WaitforDelayValue FROM Utility.WaitForSync; --if a value was not already stored, we are just starting IF @StartTime IS NULL BEGIN --set the startTime to the current time + the number of seconds in parame SET @StartTime = DATEADD(second,@waitSeconds,SYSDATETIME()); --then I use a good old style convert with a format to get the time for the delay SET @WaitforDelayValue = CONVERT(nchar(8),@starttime, 108); --insert the value into the WaitForSyncing table. StartTime willbe used to make sure --the time is later, even if it crosses the day boundry INSERT INTO Utility.WaitForSync(WaitforDelayValue, StartTime) VALUES (@WaitforDelayValue,@StartTime); END --if the time has already passed, we raise an error to the client that you can't piggy back on the --existing session, reset ELSE IF @StartTime <= SYSDATETIME() BEGIN DECLARE @msg nvarchar(1000) SET @msg = N'Too late to start another WAITFOR session. Last session DELAY time was ' + CAST(@startTime AS NVARCHAR(20)) + '.' + N' Use Utility.WaitForSync$reset to start new sessions'; RAISERROR (@msg,16,1); END --finally, we take the delay value we created and use it in an execute statement --note that SSMS won't how the SELECT until after the batch resumes. DECLARE @queryText NVARCHAR(100) = 'WAITFOR TIME ' + QUOTENAME(@WaitforDelayValue,''''); EXECUTE (@queryText); PRINT 'WaitForSync: Starting at: ' + CAST(@startTime AS NVARCHAR(20)) + ' Waited for: ' + CAST(DATEDIFF(SECOND,@procStartTime, SYSDATETIME()) AS VARCHAR(10)) + ' seconds.' END TRY BEGIN CATCH --benign output that won't stop anything. Simply keep going PRINT 'WaitForSync: An error occurred. Message: ' + CHAR(13) + CHAR(10) + ERROR_MESSAGE() RETURN -100 --The caller can use the return value to decide if they want to stop what they are doing END CATCH GO Finally, there is a viewing procedure that lets you see what the time that the procedure starts is. Note that it has a 1 return value if there is no session started, and a –100 if an expired session is out there or there is an error.
|
-
So, I have been away from blogging about technical stuff for a long time, (I haven’t blogged at all since my resolutions blog, and even my Simple Talk “commentary” blog hasn’t had an entry since December!) Most of this has been due to finishing up my database design book, which I will blog about at least one more time after it ships next month, but now it is time to get back to it certainly in a bit more regularly. For SQL Rally, I have two sessions, a precon on Database Design, and another session on Sequence Objects and as I was building a demo with concurrent connections, I found myself again struggling with getting WAITFOR statements synchronized across multiple statements and I built a quick tool to make it easier to get multiple connections to start simultaneously. As I did this, I realized that I have a ton of utility objects that may (or may not) be interesting to the rest of the community. Other concepts I have in store are objects to grant rights, do DDL that SQL Server doesn’t implement (like dropping a schema with all objects, or dropping a column and constraints), metadata viewing etc. So, I will present a series (maybe long, maybe short, who knows) of T-SQL utilities that I find useful, and perhaps you will too. If you have ideas for utilities you are interested in having built (or want me to share/link to,) let me know either as a comment or email me at louis@drsql.org. And hey, if you want a day filled with database design, be sure and register for SQL Rally and my precon in May in Dallas, TX…
|
-
I skipped last year making blog resolutions, but this year I need to get myself back on the straight and narrow and encourage myself to do a few things. 1. Finish my book quickly and efficiently – Well, duh, I supposed, but the quickly and efficiently is the biggest important part. I have no idea when SQL Server 2012 will be released, but with RC0 having been publicly released, it isn’t going to be SQL Server 2013 now is it? 2. Blog regularly – Note I didn’t exactly say more, but I do want to be regular. Perhaps a substantive blog (more than telling people about some event) once a month here on SQL Blog while working on a book, and 2 or 3 times otherwise. On simple-talk my goal is to average 2 What Counts For A DBA blog entries. 2.1 Blog about my other (computer) love occasionally – I love gadgets (I now carry 6 music/video players, including 1 Windows Phone 7, 4 Zunes, my new Android player, and a Nintendo 3DS) and software (user interfaces are my favorite pet peeve), and I occasionally I feel like talking about them. 3. Do 1 or 2 new presentations, and reduce doing the ones on database design. I love talking about database design, and I will still put my main db design session in for conferences, but I have 3 new ideas for sessions that are about 50% code and 50% design (for example, a session on triggers, which I want to show not only how they can be used, but why they should be used (or honestly not be used, which is a far larger list indeed). 4. Develop and present a DB design a variable length (1-3 day?) seminar. I am pretty happy how the book is organized, based on how I forced myself to work on it this time. I had a bunch of ideas how I wanted to change things us and shorten the book, but during the process, I realized the natural progression of how it ought to be done. I feel like it translates well to a class, even a bit more than the pre-con I have written that I did for the Orlando SQL Saturday (and which I will be happy to do for any SQL Saturday for any size group (I don’t care so much about making money, as long as my expenses are covered after the room is paid for…though I wouldn’t mind making a buck :) 5. Get more involved with our local group. I attend, I help out, I did some work with trying to SQL Rally in Nashville, but I think I could do more to help out there. It helps that I work with Christine Leo, Kevin Kline, and Joe Webb, all super dynamic folks who kick butt. In my personal life, I have to make a number of changes that are probably apparent to anyone who has met me or seen a picture of me… That will probably be the hardest of them all, though possibly the most important. Wow, looking at the list I realize why I stopped making resolutions… Maybe I should just resolve to go to Disney World more? Well, that would be way too easy!
|
-
So it is Tuesday night, just a few days until my presentation entitled What Counts For A DBA and I am still not completely sure exactly what is going to go on. In fact, I don’t exactly plan to know what is going on until the presentation is over. On paper it seems like a simple idea. I am going to use 9 of the topics I have posted on my simple-talk blog about What Counts For a DBA (http://www.simple-talk.com/community/blogs/drsql/), and write them on my spinning wheel that looks like this:  The extra three spaces will be prize slots, which will (the first time they are landed on, earn the spinner a book: I will bring a copy of MVP Deep Dives 2, a copy of my relational design book, and a copy of Tom Larock’s DBA Survivor book too. In these thee spots I will have a Joker that lets the spinner choose their own topic, and two envelope spots that will let them choose an envelope that contains one of my queued up topics that I will ask the spinner to comment on, as well as anyone else in attendance. Any repeat topics will merit an envelope open as well. Will it work? I don’t have a clue. It will take me really stepping up my looseness when I speak and an audience that isn’t prepping their minds for the Women In Technology lunch. Either I dropped 120 bucks on a spinning wheel that takes up garage space and is never used or that is used a few times a year. Either way, hope to see you in Louisville this weekend!
|
-
 Well, we are finally here at what is the secular version of the holiday season for Microsoft SQL Server nerd types, the week of the SQLPASS Summit. This year, I am speaking 3 times and will also be doing the Quiz Bowl at the Welcome Reception, so I am going to be busy. If you are here and are interested in database design, please do stop by and check out my sessions. Monday and Tuesday I will be in side sessions that are NDA for much of the day, and that is probably all that I can say. Tuesday night will be the Quiz Bowl where Tim Ford and I will attempt to one up ourselves and attempt to (along with some unwitting (but hopefully full of wit) experts) entertain you with our yearly Jeopardy-esque wanna be game show. Wednesday, from 1-1:30, a bunch of the writers of the SQL Server MVP Deep Dives 2 book will be signing copies for you (there is a second signing at 7:15 - 8 AM on Friday, if you can stomach the earliness). If you want to see what is in the book, check http://www.manning.com/delaney/. The book will be on sale starting Tuesday afternoon for you if you want is signed or not. Note that all author proceeds of the book are going to charity (this edition’s proceeds going to http://www.operationsmile.org/), so you get a good book with lots of different subjects, and make a donation to a worthy cause. I am almost certain that you get an ebook edition of the book for free with the paper version that you can read on your portable device as well. At 4:45 - 6 PM, I will be doing my "Characteristics of a Great Relational Database " session that was picked up as an alternate . This needs to be a very interactive session, and my hope is to learn a bit from your ideas as well as the slides I have prepared. The session is fairly light and a bit humorous, so if you are feeling particularly serious and sour, well, my Thursday session is far less fun: Thursday afternoon, from 3-4:15 PM, I will be presenting "Database Design Fundamentals" which presents a more deep dive on the concepts that go into designing a relational database. My goal here is to present the basics of the process of creating a database from conception until you are ready to start typing CREATE TABLE statements. Friday, I will be in a panel discussion called "Are you a Linchpin? Career management lessons to help you become indispensible.", representing the corporate developers in the world who like that they have one set of problems to solve that while the basis never varies, can never really be solved because the demand outstrips the realities of the day. Other members of the panel include Jeremiah Peschka, Stacia Misner, Kevin Kline, Brent Ozar, Thomas LaRock, Andy Warren, Andy Leonard Saturday I go home...
|
-
So I have been a bit remiss on my blogging the book duties. The fact is, the first 8 chapters were fairly heavy rewrites and reworkings, and even a good amount of new material. But when doing a new version of a book that has already existed, you do need to reuse a good deal of the material from previous version. Chapters 9, 10, and 11 are these chapters for this edition of the book. The chapters are: - Chapter 9 - Database Security and Security Patterns – The biggest change to this chapter was the differences that Contained Database bring to the picture.
- Chapter 10 - Table Structures and Indexing – Minimal changes to this chapter, mostly concerning some of the changes to compression. Since columnstore indexes aren’t really pertinent to OLTP databases, I didn’t do anything with them. An example of a columnstore index will appear in Chapter 12
- Chapter 11 - Coding for Concurrency – Again, not a tremendous difference for Denali, so just a bit of touch up.
Then comes Chapter 12. Back in the original version of the book, I had a reporting chapter, and I wasn’t amazingly pleased with it. So in 2005 I cut it, and decided to leave it out. But it always felt like a bit of a hole in the book, having to say to “don’t denormalize and do reporting in your OLTP database, build a data warehouse, which I won’t talk about”. Since those early years, I have learned a lot about data warehousing, attended a Kimball class on on dimensional modeling, and have designed our corporate data warehouse (with the requisite original “failure” before training, naturally.) But I had to face facts, I was not the right person to write even a chapter on reporting/dimensional modeling. I had made a decision to not have cowriters for this version, a bit for space reasons (I loved having Kevin Kline as a cowriter for the past two versions, but I just didn’t have space last time, and we made it a download), a bit for coordination reasons (I had a devil of a time with a spatial section for the last book), and a bit for selfish reasons (you can figure that out for your own self.) But a few months ago, as the outline gelled, I decided I just needed something about dimensional modeling… So I reached out to a particular writer named Jessica Moss (@jessicammoss) that I have known for a while, and had recently worked with on my data warehouse project as a mentor to help our team grow quite a bit in our ETL skills and to help mold our design. So I asked her if she wanted to write a chapter on dimensional modeling, and she accepted. I have seen her early version of the chapter, and I am very excited to add it as chapter 12. Only one more chapter to write, number 13, which I will blog about in the next day or two once I get my ideas down on “paper” (and if assuming I don’t decide to split the chapter, will annoy my editor, so probably not!)
|
-
You probably have noticed that I haven’t blogged all that much these days. Part of the reason has been taking on way too many projects/speaking engagements/writing projects etc. I am generally proud of all of these things, but the most project that I am probably the most proud of is the MVP Deep Dives 2 book. The project was helmed by the one and only Kalen Delaney, whom I have always admired for many reasons. Working with her on the project has been awesome, even when she had to crack the whip on us pokey editors. The other editors are a group of names that, let’s face it, if you have heard of me, you have heard of them and probably long before me (well, unless you are a family member of mine!) Greg Low who rules www.sqldownunder.com), Brad McGehee the smiling face of many Red Gate ads and blogger at http://www.bradmcgehee.com/, Paul Nielsen, writer of the SQL Server Bible series until 2008 and a good friend of mine who loves database design like I do and Paul Randal and Kimberly Tripp of SQLSkills fame. It is an honor to be even mentioned along with these names. You see the table of contents and purchase the book here: http://www.manning.com/delaney/. Just like last time, the book starts out with a section on Architecture, edited by me. It is the smallest section, which is probably to be expected. The 6 chapters in Architecture focus on the softer topics, including constraints/uniqueness, storage, generalization of designs, and general characteristics of designs. I hope you like it, but one thing that is excellent about a book like this is that there are 60 chapters with something for everyone. If you are going to be at SQLPASS, there is scheduled to be a good number of books for purchase there, and we will have a mass signing again. I always feel funny about signing books, but the book signing was a lot of fun last time. The charity for this book is Operation Smile (http://www.operationsmile.org/). All author and editor proceeds will go to them. Last time we raised well into a 5 digit sum for War Child, so it isn’t chicken feed at the least.  One last note, I should note that over the project I lost an author due to his time commitments and some difficultly with the size of the chapter. I feel bad that we couldn’t work it out and I hope that his material (that was really good in its original form, though far too large for this format, and when we cut it down it just couldn’t be saved. The terrible part of being an editor is occasionally having to make a hard decision. In any case, this person’s writing, reasoning, etc were all good and if I see it posted later I will link to it for you.
|
-
I have seen a lot of other people giving advice about what to do on your first trip to the SQL PASS Conference and I want to give you my two cents worth as well. Many people will be pushing the social aspects of the conference and that is excellent advice which I too will emphasize, but in my mind there is one main thing you need to do: Make it worth it. Someone has shelled out a pretty large sum of money to get you there, and they want to see some return on investment in order for you or your coworkers to do it again. There are far too many choices out there for training, and PASS is a great bet to really learn a breadth of information in a short amount of time. Add to that a pre-con or two and you can get some deep insight to in the short period of time. One thing that I love about it is that there are so many sessions that even after 10 years I can get some extremely deep information from super geniuses Conor Cunningham and Bob Ward (and not of the Wile E Coyote variety!) as well as some very deep information from a host of others, all on topics that I am already quite good at and still learn a very valuable thing or two (and sometimes even more). Then I can pick up get beginner and intermediate topics on stuff that I am just interested in. If you have problems you need solved, write them down and bring them with you. Bring your laptop with demonstrations of your problem. I know I love to help out people with design problems if they have enough information to make it easy to see what they are trying to do. The SQLCat team (http://sqlcat.com/) usually has a great presence and will talk to you about problems, and there are labs to try out features that you might not usually have access to. Add to that the lounge with a bunch of current and future MVPs hanging out willing to give you some time talking about SQL Server related topics. Just don’t come to most technical sessions and expect to ask a question that takes 10 minutes of explaining and get your solution while everyone else waits… So take it somewhat seriously and learn something to take back to your company and show that the investment was worth it. And try not to quit and change jobs the week after the Summit, if you can. Nothing kills a training budget like people getting the feeling that they are paying their employees to go to a job fair for a week. Now, as long as you can make the investment pay off for whomever has paid for you to come to the Summit (even if it is you!), now have fun. There are tons of opportunities to have fun at the Summit. On the opening night we have a Quiz Bowl game where we quiz some of the smartest (goofiest) people in the SQL community on various insane topics. There is a PASS Party one night, a dinner you can sign up to attend on Monday and if you keep your ears open, plenty of other happenings around the Summit. Right around the conference center there is a theater, numerous restaurants, an excellent arcade, so there are plenty of places around to hang out with your new PASS friends you might make. And if you take one of the shuttles to your hotel with other people, you will probably meet a few people heading to the conference right after you get off of the plane (if you don’t bump into someone on the plane!) One of the best things about attending a large conference like this is that you can meet a lot of people you probably read/watch on the Internet and find out that they are just people (albeit people who spend a good amount of free time punishing various keyboard devices a little extra). I said I would mention it, and social networking is a very useful tool, especially at conferences. My suggestion is to (at least a few weeks prior to the Summit,) sign up for twitter, get a twitter client and follow @sqlpass at a minimum (feel free to follow @drsql too!) Also use your twitter client (or if you refuse, a browser) to periodically watch a search of sqlpass: http://twitter.com/#!/search/sqlpass. All of the twitter types will be telling everything that is going on, so if you go to a session and don’t like it, you can find out another that is good. If you want to find a group of people out one night to hang with, there is always something going on. A handy tool I have started using on my Windows Phone 7 is an app called Spout (there is an iPhone and Android version too) that lets you watch a twitter stream, twitter search, facebook account, google reader, and several others in cool looking rotating display. I used it at Devlink last week and it was cool watching what everyone was saying about stuff based on a twitter search of “devlink”. And the best part of using twitter? The friends you make at the conference go home with you and become close friends over time, sometimes even those you never even physically meet. In the end, you can either go to the conference, attend some sessions and go home, or…end up with a head full of knowledge, some real new friends, a host of virtual friends, and a community that you can lean on when you have needs (of course, they will lean back too.) And if you really like this conference stuff, there are lots of user groups and one day little PASS conferences all over the country world these days called SQL Saturday that you can go to and see some of the same people and lots of new faces. Who knows, you might even find yourself compelled to speak at next year’s event!
|
-
When I was editing my chapter on implementing a database, I noticed a really nice improvement in the error message I had from the previous edition of the book. Instead of just telling me that there was a value in my modification statement that duplicated an existing value (or multiple values affected by the statement), it told me the duplicated value. To demo, I created the following quickie table in tempdb. USE tempdb GO --drop the object if it initially existed if object_id('test.testErrorMessage') IS NOT null DROP TABLE test.testErrorMessage IF schema_id('test') IS NOT NULL DROP SCHEMA test go CREATE SCHEMA test GO CREATE TABLE test.testErrorMessage ( testErrorMessageId INT NOT NULL CONSTRAINT PKtestErrorMessage PRIMARY KEY, otherColumn varchar(10) NOT NULL CONSTRAINT AKtestErrorMessage UNIQUE (otherColumn), ) GO INSERT INTO test.testErrorMessage (testErrorMessageId, otherColumn) VALUES (1,'First') GO Then, inserting a duplicate row for the primary key value: INSERT INTO test.testErrorMessage (testErrorMessageId, otherColumn) VALUES (1,'First') GO
And on 2008 R2, I get: Msg 2627, Level 14, State 1, Line 1 Violation of PRIMARY KEY constraint 'PKtestErrorMessage'. Cannot insert duplicate key in object 'test.testErrorMessage'. Now on Denali CTP3, you get a little bit more: Msg 2627, Level 14, State 1, Line 1 Violation of PRIMARY KEY constraint 'PKtestErrorMessage'. Cannot insert duplicate key in object 'test.testErrorMessage'. The duplicate key value is (1). Then, to show the same thing for the UNIQUE CONSTRAINT: INSERT INTO test.testErrorMessage (testErrorMessageId,otherColumn) VALUES (2,'First') On 2008 R2, you get the following Msg 2627, Level 14, State 1, Line 4 Violation of UNIQUE KEY constraint 'AKtestErrorMessage'. Cannot insert duplicate key in object 'test.testErrorMessage'. And again on Denali CTP3: Msg 2627, Level 14, State 1, Line 4 Violation of UNIQUE KEY constraint 'AKtestErrorMessage'. Cannot insert duplicate key in object 'test.testErrorMessage'. The duplicate key value is (First). You can see if you duplicate > 1 value, it gives you one of the items. It might be better if the message didn’t imply that it was the only duplicate value, but hey, it is a great improvement. If you think it ought to be tweaked to say “A duplicated key value is (…) or something, click here) INSERT INTO test.testErrorMessage (testErrorMessageId,otherColumn) VALUES (5,'Third'),(6,'Third'),(3,'Second'),(4,'Second') Msg 2627, Level 14, State 1, Line 1 Violation of UNIQUE KEY constraint 'AKtestErrorMessage'. Cannot insert duplicate key in object 'test.testErrorMessage'. The duplicate key value is (Third). It works with indexes also: ALTER TABLE test.testErrorMessage DROP CONSTRAINT AKtestErrorMessage CREATE UNIQUE INDEX UXtestErrorMessage ON test.testErrorMessage(otherColumn) INSERT INTO test.testErrorMessage (testErrorMessageId,otherColumn) VALUES (5,'Third'),(6,'Third'),(3,'Second'),(4,'Second') You get the following: Msg 2601, Level 14, State 1, Line 1 Cannot insert duplicate key row in object 'test.testErrorMessage' with unique index 'UXtestErrorMessage'. The duplicate key value is (Third).
Much nicer!
|
-
In this last kind of “creative” chapter, I will look at some of the ways you implement common problems in your relational database, and some of the ways you probably shouldn’t. The “should” sections will deal with: - Uniqueness – Beyond the simple uniqueness we have covered in the first chapters of the book, looking at some very realistic patterns of solutions that cannot be implemented with a simple uniqueness constraint.
- Data Driven Design – The goal of data driven design is that you never hard code values in your code that don’t have a fixed meaning. You break down your programming needs into situations that can be based on sets of data values that can be modified without affecting code.
- Hierarchies – A very common need is to implement hierarchies in your data. The most common example is the manager-employee relationship. In this section I will demonstrate the two simplest cases, and summarize other methods that you can explore
- Images, Documents, and Other Files – There are quite often a need to store documents in the database, like for a web users’ avatar picture, or a security photo to identify an employee, or even documents of many types. We will look at some of the methods available to you in SQL Server.
- Storing User-Specified Data – You can’t always design a database to cover every known future need. In this section I will cover some of the possibilities for letting the user extend their database themselves in a manner that can be somewhat controlled by the administrators.
- Generalization – In this section we will look at some ways that you will need to be careful with how specific you make your tables so that you fit the solution to the needs of the user.
This marks an increase of a 4 sections from the last book, when I added this Patterns chapter. I did take away a few bits about sequence and calendar tables, but I do plan to move this to a later chapter on development, where I will discuss the sorts of objects that I find nice to have in each database and why. For the anti-patterns, I am adding one more in this edition, on undecipherable data. - Undecipherable data – Too often you find the value 1 in a column with no idea what 1 means without looking into copious amounts of code.
- One-size-fits-all domain – One domain table to implement all domains rather than individual tables that are smaller and more precise
- Generic key references – Having one column where the data in the column might be the key from any number of tables, requiring you to decode the value rather than know what it is.
- Overusing unstructured data – The bain of existances of the dba, the blob of text column that the user swears they put well structured data in for you to parse out. Can’t eliminate a column for notes here and there, but overuse of such constructs lead to lots of dba pain.
This is one of my favorite chapters because it really gets to the core of design. Up until now we have stuck mostly to basics and fundamentals, building very basic structures and working with the object types available to us. Here we build practical solutions to solve common problems. From here, the next chapter we will move along to security, structures, and then putting the finishing touches on things. Hopefully soon I will have some exciting new about a final chapter that will tie it all together once I get final approval and acceptance from the guest writer on this chapter.
|
-
As the book progresses, I find myself veering from the original stated outline quite a bit, because as I teach about this more (and I am teaching a daylong db design class in August at http://www.sqlsolstice.com/… shameless plug, but it is on topic :) I start to find that a given order works better. Originally I had slated myself to talk more about modeling here for three chapters, then get back to the more implementation topics to finish out the book, but now I am going to keep plugging through the implementation tasks, then finish up with modeling task (which I hope I might end up getting some help with…emails are going out once I talk it over with my editor). In the last edition, the chapter on data protection was more inclusive, including programmatic data protection, including client code and stored procedures. But, keeping with the basic, implementation type chapters (and trying my best to shorten chapters to more realistic chunks (the free chapter shouldn’t be 1/2 of the book, or so I am told), I will put that off to probably the final chapter. This chapter was broken up into two main sections, Check Constraints and Triggers. I will demonstrate the following scenarios, and if you see anything missing, please do make suggestions Check Constraints - Simple value checks – Like when you want to make sure there are no blank string values CHECK (len(value) > 0)
- Value reasonableness checks – Like if a value should be a non-negative integer, CHECK (value >= 0)
- Checks using different tables – Like setting up a data driven format check
Triggers – Broken down by AFTER and INSTEAD OF Triggers - AFTER
- Range checks on multiple rows – Like when you want to make sure that the sum of rows related to (and including) the newly inserted rows is > 0
- Maintaining summary values (only as necessary) – Denormalization, pure and simple, but if you are going to do it, triggers are the way to go (you really shouldn’t)
- Cascading inserts – Like creating child rows to ensure a 1 to at least 1 row relationship is met, or creating a parent
- Child-to-parent cascades – Like deleting parent rows when the last child row is deleted
- Maintaining an audit trail – Also something that will come up in security, but implementing an audit trail of actions on a table. Less needed these days with auditing, but
- Relationships that span databases and servers – sometimes you just have to implement RI between databases, so it is back to 6.0 style RI
- INSTEAD OF
- Automatically maintaining values – For example, if you want to implement a bulletproof rowLastUpdatedTime column to know when the row last changed, but don’t trust the client (who does?)
- Formatting user input – Like formatting words in all caps, or proper case. Another thing that might be better done outside of SQL Server, but it is very straightforward to implement
- Redirecting invalid data to an exception table – For example, eliminating data that is outside of the norm. Possibly done better outside of SQL Server code, but if you really want to build something that takes previous data into consideration, this might be a reasonable way.
- Forcing no action to be performed on a table, even by someone who technically has proper rights – Simple do nothing trigger that works because in an instead of trigger you have to replicate the action, so you don’t.
It might seem weird to consider formatting data or redirecting invalid data to another table as data protection, but the point of data protection is to make sure that they data ends up in a reasonable state, and triggers can do some “magical” seeming stuff. Admittedly, triggers are not a fan favorite with many programmers because they do those magical stuff that they cannot directly control, but in many ways that is the point. If the dev forgets to update the last update date, the ETL may not see the row, and oops your data is out of sync. In any case, I do my best to make it clear that you don’t in fact have to do any of this, but here are the tools in the tool bag.
|
-
So I was notified a few days ago that one of my sessions was selected, and one is an alternate. Luckily, it was the one that I have the most experience with, and the alternate is my latest session that I am really quite happy with after doing it virtually and now at the SQL Saturday in Columbus. The selected session is: Database Design Fundamentals In this session I will give an overview of how to design a database, including the common normal forms and why they should matter to you if you are creating or modifying SQL Server databases. Data should be easy to work with in SQL Server if the database has been organized as close as possible to the standards of normalization that have been proven for many years. Many common T-SQL programming "difficulties" are the result of struggling against the way data should be structured and can be avoided by applying the basic normalization techniques and are obvious things that you find yourself struggling with time and again (i.e. using the SUBSTRING function in a WHERE clause meaning you can't use an index efficiently). | And I really look forward to seeing how it goes this year. Since last year’s version at PASS, where I finished out a year in less than great form, I have worked on this presentation quite a lot. Some of the examples were hard to read, and the data models just didn’t seem to bring it across, so I added tabular examples that really seemed to give it an extra push of interest when I did it for the Hampton Roads User Group earlier in the year. I am also hoping that they put me in the smallest room possible so the 40-60 people that I average every year with this session are all crammed on top of each other (why? Check answer 7 here: http://sqlpeople.net/admin/2011/05/09/louis-davidson/) The irony is, I am really glad that the other sessions didn’t get picked up because frankly I am swamped and the other ones would have been totally new sessions that I would have spent at least 100 hours on. I figured the Database Structures session wouldn’t make it because it would never compete with Kalen, Kim, Paul, and many many others… When I put it in, I was planning on doing Devlink in Chattanooga in August where it was picked up. However, I am now speaking at SQL Solstice in Raleigh the very same week, and there I am doing the full day Database Design Workshop that I did last year in Orlando for SQL Saturday. I will blog more about that later, but suffice it to say, there are 5 day long sessions by some awesome names: Ed Wilson (PowerShell), Andy Leonard (SSIS), Jessica Moss (BI), Andrew Kelly (Performance). I feel honored just be there with them, much less speaking at the same time. So if you are looking for some learning in mid August in the South/Southeastern US, SQL Solstice is shaping up to be pretty awesome, and for a quality/value ratio that is almost unbelievable, you can’t beat Devlink for a developer oriented conference with a solid DB track (including Joe Webb, Brad McGehee, Robert Cain and others). And duh, the PASS Summit is always awesome, though clearly not in the South/Southeastern US (at least not until 2013!) So I am a bit hesitant to post this because the past few times I have posted about something I have submitted they have failed to occur. But hey, I might as well. If I don’t I might get all 5 sessions I submitted and have to do them.. and that would be super tiring. So I put in 5, including a 1/2 day session and a pre-con and if I could choose only one, it would be the pre-con. Before you start thinking I am stating the obvious, I have done one pre-con by myself at the Orlando SQL Saturday last year and it was great to have enough time to actually do the topic of relational database design, including time to actually get the people in the session to create designs on their own. The 1/2 day session is intriguing. I am thinking that a 1/2 day would still give us time to do some database designs, though not likely on the whiteboard in teams. In either case, if I get to do either of these, I figure I would bring my Wacom Cintiq 12x monitor to interact onscreen… So, the sessions I submitted are, starting with the regular sessions: Database Design Fundamentals – A session I have done quite a few times over the years, including quite a few SQL Saturdays recently with major changes to the normalization bits based on newer, friendlier example code. Characteristics of a Great Relational Database – A new session that I have only done once for the AppDev virtual chapter. It is a basically a list of seven characteristics that make a database great, rather than just functional. Database Structures for Programmers – A session that I am writing for Devlink this year (and probably the Nashville SQL Server User Group in July) that is probably encroaching on a few of my idol’s sessions who talk about structures. My goal in suggesting it was to branch out a bit and start talking a bit more about non-relational stuff. A great lacking I think is sessions about the physical structures, particularly at the level I tend to give talks at (which is to talk about complex subjects in less than complex manners.) A lot of developers don’t want to get so deep it gets confusing, but they do need to know about how the physical implementation of SQL Server can be manipulated for your benefit. Database Design Patterns In Practice (1/2 day) – In this session, my goal will be to cover constraints, triggers, etc and how you can put them together to make interesting objects in your design. It will be highly interactive, more or less consisting of an overview of data protection/implementation objects, then a set of scenarios that we will implement together. Database Design Workshop (pre-con) – This session will cover database design from the basics to actually putting it into practice. It is a bit like drinking from a firehose, but it very practical in nature as we will be looking at stuff that any person who has to do database design will do regularly. So there you have it. If I get even one of them I will be happy to do it. Speaking at PASS may be the most stressful of all of the speaking I do since the competition is strong, there are lots of tracks,and I have regularly been in the final slot of the conference where I probably would have been too tired to attend a session myself. If you want to get a solid idea of why this gets to me, watch for my SQL People entry comes up sometime this month.
|
|
|
|
|
|