A place for my thoughts and experiences on SQL Server, Business Intelligence and .NET
The topic is really not new but since it’s the second time in few days that I had to explain it different customers, I think it’s worth to make a post out of it.
Many times developers would like to insert a new row in a table or, if the row already exists, update it with new data. MySQL has a specific statement for this action, called REPLACE:
or the INSERT …. ON DUPLICATE KEY UPDATE option:
With SQL Server you can do the very same using a more standard way, using the MERGE statement, with the support of Row Constructors.
Let’s say that you have this table:
CREATE TABLE dbo.MyTargetTable
id INT NOT NULL PRIMARY KEY IDENTITY,
alternate_key VARCHAR(50) UNIQUE,
INSERT [dbo].[MyTargetTable] VALUES
('GUQNH', 10, 100, 1000, 10000, 100000),
('UJAHL', 20, 200, 2000, 20000, 200000),
('YKXVW', 30, 300, 3000, 30000, 300000),
('SXMOJ', 40, 400, 4000, 40000, 400000),
('JTPGM', 50, 500, 5000, 50000, 500000),
('ZITKS', 60, 600, 6000, 60000, 600000),
('GGEYD', 70, 700, 7000, 70000, 700000),
('UFXMS', 80, 800, 8000, 80000, 800000),
('BNGGP', 90, 900, 9000, 90000, 900000),
('AMUKO', 100, 1000, 10000, 100000, 1000000)
If you want to insert or update a row, you can just do that:
(SELECT * FROM (VALUES ('ZITKS', 61, 601, 6001, 60001, 600001)) Dummy(alternate_key, col_1, col_2, col_3, col_4, col_5)) S
T.alternate_key = S.alternate_key
NOT MATCHED THEN
INSERT VALUES (alternate_key, col_1, col_2, col_3, col_4, col_5)
MATCHED AND T.col_1 != S.col_1 THEN
T.col_1 = S.col_1,
T.col_2 = S.col_2,
T.col_3 = S.col_3,
T.col_4 = S.col_4,
T.col_5 = S.col_5
If you want to insert/update more than one row at once, you can super-charge the idea using Table-Value Parameters, that you can just send from your .NET application.
Easy, powerful and effective
Yesterday has been published my Q&A interview on my Pre-Conference Workshop at SQL Pass 2012. It’s a good way to understand what will be discussed in the workshop, so if you’re interested or you’re into the Data Warehouse / Business Intelligence field and want to understand how the Agile approach can help you, you can read it here:
I’ve received a buch of emails from PASS Summit “First Timers” that are also somehow new to SQL Server (for “somehow” I mean people with less than 6 month experience but with some basic knowledge of SQL Server engine) or are catching up from SQL Server 2000. The common question regards the session one should not miss to
- have a broad view of the entire SQL Server platform
- have some insight into some specific areas of SQL Server
Given that I’m on (semi-)vacantion and that I have more free time (not true, I have to prepare slides & demos for several conferences, PASS Summit - Building the Agile Data Warehouse with SQL Server 2012 - and PASS 24H - Agile Data Warehousing with SQL Server 2012 - among them…but let’s pretend it to be true), I’ve decided to make a post to answer to this common questions. Of course this is my personal point of view and given the fact that the number and quality of session that will be delivered at PASS Summit is so high that is very difficoult to make a choice, fell free to jump into the discussion and leave your feedback or – even better – answer with another post. I’m sure it will be very helpful to all the SQL Server beginners out there.
I’ve imposed to myself to choose 6 session at maximum for each Track. Why 6? Because it’s the maximum number of session you can follow in one day, and given that all the session will be on the Summit DVD, they are the answer to the following question: “If I have one day to spend in training, which session I should watch?”.
Of course a Summit is not like a Course so a lot of very basics concept of well-established technologies won’t be found here. Analysis Services, Integration Services, MDX are not part of the Summit this time (at least for the basic part of them).
Enough with that, let’s start with the session list ideal to have a good Overview of all the SQL Server Platform:
for what concern Database Development, I recommend the following sessions
For Business Intelligence Information Delivery
and for Business Intelligence Architecture & Development
Beside the listed sessions, First Timers should also take a look the the page PASS set up for them:
See you at PASS Summit!
After saveral years of silence, Fabian Pascal, has started to work again on its DB Debunkings website! The site has now been moved on blogspot and can be found here
The website is just a *great* resource for everyone involved in the DB space, and if you never heard of it before, it’s now time to discover it. Fabian has done a tremendous work to educate people to the Relational Theory and the Relational Databases, and many of the myths, doubt and “obscurity” that surrounds Relational Database can be debunked reading it’s post, papers and book.
Want to have an example? Ever had some doubt on the NULL usage and purpose? If yes (and I’m sure you had!) read here:
Welcome back Fabian!!!!!!
The “old” website is still active in case you want to visit it:
I’m very pleased to announce that I’ll be delivering a Pre-Conference at PASS Summit 2012. I’ll speak about Business Intelligence again (as I did in 2010) but this time I’ll focus only on Data Warehouse, since it’s big topic even alone. I’ll discuss not only what is a Data Warehouse, how it can be modeled and built, but also how it’s development can be approached using and Agile approach, bringing the experience I gathered in this field.
Building the Agile Data Warehouse with SQL Server 2012
I’m sure you’ll like it, especially if you’re starting to create a BI Solution and you’re wondering what is a Data Warehouse, if it is still useful nowadays that everyone talks about Self-Service BI and In-Memory databases, and what’s the correct path to follow in order to have a successful project up and running.
Beside this Preconference, I’ll also deliver a regular session, this time related to database administration, monitoring and tuning:
DMVs: Power in Your Hands
Here we’ll dive into the most useful DMVs, so that you’ll see how that can help in everyday management in order to discover, understand and optimze you SQL Server installation, from the server itself to the single query.
See you there!!!!!
To proceed with the installation of Wordpress on SQL Server and IIS, first of all, you need to do the following steps
- Create a database on SQL Server that will be used by Wordpress
- Create login that can access to the just created database and put the user into ddladmin, db_datareader, db_datawriter roles
- Download and unpack Wordpress 3.3.2 (latest version as of 27 May 2012) zip file into a directory of your choice
- Download the wp-db-abstraction 1.1.4 (latest version as of 27 May 2012) plugin from wordpress.org website
Now that the basic action has been done, you can start to setup and configure your Wordpress installation.
Unpack and follow the instructions in the README.TXT file to install the Database Abstraction Layer. Mainly you have to:
- Upload wp-db-abstraction.php and the wp-db-abstraction directory to wp-content/mu-plugins. This should be parallel to your regular plugins directory. If the mu-plugins directory does not exist, you must create it.
- Put the db.php file from inside the wp-db-abstraction.php directory to wp-content/db.php
Now you can create an application pool in IIS like the following one
Create a website, using the above Application Pool, that points to the folder where you unpacked Wordpress files.
Be sure to give the “Write” permission to the IIS account, as pointed out in this (old, but still quite valid) installation manual:
Now you’re ready to go. Point your browser to the configured website and the Wordpress installation screen will be there for you.
When you’re requested to enter information to connect to MySQL database, simply skip that page, leaving the default values. If you have installed the Database Abstraction Layer, another database installation screen will appear after the one used by MySQL, and here you can enter the configuration information needed to connect to SQL Server.
After having finished the installation steps, you should be able to access and navigate your wordpress site. A final touch, and it’s done: just add the needed rewrite rules
and that’s it!
Well. Not really. Unfortunately the current (as of 27 May 2012) version of the Database Abstraction Layer (1.1.4) has some bugs. Luckily they can be quickly fixed:
Select Top 0 Fix
Make the change to the file “.\wp-content\mu-plugins\wp-db-abstraction\translations\sqlsrv\translations.php” suggested by “debettap” http://sourceforge.net/tracker/?func=detail&aid=3485384&group_id=315685&atid=1328061
And now you have a 100% working Wordpress installation on SQL Server!
Since I also wanted to take advantage of SQL Server Full Text Search, I’ve created a very simple wordpress plugin to setup full-text search and to use it as website search engine:
Making PHP running on IIS and configuring it to be able to connect and query a SQL Server database is quite easy. If just need to get pointed to the correct direction, since information are spread all across the web but sometimes is not easy to understand to which version they apply.
So, since our primary goal is to have Wordpress 3.3.2 (the latest version as of 07 May 2012) and PHPBB 3.0.12 (the latest version as of 07 May 2012) up and running, we’ll download the latest PHP 5.3 version (PHP 5.4 has been reported to have some problems with the current version of Wordpress so we won’t go for the latest one).
So the first step is to download PHP 5.3.12 version from PHP.NET web site:
As you may notice there are tow flavor of the PHP distribution: Thread Safe and Non Thread Safe (NTS).
I’ve googled/binged a bit to understand what’s the advised flavor and as you can read here
the best practice is to use the NTS flavor.
I downloaded the Zip package, and unzipped the files in a newly created PHP directory under C: drive (C:\PHP)
The above link also pointed me to a resource page very helpful for everyone trying to install PHP and configure on IIS:
The following link give also additional information
I also found a very nice IIS Add-on that allows you to manage PHP directly from IIS:
One installed the PHP Manager, you can check that everything is working simply checking that PHPINFO() works:
To be able to use SQL Server from PHP you need the PHP SQL Server Drivers provided by Microsoft:
Once the file has ben downloaded I extracted it to a C:\PHPSQLSRVfolder.
Now you must acknowledge PHP that they exists and should be used. You can do it directly modifying your <php install folder>\php.ini file, adding the reference to the extension,
begin sure to have copied the correct .dll file into the <php install folder>\ext folder.
if you have installed the PHP Manager, you can also enable/disable extension from here (.dll file must be manually copied into the /ext directory anyway)
As you may have noticed, after having unzipped the PHP SQL Server Drivers you have a bunch of files to choose from. Which is the one right for you? For the configuration we’ve chosen we have to go for the “standard” (which means not the “pdo” driver), non-thread safe PHP 5.3 driver. As you can guess the file we need is
After having restarted your website, you can check in the phpinfo() page if the extension was loaded correcty:
I strongly suggest to take a look at the help file that comes with the PHP SQL Server Drivers, so that you can start to get used to access SQL Server from PHP. In particular use the code shown in the
“How to: Connect Using SQL Server Authentication”
help article in order to create a .php page to test that your environment can correctly connect to SQL Server.
Of course be sure to have at least the SQL Server Native Client installed on the web server, otherwise you won’t be able to connect to SQL Server from PHP.
That’s it! You now have a PHP environment on you IIS 7 / 7.5 capable of using SQL Server as RDBMS.
In the last months, in the spare time, I started to study PHP in order to use it on a Windows + SQL Server box. Why you would do such thing you may be wondering. The point is that Wordpress is actually, IMHO, the state of the art of a free CMS that must be used as the backend for a community site: it’s feature-rich, it has a *lot* of plugins and themes, it can be used to host blogs and to empower a “thematic” website. In my case I’d like to refresh the engine used to publish the Italian SQL Server User Group website.
I’ve looked for a lot of alternatives in the .NET World, and I evaluated in the last year
- Community Server
For one reason or another, none of the mentioned platforms, which are great platforms BTW, was the right for us. We needed something
- Capable of managing a community portal with news, articles, events, calendars and so on
- Capable of managing the blogs of members, allowing the generation of new blog sub site on the fly
- Fully customizable with a minimum effort for the end user
- Enabled to use HTML5 and CSS3
- Stable and Mature, with a good documentation and/or forum support
- Easy to be extended/modified adapted to our needs
- Compatible with MSN Live Writer
- Compatible with SQL Server
- Capable of hosting forums or capable of be integrated with a 3rd party forum platform
And at the end the platform that suits all our need is…Wordpress!
Of course this decision bring some challenges in the game:
- I need to be sure that Wordpress can work *well* with SQL Server
- I need to integrate Wordpress with a forum software.
Luckily Microsoft has written a cool abstraction layer for Wordpress, that make it compatible with SQL Server. And, even more luckily, there is a mainstream forum solution, PHPBB, natively compatible with SQL Server.
Of course not everything is as smooth as one would like it to be, so there are some “attention point” that one need to take into account when going in this way. And since there isn’t a lot of documentation available on running Wordpress together with PHPBB on SQL Server, I though that writing some post can be helpful to the community. After all Wordpress and PHPBB are two *great* solution and having them available on SQL Server is something desirable in my opinion.
So, in the next months, I’ll write a series of four (maybe five) posts to describe how to have a Wordpress + PHPBB on IIS + SQL Server solution up and running.
Here’s the agenda of the next posts:
I’ll hope you’ll enjoy the topics!
Already 10 days has passed since SQL Bits X in London. I really enjoyed it! Those kind of events are great not only for the content but also to meet friends that – due to distance – is not possible to meet every day. Friends from PASS, SQL CAT, Microsoft, MVP and so on all in one place, drinking beers, whisky and having fun. A perfect mixture for a great learning and sharing experience!
I’ve also enjoyed a lot delivering my session on Temporal Snapshot Fact Tables. Given that the subject is very specific I was not expecting a lot of attendees….but I was totally wrong! It seems that the problem of handling daily snapshot of data is more common than what I expected.
I’ve also already had feedback from several attendees that applied the explained technique to their existing solution with success. This is just what a speaker in such conference wish to hear! :)
If you want to take a look at the slides and the demos, you can find them on SkyDrive:
The demo is available both for SQL Sever 2008 and for SQL Server 2012. With this last version, you can also simplify the ETL process using the new LEAD analytic function. (This is not done in the demo, I’ve left this option as a little exercise for you :) )
If you’re using MDS and DQS with the Excel Integration you may get an error when trying to use the “Match Data” feature that uses DQS in order to help to identify duplicate data in your data set.
The error is quite obscure and you have to enable WCF error reporting in order to have the error details and you’ll discover that they are related to some missing permission in MDS and DQS_STAGING_DATA database.
To fix the problem you just have to give the needed permession, as the following script does:
GRANT SELECT ON mdm.tblDataQualityOperationsState TO [VMSRV02\mdsweb]
GRANT INSERT ON mdm.tblDataQualityOperationsState TO [VMSRV02\mdsweb]
GRANT DELETE ON mdm.tblDataQualityOperationsState TO [VMSRV02\mdsweb]
GRANT UPDATE ON mdm.tblDataQualityOperationsState TO [VMSRV02\mdsweb]
ALTER AUTHORIZATION ON SCHEMA::[db_datareader] TO [VMSRV02\mdsweb]
ALTER AUTHORIZATION ON SCHEMA::[db_datawriter] TO [VMSRV02\mdsweb]
ALTER AUTHORIZATION ON SCHEMA::[db_ddladmin] TO [VMSRV02\mdsweb]
Where “VMSRV02\mdsweb” is the user you configured for MDS Service execution. If you don’t remember it, you can just check which account has been assigned to the IIS application pool that your MDS website is using:
I think I can say that we all agree that SQL Server 2012 brings with revolution in our database world, since the number of new feature and new possibilities, IMHO, is comparable to what SQL Server 2005 brought several years ago.
Interesting enough, even the well-known IDC firm thinks that SQL Server 2012 can be a game changer. From a technical point of view the highlighted technologies are:
- ColumnStore Index
- Windows Server Core Support
- Power View
- BI Sematic Model
- Data Quality Services
- Hadoop Support
- Cloud & On-Premise Integration
- Support for PHP, Java and Linux
The full document (less than 10 pages) is very interesting – definitely worth reading – and can be extremely helpful to bring the awareness of what SQL Server 2012 can offer and how it can help business, also to non technical people:
IDC WHITE PAPER - Microsoft SQL Server 2012- Potential Game Changer
I’ve released couple of hours ago the SP2 of my DTLoggedExec tool.
For those who don’t know it, it’s a DTEXEC replacement, useful to execute SSIS and having logging provided right from the engine and not from the package itself.
More info can be found here:
This SP2 release add an important feature to the CSV Log Provider. It's now possible to store a personalized label into each log, in order to make it easy to identify or group logs.
Let's say, for example, that you have 10 packages in your ETL solution, and each time you have to load your data, you need to execute all those 10 packages. In other words , you have a batch made of 10 packages. It would be nice if all logs - one for each package - can be identified as a whole and grouped together, in order to quickly identify all the log of a articular batch.
This will make things easier when you want to know the overal time consumed by each batch execution.
The new "ExecutionLabel" attribute will help to achieve this. A useful ExecutionLabel can be obtained using SQL Server Agents Token. For example:
- Updated the CSV Log Provider in order to write the status of the log file in the header.
- An OPEN status means that the log file is being written.
- A CLOSED status means that the log files has been written correctly.
- A file can be loaded into the database log only if is in the CLOSED state.
- CSV Log files have 2 additional rows in the header: one for the FileStatus and one for the ExecutionLabel values.
- File format has been update to 4 from 3. (Only the header section of the file has been changed).
- CSV Log Provider will now display EndTime value to the Console
- Updated the import-log-data.sql to correctly load file with format 3 (the old one) and 4 (the new one).
- Updated database schema to version 19 in order to store the new ExecutionLabel value
- Update samples in order to show how to use the new ExecuteLabel option
As usual the download is available for free here: