THE SQL Server Blog Spot on the Web

Welcome to SQLblog.com - The SQL Server blog spot on the web Sign in | Join | Help
in Search

SQLBI - Marco Russo

SQLBI is a blog dedicated to building Business Intelligence solutions with SQL Server

DateTool dimension: an alternative Time Intelligence implementation

Year-To-Date and Difference-Over-Previos-Year (or Year-Over-Year Growth) are among the most required features of any user. Some OLAP client (like ProClarity) offers features that try to solve this problem client-side, but I don’t like this approach given that you might have a server-side feature doing that (so you don’t discriminate Excel users).

One trivial approach is creating a calculated measure for each combination of measure and date-calculation. While this works, it results in a multiplication of the number of measures, making the cube hard to use. Analysis Services offers a feature called Time Intelligence Wizard that creates some calculated members on dedicated Date attributes. This works in MDX but has several limitations:

· It limits the measures on which the calculation are applied (each time you add a new measure, you need to update the wizard-generated MDX scripts)

· It only applies to selected hierarchies – it doesn’t work, for example, if you cross Month and Years on rows and columns.

· It doesn’t work well with Excel 2007 after you install Analysis Services 2005 SP2 – see my rants here.

While the first two issues can be solved by using the same architectural approach of Time Intelligence Wizard, simply writing a different MDX Scripts, solving the third issue (Excel 2007 compatibility) requires a different architecture. For the sake of Google/MSN Live desperate users, I’m going to describe “how to make calculated members working on non-measures dimension with Analysis Services 2005 SP2 and Excel 2007”. I hope that Microsoft will consider a similar approach on a future version of Time Intelligence Wizard…

You can download the solution I'm going to describe on SQLBI.EU web site, under DateTool Project.

The issue

Let’s start describing the Excel 2007 SP2 issue. I created a small subset of Adventure Works sample cube. Calendar Date Calculations is the wizard-generated attribute. With BIDS browser (built using OWC 11) you can put this dimension on the columns of a pivot table with Calendar hierarchy placed on rows, and you can select any set of members from the Calendar Date Calculations attribute. In this example I unchecked one of the members generated by Time Intelligence Wizard.

FIG01 - Calendar Date PivotTable OWC11

With Excel 2007 the same cube is usable only if you enable the “Show calculated members from OLAP server” on the PivotTable Options dialog box (it’s unchecked by default).

FIG02 - PivotTable options for Show Calculated Members

Now we can generate a PivotTable similar to the one generated with BIDS browser.

FIG03 - Excel 2007 PivotTable Time Intelligence Wizard

Unfortunately, we cannot select single members from the Calendar Date Calculations attribute. We only have an “all or nothing” option based on the PivotTable Options settings we’ve seen above. This is the resulting Excel 2007 PivotTable.

FIG04 - Excel 2007 calculated members cannot be unchecked

The solution

I defined a dedicated dimension for time-related calculated members. Each formula has its own “real” members, instead of calculated ones. This solves the Excel 2007 issue at the price to require a dimension process instead of a simple MDX deployment (changing MDX Scripts for new calculated members wouldn’t require cube reprocessing). Since I don’t want a cube space growth, I simply put the dimension in the cube without a relationship with any measure group.

FIG05 - Dimension Usage for DateTool dimension

This is the real tricky and most non-intuitive part of the game. The DateTool dimension is seen as a regular dimension by any client, but it can be changed and reprocessed without any need to reprocess any measure group (because it has no relationship!).

The DateTool dimension could contain a single attribute with all time-related calculated members. However, I prefer an approach that uses two sets of different calculated members, which are orthogonal and might be crossed together. For example, I would put Year-To-Date calculation in one attribute and Year-Over-Year Growth in another one, so that I can obtain the Year-Over-Year Growth of a Year-To-Date calculation without the need to create a dedicated calculated member. In other words, I use these two sets to separate aggregations from comparisons formulas.

In practice, I would need two independent dimensions, and I do that from a relational point of view. I define two views in a separate schema on my data source (I could also use a separate Data Source View with self-contained named queries, but I prefer using relational views as a general way to decouple relational schema from multidimensional one – reasons for this would fill a whole dedicated article).

CREATE SCHEMA DateTool
GO

CREATE VIEW [DateTool].[DateAggregation] AS
SELECT 0 AS ID_Aggregation, 'Regular' AS Aggregation
UNION ALL
SELECT 1 AS ID_Aggregation, 'Year To Date' AS Aggregation
UNION ALL
SELECT 2 AS ID_Aggregation, 'Last 12 Months' AS Aggregation
UNION ALL
SELECT 3 AS ID_Aggregation, 'Total Current Year' AS Aggregation
GO

CREATE VIEW [DateTool].[DateComparison] AS
SELECT 0 AS ID_Comparison, 'Regular' AS Comparison
UNION ALL
SELECT 1 AS ID_Comparison, 'Previous Year' AS Comparison
UNION ALL
SELECT 2 AS ID_Comparison, 'Diff. Over Previous Year' AS Comparison
UNION ALL
SELECT 3 AS ID_Comparison, 'Diff. % Over Previous Year' AS Comparison
GO

 

Since I want to build a single dimension with two attributes, I create another view to build a junk dimension with the desired cardinality.

CREATE VIEW [DateTool].[DateTool] AS
SELECT 
    a.ID_Aggregation, 
    s.ID_Comparison, 
    CAST( ID_Comparison AS VARCHAR ) + ' - ' + CAST( ID_Aggregation AS VARCHAR ) AS Description
FROM DateTool.DateAggregation a
CROSS JOIN DateTool.DateComparison s
GO

At this point I can import these views in the Data Source View. I specify Primary Keys and Relationships manually because they can’t be inferred by relational metadata (they don’t exist at that level).

FIG06 - DateTool Data Source View

The resulting DateTool dimension is very simple.

FIG07 - DateTool Data Dimension

At this point there is the necessary MDX Script that solves all DateTool members. This single part would require another post or two and I don’t have time to explain that now, but you can see the complete AdventureWorks based solution downloading it from SQLBI.EU.

The interesting part is the result provided by Excel 2007 with this solution. Here is an example of the Calendar hierarchy crossed with a selection of the calculated members available in the Aggregation attribute of the DataTool dimension.

FIG08 - Excel 2007 DataTool.Aggregation sample

With this dimension, there are no limitations like those involved by calculated members (see the previous example made using Time Intelligence Wizard). In the previous PivotTable there is a member unchecked, like you can see in the following picture.

FIG09 - Excel 2007 DataTool.Aggregation selection

Just using this technique you could translate each calculated measure generated by Time Intelligence Wizard into a “regular” dimension fully usable by Excel 2007 users. The presence of two independent attributes makes the user able to build more complex PivotTables like the following one: the highlighted column displays the Year Over Year Growth over the Year To Date value.

FIG10 - Excel 2007 DataTool.Aggregation cross YTD-grow

Undoubtedly, such a feature is comfortable more to advanced users that to inexperienced one, since often a rename of the resulting report is required to better describe query results.

Finally, one of the limitations of Time Intelligence Wizard is that generated formulas work on a limited part of the cube: only one time hierarchy and only a selected set of measures. With the MDX Scripts I used, there are no similar constraints, making the cube easier to navigate. In the following example, I put years on columns and months on rows, displaying the Year-To-Date value and the difference with the previous year for selected data.

FIG11 - Excel 2007 Year cross months

Unfortunately, with this approach we lose compatibility with other clients, like the Browser provided by BIDS and SQL Server Management Studio. A PivotTable like the previous one would not display any data: the different way Excel 2007 and OWC 11 manage dimensions metadata produces this discrepancy between results obtained by difference OLAP clients. I am not sure if this can be solved just changing MDX Scripts definition – I made many tests until now and I described the best compromise I obtained until now, but I’d like to get feedback about issues, possible improvements and eventually alternative architectural solutions.

If you have time to take a look at the solution, please send me your feedback and/or any other comments at marco.russo (at) sqlbi.eu or using the comments for this post.


Published Sunday, September 02, 2007 4:41 PM by Marco Russo (SQLBI)

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

 

SQLBI - Marco Russo said:

I fixed an issue of the DateTool dimension I presented in a previous post . The DateTool was unusable

September 9, 2007 4:18 PM
 

SQLBI - Marco Russo said:

Today I discovered that DefaultMember might result in a member other thant the default dimension member.

October 1, 2007 5:35 PM
 

rptodd said:

So where's the MDX to go with this?  I'm not sure if this is our solution or not, so I don't really want to struggle through the MDX just to test it out.  I wish I could just bust it out, but I still find that stuff confusing.

Thanks Marco!

April 3, 2008 10:05 PM
 

Marco Russo (SQLBI) said:

You can download the demo project from http://www.sqlbi.eu/datetool.aspx

Marco

April 4, 2008 4:09 AM
 

Jason said:

Thanks Marco for the post!  These types of tutorials are exactly what the BI community needs.  

Jason

September 9, 2008 3:04 PM
 

SQLBI - Marco Russo said:

I was tempted to start this post by this sentence “Analysis Services 2008 is not a new full release of

December 8, 2008 10:28 AM
 

gene said:

Marco...great post.  saved me some time.

January 15, 2009 5:15 PM
 

Alberto Ferrari said:

The security model of SSAS prevents us to apply security limitations to calculated members. We can hide

February 5, 2009 5:18 AM
 

Bruce Lester said:

Marco, thanks for posting this solution.  Your documentation is excellent.  

Bruce Lester

February 26, 2009 2:36 PM
 

Luke said:

Marco...thank you very much for your post.  You did an excellent job explaining this behavior of AS and Excel.  It is also nice to have an example of code for something we are trying to accomplish with our data calculations.  One question.  In your overwrites of calculations using [Quarter of Year] and [Month of Year] you used StrToMember and Key properties to link the attribute hierarchies .  Is there a significant performance increase doing this instead of using exists() or is this just another way to do it?  Or am I missing something?

March 4, 2009 2:59 PM
 

Marco Russo (SQLBI) said:

Luke, I don't get your point. We overwrite calculations using StrToMember because we want to be able to use those attributes that are outside a natural hierarchy. I don't get how you would use EXISTS functions to do the same. If you have an alternative MDX Script to get the same result, please send it to me and I'll check it.

Thanks

Marco

March 4, 2009 4:24 PM
 

Luke said:

Marco,  Thank you for your response.  You have created an elegant solution and I'm trying to apply it to my cube.  I should a stated that I'm not working with the Adventure Works cube.  My cube has attribute relationships similar to the diagram here:

http://www.sqljunkies.com/WebLog/mosha/archive/2007/06/07/katmai_ctp_attribute_relationship_tab.aspx

Using the following code I'm able to get a specific [Month] at the intersection of [month of year] and [year]

exists([Date].[Month].[Month]

      ,([Date].[Month Of Year].currentmember,[Date].[Year].currentmember)).Item(0)

I was just wondering if using the Key properties was better from a performance perspective.  I'm still getting up to speed on MDX, so please excuse any oversight on my part.

Thanks... Luke

March 4, 2009 5:45 PM
 

Marco Russo (SQLBI) said:

Now I understand - it is interesting, it could be more efficient but I never tried to model it in this way.

I don't have time these days - if you make some performance test, please keep me updated.

Thank you

Marco

March 6, 2009 11:35 AM
 

Luke said:

Marco, Thank you again for your response.  Could not tell that much difference in performance.  Ended up using exists() because getting to a specific [date] just using key properties of [day of year] and [year] was challenging because our [date] has an integer key that includes month(YYYYMMDD).

Again, I really appreciate that you shared your knowledge and code here. Your technique to scope a set of calculated members using except() was especially informative. Thank you...

Luke Pargiter

March 6, 2009 4:09 PM
 

Sumeet said:

Hello Marco,

Many thanks for putting this up.  This is the best thing since sliced bread.  I work for wine.com and this really saved me.  I don't want to put my email address here but if you call me at 925-577-1924, I would love to send you a bottle of wine.  At any rate, there is some of the best creative work I have seen.  Well Done indeed.

Regards,

Sumeet

April 10, 2009 8:22 PM
 

Marco Russo said:

Hi Sumeet,

thank you for your appreciation :)

You can write me at marco.russo (at) sqlbi.com and you can give me any information you don't want to publish here.

Best,

Marco

April 11, 2009 5:01 AM
 

Simon L-Deslauriers said:

hi Marco,  thank you for the DateTool dimension.  I used it in all of my clients.  

However, one of them have a particulary complex set of calculated member.  

Long story short : I need to make the Aggregate member being calculated AFTER other Calc.  I do need to change the SOLVE_ORDER or the "physical" member of the aggregate dimension in order to have my calculation done correctly.  I am not able to do that.

any help?

thanks!

April 14, 2009 12:41 PM
 

Simon L-Deslauriers said:

sorry, error in last post : we should read "I do need to change the solve_order OF the "physical" members of the aggregate dimension (YTD, MTD and WTD) ....

April 14, 2009 12:42 PM
 

Marco Russo (SQLBI) said:

You could simply move the MDX Script of the involved calculation - they are executed in the order they've been written... Of course, when there are no inter-dependencies into the MDX Script... Doesn't it work for you?

April 14, 2009 4:57 PM
 

Mikkel said:

Does this work with office 2003 as well?

May 23, 2009 10:20 AM
 

Marco Russo (SQLBI) said:

It should work witk Excel 2003 (and OWC11) as well.

May 23, 2009 10:26 AM
 

Mikkel said:

That was fast! Thanks, then this is exatly what I was looking for.

May 23, 2009 10:39 AM
 

Michael said:

Hi,

First off, thanks for this magnificent tool Marco!

I have a little problem with aggregation abbreviations below the year level, i.e. quarter to date, month to date. How do I get it to roll up on the above levels also? If I make a quarter to date, it only rolls up to the quarter, but not the year?

Thanks in advance

Best Regards,

Michael

June 9, 2009 5:06 AM
 

Mark said:

HI Marco,

We are just getting started with SSAS 2008.  I upgraded the datetool solution and it works perfectly there.  Then I tried to incorporate the same technique in our cube and I ran into this error with the MDX:  Error 6 MdxScript(AGF1) (43, 36) The level '[All Periods]' object was not found in the cube when the string, [Date].[Month of Year].[All Periods], was parsed. 0 0

Everywhere All Periods is mentioned this error is thrown.  Any thoughts on where to find the root cause of this issue?

Thanks

mark

June 9, 2009 4:47 PM
 

Marco Russo (SQLBI) said:

To Michael: a "quarter to date" should be an aggregation at a quarter level, what do you mean when you want to roll-up to the year level?

To Mark: The [All Periods] member should be adapted to your cube - what is the member name at the [All] level of the dimension hierarchy/attribute? Use that name instead of "All Periods" in the MDX script.

June 9, 2009 5:02 PM
 

Mark said:

Marco, Nice!  That worked -- I wasn't understanding where the [All Periods] reference was, but I finally found it in the the Dimension Browser.  

I looked at another solution that also provided YTD % change, etc. but it involved a lot more code for the mdx and the components of it were still grayed out in Excel because it created the members of the shell dimension through MDX.  This is a really powerful tool, I can't say enough for how excited I am about it.

Excellent work!  I can't thank you enough.

Mark

June 10, 2009 12:32 PM
 

Marco Russo (SQLBI) said:

Mark, you're welcome!

June 10, 2009 12:37 PM
 

Ken said:

GREAT work here.. so glad you had the time to post such a well written example.  

I am attempting to use this in my cube.... a few questions if you have time:

1. When I add

   ( [DateTool].[Aggregation].[Month To Date] )

         = Aggregate(

                    { [DateTool].[Aggregation].DefaultMember } *

                      PeriodsToDate(

                                   [Dates].[Time].[MONTH NAME],

                                   [Dates].[Time].CurrentMember

                    ) );

below the year to date definition under calcs... when i try to process the database I get an error that:

Error 25 MdxScript(Sales Cube) (57, 5) The level '[Month To Date]' object was not found in the cube when the string, [DateTool].[Aggregation].[Month To Date], was parsed.

Does not make sense.. I am defining the object??

2:  Is there a way to build a True Month to date value.. meaning JUST show the current month up to whatever TODAY is...Same question for year to date.. I want to stop the calc at "today"

Again .. thanks for the great work... and your answers to the questions have been very helpful... hope you enjoyed the wine...

-Ken

June 10, 2009 6:15 PM
 

Marco Russo (SQLBI) said:

Ken,

1) you have to define the member 'Month to Date' in the SQL view DateTool.DateAggregation

2) more complex issue, because what do you want to do for the previous years? If you want the same timeframe, you should define a dedicated member with a calculation that uses the range starting from the Jan 1st (or the beginning of the period) to the current day, whatever the selection is (and what do you do if the end user deselect some months?)

June 10, 2009 6:20 PM
 

Ankit Mittal said:

Hi Marco,

I recently learned how to use SSAS 2005, in order to meet our clients requirements. In the same process, we created 300 measures in the AS cube and appx. 200 measures required "Year-Over-Year" analysis.

I used "Time Intelligence" feature of SSAS but we ran into performance issues, though report is quite bulky in nature but still "Year-Over-Year" analysis takes unacceptable amount of time.

Kindly let me know what you think about "Time Intelligence" feature of SSAS and provide me some other approach if possible.

Thanks

Ankit

June 16, 2009 1:11 AM
 

Marco Russo (SQLBI) said:

Ankit,

DateTool has been created just to workaround Time Intelligence limitations. It creates calculated members and it creates too many members - DateTool is cleaner in this way, even if MDX Scripts is smaller but more complex.

Time Intelligence wizard is good to start and in simple projects, but is not good in large projects, just my opinion, of course!

Marco

June 16, 2009 1:39 AM
 

Andrei Rjeousski said:

Hi Marco,

Just wondering if you ever worked/created a cube that used similar concept to DateTool for creating Math Calculations (in addition to TimeCalcs)?

I have cube like that working with great success on SSAS 2005 but am having some issues porting it to 2008 as it seems that the order the calculations are done have changed in 2008.

Let me know.

Thanks,

Andrei

June 16, 2009 2:51 PM
 

Marco Russo (SQLBI) said:

I never created a Math Calculations dimensions, but I understand it could be useful in some cases.

It seems strange the problem you described about migration from 2005 to 2008 - do you have more details?

Marco

June 16, 2009 7:08 PM
 

Andrei Rjeousski said:

Marco,

Please see this thread for more information:

http://social.msdn.microsoft.com/Forums/en-US/sqlanalysisservices/thread/389a2328-bcba-4eff-bec2-5bbc97ffccb2

I was able to come up with a query against AdventureWorks that would return different results when ran against 2005 and 2008 servers, but am unable to find any documentation on changes that would cause it to run this way. I believe that 2005 version runs more correct as it gives accurate results.

If you would like to contact me for more information please email arjeousski at gmail dot com

Thanks,

Andrei

June 17, 2009 12:14 PM
 

Ankit Mittal said:

Hi Marco

Thanks a lot for such prompt reply and I will try DateTool on my project for sure.

Actually while analyzing cube's performance I came to a conclusion that query results much faster when it has high cache hits.

So is there any way in SSAS I can increase the amount of cache and keep it available all the time?

Thanks

Ankit Mittal

June 17, 2009 10:47 PM
 

Ankit Mittal said:

Marco

There is one more problem with our SSAS implementation, it creates many "SQlDmpr" files only on production server. It seems that every time a report hits cube, it creates one SQLDmpr file but this never happen on my local and any internal machines.

It's a kind of strange problem but any suggestion on this will be of great help.

Thanks

Ankit Mittal

June 17, 2009 10:56 PM
 

Marco Russo (SQLBI) said:

By default, SSAS already has much of the memory available for cache. You might want to warm-up the cache (ie after a process) by using some script that makes the necessary query to warm-up the cache.

Marco

June 18, 2009 4:30 AM
 

Giancarlo Ferrara said:

Hi Marco.

Try to set the standard format used for saving as Excel97-2003.

This forces excel to use the old pivottable format and in 2003 the calculated members works.

Note that you can seve the excel file also in 2007 format: is just the standard save option that must be modified.

June 18, 2009 6:44 AM
 

Ken said:

When I add a row to the agg view with a description that is slightly larger such as 'Total Current Quarter'

I can not get the datetool dim to process...even after I remove and re-add the agg view...remove and add the dim to the cube?

I get this error...  

Error 45 MdxScript(Sales Cube) (85, 5) The level '[Total Current Quarter]' object was not found in the cube when the string, [Time Calculations].[Time Periods].[Total Current Quarter], was parsed. 0 0

I added this code for quarter (i had to change your naming conventions)

   ( [Time Calculations].[Time Periods].[Total Current Quarter] )

         = ( [Time Calculations].[Time Periods].DefaultMember,

             Ancestor( [Dates].[Time].CurrentMember,

                       [Dates].[Time].[Quarter] ) );

Unless I remove everything and re-add it will not work?

any ideas?

Thanks

June 18, 2009 9:31 AM
 

Marco Russo (SQLBI) said:

Giancarlo,

the user that had this problem has been updated SQL Server and Excel just to get Excel 2007 PivotTable functionality - it was really the worst thing to suggest the rollback to the previous version of the PivotTable :)

Marco

June 18, 2009 6:03 PM
 

Marco Russo (SQLBI) said:

Andrei,

I've seen the thread you mentioned - I don't have time in these days to make some test, but I've seen other people interested to the issue and I'll update you on this as soon as I will have any new info about it.

Marco

June 18, 2009 6:05 PM
 

Marco Russo (SQLBI) said:

Ken,

when you create the view [to populate the attribute] for the first time, you implicitly set the the maximum length of the name of that attribute. At this point, if you later add a string longer than others, you need to check that the size of key/name used in SSAS is large enough, otherwise the name of new items will be truncated if longer than the previous maximum size.

Another solution might be to cast these columns to a NVARCHAR/VARCHAR type with a longer size just at the beginning.

Marco

June 18, 2009 6:09 PM
 

Ken said:

The issue I was having was also that when I added a new row to the view it would NOT be found when I tried to process the database..

I found that I have to process the time calc dimension separately then I can process the database...

Great tool... and thanks for being so responsive... nice to see.

-Ken

June 23, 2009 5:00 PM
 

Ken said:

We are seeing the time calcs not working for a dimension that has a parent / child hierarchy (recursive relationship) when isAggregratable is set to False.. if it is set to True it seems to work.. but that breaks other things... Any way around or thoughts?

Thanks

-Ken

June 24, 2009 1:40 PM
 

Marco Russo (SQLBI) said:

Not a good thing - having IsAggregatable set to False might have a lot of side effects. I didn't tested MDX queries of DateTool in such a scenario.

But why do you need IsAggregatable set to False?

June 24, 2009 1:46 PM
 

Ken said:

We are providing a unary operator....from a column that has a + or - in it..  

To be honest I don't really know what isAgg will turn on or off..

Is there any info on what it will or will not do?

-Ken

June 24, 2009 2:25 PM
 

Marco Russo (SQLBI) said:

Ken,

the IsAggregatable only removes the first level in the hierarchy (the one with the "All" element).

June 24, 2009 5:09 PM
 

Ankit Mittal said:

Marco

I have implemented DateTool in my project and it has given tremendous performance boon. Thanks a lot!

Actually in my report there are more than 300 measures and all of them do not need Year-Over-Year calculations. So Is it possible to restrict DateTool on those measures where it is applicable?

Ankit

June 25, 2009 2:24 PM
 

Marco Russo (SQLBI) said:

You can use the SCOPE in the MDX Script to limit the Year-over-Year calculation, but you have to choose whether to leave the cell blank or with another value (but what? you will be in a cell corresponding to a Year-over-Year calculation, after all... a blank cell might be the better idea...)

June 25, 2009 5:34 PM
 

Ankit Mittal said:

Thanks Marco, but I will use multiple query approach in one report that is much faster then getting all 300 measures in one query.

As I told you there are 100 measures that don't need Year-over-Year calculations, so it is much better to get it in separate query with a different grain.

Thanks for your help.

Ankit  

June 25, 2009 10:26 PM

Leave a Comment

(required) 
(optional)
(required) 
Submit

About Marco Russo (SQLBI)

Marco Russo is a consultant, writer and trainer specialized in Business Intelligence with Microsoft technologies. He runs the SQLBI.EU website, which is dedicated to distribute resources useful for BI developers, like Integration Services components, Analysis Services models, tools, technical information and so on. Marco is certified as MCT, MCDBA, MCSD.NET, MCSA, MCSE+I.

This Blog

Syndication

Powered by Community Server (Commercial Edition), by Telligent Systems
  Privacy Statement