I’ve always stated that creating a software solution is half and art and half an engineering process. Just like building an house, or a car, or any mechanical or electrical device, the art is that part of the work where someone has to create something new to satisfy the customer need.
Architects are the figures that plays the artistic role. Sure, they don’t re-invent everything from scratch every time: Architectural Design Pattern are here to demonstrate that ideas can be reused successfully. But, still, architects put patterns and new ideas together in order create something new.
Engineers are the more practical side of this role-playing game and they are more concerned with processes and implementation of the solution so that it can feasible in first place, but also be maintainable, predictable, replicable and cost-effective.
In the software industry we have Architects but my feeling is that we’re missing the Engineers. This means that a lot of what is being produced from the software industry is more an art than a science and this drives to the fact that the quality, and the maintainability of the final product is completely dependent on the people working on it. In addition to that, since there are no standards to follow, each one will do as it wish. If you put several people working together you’ll obtain a cauldron of different styles that leads inevitably to higher maintenance cost and low stability and predictability. (I don’t know if you have the same feeling, but I would like to have a feedback on this, if you care to share.)
Of course I’m generalizing a lot here – I complete discussion on this matter would easily fill a book – and there are several exception to this, but I think I can say without any doubt that if I had to make a picture of my perception of the software industry I’d draw a conglomerate of craftsman workshops and not a factory district.
In the last years I struggled a bit with this lack of the “engineering” discipline: particularly I found an evidence of this situation in BI projects. While there’s at least two well-know architectural approach (Kimball and Inmon), with other options rising (like the Data-Vault), but still there are no rules and tools that can make the creation of a BI solution more standard.
In order to be able to add people to an existing project quickly, being sure that they would create something that anyone of the existing team can easily understand, that the infrastructure of all of our BI solution would be similar in order to make its development faster and easier to maintain, I’ve defined a sets of rules and tools that helps to make the work easier and higher in quality.
The idea is that the rules will define the standards, and once we have the rules, we can also create the tools to apply them quickly. This will shorten a lot the time need to build the solution (which is the part with the lowest added-value) and leave much more time to design it (where there is the “core” value of it).
I’ve got this idea watching “How it’s made” on Science Discovery channel, watching people building the awesome Airbus 380. If such complex machine can be assembled by several people everyone doing a small part of the work, we should be able to do it also with BI solution! We “only” need rules that will drive organization of work and tools to make work easier and faster for anyone. Of course this also leads to a solution that is flexible enough to accomodote quickly new customers requests or changes of direction.
Apply the idea to Business Intelligence and you’ll have and “Engineered BI Solution” which will be both sound and agile. I like to call it “Adaptive”. That’s what I’ve already presented on PASS European Summit 2010 in an specific session that had very good feedbacks and that’s what I’m going to show in my “Creating a BI Solution from A to Z” post-conference seminar (8 hours of deep dive, blending theory and practice, architecture and engineering, making it a unique session of its kind), in my next BI Virtual Chapter meeting (more on this in another post) and also in the next posts of this series.
If you’re new to BI or you’ll starting a new BI project in the next months, or you simply want to learn more on this approach, don’t miss the opportunity, and share your feedback!