A Tale of Two Bens

As a general rule, the most successful man in life is the man who has the best information.

– Benjamin Disraeli

By way of introduction, I should probably tell you that plowing through information gets me sort of excited.  My wife will tell you that this is not normal. Maybe it’s a mutated gene that all analysts have.  The only thing that might get me salivating even more is a really well-defined and documented process. My wife will tell you that this goes beyond not being normal and crosses into the realm of wonkish-ness. Show me something that puts information and process in harmony and you may see me go into some dervish like frenzy.  Now you have the reason I got so excited with Kalido that I had to come be a part of it.  Thankfully she doesn’t find that weird at all. Read more

How to Scope Rules and Policies for Data Governance?

“The powers not delegated to the United States by the Constitution, nor prohibited by it to the States, are reserved to the States respectively, or to the people.” – Tenth Amendment, United States Constitution

A large insurance conglomerate has been doing data governance for many years with a lot to show for it. They instituted data quality policies for financial data coming from various divisions into the corporate finance department. Because of this, they were able to improve the efficiency of the quarterly close process. As a result of their data governance efforts, this company now can close in under 10 days rather than an average of 27. Read more

Who in the world needs a data warehouse with bad data in it?

I just read a recent article in Information Management entitled “Who in the World Needs a Data Warehouse?” On seeing the title, I immediately thought this would be an article about how companies can simply load their data in memory to avoid building a data warehouse entirely. As I suspected, that is the author’s proposed option. The article listed off a variety of issues with building a data warehouse, including capturing requirements and dealing with scope creep, ETL design complexity, data integration and data quality. In several prior posts, I discussed ways to handle capturing requirements and the ETL through business modeling and automation. But in this post I want to discuss data quality. The article says “data that is housed in the data warehouse is often either incorrect or inconsistent.” If in-memory analytics is the answer to this problem, I ask: what’s the difference if you load incorrect and inconsistent data into memory? How does simply moving it off disk fix this problem? The answer is: it doesn’t! Read more

Can You Handle a Single Version of the Truth?

“We worked long and hard to get to a single version of the truth. Then we realized that we can’t handle a single version of the truth,” said a data architect at a global manufacturing company. This company built a beautifully architected and perfectly conformed enterprise data warehouse with a single unified portal for BI and reporting. But business units everywhere, from Europe to Latin America, expressed great frustration because they can only get information out of the system in one way. As a result, spread-marts proliferated. Read more