The complexity of information projects (1/5)
Never before has there been such intense interest in data. At the same time, expectations about what you can do with data have also never been quite so overheated. Over the years, the recurring complaint of managers has been the question: Why does it take so long to make information available?
The recurring complaint of many BI professionals over the years has been: Users don’t get the complexity of what is happening under the bonnet, they think that it’s just a matter of running off a new report.
The recurring solution over the years has been: we’re going to replace the tooling, the software tools which build the information. Over the last few years, the words ‘Big Data’ and ‘Hadoop’ have crept into these replacement projects. The pattern here is iterative as well: technology X will solve any problem just like that. Then during the replacement project there are complaints about it all taking so long and being so expensive.
This series of articles is an investigation into the causes of this. Why do information projects take so long? Where does the complexity stem from?
Part of the answer to these questions is organisation: more agility in the production process and better collaboration between all actors in the process. I’ve discussed this aspect at length in the series ‘Agile with Architecture’.
In this series, a dilemma becomes very evident: the relationship between decentral (autonomous) information production and the need for central governance (alignment) between the teams.
The alignment issue can also be seen in the way in which various types of information issues can be approached. It’s possible to classify the aims of an information issue resulting in several use patterns. Each use pattern requires suitable means to best answer these questions, as well as specific skills needed from both the information producers, information professionals who produce the information, as well as the information consumers.
Underlying this production issue, lies a deeper issue to do with the extent to which the information is consumed. The greater the diversity and size of the population of information consumers becomes, the more uniformity is needed in information interpretation and the greater the challenge in the information production process. This is the point at which complexity enters the stage and where organisations run into problems.
This series of articles examines what this complexity consists of. The aim of this series is to go into this at depth, so that it becomes easier to understand. This means that I’m also giving you insights which can help you to maintain your grip on an ever-expanding data landscape.