The role of context: the source of all complexity (4/5)
The question of what differentiates BI or analytics from other data uses has always interested me. I think that I am now able to answer the question quite simply: we deal with context differently.
Data is used to gain insights into where we stand with our organisations, how effortless or cumbersome the operational processes are, which opportunities and threats these organisations are encountering. Insights emerge from analysing the collected data and interpreting it and from discussing the results of interpretations with one another to answer the “What do we need to do?” questions.
The problem is, not everyone interprets data in the same way because their personal contexts differ. The great challenge for BI and analytics projects is making the shared contexts explicit, so that interpretations can be distributed along with the information.
The process of arriving at agreement about interpretations and expressing this in a shared context leads to a lot of confusion and delays in projects. This means that you need to think carefully about how to deal with context. When taking a decision how to deal with context, three subjects come into play:
- The difference in context between how information is produced and how it is interpreted.
- The distribution level of the information.
- The information life cycle: how and the reason the information is being used, changes over time.
I’m going to discuss these three subjects briefly, in relation to the five use patterns.
The difference between context of operational processes and context of interpretation
The context of an interpretation changes faster than the context of an operational process. To put it simply, what you think the information from a cross section of your operational processes means for you, where the operational process itself remains unchanged. In BI, we often speak of one version of the facts (information in the context of an operational process) and the different points of views about it (information in the context of an interpretation).
You need to keep these two contexts separate in your solutions, so that changes in interpretation have limited impact. By designing this into your solutions, you can prevent delays in implementing changes.
Use patterns deal with context differently:
- Monitoring — Information about the execution of the operational process itself. The context of interpretation is the operational process itself and therefore the information is often limited to the facts.
- Accountability Reporting – The context of interpretation has been formally recorded and is broadly supported. Context is modelled into the information solutions and is distributed with the facts.
- Analysis — Analysis is mainly research into the validity of the agreed interpretation. In analysis, extending the context takes place by adding (external) reference data. Facts are placed in a broader perspective and the interpretation is re-evaluated.
- Prediction — Factual information is combined with behavioural information about the actors, such as customers, suppliers, members of staff, website visitors or physical machines used in a production process. Making the predictive model is itself the interpretation. The results of the predictive model are validated against the facts and the quality of the interpretation is identified with statistical uncertainty.
- Data exploration — The facts are placed in various contexts in all kinds of ways, looking for new insights. Data exploration is often actually context exploration. It is the first step towards defining a (new) jointly shared interpretation of the facts.
The implication of this is that you can’t offer information with context in one universal manner. The difference in the amount of flexibility about how to deal with contexts leads to contradictory demands being made of the solutions which support them. One cause of delays in projects is that this isn’t recognised on time or that solutions are not explicitly designed for this.
The distribution level of information
The greater the reach that information has, the simpler it has to be. What is simplicity? It is making the context of interpretation explicit so that you have a greater chance that each user has the same interpretation. At least, that is what you try to achieve. You try to work towards the point at which information is self-explanatory for the target group.
The principle is simple: if I make a report, then I know what I have done and for which use context I have made it. If I give that report to a colleague and my colleague has a question about it, then they can ask me. The greater the distance, the lesser the chance that I would know where the report came from and who I should ask questions about it.
The relationship with the use patterns is complex, but distribution level does always play a role. The lead time of a project becomes inherently longer as the distribution level increases. Making information simple is a slow process which requires a lot of communication and testing.
The information life cycle
Usage of the same information changes over time. A simple example: data exploration leads to useful information which is of value to a larger group of people and which needs to be produced at set times. At a given moment, the decision is taken that the information is going to be added to a dashboard.
The use pattern to which the information belongs, changes because of the change in the interaction pattern between information consumers and information producers. The change in use pattern leads to other demands in terms of the robustness of the production and distribution process. The distribution level increases, which means that the information needs to be delivered in a simpler way to information consumers. The result is that the technical means which produce the information need to be made to fit the operational load suitable for the use pattern of accountability reporting.
The information life cycle is a forgotten subject in the information landscape. Organisations are often not prepared to invest in rebuilding information which has already been made available. Over time, the maintenance load increases and the lead time for changes or new information gets longer and longer, because too much information is being produced with the wrong means. The unambiguity of information diminishes because it is not being changed to fit actual usage. This reduces the benefits of the information valorisation. The countermeasure is simple: Start by thinking about the criteria which you are using to determine the circumstances in which information may be distributed. Keeping to this is, however, not easy.