How connected architecture came into being
The story which we tell in “Connected Architecture: the end of information chaos” is founded in the convictions we have built up by practical experience in the many projects which we have carried out.
Our view on the world of data
Changeability in organisations and their environments has grown. That is unlikely to decrease for the time being, rather to increase. This will make the demand for data grow. Organisations will lose control of their data landscapes in their drive to keep up with the demand for information.
Our conviction can be summarised in two statements:
- Change is the only constant.
- The human factor determines your management of your data landscape.
You can only keep control in a dynamic environment if the steps you take don’t exceed the ability of human beings to understand them. It’s not so much about the steps which an individual can take, but the steps that need to be taken jointly to apply new information in a valuable way. You can take increasingly large steps as the maturity of an organisation in applying information grows. To get there, however, you will first need to return to more easily manageable changes so as to tame the chaos.
Everyone’s favourite cliché: speed of technological change
Where does this changeability come from? Are we causing it ourselves or are we just powerless little paper boats being tossed around by the current?
Let’s look at “digital transformation” as we now know it: this is the result of a development which was initiated in the nineteen sixties. Technological developments in computational power, data storage and connectivity have led to a massive reduction in data
processing costs.
If you read the paper, you might think that technological change dropped right out of the sky. At this moment, we are in the acceleration phase of technological development. It’s difficult to predict how long the acceleration phase is going to last. It’s also tricky
to predict the direction of this technological change.
The only thing you can do is to go with the flow, as best you can.
The curse of Agile
The best illustration of the need to go with the flow is the hype which has emerged around “Agile”. I have several reservations about the claims made in this hype. Whenever there is uncertainty, people appear who are more than happy to sell certainty. Enthusiastic agile amateurs are largely responsible for aversion to the term.
There is a real need for agility in organisations so that they can join in with the acceleration of technology; however, the term you use to express this need is unimportant.
You can only achieve more agility if you are willing to accept two truths:
- The future is uncertain, and you cannot predict it.
- The only way to deal with this is to keep supporting one another and to take comprehensible, forward steps together into the unknown territory.
The principles on which agile forms of co-operation are based are very simple. The most important one is that feedback needs to be organised to determine if taking one step forward is going to achieve the desired result or if the circumstances have changed to such an extent that it would be better to retrace your steps and choose a different direction. You need to institutionalise this behaviour and manage it.
The consequence of this is that you need to make it possible to take small steps backwards and forwards. These might be in terms of how the relevant technologies are implemented, the structure, processing or distribution of data storage, but also the data processing architecture and the data modelling.
This won’t happen by itself, because before you know it, you’ll be trying to control the complexity of the evolving data landscape by increasing structure.
If you neglect to make the human factor your guiding principle when organising the way in which information users and the people who make information solutions work together, you will get bogged down and the agility of your data landscape will be reduced.
Why change is the only constant factor
Long term technological change leads to structural changes in organisational environments. Structural market changes take places over decades, stopping and starting as they do so. Markets change due to new entrants and production chains being broken open. The interaction with consumers and suppliers has changed, intermediaries are disappearing off the market, technology is making requests for individual solutions possible.
Business models are changing, empires have emerged which earn money selling customer profiles; customers provide this information in exchange for software and communication options. Technological changes are accelerating. This means that the rate of change is expected to increase in the future.
Technological development gives us new sources of data. On the one hand, there is log data from machines, sensors and appliances connected to the internet and on the other hand, patterns of human interaction and communication.
This has led to an explosion of apps on mobile appliances generating data and which need data as input. Providing people with information has become child’s play. These new sources and interaction patterns are leading to new insights and stimulating the demand for information.
The ever-present connectivity and computational power have ensured that powerful algorithms can process this data on mobile phones. A modern telephone has more computing power than a laptop did only 3 years ago.
Combining layers of data, such as satellite images, measurements, signals from appliances and the automated analysis of visual images to register changes, means that we can construct information applications in an office and make them usable anywhere in the world.
Imagine areas such as food production and generating and selling energy. We have only just started to imagine what these new data sources may be able to do one day.
Four types of stress caused by constant change
Technological change and the resulting structural marketing changes are leading to new and unchartered territories. It is always difficult to make progress in unknown territories. You need to experiment. You can try out different solutions at the same time, many won’t be successful.
This leads to pressure on the demand for information, due to both its changeability, as well as the lack of clarity of what is needed.
The need for agility to change along with the markets and parallel testing of new products and services in unknown territories, leads to problems in alignment: everything is just as important as everything else and just as volatile. It is difficult to keep an overview in fast changing demands for information.
New sources of information which have become available through new technologies can lead to choice stress: which technologies should your organisation bet on? What can I do with these new sources of information? New insights develop from adding different kinds of information and new insights lead in turn to new demands for information. This makes the demand for information very unpredictable.
How to structure your data landscape
Information provision needs to address two important challenges as a result of choice stress:
- Quicker delivery times for new information;
- Being able to support parallel development.
This is only possible if you are able to let go. Letting go means that you don’t try to centralise everything at any cost, but that you make autonomous development possible, without losing sight of alignment.
This is possible if you elevate reducing complexity to a guiding principle. You can do this with a decomposition of the entire data landscape. This means that you break the entire data landscape down into parts which each only have to meet a limited number of demands. Each part can be developed and maintained autonomously.
To prevent double work and the creation of data silo’s, you need to agree the limits of this autonomy as well as how you will ensure coherence, while trying to restrict mutual dependence.
You need to do this while never losing sight of the human factor in the way information users and information producers work together. Keeping everyone involved makes it possible to take tiny steps into unchartered territories quickly. This makes it possible to discover if something is going to be successful, which information and technologies are worthwhile investing energy into and when you should decide to retire redundant information products.
You need a clear framework for testing this, in which every voice is heard and where the consequences of decisions become transparent.
The framework of connected architecture is such a test framework. That’s why we set it up. It helps you organise for change, through simplification, structuring the way people work together and using a transparent architectural decision-making process to create workable arrangements.