From fractal geometry, we know such fascinating structures as the Koch Snowflake or broccoli. In fact, many structures in nature are fractal. An outstanding feature is their self-similarity on various levels of granularity. Consider a single broccoli floret which is similar to the larger stem and even to the entire head of broccoli. Moreover, processes of growth may be facilitated by fractal structures. What if we applied these principles to an IT landscape? Imagine a number of microservices growing like broccoli florets on infrastructural ground, evolving into larger conglomerates repeating the same basic integration pattern over and over. Those larger conglomerates represent »more of the whole«, e.g. the entire enterprise. Sounds bizarre? Correct, but wouldn’t you agree that today’s IT landscapes tend to fall apart into too autonomous microservices reigned by too autonomous agile teams? Alignment concerns tend to lose out, be it an integrated user experience, traceability of business processes or holistic scaling in the cloud. Couldn’t this bizarre Fractal Architecture for IT (FractArc.IT) facilitate a balanced trade-off between autonomy and alignment concerns in an IT landscape?
Of course, there are many open questions about FractArc.IT. Where does it come from historically? What are autonomy and alignment concerns anyway? How do software units relate to the business? What about the tech stack? This article will explain the background, the actual approach and the prospects in more detail. It is meant as a complete yet compact introduction, illustrated by an end-to-end example of an imaginary e-commerce company.
- What is Fractal Architecture for IT? What not?
- Architecture disciplines and systems theory
- Why MECE prevents a balanced trade-off
- The what ain’t none of your business, dude!
- From integration patterns to the fractal pattern
- The domain-specific event format
- Pulling it all together: A fractal IT landscape
- Is our future fractal, Mr. Mandelbrot?
- References
If you are a software implementer or an IT manager, particularly an enterprise architect, this basic work is for you. However, the main line of argumentation is bottom-up, i.e. »How can autonomous software units such as microservices be aligned into the greater whole of an enterprise?« This way, software implementers should be able to take root in the strategic realm of IT landscapes – where they will encounter enterprise architects. They in turn may learn something about autonomy-focused software application architecture. It will become clear why it does not work to insist on both a short time to market and large scale-synergies at the same time. And of course, both software implementers and IT managers will have the opportunity to establish a common understanding about autonomy and alignment in IT landscapes.
If you are in a hurry, you may concentrate on the graphics. They are sufficient for an overview. Alternatively, you may focus on the paragraphs highlighted by green vertical bars. They summarize the central statements of FractArc.IT. Furthermore, if you want to print this article on paper or as a PDF, please use the print function of your browser. You may have to enable background images to include the graphics.
What is Fractal Architecture for IT? What not?
Fractal Architecture for IT (FractArc.IT) is a conceptual framework centered around a technical integration pattern which is repeated recursively on several levels of granularity in an IT landscape. It facilitates a balanced trade-off between autonomy concerns on the one hand and alignment concerns on the other hand. This is especially relevant in today’s IT landscapes in which autonomy-focused approaches like microservice architectures and team agility are not seldom employed unthinkingly as the one and only way to go. From a business perspective, they allow for prompt movements often vital in saturated target markets. Being faster than competitors is the name of the game. Ultimately, businesses will lose this game if they are fast with the »wrong« thing.
Now in the 2020s, we have the chance to do the »right« thing fast. We know how to decentralize the decision-making, how to confine cascades of changes, how to increase the flexibility and decrease the time to market as a result. From the 1990s and early 2000s, we also know how to align IT landscapes towards the purpose of the business, including, but not limited to, making money. As a means to this end, we know how to consolidate IT landscapes and maximize synergies, be it through standardization of technology stacks, the procurement of volume software licenses or by merging organizational units. Now in the 2020s, will we go for another extreme or a balanced trade-off instead? The following graphic illustrates the dilemma as well as the concept of integrating both kinds of approaches – autonomy-focused and alignment-focused ones.
Granting a sneak preview of a fractal IT landscape, the following graphic illustrates the overall benefit of FractArc.IT. It is based on the example of an imaginary e-commerce company which will be discovered step by step.
For a more complete overview of the framework, the following graphic summarizes its core elements.
Of equal importance are the limitations of FractArc.IT which are outlined below.
- Sufficient complexity of an landscapes: FractArc.IT looks at IT landscapes which are sufficiently complex, i.e. landscapes of large areas of business, large enterprises or even ecosystems spanning several legal entities. It is not suited for comparably small IT installations as may be common in start-ups.
- Software units as deployment units: It has an integrative take on the disciplines of software application architecture and enterprise architecture. However, the design within one software unit, e.g. one microservice, is out of scope. Instead, the framework stops at the level of deployment units which are visible in an IT landscape.
- Neutrality towards infrastructure: FractArc.IT does not delve into Infrastructure details such as the distribution between on-premise IT and public cloud providers. Neither does it try to revive centralized middleware approaches such as the Enterprise Service Bus (ESB). However it encourages a fresh look on the integration of software units across a distributed IT landscape.
- Focus on enterprise applications: The framework targets enterprise applications, i.e. applications focusing on business processes and value streams, making the right sets of data available to the right users. By contrast, it does not specifically target applications focusing on performance-optimized algorithms, Big Data, IoT or Machine Learning.
While the concepts and the examples are derived from an early prototype, FractArc.IT is still emerging. See the final section Is our future fractal, Mr. Mandelbrot? for an overview of its future prospects.
Architecture disciplines and systems theory
Generally speaking, Fractal Architecture for IT (FractArc.IT) draws knowledge and best practices from the following disciplines:
- Enterprise Architecture Management (EAM)
- Service-Oriented Architecture (SOA)
- Domain-Driven Design (DDD), specifically its strategic design and event-driven approach
- Software application architecture, with an inclination towards microservice architectures
- General systems theory and fractal geometry (in a broad sense)
While there is no generally accepted definition for any of the mentioned disciplines, microservice architectures is a term requiring some extra words. In this article, by microservice we mean a deployment unit in a larger software application which exposes the features outlined by Martin Fowler and James Lewis in their 2014 article »Microservices« Fowl. Microservice architectures, in the way the term is used here, include several styles of IT architectures with comparably small deployment units, distributed across a network, addressing autonomy concerns within distinct realms of responsibility. Such a realm can be defined in technical or business terms or both and is often governed by an autonomous agile team. Hence, Self-Contained Systems (SCS) and the Cloud Native architectural style are included in this definition. Now what are examples of the autonomy concerns they address?
- Decentralized decision-making: As the term »realm of responsibility« may imply, it encourages decentralized local decision-making. The microservice community speaks of the micro and the macro architectures when it wants to separate the realm in which a team may decide autonomously from the realm in which a team is inclined to align with others. As to the micro architecture, decisions typically relate to programming languages, paradigms and technology stacks.
- Better control of changes: There are several triggers for changes in an IT landscape, first and foremost changes in customer needs, legal changes and internal economic drivers. Such changes can be controlled more easily within a confined realm. In other words, unwanted cascades of consecutive changes can be confined more easily if all the deployment units and the team needed to handle them are within the same realm.
- Flexibility in the evolution: Confining unwanted cascades of changes is one thing, but using changes for the better is another. As a prerequisite, an IT landscape needs to be flexible. Generally speaking, an IT landscape can evolve more easily if it consists of comparably small and flexible deployment units rather than large and rigid ones. Besides their comparably small size, the flexibility stems from the exchangeability of deployment units as long as their technical interfaces remain stable.
- Shorter time to market: A shorter time to market is probably the most cited motivation for adopting microservices and team agility. It may provide an advantage over competitors, especially in saturated markets. Ultimately, a shorter time to market is a mediate or immediate consequence of the mastering of the aforementioned concerns.
Maybe you have stumbled across the phrase »a team is inclined to align«. It hints at a bottom-up approach to alignment in contrast to a top-down approach. So to speak, FractArc.IT views alignment as a vertical motion in an IT landscape. Now what are examples of the alignment concerns which tend to lose out if microservice architectures and team agility are employed in a one-sided way?
- Cost savings through synergies: The classical management body of thinking largely focuses on cost savings through synergies. Reuse of resources, standardization and avoidance of redundancy should lead to significant cost savings. Classical Enterprise Architecture Management (EAM) is based on that way of thinking, too. Likewise, Enterprise Application Integration (EAI) and Service-Oriented Architecture (SOA) try to reuse existing IT resources by centralizing them. You may deem these approaches outdated. However, cost savings are still relevant in today’s business strategies. Take software licenses as an example. Imagine several autonomous teams negotiating individual contracts for licenses of the same Document Management System (DMS) offered by a major corporation. Can this possibly be cost-efficient on a large scale? A roll-out of volume software licenses will likely be more cost-efficient. Please note that FractArc.IT does not specifically address this concern, at least not in the sense of classical business management. In the scope of FractArc.IT, cost savings are no end in itself.
- Integrated user experience: Surely, user-centricity is one of the most indispensable axioms in all of today’s IT and business endeavors. While the absoluteness of this axiom is questionable, it is taken as a given here. Even within the microservice community, integrating autonomous microservices into a consistent user frontend is discussed as something precarious. But integrated user experience is more than a consistent frontend. It relates to entire value streams and integrates various interaction points with users.
- Traceability of business processes: A business process, for instance the fulfillment of customer orders, most often spans multiple realms of responsibility and multiple teams. Hence, whenever an overall business process must be tracked, this requires an aligned effort of multiple teams. Such a need can arise when the legal compliance of processes must be proven to authorities via an audit trail. It may also be the very self-interest of an enterprise to monitor business processes for error handling, prediction and continuous improvement.
- Holistic scaling in the cloud: Scaling in the cloud is comparably easy for individual deployment units, e.g. microservices in pods in a Kubernetes cluster. Scaling from a business point of view is more difficult. Imagine an IT manager requesting an increased IT capacity for a particular use case, e.g. the escalation of failing orders for VIP customers. There might be a couple of dependencies to consider, not just scaling a set of autonomous microservices. This is a misery, as (elastic) scaling is one of the major benefits of the cloud.
Regarding autonomy and alignment concerns, the central statement of FractArc.IT is the following:
Autonomy and alignment concerns can and must be addressed on all levels of granularity in an IT landscape. They are not reserved to software application architecture and Enterprise Architecture Management (EAM), respectively. They rather must be integrated in a common discipline of IT architecture.
The following graphic makes this statement more tangible by example of an imaginary e-commerce company. As indicated previously, vertical and horizontal motions pertain to alignment and autonomy concerns, respectively. You may recognize a first analogy to the introductory reverse broccoli model.
What is meant by levels of granularity? Microservice architectures propagate rather fine-grained deployment units compared to coarse-grained monoliths common in the 1990s and partially in the 2000s. This is reflected at the bottom level in the preceding graphic. As will be explained more comprehensively in the next section, these small sizes of deployment units correlate with autonomy concerns. This is expressed in the horizontal dimension. Levels of granularity change alongside the vertical dimension. If we follow the commissioning service and its three neighbors upward we see them aligning into the coarse-grained business capability of Logistics. Eventually, fine-grained units at a lower level represent »more details of a selected part«. Coarse-grained units at a higher level represent »more of the whole«. Correlated with this vertical dimension is a motion of alignment as indicated by the upward arrow. It is ultimately headed to the overall purpose of the business expressed by its vision, mission and strategic goals.
For those familiar with the basic concepts of general systems theory, the preceding explanations are not new. However, FractArc.IT uses these concepts in a rather broad sense and not in a very strict sense. For further reference, the holon theory by Ken Wilber is a good starting point Wil01. It is fundamentally based on works by Arthur Koestler. Having said that, the section Pulling it all together: A fractal IT landscape hints at a more formal way of measuring the granularity of a unit.
Why MECE prevents a balanced trade-off
Up to now, the main line of argumentation was bottom-up, i.e. from autonomous software units to the greater whole of an area of business, an enterprise or an entire ecosystem. If we only had this bottom-up kind of motion, then hitting the overall business purpose was nothing more than a question of chance. Hence, how can we operationalize the business purpose in a more systematic way? Such diverse disciplines as, Enterprise Architecture Management (EAM), Business Architecture and Design-Driven Domain (DDD) help to break down the vision, mission and strategic goals of a business into the necessary units. The following table compares the concepts of a business capability, a bounded context and a domain.
Business capability | Bounded context | Domain | |
---|---|---|---|
Perspective | normally top-down (management’s perspective) | bottom-up (implementers’ perspective) | normally top-down (but can be bottom up in DDD) |
Aim | redundancy-free, consistent overview of an area of business or even an entire enterprise BAG20 | clear segregation of language areas Eva03 in software implementation projects with speed and evolvability in mind | in software development: definition of the business scope for the software under development; in DDD: problem space analysis Ver16; in TOGAF: definition of a target realm for architecture descriptions, i.e. business, data, application and technology TOG18 |
Decomposition | into finer-grained business capabilities BAG20 | into software units (for the purposes of Fractal IT Architecture) | into finer-grained domains and/or business capabilities |
As may be seen, the semantics of »domain« vary substantially by discipline. Commonly in software development, a domain defines the business scope for the software under development. In this sense, a packaging service could be a software unit in the logistics domain. In DDD, domains are more specifically considered a sub-part of an overall domain for the purpose of problem space analysis. Unless your company is a shipping provider, logistics would not be a core domain but a generic subdomain. Hence, you would rather outsource logistics services than build your own software around it Ver16. In EAM, generally speaking, domains often denote clusters of business capabilities. However The Open Group Architecture Framework (TOGAF) refers to »Architecture Domains« meaning the target realms of architecture descriptions – these being business, data, application and technology TOG18. For those perplexities, Fractal Architecture for IT (FractArc.IT) does not particularly dwell on domains. It rather uses the more specific concept of business capabilities as described by the BIZBOK® Guide BAG20.
In Business Architecture, business capabilities are the foundation for strategic planning, change management and impact analysis. They describe »what a business does« and deliberately separate it from how it is done or by whom it is done BAG20. TOGAF’s take on business capabilities is similar but calls for a 3rd-party »Business Capability Management« framework when it gets down to the nitty-gritty of actually defining the capabilities TOG18. For the purposes of FractArc.IT, the most relevant features of business capabilities are:
- Capabilities decompose into more capabilities, the latter being on a finer-grained level.
- Capabilities map to other views of the business, e.g. organizational units, value streams and IT assets.
Sometimes, when enterprise or business architects talk to software architects, they would lump together business capabilities and bounded contexts from DDD. Chances of them being at cross purposes are high. FractArc.IT recognizes the differences between these concepts and at the same time adopts them for their similarities. What are bounded contexts anyway? Generally speaking they are language borders. That is, referring to Eric Evan’s initial remarks on the matter, a bounded context is defined by a ubiquitous language shared by a team within their respective area of business Eva03. »Ubiquitous« in this case means a language that is shared throughout conversations, sketches and even the source code of the software produced by the team. For instance, let’s assume a team is associated with a bounded context named outbound logistics. The teams’ language is made up of terms like commissioning, packaging, physical goods, delivery notes and shipping providers. Let’s further assume there is another team in the same enterprise associated with a bounded context named warehousing. Now both teams might deal with physical goods – but the latter team more in the sense of raw material, inventory, demand forecasting and restocking.
An important thing to notice is the upward direction of the alignment arrow in the preceding graphic in contrast to the downward direction of the decomposition. This can be interpreted as: »Managers try to impose alignment goals by decomposing a business from top to bottom.«
Other than business capabilities, bounded contexts may not be nested but contain software units. While software units may take different shapes, let’s consider them as microservices for now. When Martin Fowler and James Lewis reported on the comparably new phenomenon of microservices back in 2014, they characterized them as »organized around business capabilities« Fowl. Voilà. Doing away with canonical data models, the two thought leaders recognized bounded contexts as a useful way of thinking about »Decentralized Data Management«. While this goes in line with DDD’s original object-oriented modeling of static structures, today’s notion of bounded contexts also encompasses paradigms beyond object orientation as well as dynamic structures. For the purpose of decomposing an IT landscape, we view each software unit as contained in one and only one bounded context. All things considered, there are two important features of the association of bounded contexts and software units:
- Bounded contexts and their contained software units create discrete language scopes for both business and IT.
- At the same time, they create a space in which business and IT may evolve independently from other such contexts.
As for the latter aspect, evolvability through autonomy, Martin Fowler and James Lewis state: »The key property of a component is the notion of independent replacement and upgradeability – which implies we look for points where we can imagine rewriting a component without affecting its collaborators.« Fowl Now what is the kind of relationship between business capabilities and bounded contexts? Consider the following graphic which adds some capabilities on level 2:
There is a zone in the vertical dimension where the decomposition breaks. That is, the MECE principle behind the decomposition by business capabilities breaks. MECE is an acronym for »mutually exclusive and collectively exhausting« coined by Barbara Minto in the 1960s Wiki. Looking at the graphic, think of Customer Relationship Management and Logistics as well-defined capabilities without any semantic overlap. Together with all other capabilities on the same level, such as Procurement, they completely cover that level of granularity. When repeating this principle recursively on lower levels, the result will be a structural breakdown of the entire business. Normally this recursion stops at a level where the units are manageable by software and people. Enter: Bounded contexts. That is their very purpose. Creating units that are manageable by software and people. However, bounded contexts are usually approached in a bottom-up kind of motion whereas business capabilities are broken down in a top-down kind of motion. This leads to upheavals in the zone where those motions collide. Look at the communication relationships between the level 2 capabilities and the bounded contexts underneath. As bounded contexts are created with autonomy in mind, they implicitly accept semantic redundancy across contexts on one level. Only within a particular context are the semantics unique. Hence, when mapping bounded contexts to business capabilities designed by the MECE principle, n:m communication relationships will arise. The unambiguousness breaks. MECE breaks.
MECE is a principle which, when consistently applied to an IT landscape, will prevent a balanced trade-off between autonomy and alignment concerns. The reason is that MECE stems from a classical management body of thinking which focuses on synergies gained by reuse, standardization and avoidance of redundancy. While this is perfectly legitimate, autonomy-focused approaches deliberately accept redundancy.
In philosophical terms, we need to transcend Aristotle’s stringent logic and embrace a paradox Zen-like Art of Architecture Maintenance. In more mundane words, it’s about communication. Human communication is normally not confined to the paths of a MECE breakdown structure – something it was not intended for anyway.
The what ain’t none of your business, dude!
As you may have noticed, software units are consistently named »service« in the preceding graphics, e.g. commissioning service. Please do not mistake these examples as a general principle in Fractal Architecture for IT (FractArc.IT). Microservices are rather one possible manifestation of software units. Generally speaking, FractArc.IT looks at software units as deployment units which are visible in an IT landscape. It does not get involved in their »interior design«, i.e. in programming languages, paradigms, persistence strategies and frameworks used within those units. However when it comes to interfaces and middleware addressing alignment concerns, FractArc.IT does get involved in technical aspects.
A software unit may take the form of a Function as a Service (FaaS), a microservice, a Self-Contained System or even a monolith. At any rate, it can be viewed both in a business light and a technical light. In the first, a software unit is perceived as a set of functionality within a specific bounded context, assuming a unique responsibility within that context. In a technical light, the interface properties of a software unit shine, i.e. the interface technologies, protocols and payload formats it supports. In this sense, a software unit has a dual business/technical nature.
It makes sense to subdivide interface technologies in synchronous and asynchronous variants. For instance, Representational State Transfer (REST) is the prevailing synchronous variant today. And yes, »technology« is not the best term in any case, as REST is rather an architectural style but no technology. To not confuse them with architectural disciplines and styles such as software application architecture and microservice architectures, they are just called »interface technologies«. On the asynchronous side, we face the large world of messaging, with »event-driven« being more or less a subvariant.
Considering the dual business/technical nature of software units, they are not limited to the lowest level of granularity in an IT landscape. For this to understand entirely, we need to look at what needs to be done in a business versus how it is done. The distinction of the two is a major premise of business capabilities, as outlined in Why MECE prevents a balanced trade-off. What is more, it is a typical cause of dispute between a business analyst and a software implementer. »The what ain’t none of your business, dude!« might be an utterance from the former to the latter. But let’s see if such a dispute is justified from the point of view of FractArc.IT. Naturally, software units are expected to populate the solution space describing how business needs are implemented. But let’s look consider the levels of granularity.
The top row in the graphic obviously shows what the business does in terms of first level capabilities, e.g. Customer Relationship Management (CRM). So far so good. On the next level, couldn’t one argue that Complaint management and Customer order management describe how CRM is implemented? If this seems a bit far-fetched, please consider that there might be other modeling options. Another enterprise might consider customer order management more closely related to logistics. Still another enterprise might consider Complaint management a first level capability. So to some extent, the modeling of the what implies how it is implemented on the next lower level. While this might still seem a bit far-fetched, the next levels down to the software units make the case. We recognize that Customer order management is implemented, inter alia, by outbound logistics which in turn is implemented through commissioning, packaging, shipping and returns handling. Considering this, we can state: The what on one level is the how on the next lower level. To some extent, this relates to the dual business/technical nature of software units. The business traditionally defines the what, IT defines the how.
Software units possess a dual business/technical nature, as well as a dual what/how nature. This makes it possible to utilize them on several levels of granularity, not just the lowest level.
From integration patterns to the fractal pattern
If we can utilize software units not only at the lowest level of granularity, there must be more coarse-grained software units. Monoliths? Well, not necessarily. As outlined in section Architecture disciplines and systems theory, units at a higher level of granularity represent »more of the whole«. This generic notion from systems theory can be put in the concrete terms of middleware and integration concepts in IT. They deal with the integration of software units across a distributed IT landscape. The Service-Oriented Architecture (SOA) from the 2000s is an example.
As you may have noticed, software units are labeled »service« in all the preceding graphics, e.g. commissioning service or shipping service. While this labeling is in no way mandatory, it is typical of microservice architectures and their predecessor SOA. In fact, the conception of software units outlined in the previous section stems from SOA which defines a »service« as both a unit of business and a unit of software. Services can be composed of each other in a hierarchical way. For instance, according to Krafzig, Slama et al., a basic service is a simple data-centric unit with create, read, update and delete (CRUD) functionality Kra04. A so-called intermediary service depends on one or more basic services, while a process-centric service can call one or more intermediary and/or basic services. Back in the 2000s. the technical integration concepts of SOA were more popular than the business concepts. Message-oriented middleware (MOM) evolved to Enterprise Service Buses (ESB) and the latter evolved to Business Process Management Suites (BPMS).
Following this hierarchical integration of software units, we arrive at the basic fractal pattern of FractArc.IT. The following graphic dwells on the example of bounded contexts and software units from the imaginary e-commerce company.
On the lower left, the outbound logistics bounded context shows the software units already known from the preceding sections. They communicate over the network via explicitly defined interfaces. FractArc.IT assumes that the communication within one bounded context is synchronous by default, e.g. via REST or gRPC. However across two bounded contexts, asynchronous communication is mandatory to support their autonomy through decoupling. FractArc.IT does not mandate a specific protocol, but since it focuses on enterprise applications JMS and AMQP are preferable. An example of asynchronous communication is the shipping service connecting to the goods movement channel in the more coarse-grained goods movement monitoring bounded context. The asynchronous nature of the communication is implied by the interface label »publish«. That is, the shipping service publishes messages to the goods movement channel, e.g. saying that a package has been shipped. Other software units like the forecast service subscribe to messages of that kind via the interface labeled »subscribe«. This so called publish/subscribe integration pattern ensures some degree of decoupling between the message sender and receivers via a message channel as an intermediary. For the purposes of FractArc.IT, it is a concrete asynchronous integration pattern which helps to maintain the autonomy of the bounded contexts involved. A good reference for asynchronous integration patterns is ente.
Within one bounded context containing several software units, FractArc.IT assumes synchronous communication as a default. Across two bounded contexts, it mandates asynchronous communication to keep a chance of maintaining their autonomy.
On the lower right, the inbound logistics bounded context also contains software units publishing messages. Let’s assume the returns handling service and the purchase order service announce new incoming packages. This makes goods movement monitoring an integrative bounded context, as it receives messages from other more fine-grained bounded contexts. Semantically, it receives messages pertaining to goods movements and distributes them to other bounded contexts on any level of granularity, i.e the same, a more fine-grained and/or a more coarse-grained level. For instance, the warehousing bounded context subscribes to messages of that kind, obviously to forecast the demand and to automate the restocking of the warehouse. Likewise, the more coarse-grained customer order tracking and purchase order tracking bounded contexts subscribe to those messages. An integrative bounded context like goods movement monitoring is special due to its alignment focus. By contrast, a bounded context in the original sense of the strategic design of Domain-Driven Design is autonomy-focused. However, an integrative bounded context is still a language border. That is, the alignment concerns it addresses also center around a language of their own – in this case a »goods movement« language. Hence, there might be other software units in an integrative bounded context as long as they adhere to that language, not just the message channel with its intermediary role.
An integrative bounded context is alignment-focused and contains at least one message channel to receive messages from more fine-grained bounded contexts. It is still a bounded context in the sense of a language border. In fact, the alignment concerns it addresses center around a language of their own.
Repeating this basic fractal integration pattern in a recursive way leads to a fractal IT landscape. The entire picture is shown in section Pulling it all together: A fractal IT landscape. But wait, one piece is still missing: Events.
The domain-specific event format
Nowadays, asynchronous communication is often used as a synonym for »event-driven«. FractArc.IT too builds on an event-driven approach on top of the messaging approach described in the preceding section. That is, the payload of a message contains an event. An event reports something that has happened. The following is a technical representation of an event originating from the shipping service.
{
"id": "wh_e42a3b98f033a12b",
"type": "ItemShipped",
"createTime": "2021-May-01 12:54:08"
"source": {
"boundedContext": "Outbound logistics",
"userPseudonym": "Anne Doe",
"softwareUnit": "HardShipper v2.31"
},
"correlationIds": {
"CustomerOrderId": "7af7-903c",
},
"revisionIds": {
"itemRev": "2fe4-7a72-0210428-104711",
"shipmentRev": "20160402-104710-fe"
},
"goodsMovementTime": "2021-May-01 12:53:59",
"eventBody": "Shipped with express service."
}
This event adheres to a domain-specific event format, targeted at the goods movement monitoring integrative bounded context. First of all, it contains an event type, ItemShipped
in this case. The goodsMovementTime
field states when the item has been shipped and the itemRev
field refers to that item. The more generic fields comprise the createTime
field and the source.boundedContext
field stating the bounded context from which the event originates. You may notice that no detailed data of the shipped item is contained, only a reference in the revisionIds.itemRev
field. More specifically, the event refers to a historicized revision of an item with the ID 2fe4-7a72-0210428-104711
. There is another reference to a shipment revision with the ID 20160402-104710-fe
. What is more, the IDs in the correlationIds
set allow for the correlation of individual events which belong to the same customer order. This is helpful for the overall monitoring of customer orders, as well as for error handling and testing. The eventBody
field just contains some extra human-readable information.
FractArc.IT presumes one such domain-specific event format per integrative bounded context, supporting the alignment concern(s) of that respective bounded context. The revision Ids are intended to keep the events lightweight, supporting the autonomy of the corresponding bounded contexts and the scalability of the overall solution. As a possible drawback, the integrative bounded context must look up the detailed data through an interface of the corresponding software unit. Its underlying data store must support the historicisation of data. To allow for a balanced trade-off in a particular use case, lightweight bits of detailed data may also be sent within the eventBody
field directly within the event.
Although JSON is used as the payload metaformat here, FractArc.IT does not mandate a specific one. For instance, XML or Protobuf can be used, too. Admittedly, there are a lot more technical implications of this approach and some additional learnings from an early prototype. However as this article provides a conceptual overview, it does not go into any more detail at this point.
Pulling it all together: A fractal IT landscape
Before looking at an entire IT landscape, it makes sense to summarize the features of the basic fractal pattern.
Feature of the fractal pattern | Description |
---|---|
1. Integrative bounded context | An integrative bounded context is a language border like any other »normal« bounded context. However, it addresses at least one alignment concern like the traceability of a business process. As a consequence, it is dependent on the communication with other, usually more autonomy-focused and more fine-grained bounded contexts. To this end, it contains at least one message channel based on the publish/subscribe integration pattern. This makes up its »integrative« nature. |
2. Domain-specific event format | The messages published to an integrative bounded context are formatted as domain-specific events. The format is specifically targeted at the alignment concern(s) of that integrative bounded context. Normally, there is one domain-specific event format per integrative bounded context. It mandates some common fields such as the originating bounded context, a timestamp and business-related IDs. A domain-specific event format has at least one technical representation, e.g. in XML or JSON. |
3. References to detailed data | Generally speaking, the domain-specific format for an integrative bounded context contains a set of revision IDs. Each of those refers to a historicized revision of a record in another bounded context, mostly the originating bounded context of the corresponding event. Together these referenced records make up the detailed data of the event. This keeps the event lightweight, supporting the autonomy of the corresponding bounded contexts and the scalability of the overall solution. However, it is perfectly fine to carry lightweight bits of detailed data in the event itself if this helps to keep a use case implementation simple. |
4. Synchronous retrieval of detailed data | While events are always published and subscribed through an asynchronous message channel in an integrative bounded context, the latter may retrieve the detailed data of an event through a synchronous interface. The main request parameter of this interface is the revision ID of that detailed data record. |
The example of the basic fractal pattern shown previously is an excerpt from the larger IT landscape of our imaginary e-commerce company. If we zoom out, we view more of the surroundings.
As you may have recognized, the graphic centers around the goods movement monitoring integrative bounded context. Similarly, order processing is an integrative bounded context on the left. Both of them connect to to the outbound logistics bounded context which may be counted to the Logistics capability in the first place. This is what the gray shade signifies – a communication relationship crossing the border of a classical MECE-oriented business capability. This pattern is mirrored on the right-hand side with the order transmission bounded context. Additionally, the pattern is repeated upwards on coarser levels of granularity. Look at customer order tracking, for instance. It communicates with order processing and goods movement monitoring, e.g. to summarize and track the status of individual customer orders. One step further up, SLA management in turn connects to customer order tracking. It also pulls in information from another bounded context, customer scoring. This makes sense as customers with a higher score may receiver better service-level agreements (SLAs) than customers with a lower score, resulting in early incident escalation. There are two more interesting aspects to note about this overall picture: First, the basic fractal pattern is not repeated in the exact same way, but in a similar way. This undogmatic approach allows for a balanced trade-off between autonomy and alignment concerns. Conceptually, it corresponds to the self-similarity of fractal structures. Second, the meaning of granularity becomes clearer. The actual metric is the sum of downward bounded contexts connected directly and transiently. Customer order acceptance is fine-grained as it maintains communication relationships to none other downward bounded context. By contrast, SLA management is coarse-grained as it maintains 9 such direct and transient relationships.
At this point, we are able to recap the exemplary alignment concerns addressed by this kind of fractal architecture.
First of all, the conceptual framework of Fractal Architecture for IT (FractArc.IT) is a major catalyst for the understanding between software implementers and IT managers. Namely, the core concepts are:
- Autonomy and alignment concerns and why they may conflict with each other
- Business capabilities, bounded contexts, the MECE principle and how they relate to each other
- Levels of granularity of business and software units in an IT landscape
- The questionable dispute of what needs to be done in a business versus how it is done
- The dual nature of software units regarding business/technology and what/how
- Asynchronous communication as a basic means for balancing autonomy and alignment concerns
- How all this fits together in a fractal IT landscape, and how to benefit from it
If software implementers and IT managers succeed in establishing such a common understanding, they may also succeed in mastering the following alignment concerns.
Integrated user experience: In the example of the e-commerce company, its end users should be provided with an overall view of their orders. By contrast, if end users had to gather all their order information from different screens in the user frontend, e.g. order submission, cancellation and shipping status, user experience rates would probably go down. The customer order tracking bounded context caters to that need. It communicates with more fine-grained bounded contexts to summarize and track the status of individual customer orders. Of course, this comes with the cost of lessened autonomy of the integrated bounded contexts – in this case order processing and goods movement monitoring. However, this has some benefits over the late integration in the user frontend. First, the integration is made visible in the IT landscape so that both software implementers and IT managers can be aware of it. This is important for the strategic evolution of the IT landscape. Second, enabling an integrated user frontend is not the only purpose of the customer order tracking. As can bee seen in the landscape graphic, the higher-order SLA management also benefits from it. Third, the customer order tracking bounded context may provide yet another microservice which can be integrated into the frontend alongside other microservices on a more fine-grained level. In essence, the levels of granularity are flattened out in the user frontend. It is highly irrelevant to end users to think about the underlying IT landscape.
Traceability of business processes: The graphic does not reveal directly that the purchase order tracking bounded context includes an audit trail for purchase orders. It integrates events from downward bounded contexts, e.g. tender management and order approval. It should be quite easy to imagine that such an audit trail can be used for proving anti-corruption measures to authorities. However in the first instance, this only includes the flow of business events. The actual data of purchase orders are not included due to the lightweight event format introduced in the section The domain-specific event format. However, the detailed data are referenced by revision IDs and can be looked up in the corresponding bounded contexts, if needed.
Holistic scaling in the cloud: FractArc.IT does not provide a comprehensive solution for the scaling of larger conglomerates in the cloud. However, it makes it easier. Look at the goods movement monitoring bounded context, for instance. The communication relationships to the surrounding bounded contexts are made visible by the structure of the preceding graphic. If needed, more detail is available when zooming into the bounded contexts as shown in section From integration patterns to the fractal pattern. This knowledge of dependencies is a prerequisite for scaling a particular use case or business process, e.g. the escalation of failing orders for VIP customers. Sticking with this example, you may follow the bounded contexts and their communication relationships up to the SLA management bounded context. It is responsible for the escalation of failing orders. With this bottom-up approach, it is more likely that the relevant software units are scaled up or down in unison. By contrast, if you scaled the software units from top to bottom you would likely end up scaling almost the entire IT landscape.
There may be more alignment concerns facilitated by FractArc.IT. As to the cost savings mentioned at the beginning, FractArc.IT may help to open up the larger context in which sourcing decisions are made. However ultimately, sourcing decisions remain a trade-off. Where to buy software, where to develop software? Where to trade cost savings through synergies for a worse time to market?
Is our future fractal, Mr. Mandelbrot?
The term fractal comes from Latin »fractus« meaning »broken«. While »broken« is connoted negatively in everyday language, French-American mathematician Benoit Mandelbrot used it in a neutral sense when he described certain patterns occurring in nature, such as the British coastline. The English version of his 1975 book »Les objects fractals« reads: »Many important spatial patterns of Nature are either irregular or fragmented to such an extreme degree that (…) classical geometry (…) is hardly of any help in describing their form.« Man77 If Mandelbrot was still living and a computer scientist, he would probably state:
Many important parts of IT landscapes are either irregular or fragmented to such an extreme degree that classical IT architecture is hardly of any help in aligning them into the greater whole of the enterprise.
Truly, some managers would deem their IT landscape »broken« into a myriad of fine-grained microservices. It behaves like the coastline originally examined by Mr. Mandelbrot: The closer you look at an IT landscape the more it seems broken into small units exposing more detail. Why not accept this state as a new reality and consciously work with fractal structures in our IT? In the first instance, this involves repeating the same basic integration pattern recursively on multiple levels of granularity in an IT landscape. Fractal Architecture for IT (FractArc.IT), as outlined in the present work, suggests such a pattern to strike a balance between two extremes – these being overly decentralized autonomous software units versus an overly centralized top-down managed IT landscape. Hence, it is not its intention to do away with microservices or Functions as a Service (FaaS). How could it possibly be, considering that increased agility and a shorter time to market count to the major achievements of the past decade? In sum, FractArc.IT aims at re-establishing the alignment of comparably fine-grained software units into the greater whole of an enterprise and its business. As an early prototype and the practical experience of the author suggest, the alignment concerns that may be addressed thusly include an integrated user experience, overall legal compliance and holistic scaling in the cloud, among others.
Unfortunately, FractArc.IT comes with a cost: With it, we must accept ambiguity, as denoted by the »A« in the frequently mentioned VUCA acronym. For instance, the unambiguous MECE principle is useful for decomposing a business but will impede the aforementioned balance when applied in too a stringent way. Likewise, the demarcation line between what needs to be done in a business versus how becomes blurrier. And even »software« isn’t purely technical any more. It is soft, yes, meaning »not physical«. But it is »business« to the same extent as it is technical. Fractal new world. And yet, we have just begun to explore this new world. Some other aspects worthwhile examining are:
- Serverless functions or Functions as a Service (FaaS) can be considered the most fine-grained units in today’s IT landscapes. Is it possible and does it make sense to mold them into fractal patterns which are recursed up to the level of ecosystems spanning several legal entities?
- In what way does a fractal view of an IT landscape support the human side of communication and organization? Although degenerated to a platitude over the past years, Conway’s Law comes into play at this point. This is the reason why the relationships between bounded contexts are termed »communication relationships« in the present work. Communication can be both, human and technical.
- How far does it make sense to create new middleware paradigms and/or frameworks supporting a fractal way of integrating software units? Delving into architectural styles such as event sourcing and CQRS could be particularly worthwhile as they may help to keep the suggested event format free of heavyweight data payloads.
All in all, it should be safe to say that the mentioned structures and challenges in IT landscapes will keep us busy in the near future. Whether you deal with them in a »fractal« way, following the concepts of FractArc.IT or not, is secondary. However, it is quite certain that we cannot simply go back to the comparably one-dimensional view of IT landscapes of the past. And in this sense, our future is fractal.
References
[BAG20] Business Architecture Guild®: A Guide to the Business Architecture Body of Knowledge® (BIZBOK® Guide), 2020, see: view.businessarchitectureguild.org (cited from the version of 07/13/2020)
[Eva03] E. Evans, Domain-Driven Design: Tackling Complexity in the Heart of Software, Addison-Wesley, 2003
[Fowl] M. Fowler, J. Lewis, Microservices – a definition of this new architectural term, see: martinfowler.com/articles/microservices.html
[ente] G. Hohpe, Enterprise Integration Patterns, see www.enterpriseintegrationpatterns.com/patterns/messaging/PublishSubscribeChannel.html
[Kra04] D. Krafzig, K. Banke, D. Slama, Enterprise SOA: Service-Oriented Architecture Best Practices, Prentice Hall, 2004
[Man77] B. Mandelbrot, Fractals: Form, Chance, and Dimension, W. H. Freeman, 1977
[TOG18] The Open Group, The TOGAF® Standard, Version 9.2, 2018
[Ver16] V. Vernon, Domain-Driven Design Distilled, Addison-Wesley, 2016
[Wil01] K. Wilber, Sex, Ecology, Spirituality: The Spirit of Evolution, Shambhala Publications, 2001
[wiki] Wikipedia, MECE principle, see en.wikipedia.org/wiki/MECE_principle
Last update: 12/2020 – © 2020 Matthias Ostermaier