Looking at the system systematically, abstracting the business in a high-dimensional way, and modeling effectively are the capability models that restrict programmers from coders to architects, and these capability models are often difficult to be concretely expressed.
The author of this article proposes a very meaningful perspective to understand from the three aspects of system, architecture and complexity, and uses a very concrete metaphor to allow people to intuitively see the stratification and position between various dimensions. It is worth reading and liking every developer who wants to improve themselves.
01
Understanding of the system
1.1 Overview of the System
With the development of human society, people are faced with more and more complex problems with huge scale, complex relationships and many parameters, and the complexity of these problems has far exceeded the ability of human understanding. The computer systems we usually come into contact with include software systems, which essentially belong to the category of systems theory. Systems theory is an independent discipline that has been developed over the years and has formed a systematic theory. Some of the principles, theories, and methods in systems theory are also applicable to computer systems, and the complexity problems encountered in computer systems will definitely have principled guidance in systems theory. I believe that our predecessors must be smarter than us, and the problems we encounter must have been encountered by our predecessors, but in another form. Years of experience in the Internet industry can easily make us stick to the Internet software system, so it is easy to ignore more general rules and principles.
Therefore, we should look at the Internet software system in the software system, and put the software system in the system theory.
Coming out of the computer system, into the level of systems theory, and back to the computer system. This kind of switching of thinking is like switching back and forth between God's perspective and the human perspective.
With regard to the definition of a system: a system is an organic whole with specific functions that are composed of several components that interact and depend on each other. Systems theory emphasizes the interdependence, mutual influence and constraint between the whole and the part, the part and the part, and the system itself and the external environment.
System theory requires that things or phenomena be studied as systems, and mathematical models are used to describe and determine the structure and behavior of the system. Systematic thinking is different from systematic thinking, which requires us to look at things as a system.
The whole of the system is not the sum of the parts of the system, the whole of the system must be greater than the sum of the parts of the system. A system can support more than the sum of its components. For example, a car consists of wheels, an engine, and a frame. Cars can drive, but wheels, engines, and frames can't.
1.2 Three basic characteristics of the system
Purposefulness: Any system has a purpose.
This can be understood as the boundary of the business system. What is our system for? What can be done? What can't you do?
Dynamics: Dynamics indicate that the system will evolve.
In terms of our business system, it will evolve and move forward dynamically. The internal system changes over time, and the interaction between the system and the outside changes over time.
Ordered: Any system is inherently orderly, and if it isn't, it's not deep enough.
If our business system is still very chaotic and complex, it means that we have not yet found the deep structure of the system, and the complexity is due to our lack of mastery.
1.3 The Four Levels of Systems Thinking
Recognize the system: recognize and understand the form and function, structure and relationship of the system.
Predictive system: When an event occurs in the system, it is able to make predictions about the behavior of the system based on existing knowledge.
Decision-making system: Sufficient understanding of the system, sufficient basis, can intervene in the system, identify, analyze, weigh the system, and finally make decisions.
Assembling a system: The highest level of systems thinking, which can reorganize a system according to its components.
Methodology of Systems Thinking:
Identify entities in a system through abstract thinking and express them with conceptual models.
Through holistic thinking, the key issues are analyzed and summarized, and the appropriate entities are selected as the assembly parts of the system.
1.4 Classification of the system
There are many classifications of systems, and different systems can be divided according to different dimensions, such as artificial systems, natural systems, social systems, etc.
Here is a brief introduction to two systems, the deterministic system and the evolutionary system.
Deterministic system: There is a clear causal relationship, and a clear judgment can be made by grasping the internal laws of the system. Mechanical systems, software systems, etc., all fall into this category.
For example, airplanes, automobiles, etc., as long as they grasp the corresponding physical principles, can make airplanes fly and cars run on the road.
Evolutionary systems: more correlation, unclear or difficult to sort out cause and effect, and its difficulty is unfathomable. For example, the human immune system, although the immune system is known in medicine, it is still difficult to understand its mechanism. The human brain, the ant colony system in nature, the ecological balance system in nature, etc.
For different systems, there are different methodologies and different processing strategies.
Systems theory is a science and a philosophy, and if you are interested, you can study it in depth, and this is just a drop in the bucket.
02
Understanding of architecture
"Put the table in the room and look at it, put the room in the courtyard and look at it in the town plan."
2.1 What is Architecture
Architecture, is a description of the system.
Wikipedia defines software architecture as an abstract description of the overall structure and components of software that guides the design of all aspects of a large software system.
The three characteristics of the system are manifested in the architecture: horizontal juxtaposition, vertical derivation, and overall evolution.
The law of entropy increase in physics shows that isolated systems always tend to develop in the direction of entropy increase. The same applies to software systems, but in the form of increased complexity.
Internet software systems are always moving in the direction of increasing complexity. Therefore, the first purpose of the architecture is to control the complexity and make the system develop in a controllable direction.
2.2 What is a good architecture diagram
Concise and abstract: A good architecture diagram must be concise, concise and powerful, and able to distinguish between latitude and longitude at a glance. If there is a certain degree of abstraction, if there are various flying loops in an architecture diagram, it must be not abstract enough. Abstraction makes sense, and if there are various details in the architecture, it is a pile-up.
Explainable: A good architecture diagram must be able to explain the current state and behavior of the current system.
Guiding Actions: A good architecture diagram must be able to guide behavior, and guiding actions is the greatest value of an architecture diagram. Ability to predict the future and guide action. For a domain architecture diagram, if you don't know where to put a module according to the architecture diagram, it is a failure. A good architecture diagram should be for an inexperienced person who can divide modules according to the architecture.
Evolvable: Corresponding to the dynamic nature of the system, the architecture will also evolve over time. Structures that cannot evolve are like vases, beautiful to look at, but shatter when touched.
2.3 Architecture Diagram
There are already different frameworks for the presentation of architectures. There are 4+1 views, the C4 model, the enterprise architecture model proposed by TOGAF, etc. Regardless of the model, the core idea is to describe the software architecture from different dimensions. The following focuses on these aspects.
2.3.1 4+1 mode
The description of the logical architecture of software engineering, proposed by Philippe Kruchten, has become the de facto standard for software architecture, describing software from different perspectives such as end users, developers, system engineers, and software managers. As shown in the figure below:
Logic View: From the end-user's perspective, describing the hierarchical relationship between different functional components from a functional perspective.
Development View: From the developer's perspective, describe the composition of packages, classes, and libraries of different code from the implementation level.
Process View: The behavioral relationship between different components, usually represented in the form of a sequence diagram, with a certain degree of temporal malleability.
物理视图（Physical View）：部署视图，系统所依托的物理视图。
Scenarios: The relationship between the different objects involved in the system. It is usually presented in the form of a use case diagram.
Based on these five perspectives, five architectural models can be derived. Scenarios, functions, implementations, processes, deployments, layers of abstraction.
The 4+1 architecture view builds a framework for observing and understanding the system. It tells us that the system can be described, observed, and understood from the logical view, development view, process view, physical view, and scene view. For a system, these 5 perspectives are already very complete. It is important to note that the greater value of 4+1 is that it provides a framework for an analytical system, and in fact how different teams are presented may take different forms. A system will be understood differently from different perspectives, and it will be seen horizontally as a peak on the side of the ridge.
2.3.2 C4 model
The C4 model was created by Simon Brown between 2006 and 2011 and is based on the 4+1 model (https://c4model.com/), which is actually an abbreviation of the following 4 words:
Context: describes the relationship between the system and its surrounding systems and people. The focus is on the system's relationship with the outside world. The outside world here includes people and other systems.
Containers: A container is a unit of function, which is described from a technical level, which can be a service, a technical component or a functional module. For example, a fund system will include trading services, order services, commodity services, etc.
Components: Components are components of containers, and components are amplifications of containers, such as SKU management, market data, and derivative indicators in commodity services.
Code: This level is the code level, including the inheritance, composition, and inclusion of interfaces, classes, and objects.
The model describes the progression of a system from macroscopic to microscopic, as if looking at the Earth from space with a magnifying glass.
The first layer is to see the earth and its planet, the second layer is to see the mountains, rivers, seas and rivers on the earth, the third layer is to see different countries and cities, and the fourth layer is to see different houses and families.
The C4 model is based on the 4+1 model, but there are some differences. If we say that 4+1 is focused on the horizontal view of the ridge and the side of the peak. The C4 model is a magnifying glass to get a glimpse into the end.
The C4 model tells us that different levels of abstraction have different concerns, challenges, and problem domains, and that we need to think about the corresponding things at different levels.
Once the focus does not match that level, there will be logical confusion. Being able to distinguish the level of the problem domain has actually solved most of the problem.
Sometimes, problems that are difficult to solve at a lower level can be solved by moving up to a level.
Sometimes, a problem that is not visible at a high level can be seen at a glance when it is lowered by one level.
The higher level is the abstraction of the lower level, and the lower level is the embodiment of the higher level.
A master should be able to recognize different levels of abstraction and be able to move between them with ease.
When you are at a high level, you should not be tied up by the specifics of a low level, and when you are at a low level, you should not be too ambitious.
2.3.3 TOGAF-4A 架构
Business Architecture: Business Strategy, Governance, Organization, and Key Business Processes. From the perspective of enterprises, the emphasis is on value, information, collaboration, and multi-departmentality.
Application architecture: The blueprint of the individual applications to be deployed, their interactions, and their relationship to the organization's core business processes.
Data Architecture: The structure of an organization's logical and physical data assets and data management resources.
Technology Architecture: The logical software and hardware capabilities required to support the deployment of business, data, and application services. This includes IT infrastructure, middleware, networking, communications, processing, and standards.
TOGAF believes that the purpose of architecture is to help organizations achieve the following capabilities:
Heterogeneous to Isomorphic (Shaping Isomorphic IT), Ex-Post to Prior (Shaping Planning IT), Discrete to Unified (Shaping Unified IT), Disordered to Ordered (Shaping Ordered IT)
2.3.4 Actual Model - Internet Model
In fact, compared with traditional software systems, the Internet industry is developing relatively fast and the business life cycle is relatively short, which forms a specific architecture description method for the Internet industry. It is more presented in three forms: business architecture, technical architecture, and deployment architecture.
Business architecture: Describe the set of functions, domain boundaries, and logical relationships of each component of the system from a business perspective. Different from technical architecture, technical terms such as DB, MySQL, CMQ, synchronous, asynchronous, and concurrent should be avoided in the business architecture diagram.
Technical architecture: Describe the interaction between the components of the system from a technical point of view, and the technical architecture should reflect technical characteristics, such as synchronization, asynchronous, message, etc.
Deployment Architecture: Describes the deployment distribution of the system from a physical perspective.
2.4 Understanding of microservices
Software architecture boils down to two models: designing from the technical level and the business function level. Before understanding these two, let's distinguish between technical language and business language:
Technical language: is at the implementation level. For example, DB, MySQL, query, timeout, read/write splitting, fast and slow splitting, logical layer, caching, order creation, synchronization, asynchronous, multi-threading, and multi-process.
Business language: It is functional. Such as buying, withdrawing, fund information, quotes, fund details, assets, product lists, position lists, subscription lists, and redemption lists.
What technicians need to do is to get rid of the technical language system and enter the business system, and they cannot be limited by the technical language. Essentially, technology is for business services, so understand business first, technology second. Once you have a deep understanding of the business, you can turn to technology to implement the business. The best practice is to see no technical vocabulary in the business code, only the business.
Microservices were first proposed by Martinfowler and defined as follows:
“In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.” --- https://www.martinfowler.com/architecture/
To talk about my understanding, microservices are not small services, if it is only because of the small scale, then it can be directly called small services, more accurate is: small and complete services, where small refers to volume, complete refers to the ability to provide complete business capabilities. Microservices are the idea of using a service to express all the behaviors related to an entity, and all the connections between an entity and the outside world take place through that service.
Different from the previous services divided according to technical functions, AO is the logic layer, DAO is the storage layer, and VO is the display layer. An entity's behavior can only be expressed through the association of three services: VO, AO, and DAO. Microservices, on the other hand, are purely from the business semantic level, requiring only one service and only one external representation. Similar to a country, although small, it has its own laws, arms, taxes, etc. Microservices have their own logic, storage, domains, and so on. The core idea of microservices: service is the entity, and I am the whole. Service is the carrier of the concept of entity. The starting point is a complete decoupling from the business domain or functional level. Microservices are completely heterogeneous with each other, and there is not even a technical commonality between microservices.
For example, the microservice representing the policy, all the behaviors related to the policy are provided by the service, and the service realizes the storage and query of the policy internally, and the outside world does not need to care, and the creation of the policy, the query of the policy, and the claim policy are all realized through the policy microservice.
However, in practice, limited by the hardware level and current technical capabilities, it is difficult for a single microservice to undertake all the behaviors related to the entity. For example, the query of the policy has high performance requirements, and the writing of the policy has high consistency requirements, in this case, if it is placed in a service, it will bring difficulties in implementation. At this time, it may return to the traditional technical function division service, consider read-write splitting, and separate a policy microservice for query and read. Sometimes it is a reluctant compromise, but the general principle is to stick to the principle first and then compromise. Design according to the indivisibility of the microservice domain first, and then make adjustments and compromises when encountering technical challenges.
03
About complex understanding
"The essence of computer programming is to control complexity," --- Brian Kernighan
3.1 What is Complex?
What is the complexity of a system? Physicist Lloyd makes a point:
Describe how difficult it is;
how difficult it is to produce it;
How well is it organized?
My personal understanding is that a system can be considered complex if the following happens:
The inability to express its concepts is beyond the expressive capabilities of human language
The process cannot be described, which is beyond human comprehension.
The results can't be measured, they can't be managed without them, and they're so complex that it's hard to measure them in an accurate way.
3.2 Complex classification
- Surface complexity: The complexity of a system that is abstracted, simplified, layered, and constructed is the most intuitively understood complexity by humans. For example, the complexity of a system architecture diagram is the surface complexity.
- Requisite complexity: The necessary complexity required to support a function, that is, the theoretical complexity.
- Actual complexity: In fact, it includes unnecessary complexity caused by technical limitations, resource constraints, and process waste.
Superficial complexity is the complexity that humans can understand, and the necessary complexity and actual complexity are often beyond human comprehension.
There is a view that the purpose of system architecture is to control the surface complexity of the system within the scope of human understanding, and reduce the actual complexity to close to the necessary complexity.
3.3 CyneFin 框架
There is a lot of connotation about complexity. There is a big guy named Dave Snowden, who created a set of decision-making frameworks in 1999, the Cynefin Framework, which is a good classification of what we usually call complex. See diagram below.
Visually, this diagram is divided into five quadrants, which are viewed counterclockwise from the lower right corner.
Simle（简单）、Complicated（繁杂）、Complex（复杂）、Chaotic（混乱），中间有一个黑色区域是Disorder（无序）。
These five sets of concepts can be understood as a gradual increase in complexity. Simply put:
- Simple: If you know the cause, you will know the effect.
- Complicated: Different effects may be derived from causes, and a certain amount of domain knowledge is required to analyze cause and effect.
- Complex: I don't know the cause and effect. Intuitively, cause and effect cannot be found, and it must be reviewed afterwards.
- Chaotic: Knowing that cause and effect are no longer important, it is important to build order and stabilize the system.
- Disorder: Too complex to be defined by the above four concepts.
Any business, environment, situation, project, or system that we encounter can be described by one of these five concepts.
The development of a system will probably follow the development of a state from simple->complex-> complex-> chaotic->disorder. This process is very consistent with the concept of entropy increase in physics.
Simle (simple): Known knowledge, there is a direct cause and effect relationship, and it is clear at a glance. Needless to say, this is true of any system that we are already familiar with.
The best way to deal with this state is to adopt the perception-classification-response model. The causal relationship is very clear, we just need to categorize it.
Complicated: There are a lot of known unknowns in this system, and it is difficult for non-professionals to see the causal relationship, and it requires a lot of professional knowledge to analyze cause and effect.
For example, in the setting of government departments, there are two sets of systems, vertical lines and horizontal lines, commonly known as "strips and blocks". Both vertical and horizontal. The longitudinal line of the education bureau of a certain county should be reported to the higher education department, and the horizontal line should be reported to the local county government. Many similar departments form an intricate relationship, which is presented as a network of relationships. In the face of such a system, it is difficult for people outside the system to figure out to whom a certain department should report. At this time, if there is an old civil servant driver, he can quickly tell you the mystery.
For example, for a person who doesn't have any medical background, you give him a bunch of indicators and ask him why he has a stomachache. He must have felt that it was too complicated to start. The best way for him is to consult medical students or systematically learn medical expertise.
Therefore, in the face of complicated situations, the best way is to learn domain knowledge and get started quickly. In practice, the "Sense-Analyze-Respond" model is adopted. Note that the focus is on analysis. The prerequisite for analysis is domain knowledge. Therefore, in-depth learning + analysis is the way to deal with it. Most of the time we encounter this kind of system, what we need is to increase professional skills, supplement domain knowledge, and improve the ability to identify. This is also the best solution.
Complex: Represents the unknown of the unknown. There are uncertain circumstances, which may or may not exist, and a review is needed to find the cause. It needs to be explored step by step, and the cause can only be analyzed through feedback.
For example, you are suddenly promoted to a manager, and your task is to build team culture. Faced with this task, neither cause nor effect is known. Because team culture is a very empty thing, you don't even know how to measure it. Not to mention what approach to take to predict. For this, you can only try it out and then attribute it after the fact.
So the adoption pattern is: "Explore-Sense-Respond". The key is to explore, to explore, to keep giving feedback, and ultimately to find cause and effect.
Chaotic: A state of chaos usually refers to a moment of crisis. It means that the information you have access to is unstable. In this case, the causal relationship is not clear, and it is in a state of disorder. There is no point in trying to identify cause and effect.
In the midst of all kinds of instability, take action to stabilize the disorder. Use action to build order.
For this situation, everything is unknown, just take action, build order through action. Turn a chaotic scene into a complex one. The coping pattern in this state is: "Action-Sense-Response".
Disorder: Too complex to classify the complexity of the system. Completely disordered, blind state. This kind of coping strategy is to observe first, wait for changes, and wait until you figure out that you are in a certain state of complexity, and then think of countermeasures. If you can't figure out the complexity, don't move. Always wanted to. Until you can distinguish between the four states in which the system is in one of the above four states.
Summary:
Different levels of complexity correspond to different processing strategies. We face a variety of situations every day, and using the guidance of this framework helps us to see the essence through the phenomenon. The more capable people are, the more complex they will deal with.
04
postscript
The content of this article is based on previous studies and collections. The more I organized, the more I found that the knowledge inside was huge, and each point could be expanded. So a lot of the places here are just briefly mentioned, and I have time to sort them out further. Recommend three books :
"System Architecture - Product Design and Development of Complex Systems" is an eye-opener for me. Some of the ideas and sentences in this article are also excerpted from this book.
Software Architecture - Architectural Patterns, Characteristics and Practices
The book "Domain-Driven Design" is almost a copy in the author's department.
In addition, there are some materials appended, which can be read in depth if necessary.
Bibliography:
https://www.cs.ubc.ca/ ~gregor/teaching/papers/4+1view-architecture.pdf
https://www.yijiyong.com/architecture/basic/02-essence.html
https://www.infoq.cn/article/c4-architecture-model
https://c4model.com/
https://zhuanlan.zhihu.com/p/442963069
https://www.cnblogs.com/lex-wu/p/13305380.html
Author: Li Yanan
Source-WeChat Official Account: Tencent Cloud Developer
Source: https://mp.weixin.qq.com/s/jaXXrZsKvVhodaLKb35IwA