Distilled from the core ideas of Jin Guantao and Hua Guofan's "Cybernetics and Scientific Methodology". This article maps the basic principles of cybernetics, systems theory, and information theory into actionable guidance for AI Agent design.
0. Core Axioms
The essence of control is making directed choices within a space of possibilities.
All of an Agent's behavior -- planning, reasoning, tool invocation, self-correction -- is the process of narrowing the space of possibilities amid uncertainty. Understanding this is the starting point for designing any AI system.
1. Space of Possibilities and Control Capacity
1.1 A Task Is a Space of Possibilities
Before it begins, any task faces aspace of possibilities: the set of all possible execution paths, outcome states, and tool combinations.
- The vaguer the task, the larger the space of possibilities.
- The clearer the task, the smaller the space of possibilities.
- The Agent's goal is:to narrow the space of possibilities to the target state the user expects。
1.2 Control Capacity = M / m
- M = the size of the space of possibilities before control
- m = the size of the space of possibilities after control
- Control capacity = M / m
Design principles:
- Do not catch birds with a net's mesh. Do not try to narrow the space of possibilities to a single state in one step. Like a hunter using a net rather than a single loop of rope, an Agent should allow the target a reasonable range of precision (tolerance), rather than pursuing absolute precision at every step.
- Estimate control capacity before acting. Before executing, first estimate whether the control capacity of the current tool/method is sufficient to reach the goal. If the tool's control capacity is insufficient, no amount of repetition will help -- you should switch tools or execute in steps.
- Control capacity can be compounded. The total capacity of multi-step control is the product of the capacities of each step. What cannot be done in one step can be done through multi-step feedback.
2. Four Basic Control Methods
2.1 Random Control (Exploration / Trial-and-Error)
Essence: Try randomly within the space of possibilities, and stop when the target is hit.
Applicable scenarios: Initial exploration when you know almost nothing about the problem domain.
Key points for Agent design:
- When an Agent is completely unfamiliar with a task, random control is a legitimate way to start.
- The key: the target must be within the exploration range. If the key is not in this bunch, trying them all is useless. A common mistake is letting bias restrict the exploration range -- as with Laval fixating on "the harder the shaft, the better" while ignoring soft shafts.
- Speed determines efficiency. When the space of possibilities is large, you must increase the speed of selection (parallel search, batch verification).
- Continuously expand the exploration range. Break inherent assumptions and consider neglected possibilities like "sending the bamboo pole through the city gate lengthwise".
2.2 Control with Memory (Elimination Method / Elimination)
Essence: Remember the states already eliminated, and gradually narrow the search range.
Applicable scenarios: Tasks where impossible options can be eliminated step by step.
Key points for Agent design:
- Maintain aneliminated list, to avoid repeatedly trying schemes already known to be ineffective.
- Beware the false-memory trap: If the correct answer is mistakenly excluded and this erroneous judgment is remembered, the target will never be within the search scope.
- Beware of trap states: Some attempts may irreversibly weaken control capability (such as destructive operations). Safe options should be prioritized for exclusion, and high-risk operations should be placed last.
- Control order matters: First depositing the substance with a large content may adsorb the substance with a small content -- the order of operations affects the remaining options for subsequent choices.
2.3 Conjugate control (L^{-1}AL transformation)
Essence: Transform a problem that cannot be directly controlled into a controllable problem A through transformation L, solve it, and then transform back through L^{-1}.
This is the essence of tool use.
Key points for Agent design:
- Cao Chong weighs the elephant principle: Cannot directly weigh the elephant -> L (use a boat to convert the weight into a waterline) -> A (weigh stones) -> L^{-1} (stone weight = elephant weight).
- Every process in which an Agent invokes a tool is conjugate control:
- L = sensor (converts real-world state into signals the Agent can process)
- A = the Agent's selection/computation
- L^{-1} = effector (converts the Agent's decision back into effects on the real world)
- Design implication: When an Agent cannot directly complete a task, do not report failure; instead, look for a transformation L: can the problem be transformed into another known controllable problem?
2.4 Negative feedback control (decreasing target difference)
This is the most core control mechanism of an Agent.
Essence: Set a goal -> observe the current state -> calculate the target difference -> execute an action that reduces the target difference -> repeat.
Two necessary conditions for negative feedback:
- Once a target difference appears in the system, a response that reduces the target difference automatically appears.
- The adjustments that reduce the target difference can accumulate time after time.
Key points for Agent design:
- Hawk catching rabbit model: The hawk does not need to precompute an exact trajectory. It fixes its eyes on the rabbit, adjusts its direction each time to reduce the distance difference, and through multiple dives accumulates control capability, finally catching the rabbit.
- Marie Curie purifying radium model: She did not know what radium looked like, but she knew that "stronger radioactivity = closer to the goal." Each step moved in the direction of increased radioactivity, and after years of accumulation the goal was reached.
- Negative feedback amplifies control capability: When single-step control capability is limited, through multiple feedbacks the limited capabilities are superimposed into a sufficiently large total capability. Total control capability = the product of the control capabilities of each step.
- "Do it and see" is the correct engineering philosophy: Do not try to plan a perfect solution all at once. Observe while doing, and correct at any time.
3. Positive feedback and loss of control
Positive feedback = a process in which the target difference keeps increasing = a vicious cycle.
Manifestations of positive feedback loss of control in an Agent:
- Error fixes introduce more errors, and fixing these new errors introduces even more errors -- code quality spirals downward.
- The Agent is overconfident in its own output and keeps doubling down in the wrong direction.
- Problem decomposition is too fine-grained, causing subtasks to explode, and each subtask produces even more subtasks.
Countermeasures:
- Positive feedback and negative feedback can transform into each other. Add to the feedback loop afilter(Quality check threshold) can turn positive feedback into negative feedback.
- Add to the feedback loop adecay element(limit the number of iterations, set rollback points).
- Key design:When the target gap is detected to be continuously increasing, immediately stop the current strategy and roll back to the last known good state.
4. Information Theory Principles
4.1 Information = narrowing of the possibility space
To "know" something = the possibility space concerning it has been narrowed.
- If the weather forecast says "it may rain tomorrow or it may not," the amount of information is zero.
- Every step of an Agent's reasoning should generate positive information -- substantively narrowing the possibility space.
4.2 Channel Capacity Principle
An information transmission channel has a capacity limit, determined by three factors:
- The control speed and capability at the input end
- The number of distinguishable states of the channel
- The receiving speed and comprehension capability at the output end
Key points for Agent design:
- Channel capacity should be neither too large nor too small. An intersection only needs red, yellow, and green lights; hundreds of colors would instead make drivers dizzy. When an Agent conveys information to a user, the precision should match the user's decision-making needs.
- Precision instruments are not necessarily better. Using overly precise instruments in a rough experiment may actually decrease the amount of information transmitted per unit time. When an Agent chooses tools, matching the task precision is sufficient.
4.3 Filtering (removing the false and preserving the true)
Interference inevitably exists in information transmission. Methods for eliminating interference:
| Method | Principle | Agent application |
|---|---|---|
| Repeated transmission | Transmit multiple times through the same channel and check the results | Perform multiple verifications on key results |
| Multi-channel cross-validation | Use completely different channels to transmit the same information and compare the results | Use different methods/tools to verify the same conclusion |
| Impedance filtering | Use the essential difference between signal and interference to filter out interference | Set acceptance criteria to filter out results that do not conform to format/logic |
| Feedback filtering | Use useful signals already received to conversely suppress useless signals | Use already confirmed partial results to guide the direction of subsequent searches |
| Synchronous filtering | Open the channel only when a signal appears | Collect information only at key decision points to avoid information overload |
Core principle:
- The amount of information only decreases during transmission, never increases. The more hands it passes through, the greater the distortion. An Agent should use first-hand information sources as much as possible.
- The warning of Zeng Shen being killed: When the first link of multiple channels is the same, they are essentially the same channel. Three people all getting information from the same erroneous source is no more reliable than one person.
4.4 The subjectivity of information
The same signal has different meanings for different receivers.
- Playing the lute to a cow is not the cow's fault. The information output by an Agent must match the receiver's model of understanding.
- Kongming's fireworks strategy: The same signal may convey opposite information in different contexts. An Agent must consider the mental model of the information receiver.
5. System Design Principles
5.1 Relatively Isolated Systems (Defining System Boundaries)
It is impossible to consider all factors. Principles for defining system boundaries:
- Ignore factors whose probability of influence is sufficiently small (treat them as interference outside the system).
- The system should be aself-closing network of mutual causation(with strong correlations among core variables).
- Select main variables according to the research purpose and time scale.
Application to Agent task planning:
- Do not take all relevant factors into consideration. Identify the core causal chain and ignore weakly related factors.
- System boundaries are relative: the system boundaries for short-term tasks and long-term tasks are different.
- The Kaibab deer lesson: One cannot hold other factors fixed and adjust only one variable. In a system of mutual causation, changing one variable will chain-react and affect all variables.
5.2 Steady-State Structure (the destination of a system)
A system of mutual causation will automatically tend toward a steady-state structure.
- Steady-state structure = a state in which the interactions among all parts of the system keep the whole unchanged.
- After interference causes the system to deviate from the steady state, the system will automatically return.
Application to Agent design:
- Use steady states for prediction: There is no need to track every step of system change; one only needs to find the possible steady-state structures of the system to predict the final result. "Where will the bee be in two hours? Just look at where there are more flowers."
- Find all steady states and then compare them: Yu the Great chose "channeling" rather than "blocking" in controlling the waters, because "returning to the sea" is the most stable state for floodwater.
- The contradiction between sensitivity and stability: A system that is too stable will be insensitive (the instrument needle does not move). An Agent needs to strike a balance between stability (robustness) and sensitivity (adapting to change).
5.3 Ultrastable Systems (maintaining stability through instability)
Long-term stable systems all have self-repair mechanisms.
Ultrastable = stable -> unstable (suffering interference/change) -> repair -> restore stability
Application to Agent design:
- An Agent system cannot have only a steady state; it must also have arepair mechanism: when a deviation in state is detected, it can automatically trigger the repair process.
- The direction of repair must be to return to theoriginal good state, rather than to any arbitrary new state.
- The human body's organ manifestation model is an ultrastable system: the five viscera regulate one another through feedback, small deviations are automatically corrected, and large deviations trigger repair mechanisms.
5.4 Self-Reproduction and Chain Reactions
Self-reproduction = the larger the variable value, the faster the growth = exponential growth = a precursor to system collapse.
Risks of self-reproduction in Agents:
- Exponential growth of token consumption (each step generates more subtasks)
- Error repair loops (each repair introduces more problems)
- Context bloat (each step accumulates more redundant information)
Countermeasures:
- Set critical values: Self-reproduction occurs only after the critical value is exceeded. Keep key variables below the critical value.
- Maintain negative feedback suppression mechanisms: Self-replication often occurs because the negative feedback inhibition mechanism is disrupted (just as cancer is the failure of the inhibition mechanism of cell proliferation).
- Principle of neutron moderator: In a chain reaction, a deceleration/absorption step is added to keep the multiplication factor near 1.
5.5 Self-organizing systems
Self-organization = a system automatically forms an ordered structure from disorder.
The five characteristics of self-organization:
- Requires an organizing core -- The crystal nucleus determines the shape of the crystal; whether the first piece of wood is straight or curved determines the entire stable.
- The system must be unstable or metastable -- A system that is too stable cannot reorganize.
- There is an automatic selection chain -- One selection leads to the next selection, and the space of possibilities automatically shrinks.
- The process is irreversible -- Once an organization is formed, it is difficult to change it into another form.
- A slight difference leads to a huge error -- A tiny difference in the core leads to a huge difference in the final organization.
Agent design applications:
- Control the organizing core: The initial architecture of the system design determines the evolution direction of the entire system. The Agent's initial prompt, seed strategy, and basic toolchain are the "organizing core."
- Use self-organization to reduce control overhead: There is no need to control every detail; just choose the core well and let the system organize itself (intelligence amplification).
- The learning process is self-organization: Once an erroneous foundational concept is established, it is difficult to correct, more difficult than learning from scratch.
6. Determining qualitative change and gradual change
6.1 Principles of determination
It is not determined by speed, but by stability.
If the intermediate transitional state experienced during a qualitative change is unstable, then it is a leap. If the intermediate transitional state is stable, then it is a gradual change.
- Stealing bricks from the bottom of a tower -> the tower's stability is shaken -> at some moment it collapses (a leap).
- Stealing bricks from the top of a tower -> the tower remains stable throughout -> it gradually becomes shorter until it disappears (a gradual change).
- Dominoes falling is a leap (each falling piece affects the stability of the next), no matter how slow the speed.
- Taking grain from a grain pile is a gradual change (the remaining grain pile remains stable throughout), no matter how fast the speed.
6.2 Agent applications
- When assessing the impact of a change: Determine whether this change will shake the stable structure of the system. If it will, be prepared to deal with a "collapse"-style chain reaction. If it will not, it can be safely executed gradually.
- Refactoring vs patching: Changing the architecture from the bottom layer may trigger a leap-style qualitative change (a period when the system as a whole is unstable). Changing the API from the surface layer is usually a gradual change.
6.3 Hysteresis (overcorrection)
Hysteresis occurs only in leap processes.
- After a forward change in conditions causes a leap in state, when the conditions are reversed for recovery, the state will not immediately return to its original condition; the conditions must be reversed beyond the original degree.
- Agent application: After an Agent's strategy leaps from A to B, if it wants to return to A, simply withdrawing the last change is not enough -- a larger reverse adjustment may be needed.
7. Black-box epistemology and experimental system design
7.1 Black-box method
All objects of cognition are black boxes. A black box has two sets of variables:
- Observable variables(output): what the Agent can perceive.
- Controllable variables(input): what the Agent can manipulate.
Two methods of cognition:
| Method | Meaning | Applicable scenarios |
|---|---|---|
| Do not open the black box | Infer the internal structure through input-output relationships and build a model | The system is too complex, cannot be opened, or opening it would destroy the structure |
| Open the black box | Directly observe the internal structure and add new observable/controllable variables | Need to verify the model and deepen understanding |
Agent experiment design:
- Mendel's method (do not open the black box): Control the input (select parental traits), observe the output (offspring traits), and infer the internal mechanism (the gene hypothesis).
- Opening the black box is relative: Opening one layer of the black box reveals a new black box. Chromosomes -> DNA -> base sequences -> ... it never ends.
- Pavlov's method: One cannot open the dog's brain to see how it "sees," but through conditioned reflex experiments (controlling input/observing output) one can infer that dogs are colorblind.
7.2 Epistemological model (practice-theory-practice)
Cognition = a negative feedback regulation process:
+---------+
控制输入 ----->| 客体 A |-----> 实际结果
| (黑箱) | |
+---------+ |
v
+------------+
| 比较(鉴别) |<--- 预期结果
+------------+ ^
| |
v |
+------------+ +----------+
| 修改模型 |-->| 模型 B |
+------------+ | (理论) |
+----------+
7.3 Five obstacles to cognition approaching truth
This is the core checklist for Agent experiment system design.
Obstacle one: insufficient observable/controllable variables
- No matter how precisely van Helmont weighed the willow tree, water, and soil, he could not discover photosynthesis, because CO2 in the air was not among his observable variables.
- Agent check: Do the current observation methods and toolset cover the key variables of the problem? Are there blind spots?
Obstacle two: lack of clarity in theory (unfalsifiable)
- A forecast that "it may rain tomorrow or it may not" carries zero information and cannot be tested.
- Popper's falsifiability principle: A theory must give clear predictions that can be negated by practice.
- Agent check: Every hypothesis/plan should have clear expected results and failure criteria. "If X is observed after execution, the hypothesis holds; if Y is observed, the hypothesis fails."
Obstacle three: feedback speed cannot keep up with the speed of change
- The state of Chu measured the river depth before crossing, but a flash flood broke out.
- If the Agent modifies its strategy more slowly than the environment changes, it will always lag behind the actual situation and produce oscillations.
- Agent check: Does the decision cycle time match the task's time constraints?
Obstacle four: excessive feedback (overcorrection -> oscillation)
- The missile overshoots every time it adjusts direction, swinging back and forth around the target.
- History of optics: particle theory -> wave theory -> particle theory -> ... each correction completely negates the previous theory.
- Agent check: When revising the strategy, is an appropriate magnitude maintained? Is there a pattern of jumping from one extreme to the other?
Obstacle five: the decidable condition does not hold
- Schrödinger's first equation did not match experiment, but the reason was not that the equation was wrong, but that electron spin had not yet been discovered.
- When experimental results cannot correctly reflect the degree of correctness of a theory, the Agent's direction of correction may be wrong.
- Agent check: Is the evaluation standard itself reliable? Is there an intermediate standard/paradigm that can assist judgment?
8. Feedback Loop Design Templates
8.1 Basic Feedback Loop
loop:
observe(当前状态)
gap = compare(当前状态, 目标状态)
if gap <= tolerance:
return success
if gap 连续增大:
return fallback(上一个良好状态) // 正反馈检测
if iterations > max_iterations:
return escalate(向上级/用户求助) // 防止无限循环
action = select_action(减小 gap 的方向)
execute(action)
iterations += 1
8.2 Multi-Layer Feedback (Ultra-Stable System)
# 第一层：任务执行反馈
while not task_complete:
execute_step()
if deviation_detected():
adjust_step() // 负反馈修正
# 第二层：策略修复反馈
if strategy_broken(): // 第一层反馈失效
diagnose_failure()
repair_strategy() // 超稳定修复机制
restart_first_layer()
# 第三层：目标修复反馈
if goal_unreachable(): // 第二层修复也失效
re_examine_goal()
consult_user() // 人类介入
8.3 Experimental System Design Template
experiment:
1. 定义黑箱边界
- 可观察变量: [列出所有可观测的输出]
- 可控制变量: [列出所有可操控的输入]
- 不可控因素: [列出已知但无法控制的干扰]
2. 建立模型（假设）
- 假设: [明确、可证伪的陈述]
- 预期结果: [在条件 C 下，应当观察到 R]
- 失败判定: [如果观察到 R'，则假设失败]
3. 执行实验
- 设定输入
- 观察输出
- 记录干扰
4. 比较与修正
- 实际结果 vs 预期结果
- 目标差计算
- 修改模型（注意避免反馈过度）
5. 滤波验证
- 用不同通道/方法重复验证关键结论
- 确保多条通道的环节真正独立（避免曾参杀人陷阱）
6. 迭代或终止
- 目标差在容忍范围内 -> 接受结论
- 目标差仍大 -> 回到步骤 2
- 检测到可判定条件不成立 -> 扩展可观察/可控制变量
9. Intelligence Amplification and Task Decomposition
9.1 Intelligence Amplification = Amplification of Selection Capability
Liu Bang could not fight battles, but he was good at selecting people. He selected Zhang Liang, Xiao He, and Han Xin, and through organizing a core, amplified his own selection capability and established the Han Dynasty.
Agent's intelligence amplification strategy:
- It does not need to solve all problems itself; it only needs toselect the right tools and the right sub-Agents。
- Through self-organizing mechanisms, small-scale core selections can automatically amplify into large-scale organizational effects.
- Ore flotation model: A person selects the beneficiation agent (small-scale selection), and the beneficiation agent automatically completes large-scale beneficiation (intelligence amplification).
9.2 Cybernetic Principles of Task Decomposition
- First estimate the total control capability requirement: M_total / m_target = required total control capability
- Decompose into manageable subtasks: The control capability of each subtask is Q_i, and the total capability = Q_1 * Q_2 * ... * Q_n
- Keep the control capability of each step as even as possible: Just as when weighing balls on a balance, groups a, b, and c should be divided as equally as possible (maximizing information utilization efficiency)
- Maintain feedback between each step: The output of the previous step becomes the possibility space of the input to the next step
10. Design Checklist
When designing or reviewing an AI Agent system, check item by item:
- Is the possibility space clear? Are the initial possibility space and target state of the task clearly defined?
- Does the control capability match? Is the control capability of the available tools/methods sufficient to achieve the goal?
- Is the feedback loop closed? Is there a complete loop of observe -> compare -> correct?
- Is there positive feedback detection? Is there a circuit-breaker/rollback mechanism when the goal gap continuously increases?
- Are the information channels sufficient? Are there enough observation means to cover the key variables?
- Is the filtering mechanism in place? Are key conclusions cross-validated through independent channels?
- Is the theory falsifiable? Does every hypothesis/plan have clear criteria for determining failure?
- Is the feedback speed sufficient? Can the speed of decision correction keep up with changes in the task environment?
- Is excessive feedback prevented? Is the correction magnitude appropriate, without jumping from one extreme to another?
- Are the system boundaries reasonable? Does it include only strongly relevant core variables and exclude weakly relevant interference?
- Does an ultra-stable mechanism exist? When the strategy level fails, is there a repair mechanism that can restore it to a known good state?
- Is the self-replication risk controllable? Is there a mechanism to prevent exponential growth of Tokens/tasks/errors?
Appendix: Glossary Quick Reference
| Cybernetics Term | Corresponding Agent Domain |
|---|---|
| Space of Possibilities | Task solution space, state space |
| Control Capability M/m | Tool precision, model accuracy |
| Negative Feedback | Error correction loop, self-correction |
| Positive Feedback | Error amplification, runaway loop |
| Conjugate Control L^{-1}AL | Tool invocation, format conversion, intermediate representation |
| Amount of Information | Amount of uncertainty eliminated |
| Channel Capacity | Context window, API rate limit |
| Filtering | Result validation, noise filtering |
| Steady-State Structure | System equilibrium state, convergence state |
| Ultra-Stable System | Self-healing system, fault-tolerant architecture |
| Self-Reproduction | Exponential growth, chain reaction |
| Self-Organization | Emergent behavior, bottom-up order |
| Black Box | Controlled object with unknown internal structure |
| Falsifiability | Testability, clear criteria for success and failure |
| Critical Point | Threshold value, Phase Transition |
| Goal Difference | Distance between current state and desired state |
The reason why this world is the world it is now, and why various things can exist and engage in regular motion, is inseparable from stability. -- Jin Guantao, Hua Guofan, Cybernetics and Scientific Methodology