On Linearity II

"Why are nonlinear systems so much harder to analyze than linear ones? The essential difference is that linear systems can be broken down into parts. Then each part can be solved separately and finally recombined to get the answer. This idea allows a fantastic simplification of complex problems, and underlies such methods as normal modes, Laplace transforms, superposition arguments, and Fourier analysis. In this sense, a linear system is precisely equal to the sum of its parts." (Steven H Strogatz, "Non-Linear Dynamics and Chaos, 1994)

"When we examine the modeling literature, its most striking aspect is the predominance of 'flat' linear models. Why is this the case? After all, from a singularity theory viewpoint these linear objects are mathematical rarities. On mathematical grounds we should certainly not expect to see them put forth as credible representations of reality. Yet they are. And the reason is simple: linearity is a neutral assumption that leads to mathematically tractable models. So unless there is good reason to do otherwise, why not use a linear model?"  (John L Casti, "Five Golden Rules", 1995)

"In a linear system a tiny push produces a small effect, so that cause and effect are always proportional to each other. If one plotted on a graph the cause against the effect, the result would be a straight line. In nonlinear systems, however, a small push may produce a small effect, a slightly larger push produces a proportionately larger effect, but increase that push by a hair’s breadth and suddenly the system does something radically different." (F David Peat, "From Certainty to Uncertainty", 2002)

"Linearity means that the rule that determines what a piece of a system is going to do next is not influenced by what it is doing now. More precisely, this is intended in a differential or incremental sense: For a linear spring, the increase of its tension is proportional to the increment whereby it is stretched, with the ratio of these increments exactly independent of how much it has already been stretched. Such a spring can be stretched arbitrarily far, and in particular will never snap or break. Accordingly, no real spring is linear." (Heinz-Otto Peitgen et al, "Chaos and Fractals: New Frontiers of Science" 2nd Ed., 2004) 

"Most long-range forecasts of what is technically feasible in future time periods dramatically underestimate the power of future developments because they are based on what I call the 'intuitive linear' view of history rather than the 'historical exponential' view." (Ray Kurzweil, "The Singularity is Near", 2005)

"Linear systems do not benefit from noise because the output of a linear system is just a simple scaled version of the input [...] Put noise in a linear system and you get out noise. Sometimes you get out a lot more noise than you put in. This can produce explosive effects in feedback systems that take their own outputs as inputs." (Bart Kosko, "Noise", 2006)

"On a linear system like a scale, the whole is equal to the sum of the parts. That’s the first key property of linearity. The second is that causes are proportional to effects. […] These two properties - the proportionality between cause and effect, and the equality of the whole to the sum of the parts - are the essence of what it means to be linear. […] The great advantage of linearity is that it allows for reductionist thinking. To solve a linear problem, we can break it down to its simplest parts, solve each part separately, and put the parts back together to get the answer." (Steven H Strogatz, "Infinite Powers: The Story of Calculus - The Most Important Discovery in Mathematics", 2019)

"[...] perhaps one of the most important features of complex systems, which is a key differentiator when comparing with chaotic systems, is the concept of emergence. Emergence 'breaks' the notion of determinism and linearity because it means that the outcome of these interactions is naturally unpredictable. In large systems, macro features often emerge in ways that cannot be traced back to any particular event or agent. Therefore, complexity theory is based on interaction, emergence and iterations." (Luis Tomé & Şuay Nilhan Açıkalın, "Complexity Theory as a New Lens in IR: System and Change" [in "Chaos, Complexity and Leadership 2017", Şefika Şule Erçetin & Nihan Potas], 2019)

"With a linear growth of errors, improving the measurements could always keep pace with the desire for longer prediction. But when errors grow exponentially fast, a system is said to have sensitive dependence on its initial conditions. Then long-term prediction becomes impossible. This is the philosophically disturbing message of chaos." (Steven H Strogatz, "Infinite Powers: The Story of Calculus - The Most Important Discovery in Mathematics", 2019)

On Nonlinearity III

"Finite systems of deterministic ordinary nonlinear differential equations may be designed to represent forced dissipative hydrodynamic flow. Solutions of these equations can be identified with trajectories in phase space. For those systems with bounded solutions, it is found that nonperiodic solutions are ordinarily unstable with respect to small modifications, so that slightly differing initial states can evolve into considerably different states. Systems with bounded solutions are shown to possess bounded numerical solutions. (Edward N Lorenz, "Deterministic Nonperiodic Flow", Journal of the Atmospheric Science 20, 1963)

"We've seen that even in the simplest situations nonlinearities can interfere with a linear approach to aggregates. That point holds in general: nonlinear interactions almost always make the behavior of the aggregate more complicated than would be predicted by summing or averaging." (Lewis Mumford, "The Myth of the Machine" Vol 1, 1967)

"The structure of a complex system is not a simple feedback loop where one system state dominates the behavior. The complex system has a multiplicity of interacting feedback loops. Its internal rates of flow are controlled by non‐linear relationships. The complex system is of high order, meaning that there are many system states (or levels). It usually contains positive‐feedback loops describing growth processes as well as negative, goal‐seeking loops." (Jay F Forrester, "Urban Dynamics", 1969)

"I would therefore urge that people be introduced to [the logistic equation] early in their mathematical education. This equation can be studied phenomenologically by iterating it on a calculator, or even by hand. Its study does not involve as much conceptual sophistication as does elementary calculus. Such study would greatly enrich the student’s intuition about nonlinear systems. Not only in research but also in the everyday world of politics and economics, we would all be better off if more people realized that simple nonlinear systems do not necessarily possess simple dynamical properties." (Robert M May, "Simple Mathematical Models with Very Complicated Dynamics", Nature Vol. 261 (5560), 1976)

"Most physical systems, particularly those complex ones, are extremely difficult to model by an accurate and precise mathematical formula or equation due to the complexity of the system structure, nonlinearity, uncertainty, randomness, etc. Therefore, approximate modeling is often necessary and practical in real-world applications. Intuitively, approximate modeling is always possible. However, the key questions are what kind of approximation is good, where the sense of 'goodness' has to be first defined, of course, and how to formulate such a good approximation in modeling a system such that it is mathematically rigorous and can produce satisfactory results in both theory and applications." (Guanrong Chen & Trung Tat Pham, "Introduction to Fuzzy Sets, Fuzzy Logic, and Fuzzy Control Systems", 2001) 

"Thus, nonlinearity can be understood as the effect of a causal loop, where effects or outputs are fed back into the causes or inputs of the process. Complex systems are characterized by networks of such causal loops. In a complex, the interdependencies are such that a component A will affect a component B, but B will in general also affect A, directly or indirectly.  A single feedback loop can be positive or negative. A positive feedback will amplify any variation in A, making it grow exponentially. The result is that the tiniest, microscopic difference between initial states can grow into macroscopically observable distinctions." (Carlos Gershenson, "Design and Control of Self-organizing Systems", 2007)

"Where simplifications fail, causing the most damage, is when something nonlinear is simplified with the linear as a substitute. That is the most common Procrustean bed." (Nassim N Taleb, "Antifragile: Things that Gain from Disorder", 2012)

"Complex systems defy intuitive solutions. Even a third-order, linear differential equation is unsolvable by inspection. Yet, important situations in management, economics, medicine, and social behavior usually lose reality if simplified to less than fifth-order nonlinear dynamic systems. Attempts to deal with nonlinear dynamic systems using ordinary processes of description and debate lead to internal inconsistencies. Underlying assumptions may have been left unclear and contradictory, and mental models are often logically incomplete. Resulting behavior is likely to be contrary to that implied by the assumptions being made about' underlying system structure and governing policies." (Jay W Forrester, "Modeling for What Purpose?", The Systems Thinker Vol. 24 (2), 2013)

"There is no linear additive process that, if all the parts are taken together, can be understood to create the total system that occurs at the moment of self-organization; it is not a quantity that comes into being. It is not predictable in its shape or subsequent behavior or its subsequent qualities. There is a nonlinear quality that comes into being at the moment of synchronicity." (Stephen H Buhner, "Plant Intelligence and the Imaginal Realm: Beyond the Doors of Perception into the Dreaming of Earth", 2014)

"Exponentially growing systems are prevalent in nature, spanning all scales from biochemical reaction networks in single cells to food webs of ecosystems. How exponential growth emerges in nonlinear systems is mathematically unclear. […] The emergence of exponential growth from a multivariable nonlinear network is not mathematically intuitive. This indicates that the network structure and the flux functions of the modeled system must be subjected to constraints to result in long-term exponential dynamics." (Wei-Hsiang Lin et al, "Origin of exponential growth in nonlinear reaction networks", PNAS 117 (45), 2020)

On Linearity I

"Today it is no longer questioned that the principles of the analysts are the more far-reaching. Indeed, the synthesists lack two things in order to engage in a general theory of algebraic configurations: these are on the one hand a definition of imaginary elements, on the other an interpretation of general algebraic concepts. Both of these have subsequently been developed in synthetic form, but to do this the essential principle of synthetic geometry had to be set aside. This principle which manifests itself so brilliantly in the theory of linear forms and the forms of the second degree, is the possibility of immediate proof by means of visualized constructions." (Felix Klein, "Riemannsche Flächen", 1906)

"The conception of tensors is possible owing to the circumstance that the transition from one co-ordinate system to another expresses itself as a linear transformation in the differentials. One here uses the exceedingly fruitful mathematical device of making a problem 'linear' by reverting to infinitely small quantities." (Hermann Weyl, "Space - Time - Matter", 1922)

"Any organism must be treated as-a-whole; in other words, that an organism is not an algebraic sum, a linear function of its elements, but always more than that. It is seemingly little realized, at present, that this simple and innocent-looking statement involves a full structural revision of our language […]" (Alfred Korzybski, "Science and Sanity", 1933)

"Beauty had been born, not, as we so often conceive it nowadays, as an ideal of humanity, but as measure, as the reduction of the chaos of appearances to the precision of linear symbols. Symmetry, balance, harmonic division, mated and mensurated intervals - such were its abstract characteristics." (Herbert E Read, "Icon and Idea", 1955)

"We've seen that even in the simplest situations nonlinearities can interfere with a linear approach to aggregates. That point holds in general: nonlinear interactions almost always make the behavior of the aggregate more complicated than would be predicted by summing or averaging." (Lewis Mumford, "The Myth of the Machine" Vol 1, 1967)

"It is sometimes said that the great discovery of the nineteenth century was that the equations of nature were linear, and the great discovery of the twentieth century is that they are not." (Thomas W Körner, "Fourier Analysis", 1988)

"A major clash between economics and ecology derives from the fact that nature is cyclical, whereas our industrial systems are linear. Our businesses take resources, transform them into products plus waste, and sell the products to consumers, who discard more waste […]" (Fritjof Capra, "The Web of Life", 1996)

"The first idea is that human progress is exponential (that is, it expands by repeatedly multiplying by a constant) rather than linear (that is, expanding by repeatedly adding a constant). Linear versus exponential: Linear growth is steady; exponential growth becomes explosive." (Ray Kurzweil, "The Singularity is Near", 2005)

"Without precise predictability, control is impotent and almost meaningless. In other words, the lesser the predictability, the harder the entity or system is to control, and vice versa. If our universe actually operated on linear causality, with no surprises, uncertainty, or abrupt changes, all future events would be absolutely predictable in a sort of waveless orderliness." (Lawrence K Samuels, "Defense of Chaos: The Chaology of Politics, Economics and Human Action", 2013)

"There is no linear additive process that, if all the parts are taken together, can be understood to create the total system that occurs at the moment of self-organization; it is not a quantity that comes into being. It is not predictable in its shape or subsequent behavior or its subsequent qualities. There is a nonlinear quality that comes into being at the moment of synchronicity." (Stephen H Buhner, "Plant Intelligence and the Imaginal Realm: Beyond the Doors of Perception into the Dreaming of Earth", 2014)

Exponential Growth III

"Thus, nonlinearity can be understood as the effect of a causal loop, where effects or outputs are fed back into the causes or inputs of the process. Complex systems are characterized by networks of such causal loops. In a complex, the interdependencies are such that a component A will affect a component B, but B will in general also affect A, directly or indirectly.  A single feedback loop can be positive or negative. A positive feedback will amplify any variation in A, making it grow exponentially. The result is that the tiniest, microscopic difference between initial states can grow into macroscopically observable distinctions." (Carlos Gershenson, "Design and Control of Self-organizing Systems", 2007)

"It has long been appreciated by science that large numbers behave differently than small numbers. Mobs breed a requisite measure of complexity for emergent entities. The total number of possible interactions between two or more members accumulates exponentially as the number of members increases. At a high level of connectivity, and a high number of members, the dynamics of mobs takes hold. " (Kevin Kelly, "Out of Control: The New Biology of Machines, Social Systems and the Economic World", 1995)

"Swarm systems generate novelty for three reasons: (1) They are 'sensitive to initial conditions' - a scientific shorthand for saying that the size of the effect is not proportional to the size of the cause - so they can make a surprising mountain out of a molehill. (2) They hide countless novel possibilities in the exponential combinations of many interlinked individuals. (3) They don’t reckon individuals, so therefore individual variation and imperfection can be allowed. In swarm systems with heritability, individual variation and imperfection will lead to perpetual novelty, or what we call evolution." (Kevin Kelly, "Out of Control: The New Biology of Machines, Social Systems and the Economic World", 1995)

"A more extreme form of exponential growth was probably responsible for the start of the universe. Astronomer and physicists now generally accept the Big Bang theory, according to which the universe started at an unimaginably small size and then doubled in a split second 100 times, enough to make it the size of a small grapefruit. This period of 'inflation' or exponential growth then ended, and linear growth took over, with an expanding fireball creating the universe that we know today." (Richar Koch, "The Power Laws", 2000)

"Mathematics has given us dazzling insights into the power of exponential growth and how the same patterns recur in numbers, regardless of the phenomena being observed." (Richar Koch, "The Power Laws", 2000)

"Periods of rapid change and high exponential growth do not, typically, last long. A new equilibrium with a new dominant technology and/or competitor is likely to be established before long. Periods of punctuation are therefore exciting and exhibit unusual uncertainty. The payoff from establishing a dominant position in this short time is therefore extraordinarily high. Dominance is more likely to come from skill in marketing and positioning than from superior technology itself." (Richar Koch, "The Power Laws", 2000)

"There is a strong tendency today to narrow specialization. Because of the exponential growth of information, we can afford (in terms of both economics and time) preparation of specialists in extremely narrow fields, the various branches of science and engineering having their own particular realms. As the knowledge in these fields grows deeper and broader, the individual's field of expertise has necessarily become narrower. One result is that handling information has become more difficult and even ineffective." (Semyon D Savransky, "Engineering of Creativity", 2000)

"A chain reaction is a string of interactions, each of which creates a by-product that starts another reaction. When the number of reactions grows exponentially, you get a cascade as one reaction begets two, and each of these begets two more, and so on. […] Just so, when you create an interaction that then generates other interactions, you produce amazing impact. As one person talks to two and two talk to four, the number of conversations grows exponentially, creating powerful increases in the speed and spread of change." (Seth Kahan, "Getting Change Right", 2010)

"[...] a high degree of unpredictability is associated with erratic trajectories. This not only because they look random but mostly because infinitesimally small uncertainties on the initial state of the system grow very quickly - actually exponentially fast. In real world, this error amplification translates into our inability to predict the system behavior from the unavoidable imperfect knowledge of its initial state." (Massimo Cencini, "Chaos: From Simple Models to Complex Systems", 2010)

"We can draw several general conclusions. First, because populations of living organisms tend to grow exponentially, numbers can rise very rapidly. This explains the inevitable population pressure that helped Darwin realize the role of natural selection, Second, exponential growth must always be a short-term, temporary phenomenon; for living organisms, the growth typically stops because of predation or a lack of sufficient nutrients or energy. Third, these laws about growth apply to all species- our intelligence cannot make us immune to simple mathematical laws. This is a critical lesson, because human population has been growing exponentially for the past few centuries. Of course, our intelligence gives us one option not available to bacteria. Exponential growth can stop only through some combination of an increase in the death rate and a decrease in the birth rate." (Jeffrey O Bennett & Seth Shostak, "Life in the universe" 3rd Ed., 2012)

Exponential Growth II

"Taking no action to solve these problems is equivalent of taking strong action. Every day of continued exponential growth brings the world system closer to the ultimate limits of that growth. A decision to do nothing is a decision to increase the risk of collapse." (Donella Meadows et al, "The Limits to Growth", 1972) 

"Every day of continued exponential growth brings the world system closer to the ultimate limits of that growth." (Mihajlo D Mesarovic, "Mankind at the Turning Point", 1974)

"The world's present industrial civilization is handicapped by the coexistence of two universal, overlapping, and incompatible intellectual systems: the accumulated knowledge of the last four centuries of the properties and interrelationships of matter and energy; and the associated monetary culture which has evolved from folkways of prehistoric origin. […] Despite their inherent incompatibilities, these two systems during the last two centuries have had one fundamental characteristic in common, namely exponential growth, which has made a reasonably stable coexistence possible. But, for various reasons, it is impossible for the matter-energy system to sustain exponential growth for more than a few tens of doublings, and this phase is by now almost over. The monetary system has no such constraints, and according to one of its most fundamental rules, it must continue to grow by compound interest." (Marion K Hubbert, "Two Intellectual Systems: Matter-energy and the Monetary Culture", [seminar] 1981)

"The change from atoms to bits is irrevocable and unstoppable. Why now? Because the change is also exponential - small differences of yesterday can have suddenly shocking consequences tomorrow." (Nicholas Negroponte, "Being Digital", 1995)

"In order to have a continuing influence, the stock market has to continue rising at an accelerating pace faster than exponential. Only a faster-than-exponential stock market growth makes private investors feel richer." (Didier Sornette, "Why Stock Markets Crash" , 2003)

"Evolution moves towards greater complexity, greater elegance, greater knowledge, greater intelligence, greater beauty, greater creativity, and greater levels of subtle attributes such as love. […] Of course, even the accelerating growth of evolution never achieves an infinite level, but as it explodes exponentially it certainly moves rapidly in that direction." (Ray Kurzweil, "The Singularity is Near", 2005)

"The standard big bang model doesn’t explain the smoothness and flatness of the universe, so it’s been embellished by an additional component: inflation. A minuscule fraction of a second after the big bang, the universe was propelled into an exponential expansion that increased its size from a proton to a grapefruit. […] Moreover, if we accept the theory that the universe emerged from a quantum seed and exponentially expanded in the big bang, there is the possibility that other regions of space-time exist, remote in time or space from our universe. Due to the random nature of quantum processes, these parallel universes could have wildly different properties. This extravagant concept is called the multiverse." (Chris Impey, "The Living Cosmos: Our search for life in the universe", 2007)

"A characteristic of such chaotic dynamics is an extreme sensitivity to initial conditions (exponential separation of neighboring trajectories), which puts severe limitations on any forecast of the future fate of a particular trajectory. This sensitivity is known as the ‘butterfly effect’: the state of the system at time t can be entirely different even if the initial conditions are only slightly changed, i.e., by a butterfly flapping its wings." (Hans J Korsch et al, "Chaos: A Program Collection for the PC", 2008)

"Standard economists don't seem to understand exponential growth. Ecological economics recognizes that the economy, like any other subsystem on the planet, cannot grow forever. And if you think of an organism as an analogy, organisms grow for a period and then they stop growing. They can still continue to improve and develop, but without physically growing, because if organisms did that you’d end up with nine-billion-ton hamsters." (Robert Costanza, "What is Ecological economics", 2010)

"Exponentially growing systems are prevalent in nature, spanning all scales from biochemical reaction networks in single cells to food webs of ecosystems. How exponential growth emerges in nonlinear systems is mathematically unclear. […] The emergence of exponential growth from a multivariable nonlinear network is not mathematically intuitive. This indicates that the network structure and the flux functions of the modeled system must be subjected to constraints to result in long-term exponential dynamics." (Wei-Hsiang Lin et al, "Origin of exponential growth in nonlinear reaction networks", PNAS 117 (45), 2020) 

Exponential Growth I

"However, and conversely, our models fall far short of representing the world fully. That is why we make mistakes and why we are regularly surprised. In our heads, we can keep track of only a few variables at one time. We often draw illogical conclusions from accurate assumptions, or logical conclusions from inaccurate assumptions. Most of us, for instance, are surprised by the amount of growth an exponential process can generate. Few of us can intuit how to damp oscillations in a complex system." (Donella H Meadows, "Limits to Growth", 1972) 

"Mathematics says the sum value of a network increases as the square of the number of members. In other words, as the number of nodes in a network increases arithmetically, the value of the network increases exponentially. Adding a few more members can dramatically increase the value of the network." (Kevin Kelly, "Out of Control: The New Biology of Machines, Social Systems and the Economic World", 1995)

"It is in the nature of exponential growth that events develop extremely slowly for extremely long periods of time, but as one glides through the knee of the curve, events erupt at an increasingly furious pace. And that is what we will experience as we enter the twenty-first century." (Ray Kurzweil, "The Age of Spiritual Machines: When Computers Exceed Human Intelligence", 1999)

"The Law of Accelerating Returns: As order exponentially increases, time exponentially speeds up (that is, the time interval between salient events grows shorter as time passes)." (Ray Kurzweil, "The Age of Spiritual Machines: When Computers Exceed Human Intelligence", 1999)

"Limiting factors in population dynamics play the role in ecology that friction does in physics. They stop exponential growth, not unlike the way in which friction stops uniform motion. Whether or not ecology is more like physics in a viscous liquid, when the growth-rate-based traditional view is sufficient, is an open question. We argue that this limit is an oversimplification, that populations do exhibit inertial properties that are noticeable. Note that the inclusion of inertia is a generalization - it does not exclude the regular rate-based, first-order theories. They may still be widely applicable under a strong immediate density dependence, acting like friction in physics." (Lev Ginzburg & Mark Colyvan, "Ecological Orbits: How Planets Move and Populations Grow", 2004)

"Most long-range forecasts of what is technically feasible in future time periods dramatically underestimate the power of future developments because they are based on what I call the 'intuitive linear' view of history rather than the 'historical exponential'.” view'." (Ray Kurzweil, "The Singularity is Near", 2005)

"The first idea is that human progress is exponential (that is, it expands by repeatedly multiplying by a constant) rather than linear (that is, expanding by repeatedly adding a constant). Linear versus exponential: Linear growth is steady; exponential growth becomes explosive." (Ray Kurzweil, "The Singularity is Near", 2005)

"A quantity growing exponentially toward a limit reaches that limit in a surprisingly short time." (Donella Meadows, "Thinking in systems: A Primer", 2008)

"In physical, exponentially growing systems, there must be at least one reinforcing loop driving growth and at least one balancing feedback loop constraining growth, because no system can grow forever in a finite environment." (Donella H Meadows, “Thinking in Systems: A Primer”, 2008)

"Cyberneticists argue that positive feedback may be useful, but it is inherently unstable, capable of causing loss of control and runaway. A higher level of control must therefore be imposed upon any positive feedback mechanism: self-stabilising properties of a negative feedback loop constrain the explosive tendencies of positive feedback. This is the starting point of our journey to explore the role of cybernetics in the control of biological growth. That is the assumption that the evolution of self-limitation has been an absolute necessity for life forms with exponential growth." (Tony Stebbing, "A Cybernetic View of Biological Growth: The Maia Hypothesis", 2011)