On Uniformity (2000-)

"There is no ‘scientific worldview’ just as there is no uniform enterprise ‘science’- except in the minds of metaphysicians, school masters, and scientists blinded by the achievements of their own particular niche." (Paul Feyerabend, "Conquest of Abundance", 2001)

"When natural frequencies are transformed into conditional probabilities, the base rate information is taken out (this is called normalization). The benefit of this normalization is that the resulting values fall within the uniform range of 0 and 1. The cost, however, is that when drawing inferences from probabilities (as opposed to natural frequencies), one has to put the base rates back in by multiplying the conditional probabilities by their respective base rates." (Gerd Gigerenzer, "Calculated Risks: How to know when numbers deceive you", 2002

"Limiting factors in population dynamics play the role in ecology that friction does in physics. They stop exponential growth, not unlike the way in which friction stops uniform motion. Whether or not ecology is more like physics in a viscous liquid, when the growth-rate-based traditional view is sufficient, is an open question. We argue that this limit is an oversimplification, that populations do exhibit inertial properties that are noticeable. Note that the inclusion of inertia is a generalization - it does not exclude the regular rate-based, first-order theories. They may still be widely applicable under a strong immediate density dependence, acting like friction in physics." (Lev Ginzburg & Mark Colyvan, "Ecological Orbits: How Planets Move and Populations Grow", 2004)

"Statistical tests are just a way of working out the probability of obtaining the observed, or an even more extreme, difference among samples (or between an observed and expected value) if a specific hypothesis (usually the null of no difference) is true. Once the probability is known, the experimenter can make a decision about the difference, using criteria that are uniformly used and understood." (Steve McKillup, "Statistics Explained: An Introductory Guide for Life Scientists", 2005) 

"One of the apparent paradoxes in probability is that, while the outcome of the next roll of a die or toss of a coin may be unpredictable, there are nevertheless underlying patterns in the outcomes overall. Specifically, when a fair die is rolled many times, there is a 'settling down' effect as the proportion of each outcome (1, 2, 3, …, 6) gradually approaches 1/6. In the limiting case, as the number of rolls reaches infinity, the shape of the probability distribution becomes uniform." (Alan Graham, "Developing Thinking in Statistics", 2006)

"Let's face it, the universe is messy. It is nonlinear, turbulent, and chaotic. It is dynamic. It spends its time in transient behavior on its way to somewhere else, not in mathematically neat equilibria. It self-organizes and evolves. It creates diversity, not uniformity. That's what makes the world interesting, that's what makes it beautiful, and that's what makes it work." (Donella H Meadow, "Thinking in Systems: A Primer", 2008)

"[…] topology is the study of those properties of geometric objects which remain unchanged under bi-uniform and bi-continuous transformations. Such transformations can be thought of as bending, stretching, twisting or compressing or any combination of these." (Lokenath Debnath, "The Legacy of Leonhard Euler - A Tricentennial Tribute", 2010)

"The invariance of physical laws with respect to position or orientation (i.e., the symmetry of space) gives rise to conservation laws for linear and angular momentum. Sometimes the implications of symmetry invariance are far more complicated or sophisticated than might at first be supposed; the invariance of the forces predicted by electromagnetic theory when measurements are made in observation frames moving uniformly at different speeds (inertial frames) was an important clue leading Einstein to the discovery of special relativity. With the advent of quantum mechanics, considerations of angular momentum and spin introduced new symmetry concepts into physics. These ideas have since catalyzed the modern development of particle theory." (George B Arfken et al, "Mathematical Methods for Physicists: A comprehensive guide", 2013)

"The No Free Lunch theorems prove that under a uniform distribution over induction problems" (search problems or learning problems), all induction algorithms perform equally.  […] the importance of the theorems arises by using them to analyze scenarios involving non-uniform distributions, and to compare different algorithms, without any assumption about the distribution over problems at all. In particular, the theorems prove that anti-cross-validation" (choosing among a set of candidate algorithms based on which has worst out-of-sample behavior) performs as well as cross-validation, unless one makes an assumption - which has never been formalized - about how the distribution over induction problems, on the one hand, is related to the set of algorithms one is choosing among using" (anti-)cross validation, on the other. In addition, they establish strong caveats concerning the significance of the many results in the literature which establish the strength of a particular algorithm without assuming a particular distribution." (David H Wolpert,What is important about the No Free Lunch theorems?", 2020)

On Analysis in Dynamics

"First, strange attractors look strange: they are not smooth curves or surfaces but have 'non-integer dimension' - or, as Benoit Mandelbrot puts it, they are fractal objects. Next, and more importantly, the motion on a strange attractor has sensitive dependence on initial condition. Finally, while strange attractors have only finite dimension, the time-frequency analysis reveals a continuum of frequencies." (David Ruelle, "Chance and Chaos", 1991)"

"The recognition of symmetry is intuitive but is often difficult to express in any simple and systematic manner. Group theory is a mathematical device to allow for the analysis of symmetry in a variety of ways." (M Ladd, "Symmetry and Group theory in Chemistry", 1998)

"[...] an accurate statement of the 'weak' form of the random-walk hypothesis goes as follows: The history of stock price movements contains no useful information that will enable an investor consistently to outperform a buy-and-hold strategy in managing a portfolio. [...] Moreover, new fundamental information about a company [...] is also unpredictable. It will occur randomly over time. Indeed, successive appearances of news items must be random. If an item of news were not random, that is, if it were dependent on an earlier item of news, then it wouldn't be news at all. The weak form of the random-walk theory says only that stock prices cannot be predicted on the basis of past stock prices. [...] the weak form of the efficient-market hypothesis" (the random-walk notion) says simply that the technical analysis of past price patterns to forecast the future is useless because any information from such an analysis will already have been incorporated in current market prices." (Burton G Malkiel, "A Random Walk Down Wall Street", 1999)

"While chaos theory is, in the last analysis, no more than a metaphor for human society, it can be a valuable metaphor. It makes us sensitive to the types of organizations we create and the way we deal with the situations that surround us." (F David Peat, "From Certainty to Uncertainty", 2002)

"Computers bootstrap their own offspring, grow so wise and incomprehensible that their communiqués assume the hallmarks of dementia: unfocused and irrelevant to the barely-intelligent creatures left behind. And when your surpassing creations find the answers you asked for, you can't understand their analysis and you can't verify their answers. You have to take their word on faith." (Peter Watts, "Blindsight", 2006)

"The core of bifurcation theory of nonlinear system inevitably falls back to the dynamic analysis of linear ones. Because of that, the fundamental question one may ask is if there exist a linearized DAE system with the same qualitative behavior around fixed points of its nonlinear counterpart." (Ataíde S A.Netoa et al, "Nonlinear dynamic analysis of chemical engineering processes described by differential-algebraic equations systems", 2019)

On Dynamics: On Stability (2000-)

"The phenomenon of emergence takes place at critical points of instability that arise from fluctuations in the environment, amplified by feedback loops." (Fritjof Capra, "The Hidden Connections", 2002)

"This spontaneous emergence of order at critical points of instability is one of the most important concepts of the new understanding of life. It is technically known as self-organization and is often referred to simply as ‘emergence’. It has been recognized as the dynamic origin of development, learning and evolution. In other words, creativity-the generation of new forms-is a key property of all living systems. And since emergence is an integral part of the dynamics of open systems, we reach the important conclusion that open systems develop and evolve. Life constantly reaches out into novelty." (Fritjof  Capra, "The Hidden Connections", 2002)

"If a network has violently changing properties, it is most probably not very stable. How can we measure stability, if a network remains unchanged? The assessment of stability often requires a test, and this test comes in the form of a perturbation to the network. A stable network should try to restore its original status after a perturbation. However, this is not easy. Most networks are open systems and therefore undergo a continuous series of perturbations." (Péter Csermely, "Weak Links: The Universal Key to the Stabilityof Networks and Complex Systems", 2009)

"Network stability may be a key element in the development of multilevel, nested networks. The formation of nested networks obviously requires at least a few contacts between the bottom networks. However, evolutionary selection requires the independence and at least temporary isolation of the bottom networks themselves. Weak links are probably the only tools for solving this apparent paradox." (Péter Csermely, "Weak Links: The Universal Key to the Stabilityof Networks and Complex Systems", 2009)

"But the history of large systems demonstrates that, once the hurdle of stability has been cleared, a more subtle challenge appears. It is the challenge of remaining stable when the rules change. Machines, like organizations or organisms, that fail to meet this challenge find that their previous stability is no longer of any use. The responses that once were life-saving now just make things worse. What is needed now is the capacity to re-write the procedure manual on short notice, or even (most radical change of all) to change goals." (John Gall, "The Systems Bible: The Beginner's Guide to Systems Large and Small"[Systematics 3^rd Ed.], 2011)

"Classification is only one of the mathematical aspects of catastrophe theory. Another is stability. The stable states of natural systems are the ones that we can observe over a longer period of time. But the stable states of a system, which can be described by potential functions and their singularities, can become unstable if the potentials are changed by perturbations. So stability problems in nature lead to mathematical questions concerning the stability of the potential functions." (Werner Sanns, "Catastrophe Theory" [Mathematics of Complexity and Dynamical Systems, 2012])

"An important aspect of the global theory of dynamical systems is the stability of the orbit structure as a whole. The motivation for the corresponding theory comes from applied mathematics. Mathematical models always contain simplifying assumptions. Dominant features are modeled; supposed small disturbing forces are ignored. Thus, it is natural to ask if the qualitative structure of the set of solutions - the phase portrait - of a model would remain the same if small perturbations were included in the model. The corresponding mathematical theory is called structural stability." (Carmen Chicone, "Stability Theory of Ordinary Differential Equations" [Mathematics of Complexity and Dynamical Systems, 2012])

"This spontaneous emergence of order at critical points of instability, which is often referred to simply as 'emergence', is one of the hallmarks of life. It has been recognized as the dynamic origin of development, learning, and evolution. In other words, creativity-the generation of new forms-is a key property of all living systems." (Fritjof Capra, "The Systems View of Life: A Unifying Vision", 2014)

Bilash K Bala - Collected Quotes

"A model may be defined as a substitute of any object or system. […] A mental image used in thinking is a model, and it is not the real system. A written description of a system is a model that presents one aspect of reality. The simulation model is logically complete and describes the dynamic behaviour of the system. Models can be broadly classified as (a) physical models and (b) abstract models [..] Mental models and mathematical models are examples of abstract models." (Bilash K Bala et al, "System Dynamics: Modelling and Simulation", 2017)

"Feedback systems are closed loop systems, and the inputs are changed on the basis of output. A feedback system has a closed loop structure that brings back the results of the past action to control the future action. In a closed system, the problem is perceived, action is taken and the result influences the further action. Thus, the distinguishing feature of a closed loop system is a feedback path of information, decision and action connecting the output to input." (Bilash K Bala et al, "System Dynamics: Modelling and Simulation", 2017)

"For a scientist, a model is useful if it generates insight into the structure of [the] real system, makes correct prediction and stimulates meaningful questions for future research. For the public and political leaders, a model is useful if it explains the causes of important problems and provides a basis for designing policy to improve the behaviour of the system. Validity meaning confidence in a model’s usefulness is inherently relative concepts. One must choose between competing models." (Bilash K Bala et al, "System Dynamics: Modelling and Simulation", 2017)

"The goal of a system dynamics approach is to understand how a dynamic pattern of behaviour is generated by a system and to find leverage points within the system structure that have the potential to change the problematic trend to a more desirable one. The key steps in a system dynamics approach are identifying one or more trends that characterise the problem, describing the structure of the system generating the behaviour and finding and testing leverage points in the system to change the problematic behaviour. System dynamics is an appropriate modelling approach for sustainability questions because of the long-term perspective and feedback dynamics inherent in such questions." (Bilash K Bala et al, "System Dynamics: Modelling and Simulation", 2017)

"The model should be robust under extreme conditions. There is an important direct structure test to the robustness of the model under direct extreme conditions, and it evaluates the validity of the equations under extreme conditions by assessing the plausibility of the resulting values against knowledge/anticipation of what would happen under similar conditions in real life." (Bilash K Bala et al, "System Dynamics: Modelling and Simulation", 2017)

"[…] the system boundary should encompass that portion of the whole system which includes all the important and relevant variables to address the problem and the purpose of policy analysis and design. The scope of the study should be clearly stated in order to identify the causes of the problem for clear understanding of the problem and policies for solving the problem in the short run and long run." (Bilash K Bala et al, "System Dynamics: Modelling and Simulation", 2017)

"There is nothing in either physical or social science about which we have perfect knowledge and information. We can never say that a model is a perfect representation of the reality. On the other hand, we can say that there is nothing of which we know absolutely nothing. So, models should not be judged on an absolute scale but on relative scale if the models clarify our knowledge and provide insights into systems." (Bilash K Bala et al, "System Dynamics: Modelling and Simulation", 2017)

Ragnar A K Frisch - Collected Quotes

"I believe that economic theory has arrived at a point in its development where the appeal to quantitative empirical data has become more necessary than ever. At the same time its analyses have reached a degree of complexity that require the application of a more refined scientific method than that employed by the classical economists." (Ragnar Frisch, 1926)

"Intermediate between mathematics, statistics, and economics, we find a new discipline which, for lack of a better name, may be called econometrics. Econometrics has as its aim to subject abstract laws of theoretical political economy or 'pure' economics to experimental and numerical verification, and thus to turn pure economics, as far as possible, into a science in the strict sense of the word." (Ragnar Frisch, "On a Problem in Pure Eco­nomics", 1926)

"Certain exterior impulses hit the economic mechanism and thereby initiate more or less regular oscillations." (Ragnar Frisch, "Propagation problems and impulse problems in dynamic economics", 1933)

"In reality the cycles we have the occasion to observe are generally not damped. How can the maintenance of the swings be explained? Have theses dynamic laws deduced from theory and showing damped oscillations no value in explaining the real phenomena, or in what respect do the dynamic laws need to be completed in order to explain the real happenings? They (dynamic laws) only form one element of the explanation: they solve the propagation problem. But the impulse problem remains." (Ragnar Frisch, "Propagation problems and impulse problems in dynamic economics", 1933)

"[...] the length of the cycles and the tendency towards dampening are determined by the intrinsic structure of the swinging system, while the intensity (the amplitude) of the fluctuations is determined primarily by the exterior impulse. An important consequence of this is that a more or less regular fluctuation may be be produced by a cause which operates irregularly." (Ragnar Frisch, "Propagation problems and impulse problems in dynamic economics", 1933)

"The majority of the economic oscillations which we encounter seem to be explained most plausibly as free oscillations." (Ragnar Frisch, "Propagation problems and impulse problems in dynamic economics", 1933)

"The propagation problem is the problem of explaining by the structural properties of the swinging system what the character of the swings would be in case the system was started in some initial situation." (Ragnar Frisch, "Propagation problems and impulse problems in dynamic economics", 1933)

"When we approach the study of business cycle with the intention of carrying through an analysis that is truly dynamic and determinate in the above sense, we are naturally led to distinguish between two types of analyses: the micro-dynamic and the macro-dynamic types. The micro-dynamic analysis is an analysis by which we try to explain in some detail the behaviour of a certain section of the huge economic mechanism, taking for granted that certain general parameters are given. Obviously it may well be that we obtain more or less cyclical fluctuations in such sub-systems, even though the general parameters are given. The essence of this type of analysis is to show the details of the evolution of a given specific market, the behaviour of a given type of consumers, and so on." (Ragnar Frisch, "Propagation problems and impulse problems in dynamic economics", 1933)

"As long as economic theory still works on a purely qualitative basis without attempting to measure the numerical importance of the various factors, practically any 'conclusion' can be drawn and defended." (Ragnar Frisch, "From Utopian Theory to Practical Applications", [Nobel lecture] 1970)

"Deep in the human nature there is an almost irresistible tendency to concentrate physical and mental energy on attempts at solving problems that seem to be unsolvable." (Ragnar Frisch, "From Utopian Theory to Practical Applications", [Nobel lecture] 1970)

Fred C Scweppe - Collected Quotes

"A bias can be considered a limiting case of a nonwhite disturbance as a constant is the most time-correlated process possible." (Fred C Scweppe, "Uncertain dynamic systems", 1973)

"Changes of variables can be helpful for iterative and parametric solutions even if they do not linearize the problem. For example, a change of variables may change the 'shape' of J(x) into a more suitable form. Unfortunately there seems to be no· general way to choose the 'right' change of variables. Success depends on the particular problem and the engineer's insight. However, the possibility of a change of variables should always be considered."(Fred C Scweppe, "Uncertain dynamic systems", 1973)

"Decision-making problems (hypothesis testing) involve situations where it is desired to make a choice among various alternative decisions (hypotheses). Such problems can be viewed as generalized state estimation problems where the definition of state has simply been expanded." (Fred C Scweppe, "Uncertain dynamic systems", 1973)

"Hypothesis testing can introduce the need for multiple models for the multiple hypotheses and,' if appropriate, a priori probabilities. The one modeling aspect of hypothesis testing that has no estimation counterpart is the problem of specifying the hypotheses to be considered. Often this is a critical step which influences both performance arid the difficulty of implementation." (Fred C Scweppe, "Uncertain dynamic systems", 1973)

"Modeling is definitely the most important and critical problem. If the mathematical model is not valid, any subsequent analysis, estimation, or control study is meaningless. The development of the model in a convenient form can greatly reduce the complexity of the actual studies." (Fred C Scweppe, "Uncertain dynamic systems", 1973)

"Pattern recognition can be viewed as a special case of hypothesis testing. In pattern recognition, an observation z is to be used to decide what pattern caused it. Each possible pattern can be viewed as one hypothesis. The main problem in pattern recognition is the development of models for the z corresponding to each pattern (hypothesis)." (Fred C Scweppe, "Uncertain dynamic systems", 1973)

"System theory is a tool which engineers use to help them design the 'best' system to do the job that must be done. A dominant characteristic of system theory is the interest in the analysis and design (synthesis) of systems from an input-output point of view. System theory uses mathematical manipulation of a mathematical model to help design the actual system." (Fred C Scweppe, "Uncertain dynamic systems", 1973)

"The biggest (and sometimes insurmountable) problem is usually to use the available data (information, measurements, etc.) to find out what the system is actually doing (i.e., to estimate its state). If the system's state can be estimated to some reasonable accuracy, the desired control is often obvious (or can be obtained by the use of deterministic control theory)." (Fred C Scweppe, "Uncertain dynamic systems", 1973)

"The choice of model is often the most critical aspect of a design and development engineering job, but it is impossible to give explicit rules or techniques." (Fred C Scweppe, "Uncertain dynamic systems", 1973)

"The power and beauty of stochastic approximation theory is that it provides simple, easy to implement gain sequences which guarantee convergence without depending (explicitly) on knowledge of the function to be minimized or the noise properties. Unfortunately, convergence is usually extremely slow. This is to be expected, as 'good performance' cannot be expected if no (or very little) knowledge of the nature of the problem is built into the algorithm. In other words, the strength of stochastic approximation (simplicity, little a priori knowledge) is also its weakness." (Fred C Scweppe, "Uncertain dynamic systems", 1973)

"The pseudo approach to uncertainty modeling refers to the use of an uncertainty model instead of using a deterministic model which is actually (or at least theoretically) available. The uncertainty model may be desired because it results in a simpler analysis, because it is too difficult (expensive) to gather all the data necessary for an exact model, or because the exact model is too complex to be included in the computer." (Fred C Scweppe, "Uncertain dynamic systems", 1973)

"[A] system is represented by a mathematical model which may take many forms, such as algebraic equations, finite state machines, difference equations, ordinary differential equations, partial differential equations, and functional equations. The system model may be uncertain, as the mathematical model may not be known completely." (Fred C Scweppe, "Uncertain dynamic systems", 1973)

"The term hypothesis testing arises because the choice as to which process is observed is based on hypothesized models. Thus hypothesis testing could also be called model testing. Hypothesis testing is sometimes called decision theory. The detection theory of communication theory is a special case." (Fred C Scweppe, "Uncertain dynamic systems", 1973)

On Bifurcations I

"It is not enough to know the critical stress, that is, the quantitative breaking point of a complex design; one should also know as much as possible of the qualitative geometry of its failure modes, because what will happen beyond the critical stress level can be very different from one case to the next, depending on just which path the buckling takes. And here catastrophe theory, joined with bifurcation theory, can be very helpful by indicating how new failure modes appear." (Alexander Woodcock & Monte Davis, "Catastrophe Theory", 1978)

"The study of changes in the qualitative structure of the flow of a differential equation as parameters are varied is called bifurcation theory. At a given parameter value, a differential equation is said to have stable orbit structure if the qualitative structure of the flow does not change for sufficiently small variations of the parameter. A parameter value for which the flow does not have stable orbit structure is called a bifurcation value, and the equation is said to be at a bifurcation point." (Jack K Hale & Hüseyin Kocak, "Dynamics and Bifurcations", 1991)

"Fundamental to catastrophe theory is the idea of a bifurcation. A bifurcation is an event that occurs in the evolution of a dynamic system in which the characteristic behavior of the system is transformed. This occurs when an attractor in the system changes in response to change in the value of a parameter. A catastrophe is one type of bifurcation. The broader framework within which catastrophes are located is called dynamical bifurcation theory." (Courtney Brown, "Chaos and Catastrophe Theories", 1995)

"The existence of equilibria or steady periodic solutions is not sufficient to determine if a system will actually behave that way. The stability of these solutions must also be checked. As parameters are changed, a stable motion can become unstable and new solutions may appear. The study of the changes in the dynamic behavior of systems as parameters are varied is the subject of bifurcation theory. Values of the parameters at which the qualitative or topological nature of the motion changes are known as critical or bifurcation values." (Francis C Moona, "Nonlinear Dynamics", 2003)

"In parametrized dynamical systems a bifurcation occurs when a qualitative change is invoked by a change of parameters. In models such a qualitative change corresponds to transition between dynamical regimes. In the generic theory a finite list of cases is obtained, containing elements like ‘saddle-node’, ‘period doubling’, ‘Hopf bifurcation’ and many others." (Henk W Broer & Heinz Hanssmann, "Hamiltonian Perturbation Theory (and Transition to Chaos)", 2009)

"In dynamical systems, a bifurcation occurs when a small smooth change made to the parameter values (the bifurcation parameters) of a system causes a sudden 'qualitative' or topological change in its behaviour. Generally, at a bifurcation, the local stability properties of equilibria, periodic orbits or other invariant sets changes." (Gregory Faye, "An introduction to bifurcation theory",  2011)

"Catastrophe theory can be thought of as a link between classical analysis, dynamical systems, differential topology (including singularity theory), modern bifurcation theory and the theory of complex systems." (Werner Sanns, "Catastrophe Theory" [Mathematics of Complexity and Dynamical Systems, 2012])

"Roughly spoken, bifurcation theory describes the way in which dynamical system changes due to a small perturbation of the system-parameters. A qualitative change in the phase space of the dynamical system occurs at a bifurcation point, that means that the system is structural unstable against a small perturbation in the parameter space and the dynamic structure of the system has changed due to this slight variation in the parameter space." (Holger I Meinhardt, "Cooperative Decision Making in Common Pool Situations", 2012)

"Bifurcation theory is the mathematical study of changes in the qualitative or topological structure of a given family, such as the integral curves of a family of vector fields, and the solutions of a family of differential equations. Most commonly applied to the mathematical study of dynamical systems, a bifurcation occurs when a small smooth change made to the parameter values (the bifurcation parameters) of a system causes a sudden “qualitative” or topological change in its behavior. Bifurcations can occur in both continuous systems (described by ODEs, DDEs, or PDEs) and discrete systems (described by maps)." (Tianshou Zhou, "Bifurcation", 2013)

"The core of bifurcation theory of nonlinear system inevitably falls back to the dynamic analysis of linear ones. Because of that, the fundamental question one may ask is if there exist a linearized DAE system with the same qualitative behavior around fixed points of its nonlinear counterpart." (Ataíde S A.Netoa et al, "Nonlinear dynamic analysis of chemical engineering processes described by differential-algebraic equations systems", 2019)

On Dynamics: On Stability (1975-1999)

"Effect spreads its 'tentacles' not only forwards (as a new cause giving rise to a new effect) but also backwards, to the cause which gave rise to it, thus modifying, exhausting or intensifying its force. This interaction of cause and effect is known as the principle of feedback. It operates everywhere, particularly in all self-organising systems where perception, storing, processing and use of information take place, as for example, in the organism, in a cybernetic device, and in society. The stability, control and progress of a system are inconceivable without feedback." (Alexander Spirkin, "Dialectical Materialism", 1983)

"Structure is the type of connection between the elements of a whole. […] . Structure is a composite whole, or an internally organised content. […] Structure implies not only the position of its elements in space but also their movement in time, their sequence and rhythm, the law of mutation of a process. So structure is actually the law or set of laws that determine a system's composition and functioning, its properties and stability." (Alexander Spirkin, "Dialectical Materialism", 1983)

"Stability theory is the study of systems under various perturbing influences. Since there are many systems, many types of influences, and many equations describing systems, this is an open-ended problem. A system is designed so that it will be stable under external influences. However, one cannot predict all external influences, nor predict the magnitude of those that occur. Consequently, we need control theory. If one is interested in stability theory, a natural result is a theory of control." (Richard E Bellman, "Eye of the Hurricane: An Autobiography", 1984)

"This construction, the horseshoe, has some consequences. First, it yields the fact that homoclinic points do exist and gives a direct construction of them. Second, one obtains such a useful analysis of a general transversal homoclinic point that many properties follow, including sensitive dependence on initial conditions - 'a large number', anyway. Third, one can prove robustness of the horseshoe in a strong global sense structural stability)." (Steven Smale, "What is chaos?", 1990)

"This construction, the horseshoe, has some consequences. First, it yields the fact that homoclinic points do exist and gives a direct construction of them. Second, one obtains such a useful analysis of a general transversal homoclinic point that many properties follow, including sensitive dependence on initial conditions - 'a large number', anyway. Third, one can prove robustness of the horseshoe in a strong global sense structural stability)." (Steven Smale, "What is chaos?", 1990)

"An equilibrium is defined to be stable if all sufficiently small disturbances away from it damp out in time. Thus stable equilibria are represented geometrically by stable fixed points. Conversely, unstable equilibria, in which disturbances grow in time, are represented by unstable fixed points." (Steven H Strogatz, "Non-Linear Dynamics and Chaos, 1994)

"The qualitative structure of the flow can change as parameters are varied. In particular, fixed points can be created or destroyed, or their stability can change. These qualitative changes in the dynamics are called bifurcations , and the parameter values at which they occur are called bifurcation points." (Steven H Strogatz, "Non-Linear Dynamics and Chaos, 1994)

"All systems evolve, although the rates of evolution may vary over time both between and within systems. The rate of evolution is a function of both the inherent stability of the system and changing environmental circumstances. But no system can be stabilized forever. For the universe as a whole, an isolated system, time’s arrow points toward greater and greater breakdown, leading to complete molecular chaos, maximum entropy, and heat death. For open systems, including the living systems that are of major interest to us and that interchange matter and energy with their external environments, time’s arrow points to evolution toward greater and greater complexity. Thus, the universe consists of islands of increasing order in a sea of decreasing order. Open systems evolve and maintain structure by exporting entropy to their external environments." (L Douglas Kiel, "Chaos Theory in the Social Sciences: Foundations and Applications", 1996)

"Where did those symmetries come from? From the even more extensive set of symmetries of the (idealised) uniform state in the infinite dish. The instability of that state caused certain symmetries to be eliminated, but others persist. For target patterns, some rotations and reflections persist. For spirals, what persists is the space-time symmetries 'let time pass and then rotate back'. In a very curious sense, the patterns that we see in the spirals are evidence of other patterns that might have been - the unstable uniform state with its enormous amount of (totally boring) symmetry. They are 'caused' by something that doesn't actually happen." (Ian Stewart, "The Magical Maze: Seeing the world through mathematical eyes", 1997)

"Why don't the chemicals take up the fully symmetric uniform state? Because it is unstable. Any tiny lack of uniformity grows, and destroys the uniform pattern. And in the real world there are always tiny lacks of uniformity - dust motes, bubbles, even just a few molecules vibrating because of heat. (All molecules vibrate because of heat - or more accurately 'heat' is what you get when molecules vibrate - but it only takes a few of them to trigger instability.) The instability is not intuitively obvious, but it's what happens both in the real world and in mathematical models, and here we can take it as given." (Ian Stewart, "The Magical Maze: Seeing the world through mathematical eyes", 1997)

"Complex systems operate under conditions far from equilibrium. Complex systems need a constant flow of energy to change, evolve and survive as complex entities. Equilibrium, symmetry and complete stability mean death. Just as the flow, of energy is necessary to fight entropy and maintain the complex structure of the system, society can only survive as a process. It is defined not by its origins or its goals, but by what it is doing." (Paul Cilliers,"Complexity and Postmodernism: Understanding Complex Systems", 1998)

"Cybernetics is the science of effective organization, of control and communication in animals and machines. It is the art of steersmanship, of regulation and stability. The concern here is with function, not construction, in providing regular and reproducible behaviour in the presence of disturbances. Here the emphasis is on families of solutions, ways of arranging matters that can apply to all forms of systems, whatever the material or design employed. [...] This science concerns the effects of inputs on outputs, but in the sense that the output state is desired to be constant or predictable – we wish the system to maintain an equilibrium state. It is applicable mostly to complex systems and to coupled systems, and uses the concepts of feedback and transformations (mappings from input to output) to effect the desired invariance or stability in the result." (Chris Lucas, "Cybernetics and Stochastic Systems", 1999) 

On Generalization (2000-2009)

"The fruitful generalization in mathematics often involves starting from a commonsense concept such as a point on a line. A mathematical framework is then developed within which the particular example of a point in space is seen to be just a very special case of a much broader structure, say a point in three-dimensional space. Further generalizations then show this new structure itself to be only a special case of an even broader framework, the notion of a point in a space of n dimensions. And so it goes, one generalization piled atop another, each element leading to a deeper understanding of how the original object fits into a bigger picture." (John L Casti, "Five More Golden Rules : Knots, Codes, Chaos, and Other Great Theories of 20th Century Mathematics", 2000)

"A mental model is a representation of some domain or situation that supports understanding, reasoning, and prediction. Mental models permit reasoning about situations not directly experienced. They allow people to mentally simulate the behavior of a system. Many mental models are based on generalizations and analogies from experience." (D Gentner, "Psychology of Mental Models" [in "International Encyclopedia of the Social & Behavioral Sciences"], 2001)

"Ecology, on the other hand, is messy. We cannot find anything deserving of the term law, not because ecology is less developed than physics, but simply because the underlying phenomena are more chaotic and hence less amenable to description via generalization." (Lev Ginzburg & Mark Colyvan, "Ecological Orbits: How Planets Move and Populations Grow", 2004)

"Limiting factors in population dynamics play the role in ecology that friction does in physics. They stop exponential growth, not unlike the way in which friction stops uniform motion. Whether or not ecology is more like physics in a viscous liquid, when the growth-rate-based traditional view is sufficient, is an open question. We argue that this limit is an oversimplification, that populations do exhibit inertial properties that are noticeable. Note that the inclusion of inertia is a generalization - it does not exclude the regular rate-based, first-order theories. They may still be widely applicable under a strong immediate density dependence, acting like friction in physics." (Lev Ginzburg & Mark Colyvan, "Ecological Orbits: How Planets Move and Populations Grow", 2004)

"Mathematical truth is not totally objective. If a mathematical statement is false, there will be no proofs, but if it is true, there will be an endless variety of proofs, not just one! Proofs are not impersonal, they express the personality of their creator/discoverer just as much as literary efforts do. If something important is true, there will be many reasons that it is true, many proofs of that fact. [...] each proof will emphasize different aspects of the problem, each proof will lead in a different direction. Each one will have different corollaries, different generalizations. [...] the world of mathematical truth has infinite complexity […]" (Gregory Chaitin, "Meta Math: The Quest for Omega", 2005)

"The concept of symmetry (invariance) with its rigorous mathematical formulation and generalization has guided us to know the most fundamental of physical laws. Symmetry as a concept has helped mankind not only to define ‘beauty’ but also to express the ‘truth’. Physical laws tries to quantify the truth that appears to be ‘transient’ at the level of phenomena but symmetry promotes that truth to the level of ‘eternity’." (Vladimir G Ivancevic & Tijana T Ivancevic, "Quantum Leap", 2008)

"The reasoning of the mathematician and that of the scientist are similar to a point. Both make conjectures often prompted by particular observations. Both advance tentative generalizations and look for supporting evidence of their validity. Both consider specific implications of their generalizations and put those implications to the test. Both attempt to understand their generalizations in the sense of finding explanations for them in terms of concepts with which they are already familiar. Both notice fragmentary regularities and - through a process that may include false starts and blind alleys - attempt to put the scattered details together into what appears to be a meaningful whole. At some point, however, the mathematician’s quest and that of the scientist diverge. For scientists, observation is the highest authority, whereas what mathematicians seek ultimately for their conjectures is deductive proof." (Raymond S Nickerson, "Mathematical Reasoning: Patterns, Problems, Conjectures and Proofs", 2009)

On Chaos IV

"One of the central problems studied by mankind is the problem of the succession of form. Whatever is the ultimate nature of reality (assuming that this expression has meaning), it is indisputable that our universe is not chaos. We perceive beings, objects, things to which we give names. These beings or things are forms or structures endowed with a degree of stability; they take up some part of space and last for some period of time." (René Thom, "Structural Stability and Morphogenesis", 1972)

"'Disorder' is not mere chaos; it implies defective order." (John M Ziman, "Models of Disorder", 1979)

"Chaos and catastrophe theories are among the most interesting recent developments in nonlinear modeling, and both have captured the interests of scientists in many disciplines. It is only natural that social scientists should be concerned with these theories. Linear statistical models have proven very useful in a great deal of social scientific empirical analyses, as is evidenced by how widely these models have been used for a number of decades. However, there is no apparent reason, intuitive or otherwise, as to why human behavior should be more linear than the behavior of other things, living and nonliving. Thus an intellectual movement toward nonlinear models is an appropriate evolutionary movement in social scientific thinking, if for no other reason than to expand our paradigmatic boundaries by encouraging greater flexibility in our algebraic specifications of all aspects of human life." (Courtney Brown, "Chaos and Catastrophe Theories", 1995)

"[...] chaos and catastrophe theories per se address behavioral phenomena that are consequences of two general types of nonlinear dynamic behavior. In the most elementary of behavioral terms, chaotic phenomena are a class of deterministic processes that seem to mimic random or stochastic dynamics. Catastrophe phenomena, on the other hand, are a class of dynamic processes that exhibit a sudden and large scale change in at least one variable in correspondence with relatively small changes in other variables or, in some cases, parameters." (Courtney Brown, "Chaos and Catastrophe Theories", 1995)

"Nature normally hates power laws. In ordinary systems all quantities follow bell curves, and correlations decay rapidly, obeying exponential laws. But all that changes if the system is forced to undergo a phase transition. Then power laws emerge-nature's unmistakable sign that chaos is departing in favor of order. The theory of phase transitions told us loud and clear that the road from disorder to order is maintained by the powerful forces of self-organization and is paved by power laws. It told us that power laws are not just another way of characterizing a system's behavior. They are the patent signatures of self-organization in complex systems." (Albert-László Barabási, "Linked: How Everything Is Connected to Everything Else and What It Means for Business, Science, and Everyday Life", 2002)

"Chaos is not pure disorder, it carries within itself the indistinctness between the potentialities of order, of disorder, and of organization from which a cosmos will be born, which is an ordered universe." (Edgar Morin, "Restricted Complexity, General Complexity" [in (Carlos Gershenson et al [Eds.], "Worldviews, Science and Us: Philosophy and Complexity", 2007)])

"Chaos can be understood as a dynamical process in which microscopic information hidden in the details of a system’s state is dug out and expanded to a macroscopically visible scale (stretching), while the macroscopic information visible in the current system’s state is continuously discarded (folding)." (Hiroki Sayama, "Introduction to the Modeling and Analysis of Complex Systems", 2015)

"God has put a secret art into the forces of Nature so as to enable it to fashion itself out of chaos into a perfect world system." (Immanuel Kant)

"Science, like art, music and poetry, tries to reduce chaos to the clarity and order of pure beauty." (Detlev W Bronk)

Catastrophe Theory II

"What I am offering, is not a scientific theory, but a method; the first step in the construction of a model is to describe the dynamical models compatible with an empirically given morphology, and this is also the first step in understanding the phenomena under consideration. [...] We may hope that theoreticians will develop a quantitative model [for specific processes described by catastrophe theory ...] But this is only a hope." (René Thom, "Structural Stability and Morphogenesis", 1972)

"The catastrophe model is at the same time much less and much more than a scientific theory; one should consider it as a language, a method, which permits classification and systematization of given empirical data [...] In fact, any phenomenon at all can be explained by a suitable model from catastrophe theory." (René F Thom, 1973)

"First, nature's line patterns are not all of the same sort; the triple junctions generic in mud cracks cannot occur with caustics. Second, the geometrical optics of cylindrically symmetric artifacts such as telescopes, where departures from the ideal point focus are treated as 'aberrations', is very different from the geometrical optics of nature, where the generic forms of caustic surfaces are governed by the mathematics of catastrophe theory." Michael V Berry & John F Nye, "Fine Structure in Caustic Junctions", Nature Vol. 267 (3606), 1977)

"the claims made for the theory are greatly exaggerated and its accomplishments, at least in the biological and social sciences, are insignificant. [...] Catastrophe theory is one of many attempts that have been made to deduce the world by thought alone [...] an appealing dream for mathematicians, but a dream that cannot come true."  (Héctor J Sussmann & Raphael S Zahler, Nature, 1977)

"A catastrophe, in the very broad sense [René] Thom gives to the word, is any discontinuous transition that occurs when a system can have more than one stable state, or can follow more than one stable pathway of change. The catastrophe is the 'jump' from one state or pathway to another." (Alexander Woodcock & Monte Davis, "Catastrophe Theory", 1978)

"It is not enough to know the critical stress, that is, the quantitative breaking point of a complex design; one should also know as much as possible of the qualitative geometry of its failure modes, because what will happen beyond the critical stress level can be very different from one case to the next, depending on just which path the buckling takes. And here catastrophe theory, joined with bifurcation theory, can be very helpful by indicating how new failure modes appear." (Alexander Woodcock & Monte Davis, "Catastrophe Theory", 1978)

"The unfoldings are called catastrophes because each of them has regions where a dynamic system can jump suddenly from one state to another, although the factors controlling the process change continuously. Each of the seven catastrophes represents a pattern of behavior determined only by the number of control factors, not by their nature or by the interior mechanisms that connect them to the system's behavior. Therefore, the elementary catastrophes can be models for a wide variety of processes, even those in which we know little about the quantitative laws involved." (Alexander Woodcock & Monte Davis, "Catastrophe Theory", 1978)

"Two assumptions are needed to apply catastrophe theory as it now stands: first, that the system described be governed by a potential, and second, that its behavior depend on a limited number of control factors. Without these assumptions, the classification of the elementary catastrophes is impossible." (Alexander Woodcock & Monte Davis, "Catastrophe Theory", 1978)

"It is more a philosophy than mathematics, and even as a philosophy it doesn't explain the real world [...] as mathematics, it brings together two of the most basic ideas in modern math: the study of dynamic systems and the study of the singularities of maps. Together, they cover a very wide area - but catastrophe theory brings them together in an arbitrary and constrained way." (Steven Smale)

"While it must be granted that a number of immoderate claims in the form of 'catastrophe theory can do everything' have been made in the literature, on the basis of too little experience, it doesn't seem that the proper response is an equally immoderate claim that 'catastrophe theory can do nothing' on the basis of that same body of experience." (Robert Rosen)

Catastrophe Theory I

"[...] if the behavior points for the entire control surface are plotted and then connected, they form a smooth surface: the behavior surface. The surface has an overall slope from high values where rage predominates to low values in the region where fear is the prevailing state of mind, but the slope is not its most distinctive feature. Catastrophe theory reveals that in the middle of the surface there must be a smooth double fold, creating a pleat without creases, which grows narrower from the front of the surface to the back and eventually disappears in a singular point where the three sheets of the pleat come together. It is the pleat that gives the model its most interesting characteristics. All the points on the behavior surface represent the most probable behavior [...], with the exception of those on the middle sheet, which represent least probable behavior. Through catastrophe theory we can deduce the shape of the entire surface from the fact that the behavior is bimodal for some control points." (E Cristopher Zeeman, "Catastrophe Theory", Scientific American, 1976)

"Catastrophe Theory is - quite likely - the first coherent attempt (since Aristotelian logic) to give a theory on analogy. When narrow-minded scientists object to Catastrophe Theory that it gives no more than analogies, or metaphors, they do not realise that they are stating the proper aim of Catastrophe Theory, which is to classify all possible types of analogous situations." (René F Thom," La Théorie des catastrophes: État présent et perspective", 1977)

"'Catastrophe theory' denotes both a purely mathematical discipline describing certain singularities of smooth maps, as well as the concerted effort to apply these theorems to a wide variety of problems in fields ranging from linguistics and psychology to embryology, evolution, physics, and engineering." (Héctor J Sussmann & Raphael S Zahler, "Catastrophe Theory as Applied to the Social and Biological Sciences: A Critique" Synthese Vol. 37 (2), 1978)

"Because of its foundation in topology, catastrophe theory is qualitative, not quantitative. Just as geometry treated the properties of a triangle without regard to its size, so topology deals with properties that have no magnitude, for example, the property of a given point being inside or outside a closed curve or surface. This property is what topologists call 'invariant' -it does not change even when the curve is distorted. A topologist may work with seven-dimensional space, but he does not and cannot measure (in the ordinary sense) along any of those dimensions. The ability to classify and manipulate all types of form is achieved only by giving up concepts such as size, distance, and rate. So while catastrophe theory is well suited to describe and even to predict the shape of processes, its descriptions and predictions are not quantitative like those of theories built upon calculus. Instead, they are rather like maps without a scale: they tell us that there are mountains to the left, a river to the right, and a cliff somewhere ahead, but not how far away each is, or how large." (Alexander Woodcock & Monte Davis, "Catastrophe Theory", 1978)

"But there is another kind of change, too, change that is less suited to mathematical analysis: the abrupt bursting of a bubble, the discontinuous transition from ice at its melting point to water at its freezing point, the qualitative shift in our minds when we 'get' a pun or a play on words. Catastrophe theory is a mathematical language created to describe and classify this second type of change. It challenges scientists to change the way they think about processes and events in many fields." (Alexander Woodcock & Monte Davis, "Catastrophe Theory", 1978)

"Catastrophe theory is a controversial new way of thinking about change - change in a course of events, change in an object's shape, change in a system's behavior, change in ideas themselves. Its name suggests disaster, and indeed the theory can be applied to literal catastrophes such as the collapse of a bridge or the downfall of an empire. But it also deals with changes as quiet as the dancing of sunlight on the bottom of a pool of water and as subtle as the transition from waking to sleep." (Alexander Woodcock & Monte Davis, "Catastrophe Theory", 1978)

"Catastrophes are often stimulated by the failure to feel the emergence of a domain, and so what cannot be felt in the imagination is experienced as embodied sensation in the catastrophe. (William I Thompson, "Gaia, a Way of Knowing: Political Implications of the New Biology", 1987)

"A catastrophe is a universal unfolding of a singular function germ. The singular function germs are called organization centers of the catastrophes. [...] Catastrophe theory is concerned with the mathematical modeling of sudden changes - so called 'catastrophes' - in the behavior of natural systems, which can appear as a consequence of continuous changes of the system parameters. While in common speech the word catastrophe has a negative connotation, in mathematics it is neutral." (Werner Sanns, "Catastrophe Theory" [Mathematics of Complexity and Dynamical Systems, 2012])

"Catastrophe theory can be thought of as a link between classical analysis, dynamical systems, differential topology (including singularity theory), modern bifurcation theory and the theory of complex systems. [...] The name ‘catastrophe theory’ is used for a combination of singularity theory and its applications. [...] From the didactical point of view, there are two main positions for courses in catastrophe theory at university level: Trying to teach the theory as a perfect axiomatic system consisting of exact definitions, theorems and proofs or trying to teach mathematics as it can be developed from historical or from natural problems." (Werner Sanns, "Catastrophe Theory" [Mathematics of Complexity and Dynamical Systems, 2012])

"Classification is only one of the mathematical aspects of catastrophe theory. Another is stability. The stable states of natural systems are the ones that we can observe over a longer period of time. But the stable states of a system, which can be described by potential functions and their singularities, can become unstable if the potentials are changed by perturbations. So stability problems in nature lead to mathematical questions concerning the stability of the potential functions." (Werner Sanns, "Catastrophe Theory" [Mathematics of Complexity and Dynamical Systems, 2012])

Carl G J Jacobi - Collected Quotes

"It is true that Fourier had the opinion that the principal object of mathematics was public use and the explanation of natural phenomena; but a philosopher like him ought to know that the sole object of the science is the honor of the human spirit and that under this view a problem of [the theory of] numbers is worth as much as a problem on the system of the world." (Carl G J Jacobi [letter to Adrien-Marie Legendre], 1830)

"Any progress in the theory of partial differential equations must also bring about a progress in Mechanics." (Carl G J Jacobi, "Vorlesungen über Dynamik" ["Lectures on Dynamics"], 1843)

"Dirichlet alone, not I, nor Cauchy, nor Gauss knows what a completely rigorous mathematical proof is. Rather we learn it first from him. When Gauss says that he has proved something, it is very clear; when Cauchy says it, one can wager as much pro as con; when Dirichlet says it, it is certain."(Carl G J Jacobi, [letter to Alexander von Humboldt] 1846)

"Wherever Mathematics is mixed up with anything, which is outside its field, you will find attempts to demonstrate these merely conventional propositions a priori, and it will be your task to find out the false deduction in each case." (Carl G J Jacobi, "Vorlesungen über analytische Mechanik" ["Lectures on Analytical Mechanics"], 1847/48)

"God ever arithmetizes." (Carl G J Jacobi)

"Mathematics exists solely for the honour of the human mind." (Carl G J Jacobi)

"Mathematics is slow of growth and only reaches the truth by long and devious paths, that the way to its discovery must be prepared for long beforehand, and that then the truth will make its long-deferred appearance as if impelled by some divine necessity." (Carl G J Jacobi)

"Mathematics is the science of what is clear by itself." (Carl G J Jacobi)

"One should always generalize." (Carl G J Jacobi)

"The God that reigns in Olympus is Number Eternal." (Carl G J Jacobi)

"The real end of science is the honor of the human mind." (Carl G J Jacobi)

"[...] the sole object of science is the honor of the human spirit and that under this view a problem of numbers is worth as much as a problem on the system of the world." (Carl G J Jacobi [letter to Legendre])

Complex Systems VI

"In short, complex adaptive systems are characterized by perpetual novelty." (M Mitchell Waldrop, "Complexity: The Emerging Science at the Edge of Order and Chaos", 1992)

"[...] it's essentially meaningless to talk about a complex adaptive system being in equilibrium: the system can never get there. It is always unfolding, always in transition. In fact, if the system ever does reach equilibrium, it isn't just stable. It's dead." (M Mitchell Waldrop, "Complexity: The Emerging Science at the Edge of Order and Chaos", 1992)

"If universality is one of the observed characteristics of complex dynamical systems in many fields of study, a second characteristic that flows from the study of these systems is that of emergence. As self-organizing systems go about their daily business, they are constantly exchanging matter and energy with their environment, and this allows them to remain in a state that is far from equilibrium. That allows spontaneous behavior to give rise to new patterns." (Terry Cooke-Davies et al, "Exploring the Complexity of Projects", 2009)

"The difference between complex adaptive systems and self-organizing systems is that the former have the capacity to learn from their experience, and thus to embody successful patterns into their repertoire, although there is actually quite a deep relationship between self-organizing systems and complex adaptive systems. Adaptive entities can emerge at high levels of description in simple self-organizing systems, i.e., adaptive systems are not necessarily self-organizing systems with something extra thrown in." (Terry Cooke-Davies et al, "Exploring the Complexity of Projects", 2009)

"Most systems in nature are inherently nonlinear and can only be described by nonlinear equations, which are difficult to solve in a closed form. Non-linear systems give rise to interesting phenomena such as chaos, complexity, emergence and self-organization. One of the characteristics of non-linear systems is that a small change in the initial conditions can give rise to complex and significant changes throughout the system. This property of a non-linear system such as the weather is known as the butterfly effect where it is purported that a butterfly flapping its wings in Japan can give rise to a tornado in Kansas. This unpredictable behaviour of nonlinear dynamical systems, i.e. its extreme sensitivity to initial conditions, seems to be random and is therefore referred to as chaos. This chaotic and seemingly random behaviour occurs for non-linear deterministic system in which effects can be linked to causes but cannot be predicted ahead of time." (Robert K Logan, "The Poetry of Physics and The Physics of Poetry", 2010)

"Complex systems seem to have this property, with large periods of apparent stasis marked by sudden and catastrophic failures. These processes may not literally be random, but they are so irreducibly complex (right down to the last grain of sand) that it just won’t be possible to predict them beyond a certain level. […] And yet complex processes produce order and beauty when you zoom out and look at them from enough distance." (Nate Silver, "The Signal and the Noise: Why So Many Predictions Fail-but Some Don't", 2012)

"We forget - or we willfully ignore - that our models are simplifications of the world. We figure that if we make a mistake, it will be at the margin. In complex systems, however, mistakes are not measured in degrees but in whole orders of magnitude." (Nate Silver, "The Signal and the Noise: Why So Many Predictions Fail-but Some Don't", 2012)

"If an emerging system is born complex, there is neither leeway to abandon it when it fails, nor the means to join another, successful one. Such a system would be caught in an immovable grip, congested at the top, and prevented, by a set of confusing but locked–in precepts, from changing." (Lawrence K Samuels, "Defense of Chaos: The Chaology of Politics, Economics and Human Action", 2013) 

"Simplicity in a system tends to increase that system’s efficiency. Because less can go wrong with fewer parts, less will. Complexity in a system tends to increase that system’s inefficiency; the greater the number of variables, the greater the probability of those variables clashing, and in turn, the greater the potential for conflict and disarray. Because more can go wrong, more will. That is why centralized systems are inclined to break down quickly and become enmeshed in greater unintended consequences." (Lawrence K Samuels,"Defense of Chaos: The Chaology of Politics, Economics and Human Action", 2013)

"One of the remarkable features of these complex systems created by replicator dynamics is that infinitesimal differences in starting positions create vastly different patterns. This sensitive dependence on initial conditions is often called the butterfly-effect aspect of complex systems - small changes in the replicator dynamics or in the starting point can lead to enormous differences in outcome, and they change one’s view of how robust the current reality is. If it is complex, one small change could have led to a reality that is quite different." (David Colander & Roland Kupers, "Complexity and the art of public policy : solving society’s problems from the bottom up", 2014)

Complex Systems IV

"With the growing interest in complex adaptive systems, artificial life, swarms and simulated societies, the concept of 'collective intelligence' is coming more and more to the fore. The basic idea is that a group of individuals (e. g. people, insects, robots, or software agents) can be smart in a way that none of its members is. Complex, apparently intelligent behavior may emerge from the synergy created by simple interactions between individuals that follow simple rules." (Francis Heylighen, "Collective Intelligence and its Implementation on the Web", 1999)

"A system may be called complex here if its dimension (order) is too high and its model (if available) is nonlinear, interconnected, and information on the system is uncertain such that classical techniques can not easily handle the problem." (M Jamshidi, "Autonomous Control on Complex Systems: Robotic Applications", Current Advances in Mechanical Design and Production VII, 2000)

"Bounded rationality simultaneously constrains the complexity of our cognitive maps and our ability to use them to anticipate the system dynamics. Mental models in which the world is seen as a sequence of events and in which feedback, nonlinearity, time delays, and multiple consequences are lacking lead to poor performance when these elements of dynamic complexity are present. Dysfunction in complex systems can arise from the misperception of the feedback structure of the environment. But rich mental models that capture these sources of complexity cannot be used reliably to understand the dynamics. Dysfunction in complex systems can arise from faulty mental simulation-the misperception of feedback dynamics. These two different bounds on rationality must both be overcome for effective learning to occur. Perfect mental models without a simulation capability yield little insight; a calculus for reliable inferences about dynamics yields systematically erroneous results when applied to simplistic models." (John D Sterman, "Business Dynamics: Systems thinking and modeling for a complex world", 2000)

"Much of the art of system dynamics modeling is discovering and representing the feedback processes, which, along with stock and flow structures, time delays, and nonlinearities, determine the dynamics of a system. […] the most complex behaviors usually arise from the interactions (feedbacks) among the components of the system, not from the complexity of the components themselves." (John D Sterman, "Business Dynamics: Systems thinking and modeling for a complex world", 2000)

"To avoid policy resistance and find high leverage policies requires us to expand the boundaries of our mental models so that we become aware of and understand the implications of the feedbacks created by the decisions we make. That is, we must learn about the structure and dynamics of the increasingly complex systems in which we are embedded." (John D Sterman, "Business dynamics: Systems thinking and modeling for a complex world", 2000) 

"Falling between order and chaos, the moment of complexity is the point at which self-organizing systems emerge to create new patterns of coherence and structures of behaviour." (Mark C Taylor, "The Moment of Complexity: Emerging Network Culture", 2001)

"[…] most earlier attempts to construct a theory of complexity have overlooked the deep link between it and networks. In most systems, complexity starts where networks turn nontrivial." (Albert-László Barabási, "Linked: How Everything Is Connected to Everything Else and What It Means for Business, Science, and Everyday Life", 2002)

"[…] networks are the prerequisite for describing any complex system, indicating that complexity theory must inevitably stand on the shoulders of network theory. It is tempting to step in the footsteps of some of my predecessors and predict whether and when we will tame complexity. If nothing else, such a prediction could serve as a benchmark to be disproven. Looking back at the speed with which we disentangled the networks around us after the discovery of scale-free networks, one thing is sure: Once we stumble across the right vision of complexity, it will take little to bring it to fruition. When that will happen is one of the mysteries that keeps many of us going." (Albert-László Barabási, "Linked: How Everything Is Connected to Everything Else and What It Means for Business, Science, and Everyday Life", 2002)

"A sudden change in the evolutive dynamics of a system (a ‘surprise’) can emerge, apparently violating a symmetrical law that was formulated by making a reduction on some (or many) finite sequences of numerical data. This is the crucial point. As we have said on a number of occasions, complexity emerges as a breakdown of symmetry (a system that, by evolving with continuity, suddenly passes from one attractor to another) in laws which, expressed in mathematical form, are symmetrical. Nonetheless, this breakdown happens. It is the surprise, the paradox, a sort of butterfly effect that can highlight small differences between numbers that are very close to one another in the continuum of real numbers; differences that may evade the experimental interpretation of data, but that may increasingly amplify in the system’s dynamics." (Cristoforo S Bertuglia & Franco Vaio, "Nonlinearity, Chaos, and Complexity: The Dynamics of Natural and Social Systems", 2003) 

On Automata I

"Ninety-nine [students] out of a hundred are automata, careful to walk in prescribed paths, careful to follow the prescribed custom. This is not an accident but the result of substantial education, which, scientifically defined, is the subsumption of the individual." (William T Harris, "The Philosophy of Education", 1889)

"We are automata entirely controlled by the forces of the medium being tossed about like corks on the surface of the water, but mistaking the resultant of the impulses from the outside for free will. The movements and other actions we perform are always life preservative and tho seemingly quite independent from one another, we are connected by invisible links." (Nikola Tesla, "My Inventions", 1919)

"Besides electrical engineering theory of the transmission of messages, there is a larger field [cybernetics] which includes not only the study of language but the study of messages as a means of controlling machinery and society, the development of computing machines and other such automata, certain reflections upon psychology and the nervous system, and a tentative new theory of scientific method." (Norbert Wiener, "Cybernetics", 1948)

"Automata have begun to invade certain parts of mathematics too, particularly but not exclusively mathematical physics or applied mathematics. The natural systems (e.g., central nervous system) are of enormous complexity and it is clearly necessary first to subdivide what they represent into several parts that to a certain extent are independent, elementary units. The problem then consists of understanding how these elements are organized as a whole. It is the latter problem which is likely to attract those who have the background and tastes of the mathematician or a logician. With this attitude, he will be inclined to forget the origins and then, after the process of axiomatization is complete, concentrate on the mathematical aspects." (John Von Neumann, "The General and Logical Theory of Automata", 1951)

"A world of automata – of creatures that worked like machines – would hardly be worth creating." (Clive S Lewis, "Mere Christianity", 1952)

"Cellular automata are discrete dynamical systems with simple construction but complex self-organizing behaviour. Evidence is presented that all one-dimensional cellular automata fall into four distinct universality classes. Characterizations of the structures generated in these classes are discussed. Three classes exhibit behaviour analogous to limit points, limit cycles and chaotic attractors. The fourth class is probably capable of universal computation, so that properties of its infinite time behaviour are undecidable." (Stephen Wolfram, "Nonlinear Phenomena, Universality and complexity in cellular automata", Physica 10D, 1984)

"Cellular automata are mathematical models for complex natural systems containing large numbers of simple identical components with local interactions. They consist of a lattice of sites, each with a finite set of possible values. The value of the sites evolve synchronously in discrete time steps according to identical rules. The value of a particular site is determined by the previous values of a neighbourhood of sites around it." (Stephen Wolfram, "Nonlinear Phenomena, Universality and complexity in cellular automata", Physica 10D, 1984)

"Cellular automata may be considered as discrete dynamical systems. In almost all cases, cellular automaton evolution is irreversible. Trajectories in the configuration space for cellular automata therefore merge with time, and after many time steps, trajectories starting from almost all initial states become concentrated onto 'attractors'. These attractors typically contain only a very small fraction of possible states. Evolution to attractors from arbitrary initial states allows for 'self-organizing' behaviour, in which structure may evolve at large times from structureless initial states. The nature of the attractors determines the form and extent of such structures." (Stephen Wolfram, "Nonlinear Phenomena, Universality and complexity in cellular automata", Physica 10D, 1984)

"Finite Nature is a hypothesis that ultimately every quantity of physics, including space and time, will turn out to be discrete and finite; that the amount of information in any small volume of space-time will be finite and equal to one of a small number of possibilities. [...] We take the position that Finite Nature implies that the basic substrate of physics operates in a manner similar to the workings of certain specialized computers called cellular automata." (Edward Fredkin, "A New Cosmogony", PhysComp ’92: Proceedings of the Workshop on Physics and Computation, 1993)

On Nonlinearity V (Chaos I)

"When one combines the new insights gained from studying far-from-equilibrium states and nonlinear processes, along with these complicated feedback systems, a whole new approach is opened that makes it possible to relate the so-called hard sciences to the softer sciences of life - and perhaps even to social processes as well. […] It is these panoramic vistas that are opened to us by Order Out of Chaos." (Ilya Prigogine, "Order Out of Chaos: Man's New Dialogue with Nature", 1984)

"Algorithmic complexity theory and nonlinear dynamics together establish the fact that determinism reigns only over a quite finite domain; outside this small haven of order lies a largely uncharted, vast wasteland of chaos." (Joseph Ford, "Progress in Chaotic Dynamics: Essays in Honor of Joseph Ford's 60th Birthday", 1988)

"The term chaos is used in a specific sense where it is an inherently random pattern of behaviour generated by fixed inputs into deterministic (that is fixed) rules (relationships). The rules take the form of non-linear feedback loops. Although the specific path followed by the behaviour so generated is random and hence unpredictable in the long-term, it always has an underlying pattern to it, a 'hidden' pattern, a global pattern or rhythm. That pattern is self-similarity, that is a constant degree of variation, consistent variability, regular irregularity, or more precisely, a constant fractal dimension. Chaos is therefore order (a pattern) within disorder (random behaviour)." (Ralph D Stacey, "The Chaos Frontier: Creative Strategic Control for Business", 1991)

"In the everyday world of human affairs, no one is surprised to learn that a tiny event over here can have an enormous effect over there. For want of a nail, the shoe was lost, et cetera. But when the physicists started paying serious attention to nonlinear systems in their own domain, they began to realize just how profound a principle this really was. […] Tiny perturbations won't always remain tiny. Under the right circumstances, the slightest uncertainty can grow until the system's future becomes utterly unpredictable - or, in a word, chaotic." (M Mitchell Waldrop, "Complexity: The Emerging Science at the Edge of Order and Chaos", 1992)

"There is a new science of complexity which says that the link between cause and effect is increasingly difficult to trace; that change (planned or otherwise) unfolds in non-linear ways; that paradoxes and contradictions abound; and that creative solutions arise out of diversity, uncertainty and chaos." (Andy P Hargreaves & Michael Fullan, "What’s Worth Fighting for Out There?", 1998)

"Let's face it, the universe is messy. It is nonlinear, turbulent, and chaotic. It is dynamic. It spends its time in transient behavior on its way to somewhere else, not in mathematically neat equilibria. It self-organizes and evolves. It creates diversity, not uniformity. That's what makes the world interesting, that's what makes it beautiful, and that's what makes it work." (Donella H Meadow, "Thinking in Systems: A Primer", 2008)

"Complexity theory can be defined broadly as the study of how order, structure, pattern, and novelty arise from extremely complicated, apparently chaotic systems and conversely, how complex behavior and structure emerges from simple underlying rules. As such, it includes those other areas of study that are collectively known as chaos theory, and nonlinear dynamical theory." (Terry Cooke-Davies et al, "Exploring the Complexity of Projects", 2009)

"Most systems in nature are inherently nonlinear and can only be described by nonlinear equations, which are difficult to solve in a closed form. Non-linear systems give rise to interesting phenomena such as chaos, complexity, emergence and self-organization. One of the characteristics of non-linear systems is that a small change in the initial conditions can give rise to complex and significant changes throughout the system. This property of a non-linear system such as the weather is known as the butterfly effect where it is purported that a butterfly flapping its wings in Japan can give rise to a tornado in Kansas. This unpredictable behaviour of nonlinear dynamical systems, i.e. its extreme sensitivity to initial conditions, seems to be random and is therefore referred to as chaos. This chaotic and seemingly random behaviour occurs for non-linear deterministic system in which effects can be linked to causes but cannot be predicted ahead of time." (Robert K Logan, "The Poetry of Physics and The Physics of Poetry", 2010)

"Even more important is the way complex systems seem to strike a balance between the need for order and the imperative for change. Complex systems tend to locate themselves at a place we call 'the edge of chaos'. We imagine the edge of chaos as a place where there is enough innovation to keep a living system vibrant, and enough stability to keep it from collapsing into anarchy. It is a zone of conflict and upheaval, where the old and new are constantly at war. Finding the balance point must be a delicate matter - if a living system drifts too close, it risks falling over into incoherence and dissolution; but if the system moves too far away from the edge, it becomes rigid, frozen, totalitarian. Both conditions lead to extinction. […] Only at the edge of chaos can complex systems flourish. This threshold line, that edge between anarchy and frozen rigidity, is not a like a fence line, it is a fractal line; it possesses nonlinearity." (Stephen H Buhner, "Plant Intelligence and the Imaginal Realm: Beyond the Doors of Perception into the Dreaming of Earth", 2014)

"To remedy chaotic situations requires a chaotic approach, one that is non-linear, constantly morphing, and continually sharpening its competitive edge with recurring feedback loops that build upon past experiences and lessons learned. Improvement cannot be sustained without reflection. Chaos arises from myriad sources that stem from two origins: internal chaos rising within you, and external chaos being imposed upon you by the environment. The result of this push/pull effect is the disequilibrium [...]." (Jeff Boss, "Navigating Chaos: How to Find Certainty in Uncertain Situations", 2015)

On Nonlinearity III

"Finite systems of deterministic ordinary nonlinear differential equations may be designed to represent forced dissipative hydrodynamic flow. Solutions of these equations can be identified with trajectories in phase space. For those systems with bounded solutions, it is found that nonperiodic solutions are ordinarily unstable with respect to small modifications, so that slightly differing initial states can evolve into considerably different states. Systems with bounded solutions are shown to possess bounded numerical solutions. (Edward N Lorenz, "Deterministic Nonperiodic Flow", Journal of the Atmospheric Science 20, 1963)

"We've seen that even in the simplest situations nonlinearities can interfere with a linear approach to aggregates. That point holds in general: nonlinear interactions almost always make the behavior of the aggregate more complicated than would be predicted by summing or averaging." (Lewis Mumford, "The Myth of the Machine" Vol 1, 1967)

"The structure of a complex system is not a simple feedback loop where one system state dominates the behavior. The complex system has a multiplicity of interacting feedback loops. Its internal rates of flow are controlled by non‐linear relationships. The complex system is of high order, meaning that there are many system states (or levels). It usually contains positive‐feedback loops describing growth processes as well as negative, goal‐seeking loops." (Jay F Forrester, "Urban Dynamics", 1969)

"I would therefore urge that people be introduced to [the logistic equation] early in their mathematical education. This equation can be studied phenomenologically by iterating it on a calculator, or even by hand. Its study does not involve as much conceptual sophistication as does elementary calculus. Such study would greatly enrich the student’s intuition about nonlinear systems. Not only in research but also in the everyday world of politics and economics, we would all be better off if more people realized that simple nonlinear systems do not necessarily possess simple dynamical properties." (Robert M May, "Simple Mathematical Models with Very Complicated Dynamics", Nature Vol. 261 (5560), 1976)

"Most physical systems, particularly those complex ones, are extremely difficult to model by an accurate and precise mathematical formula or equation due to the complexity of the system structure, nonlinearity, uncertainty, randomness, etc. Therefore, approximate modeling is often necessary and practical in real-world applications. Intuitively, approximate modeling is always possible. However, the key questions are what kind of approximation is good, where the sense of 'goodness' has to be first defined, of course, and how to formulate such a good approximation in modeling a system such that it is mathematically rigorous and can produce satisfactory results in both theory and applications." (Guanrong Chen & Trung Tat Pham, "Introduction to Fuzzy Sets, Fuzzy Logic, and Fuzzy Control Systems", 2001) 

"Thus, nonlinearity can be understood as the effect of a causal loop, where effects or outputs are fed back into the causes or inputs of the process. Complex systems are characterized by networks of such causal loops. In a complex, the interdependencies are such that a component A will affect a component B, but B will in general also affect A, directly or indirectly.  A single feedback loop can be positive or negative. A positive feedback will amplify any variation in A, making it grow exponentially. The result is that the tiniest, microscopic difference between initial states can grow into macroscopically observable distinctions." (Carlos Gershenson, "Design and Control of Self-organizing Systems", 2007)

"Where simplifications fail, causing the most damage, is when something nonlinear is simplified with the linear as a substitute. That is the most common Procrustean bed." (Nassim N Taleb, "Antifragile: Things that Gain from Disorder", 2012)

"Complex systems defy intuitive solutions. Even a third-order, linear differential equation is unsolvable by inspection. Yet, important situations in management, economics, medicine, and social behavior usually lose reality if simplified to less than fifth-order nonlinear dynamic systems. Attempts to deal with nonlinear dynamic systems using ordinary processes of description and debate lead to internal inconsistencies. Underlying assumptions may have been left unclear and contradictory, and mental models are often logically incomplete. Resulting behavior is likely to be contrary to that implied by the assumptions being made about' underlying system structure and governing policies." (Jay W Forrester, "Modeling for What Purpose?", The Systems Thinker Vol. 24 (2), 2013)

"There is no linear additive process that, if all the parts are taken together, can be understood to create the total system that occurs at the moment of self-organization; it is not a quantity that comes into being. It is not predictable in its shape or subsequent behavior or its subsequent qualities. There is a nonlinear quality that comes into being at the moment of synchronicity." (Stephen H Buhner, "Plant Intelligence and the Imaginal Realm: Beyond the Doors of Perception into the Dreaming of Earth", 2014)

"Exponentially growing systems are prevalent in nature, spanning all scales from biochemical reaction networks in single cells to food webs of ecosystems. How exponential growth emerges in nonlinear systems is mathematically unclear. […] The emergence of exponential growth from a multivariable nonlinear network is not mathematically intuitive. This indicates that the network structure and the flux functions of the modeled system must be subjected to constraints to result in long-term exponential dynamics." (Wei-Hsiang Lin et al, "Origin of exponential growth in nonlinear reaction networks", PNAS 117 (45), 2020)

On Nonlinearity II

"Indeed, except for the very simplest physical systems, virtually everything and everybody in the world is caught up in a vast, nonlinear web of incentives and constraints and connections. The slightest change in one place causes tremors everywhere else. We can't help but disturb the universe, as T.S. Eliot almost said. The whole is almost always equal to a good deal more than the sum of its parts. And the mathematical expression of that property - to the extent that such systems can be described by mathematics at all - is a nonlinear equation: one whose graph is curvy." (M Mitchell Waldrop, "Complexity: The Emerging Science at the Edge of Order and Chaos", 1992)

"Today the network of relationships linking the human race to itself and to the rest of the biosphere is so complex that all aspects affect all others to an extraordinary degree. Someone should be studying the whole system, however crudely that has to be done, because no gluing together of partial studies of a complex nonlinear system can give a good idea of the behaviour of the whole." (Murray Gell-Mann, 1997)

"Much of the art of system dynamics modeling is discovering and representing the feedback processes, which, along with stock and flow structures, time delays, and nonlinearities, determine the dynamics of a system. […] the most complex behaviors usually arise from the interactions (feedbacks) among the components of the system, not from the complexity of the components themselves." (John D Sterman, "Business Dynamics: Systems thinking and modeling for a complex world", 2000)

"The mental models people use to guide their decisions are dynamically deficient. […] people generally adopt an event-based, open-loop view of causality, ignore feedback processes, fail to appreciate time delays between action and response and in the reporting of information, do not understand stocks and flows and are insensitive to nonlinearities that may alter the strengths of different feedback loops as a system evolves." (John D Sterman, "Business Dynamics: Systems thinking and modeling for a complex world", 2000)

"Most physical processes in the real world are nonlinear. It is our abstraction of the real world that leads us to the use of linear systems in modeling these processes. These linear systems are simple, understandable, and, in many situations, provide acceptable simulations of the actual processes. Unfortunately, only the simplest of linear processes and only a very small fraction of the nonlinear having verifiable solutions can be modeled with linear systems theory. The bulk of the physical processes that we must address are, unfortunately, too complex to reduce to algorithmic form - linear or nonlinear. Most observable processes have only a small amount of information available with which to develop an algorithmic understanding. The vast majority of information that we have on most processes tends to be nonnumeric and nonalgorithmic. Most of the information is fuzzy and linguistic in form." (Timothy J Ross & W Jerry Parkinson, "Fuzzy Set Theory, Fuzzy Logic, and Fuzzy Systems", 2002)

"Swarm intelligence can be effective when applied to highly complicated problems with many nonlinear factors, although it is often less effective than the genetic algorithm approach [...]. Swarm intelligence is related to swarm optimization […]. As with swarm intelligence, there is some evidence that at least some of the time swarm optimization can produce solutions that are more robust than genetic algorithms. Robustness here is defined as a solution’s resistance to performance degradation when the underlying variables are changed. (Michael J North & Charles M Macal, Managing Business Complexity: Discovering Strategic Solutions with Agent-Based Modeling and Simulation, 2007)

"[…] our mental models fail to take into account the complications of the real world - at least those ways that one can see from a systems perspective. It is a warning list. Here is where hidden snags lie. You can’t navigate well in an interconnected, feedback-dominated world unless you take your eyes off short-term events and look for long-term behavior and structure; unless you are aware of false boundaries and bounded rationality; unless you take into account limiting factors, nonlinearities and delays. You are likely to mistreat, misdesign, or misread systems if you don’t respect their properties of resilience, self-organization, and hierarchy." (Donella H Meadows, "Thinking in Systems: A Primer", 2008)

"You can’t navigate well in an interconnected, feedback-dominated world unless you take your eyes off short-term events and look for long term behavior and structure; unless you are aware of false boundaries and bounded rationality; unless you take into account limiting factors, nonlinearities and delays." (Donella H Meadow, "Thinking in Systems: A Primer", 2008)

"A network of many simple processors ('units' or 'neurons') that imitates a biological neural network. The units are connected by unidirectional communication channels, which carry numeric data. Neural networks can be trained to find nonlinear relationships in data, and are used in various applications such as robotics, speech recognition, signal processing, medical diagnosis, or power systems." (Adnan Khashman et al, "Voltage Instability Detection Using Neural Networks", 2009)

"Linearity is a reductionist’s dream, and nonlinearity can sometimes be a reductionist’s nightmare. Understanding the distinction between linearity and nonlinearity is very important and worthwhile." (Melanie Mitchell, "Complexity: A Guided Tour", 2009)