💠On Problem Solving 16 (Trivia)

"Paradoxes have played a dramatic part in intellectual history, often foreshadowing revolutionary developments in science, mathematics, and logic. Whenever, in any discipline, we discover a problem that cannot be solved within the conceptual framework that supposedly should apply, we experience an intellectual shock. The shock may compel us to discard the old framework and adopt a new one. It is to this process of intellectual molting that we owe the birth of many of the major ideas in mathematics and science." (Anatol Rapoport, "Escape from Paradox", Scientific American Vol. 217 (1), 1967)

"The problems of different fields are much more alike than their practitioners think, much more alike than different." (John W Tukey, "Analyzing Data: Sanctification or Detective Work?", 1969)

"Design is really a special case of problem solving. One wants to bring about a desired state of affairs. Occasionally one wants to remedy some fault but more usually one wants to bring about something new. For that reason design is more open ended than problem solving. It requires more creativity. It is not so much a matter of linking up a clearly defined objective with a clearly defined starting position (as in problem solving) but more a matter of starting out from a general position in the direction of a general objective." (Edward de Bono, "Lateral Thinking: Creativity Step by Step", 1970)

"A system represents someone's solution to a problem. The system doesn't solve the problem." (John Gall, "General Systemantics: How systems work, and especially how they fail", 1975)

"[...] any hope that we are smart enough to find even transiently optimum solutions to our data analysis problems is doomed to failure, and, indeed, if taken seriously, will mislead us in the allocation of effort, thus wasting both intellectual and computational effort." (John W Tukey, "Choosing Techniques for the Analysis of Data", 1981)

"We will treat problem solving as a process of search through a state space. A problem is defined by an initial state, one or more goal states to be reached, a set of operators that can transform one state into another, and constraints that an acceptable solution must meet. Problem-solving methods are procedures for selecting an appropriate sequence of operators that will succeed in transforming the initial state into a goal state through a series of steps." (John H Holland et al, "Induction: Processes Of Inference, Learning, And Discovery", 1986)

"Delay time, the time between causes and their impacts, can highly influence systems. Yet the concept of delayed effect is often missed in our impatient society, and when it is recognized, it’s almost always underestimated. Such oversight and devaluation can lead to poor decision making as well as poor problem solving, for decisions often have consequences that don’t show up until years later. Fortunately, mind mapping, fishbone diagrams, and creativity/brainstorming tools can be quite useful here." (Stephen G Haines, "The Managers Pocket Guide to Systems Thinking & Learning", 1998)

"A problem is defined as a gap between where you are and where you want to be that is confronted with obstacles that make closing the gap difficult. It is actually the obstacles that make the gap a problem." (James P Lewis, "Project Planning, Scheduling, and Control" 3rd Ed., 2001)

"Complex problems are the problems that persist - the problems that bounce back and continue to haunt us. People often go through a series of stages in dealing with such problems - from believing they are beyond hope, to galvanizing collective efforts of many people and dollars to address the problem, to despair, retreat, and rationalization."(Yaneer Bar-Yam, "Making Things Work: Solving Complex Problems in a Complex World", 2004)

"Design thinking taps into capacities we all have but that are overlooked by more conventional problem-solving practices. It is not only human-centered; it is deeply human in and of itself. Design thinking relies on our ability to be intuitive, to recognize patterns, to construct ideas that have emotional meaning as well as functionality, to express ourselves in media other than words or symbols." (Tim Brown, "Change by Design: How Design Thinking Transforms Organizations and Inspires Innovation", 2009)

"Mostly we rely on stories to put our ideas into context and give them meaning. It should be no surprise, then, that the human capacity for storytelling plays an important role in the intrinsically human-centered approach to problem solving, design thinking." (Tim Brown, "Change by Design: How Design Thinking Transforms Organizations and Inspires Innovation", 2009)

"The basic idea behind all of these techniques is to simplify problem solving by concentrating on its essentials. Consolidate and simplify the objectives. Focus on the things with the highest impact, things that determine other things. Put to one side minor issues likely to be resolved by the resolution of major ones. Discard the nonessentials. Model (abstract) the system at as high a level as possible, then progressively reduce the level of abstraction. In short: Simplify!" (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

"Teaching statistical thinking means giving people tools for problem solving in the real world. It should not be taught as pure mathematics. Instead of mechanically solving a dozen problems with the help of a particular formula, children and adolescents should be asked to find solutions to real-life problems. That’s what teaches them how to solve problems, and also shows that there may be more than one good answer in the first place. Equally important is encouraging curiosity, such as asking for answers to questions by doing experiments." (Gerd Gigerenzer, "Risk Savvy: How to make good decisions", 2014)

"Building a comprehensive problem-solving framework is about leveraging a structured methodology that allows you to frame problems systematically and solve problems creatively." (Pearl Zhu, "Problem Solving Master: Frame Problems Systematically and Solve Problem Creatively", 2017)

"[...] strategy is about determining the problems and opportunities in front of you, defining them properly, and shaping a course of action that will give your business the greatest advantage. Balancing problem solving with creating and exploiting new opportunities through imagination and analysis is the cornerstone of a great strategy." (Eben Hewitt, "Technology Strategy Patterns: Architecture as strategy" 2nd Ed., 2019)

"The most fruitful research grows out of practical problems."  (Ralph B Peck)

"Understand the reality of the problem first, rather than wondering what its cause may be. The first step of problem solving is to understand the existing conditions." (Kaoru Ishikawa)

Previous <<||>> Next

S Ian Robertson - Collected Quotes

"An internal model corresponds to a specific concrete situation in the external world and allows us to reason about the external situation. To do so you used information about the problem presented in the problem statement. The process of understanding, then, refers to constructing an initial mental representation of what the problem is, based on the information in the problem statement about the goal, the initial state, what you are not allowed to do, and what operator to apply, as well as your own personal past experience." (S Ian Robertson, "Problem Solving", 2001)

"Heuristics are rules of thumb that help constrain the problem in certain ways (in other words they help you to avoid falling back on blind trial and error), but they don't guarantee that you will find a solution. Heuristics are often contrasted with algorithms that will guarantee that you find a solution - it may take forever, but if the problem is algorithmic you will get there. However, heuristics are also algorithms." (S Ian Robertson, "Problem Solving", 2001)

"Lack of understanding of a problem (or a concept, or a system of relations) can lead to superficial answers to problems, as well as to blindly following a procedure." (S Ian Robertson, "Problem Solving", 2001)

"Problem solving starts off from an initial given situation or statement of a problem (known as the initial state of the problem). Based on the problem situation and your prior knowledge you have to work towards a solution. When you reach it you are in the goal state of the problem. On the way from the initial state to the goal state you pass through a number of intermediate problem states." (S Ian Robertson, "Problem Solving", 2001)

"The difficulty facing us when we have to make inferences is two-fold. First, we may build entirely the wrong mental model from the information we read or hear. […] The second difficulty facing us is that we may well build a reasonably correct initial representation of a problem, but this representation may be impoverished in some way because we have no idea what inferences are relevant […]" (S Ian Robertson, "Problem Solving", 2001)

"The process of abstracting out the common features of things in this way is known as induction. Applying what we have learned - generalising from one or a few examples to a whole range of new examples - is known as deduction. Induction is the process of moving from the particular to the general, and deduction is the process of going from the general to the particular." (S Ian Robertson, "Problem Solving", 2001)

"Thinking involves reasoning about a situation, and to do that we must have some kind of dynamic 'model' of the situation in our heads. Any changes we make to this mental model of the world should ideally mirror changes in the real world." (S Ian Robertson, "Problem Solving", 2001)

"This does not mean that the representation has to be 'complete' before any problem solving can take place. If you had a 'complete' representation of a problem then you wouldn't have a problem, as you would know exactly how to get from where you are now to where you want to be. A problem only exists when it is not immediately obvious how to get from where you are now to your goal. An adequate representation should at least allow you to see what moves you can possibly make and allow you to start heading towards your goal." (S Ian Robertson, "Problem Solving", 2001)

"Understanding a problem means building some kind of representation of the problem in one's mind, based on what the situation is or what the problem statement says and on one's prior knowledge. It is then possible to reason about the problem within this mental representation. Generating a useful mental representation is therefore the most important single factor for successful problem solving." (S Ian Robertson, "Problem Solving", 2001)

"We are constantly using old knowledge in new situations. When a solver can successfully use a solution procedure used in the past to solve a target problem this is known as positive transfer. […] However, it is also the case that a procedure learned in the past can impede one's learning of a new procedure. This is known as negative transfer. In this case what you have learned prevents you from solving a new problem or at least prevents you from seeing an optimal solution." (S Ian Robertson, "Problem Solving", 2001)

"When we have difficulties solving a problem, insight into its solution can come about by restructuring the problem." (S Ian Robertson, "Problem Solving", 2001)

On Chaos VII

"The central task of a natural science is to make the wonderful commonplace: to show that complexity, correctly viewed, is only a mask for simplicity; to find pattern hidden in apparent chaos. […] This is the task of natural science: to show that the wonderful is not incomprehensible, to show how it can be comprehended - but not to destroy wonder. For when we have explained the wonderful, unmasked the hidden pattern, a new wonder arises at how complexity was woven out of simplicity. The aesthetics of natural science and mathematics is at one with the aesthetics of music and painting - both inhere in the discovery of a partially concealed pattern." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"I would [...] urge that people be introduced to [chaos] early in their mathematical education. [Chaos] can be studied phenomenologically by iterating it on a calculator, or even by hand. Its study does not involve as much conceptual sophistication as does elementary calculus. Such study would greatly enrich the student's intuition. Not only in research, but also in the everyday world of politics and economics, we would all be better off if more people realised that simple nonlinear systems do not necessarily possess simple dynamical properties." (Robert May, "Simple mathematical models with very complicated dynamics", Nature 26(5560), 1976)

"Due to this sensitivity any uncertainty about seemingly insignificant digits in the sequence of numbers which defines an initial condition, spreads with time towards the significant digits, leading to chaotic behavior. Therefore there is a change in the information we have about the state of the system. This change can be thought of as a creation of information if we consider that two initial conditions that are different but indistinguishable (within a certain precision), evolve into distinguishable states after a finite time." (David Ruelle, "Chaotic Evolution and Strange Attractors: The statistical analysis of time series for deterministic nonlinear systems", 1989)

"[…] chaos and fractals are part of an even grander subject known as dynamics. This is the subject that deals with change, with systems that evolve in time. Whether the system in question settles down to equilibrium, keeps repeating in cycles, or does something more complicated, it is dynamics that we use to analyze the behavior." (Steven H Strogatz, "Non-Linear Dynamics and Chaos, 1994)

"Chaos appears in both dissipative and conservative systems, but there is a difference in its structure in the two types of systems. Conservative systems have no attractors. Initial conditions can give rise to periodic, quasiperiodic, or chaotic motion, but the chaotic motion, unlike that associated with dissipative systems, is not self-similar. In other words, if you magnify it, it does not give smaller copies of itself. A system that does exhibit self-similarity is called fractal. [...] The chaotic orbits in conservative systems are not fractal; they visit all regions of certain small sections of the phase space, and completely avoid other regions. If you magnify a region of the space, it is not self-similar." (Barry R Parker, "Chaos in the Cosmos: The stunning complexity of the universe", 1996)

"Although the shape of chaos is nightmarish, its voice is oddly soothing. When played through a loudspeaker, chaos sounds like white noise, like the soft static that helps insomniacs fall asleep." (Steven Strogatz, "Sync: The Emerging Science of Spontaneous Order", 2003)

"Chaos has three primary features: unpredictability, boundedness, and sensitivity to initial conditions. Unpredictability means that a sequence of numbers that is generated from a chaotic function does not repeat. This principle is perhaps a matter of degree, because some of the numbers could look as though they are recurring only because they are rounded to a convenient number of decimal points. [...] Boundedness means that, for all the unpredictability of motion, all points remain within certain boundaries. The principle of sensitivity to initial conditions means that two points that start off as arbitrarily close together become exponentially farther away from each other as the iteration process proceeds. This is a clear case of small differences producing a huge effect." (Stephen J Guastello & Larry S Liebovitch, "Introduction to Nonlinear Dynamics and Complexity" [in "Chaos and Complexity in Psychology"], 2009)

"Chaos is a phenomenon encountered in science and mathematics wherein a deterministic (rule-based) system behaves unpredictably. That is, a system which is governed by fixed, precise rules, nevertheless behaves in a way which is, for all practical purposes, unpredictable in the long run. The mathematical use of the word 'chaos' does not align well with its more common usage to indicate lawlessness or the complete absence of order. On the contrary, mathematically chaotic systems are, in a sense, perfectly ordered, despite their apparent randomness. This seems like nonsense, but it is not." (David P Feldman, "Chaos and Fractals: An Elementary Introduction", 2012)

"Chaos is just one phenomenon out of many that are encountered in the study of dynamical systems. In addition to behaving chaotically, systems may show fixed equilibria, simple periodic cycles, and more complicated behaviors that defy easy categorization. The study of dynamical systems holds many surprises and shows that the relationships between order and disorder, simplicity and complexity, can be subtle, and counterintuitive." (David P Feldman, "Chaos and Fractals: An Elementary Introduction", 2012)

 "Understanding chaos requires much less advanced mathematics than other current areas of physics research such as general relativity or particle physics. Observing chaos and fractals requires no specialized equipment; chaos is seen in scores of everyday phenomena - a boiling pot of water, a dripping faucet, shifting weather patterns. And fractals are almost ubiquitous in the natural world. Thus, it is possible to teach the central ideas and insights of chaos in a rigorous, genuine, and relevant way to students with relatively little mathematics background." (David P Feldman, "Chaos and Fractals: An Elementary Introduction", 2012)

On Linearity II

"Why are nonlinear systems so much harder to analyze than linear ones? The essential difference is that linear systems can be broken down into parts. Then each part can be solved separately and finally recombined to get the answer. This idea allows a fantastic simplification of complex problems, and underlies such methods as normal modes, Laplace transforms, superposition arguments, and Fourier analysis. In this sense, a linear system is precisely equal to the sum of its parts." (Steven H Strogatz, "Non-Linear Dynamics and Chaos, 1994)

"When we examine the modeling literature, its most striking aspect is the predominance of 'flat' linear models. Why is this the case? After all, from a singularity theory viewpoint these linear objects are mathematical rarities. On mathematical grounds we should certainly not expect to see them put forth as credible representations of reality. Yet they are. And the reason is simple: linearity is a neutral assumption that leads to mathematically tractable models. So unless there is good reason to do otherwise, why not use a linear model?"  (John L Casti, "Five Golden Rules", 1995)

"In a linear system a tiny push produces a small effect, so that cause and effect are always proportional to each other. If one plotted on a graph the cause against the effect, the result would be a straight line. In nonlinear systems, however, a small push may produce a small effect, a slightly larger push produces a proportionately larger effect, but increase that push by a hair’s breadth and suddenly the system does something radically different." (F David Peat, "From Certainty to Uncertainty", 2002)

"Linearity means that the rule that determines what a piece of a system is going to do next is not influenced by what it is doing now. More precisely, this is intended in a differential or incremental sense: For a linear spring, the increase of its tension is proportional to the increment whereby it is stretched, with the ratio of these increments exactly independent of how much it has already been stretched. Such a spring can be stretched arbitrarily far, and in particular will never snap or break. Accordingly, no real spring is linear." (Heinz-Otto Peitgen et al, "Chaos and Fractals: New Frontiers of Science" 2nd Ed., 2004) 

"Most long-range forecasts of what is technically feasible in future time periods dramatically underestimate the power of future developments because they are based on what I call the 'intuitive linear' view of history rather than the 'historical exponential' view." (Ray Kurzweil, "The Singularity is Near", 2005)

"Linear systems do not benefit from noise because the output of a linear system is just a simple scaled version of the input [...] Put noise in a linear system and you get out noise. Sometimes you get out a lot more noise than you put in. This can produce explosive effects in feedback systems that take their own outputs as inputs." (Bart Kosko, "Noise", 2006)

"On a linear system like a scale, the whole is equal to the sum of the parts. That’s the first key property of linearity. The second is that causes are proportional to effects. […] These two properties - the proportionality between cause and effect, and the equality of the whole to the sum of the parts - are the essence of what it means to be linear. […] The great advantage of linearity is that it allows for reductionist thinking. To solve a linear problem, we can break it down to its simplest parts, solve each part separately, and put the parts back together to get the answer." (Steven H Strogatz, "Infinite Powers: The Story of Calculus - The Most Important Discovery in Mathematics", 2019)

"[...] perhaps one of the most important features of complex systems, which is a key differentiator when comparing with chaotic systems, is the concept of emergence. Emergence 'breaks' the notion of determinism and linearity because it means that the outcome of these interactions is naturally unpredictable. In large systems, macro features often emerge in ways that cannot be traced back to any particular event or agent. Therefore, complexity theory is based on interaction, emergence and iterations." (Luis Tomé & Şuay Nilhan Açıkalın, "Complexity Theory as a New Lens in IR: System and Change" [in "Chaos, Complexity and Leadership 2017", Şefika Şule Erçetin & Nihan Potas], 2019)

"With a linear growth of errors, improving the measurements could always keep pace with the desire for longer prediction. But when errors grow exponentially fast, a system is said to have sensitive dependence on its initial conditions. Then long-term prediction becomes impossible. This is the philosophically disturbing message of chaos." (Steven H Strogatz, "Infinite Powers: The Story of Calculus - The Most Important Discovery in Mathematics", 2019)

Mathematical Experience V: Physics

"The supreme task of the physicist is to arrive at those universal elementary laws from which the cosmos can be built up by pure deduction. There is no logical path to these laws; only intuition, resting on sympathetic understanding of experience, can reach them."(Albert Einstein, "Principles of Research", 1918)

"The scene of action of reality is not a three-dimensional Euclidean space but rather a four-dimensional world, in which space and time are linked together indissolubly. However deep the chasm may be that separates the intuitive nature of space from that of time in our experience, nothing of this qualitative difference enters into the objective world which physics endeavors to crystallize out of direct experience. It is a four-dimensional continuum, which is neither 'time' nor 'space'. Only the consciousness that passes on in one portion of this world experiences the detached piece which comes to meet it and passes behind it as history, that is, as a process that is going forward in time and takes place in space." (Hermann Weyl, "Space, Time, Matter", 1922)

"It is not surprising that our language should be incapable of describing the processes occurring within the atoms, for, as has been remarked, it was invented to describe the experiences of daily life, and these consist only of processes involving exceedingly large numbers of atoms. Furthermore, it is very difficult to modify our language so that it will be able to describe these atomic processes, for words can only describe things of which we can form mental pictures, and this ability, too, is a result of daily experience. Fortunately, mathematics is not subject to this limitation, and it has been possible to invent a mathematical scheme - the quantum theory - which seems entirely adequate for the treatment of atomic processes; for visualisation, however, we must content ourselves with two incomplete analogies - the wave picture and the corpuscular picture." (Werner Heisenberg, "On Quantum Physics", 1930)

"But, despite their remoteness from sense experience, we do have something like a perception of the objects of set theory, as is seen from the fact that the axioms force themselves upon us as being true. I don't see any reason why we should have less confidence in this kind of perception, i.e., in mathematical intuition, than in sense perception, which induces us to build up physical theories and to expect that future sense perception will agree with them and, moreover, to believe that a question not decidable now has meaning and may be decided in future." (Kurt Gödel, "What is Cantor’s Continuum problem?", American Mathematical Monthly 54, 1947)

"Physics too deals with mathematical concepts; however, these concepts attain physical content only by the clear determination of their relation to the objects of experience." (Albert Einstein, "Out of My Later Years", 1950)

"We have thus assigned to pure reason and experience their places in a theoretical system of physics. The structure of the system is the work of reason: the empirical contents and their mutual relations must find their representation in the conclusions of the theory. In the possibility of such a representation lie the sole value and justification of the whole system, and especially of the concepts and fundamental principles which underlie it. Apart from that, these latter are free inventions of human intellect, which cannot be justified either by the nature of that intellect or in any other fashion a priori." (Albert Einstein, "Ideas and Opinions", 1954)

"The hardest problems we have to face do not come from philosophical questions about whether brains are machines or not. There is not the slightest reason to doubt that brains are anything other than machines with enormous numbers of parts that work in perfect accord with physical laws. As far as anyone can tell, our minds are merely complex processes. The serious problems come from our having had so little experience with machines of such complexity that we are not yet prepared to think effectively about them." (Marvin Minsky, 1986)

"What appear to be the most valuable aspects of the theoretical physics we have are the mathematical descriptions which enable us to predict events. These equations are, we would argue, the only realities we can be certain of in physics; any other ways we have of thinking about the situation are visual aids or mnemonics which make it easier for beings with our sort of macroscopic experience to use and remember the equations." (Celia Green, "The Lost Cause", 2003)

On Geometrical Figures: On Spirals

"The individuation process, as the way of development and maturation of the psyche, does not follow a straight line, nor does it always lead onwards and upwards. The course it follows is rather 'stadial', consisting of progress and regress, flux and stagnation in alternating sequence. Only when we glance back over a long stretch of the way can we notice the development. If we wish to mark out the way somehow or other, it can equally well be considered a' spiral', the same problems and motifs occurring again and again on different levels." (C West Churchman, "Theory of Experimental Inference", 1948) 

"Mathematics is more than doing calculations, more than solving equations, more than proving theorems, more than doing algebra, geometry or calculus, more than a way of thinking. Mathematics is the design of a snowflake, the curve of a palm frond, the shape of a building, the joy of a game, the frustration of a puzzle, the crest of a wave, the spiral of a spider's web. It is ancient and yet new. Mathematics is linked to so many ideas and aspects of the universe." (Theoni Pappas, "More Joy of Mathematics: Exploring Mathematics All Around You", 1986)

"One reason nature pleases us is its endless use of a few simple principles: the cube-square law; fractals; spirals; the way that waves, wheels, trig functions, and harmonic oscillators are alike; the importance of ratios between small primes; bilateral symmetry; Fibonacci series, golden sections, quantization, strange attractors, path-dependency, all the things that show up in places where you don’t expect them [...] these rules work with and against each other ceaselessly at all levels, so that out of their intrinsic simplicity comes the rich complexity of the world around us. That tension - between the simple rules that describe the world and the complex world we see - is itself both simple in execution and immensely complex in effect. Thus exactly the levels, mixtures, and relations of complexity that seem to be hardwired into the pleasure centers of the human brain - or are they, perhaps, intrinsic to intelligence and perception, pleasant to anything that can see, think, create? - are the ones found in the world around us." (John Barnes, "Mother of Storms", 1994)

"A single snapshot of the pattern. A fixed spiral is elegant in form, but it does not possess symmetry in the mathematical sense. However, a rotating spiral does possess a dynamic symmetry - as does any continuously rotating object. It is a symmetry that takes place in both space and time. Start with the spiral, let time pass. It rotates to a new position. Now rotate it backwards through the appropriate angle, and it looks exactly as it did to begin with. This is a space-time symmetry: translate time and rotate space. Indeed, the rotating spiral has lots of space-time symmetries, because every period of time corresponds to some spatial rotation." (Ian Stewart, "The Magical Maze: Seeing the world through mathematical eyes", 1997)

"[…] the symmetry group of the infinite logarithmic spiral is an infinite group, with one element for each real number . Two such transformations compose by adding the corresponding angles, so this group is isomorphic to the real numbers under addition." (Ian Stewart, "Symmetry: A Very Short Introduction", 2013)

"A limit cycle is an isolated closed trajectory. Isolated means that neighboring trajectories are not closed; they spiral either toward or away from the limit cycle. If all neighboring trajectories approach the limit cycle, we say the limit cycle is stable or attracting. Otherwise the limit cycle is unstable, or in exceptional cases, half-stable. Stable limit cycles are very important scientifically - they model systems that exhibit self-sustained oscillations. In other words, these systems oscillate even in the absence of external periodic forcing." (Steven H Strogatz, "Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering", 2015)

"Infinity is a Loch Ness Monster, capturing the imagination with its awe-inspiring size but elusive nature. Infinity is a dream, a vast fantasy world of endless time and space. Infinity is a dark forest with unexpected creatures, tangled thickets and sudden rays of light breaking through. Infinity is a loop that springs open to reveal an endless spiral." (Eugenia Cheng, "Beyond Infinity: An Expedition to the Outer Limits of Mathematics", 2017)

On Geometrical Figures: On Waves

"It is told that those who first brought out the irrationals from concealment into the open perished in a shipwreck, to a man. For the unutterable and the formless must needs be concealed. And those who uncovered and touched this image of life were instantaneously destroyed and shall remain forever exposed to the play of the eternal waves." (Proclus Lycaeus, cca 5th century)

"To Nature nothing can be added; from Nature nothing can be taken away; the sum of her energies is constant, and the utmost man can do in the pursuit of physical truth, or in the applications of physical knowledge, is to shift the constituents of the never-varying total. The law of conservation rigidly excludes both creation and annihilation. Waves may change to ripples, and ripples to waves; magnitude may be substituted for number, and number for magnitude; asteroids may aggregate to suns, suns may resolve themselves into florae and faunae, and floras and faunas melt in air: the flux of power is eternally the same. It rolls in music through the ages, and all terrestrial energy - the manifestations of life as well as the display of phenomena - are but the modulations of its rhythm." (John Tyndall, "Conclusion of Heat Considered as a Mode of Motion: Being a Course of Twelve Lectures Delivered at the Royal Institution of Great Britain in the Season of 1862", 1863)

"I hold: 1) that small portions of space are, in fact, of a nature analogous to little hills on a surface that is on the average fiat; namely, that the ordinary laws of geometry are not valid in them; 2) that this property of being curved or distorted is constantly being passed on from one portion of space to another after the manner of a wave; 3) that this variation of the curvature of space is what really happens in the phenomenon that we call the motion of matter, whether ponderable or ethereal; 4) that in the physical world nothing else takes place but this variation, subject (possibly) to the law of continuity." (William K Clifford, "On the Space Theory of Matter", [paper delivered before the Cambridge Philosophical Society, 1870)

"You cannot crown the edifice by this abstraction. The scientific imagination, which is here authoritative, demands as the origin and cause of a series of ether waves a particle of vibrating matter quite as definite, though it may be excessively minute, as that which gives origin to a musical sound. Such a particle we name an atom or a molecule. I think the imagination when focused so as to give definition without penumbral haze, is sure to realise this image at last." (John Tyndall, "The Scientific Use of the Imagination", 1870) 

"Ask your imagination if it will accept a vibrating multiple proportion - a numerical ratio in a state of oscillation? I do not think it will. You cannot crown the edifice with this abstraction. The scientific imagination, which is here authoritative, demands, as the origin and cause of a series of ether-waves, a particle of vibrating matter quite as definite, though it may be excessively minute, as that which gives origin to a musical sound. Such a particle we name an atom or a molecule. I think the intellect, when focused so as to give definition without penumbral haze, is sure to realize this image at the last." (John Tyndall, "Fragments of Science for Unscientific People", 1871)

"For thought raised on specialization the most potent objection to the possibility of a universal organizational science is precisely its universality. Is it ever possible that the same laws be applicable to the combination of astronomic worlds and those of biological cells, of living people and the waves of the ether, of scientific ideas and quanta of energy? [...] Mathematics provide a resolute and irrefutable answer: yes, it is undoubtedly possible, for such is indeed the case. Two and two homogenous separate elements amount to four such elements, be they astronomic systems or mental images, electrons or workers; numerical structures are indifferent to any element, there is no place here for specificity." (Alexander Bogdanov, "Tektology: The Universal Organizational Science" Vol. I, 1913)

"Meantime, there is no doubt a certain crudeness in the use of a complex wave function. If it were unavoidable in principle, and not merely a facilitation of the calculation, this would mean that there are in principle two wave functions, which must be used together in order to obtain information on the state of the system. [...] Our inability to give more accurate information about this is intimately connected with the fact that, in the pair of equations [considered], we have before us only the substitute - extraordinarily convenient for the calculation, to be sure - for a real wave equation of probably the fourth order, which, however, I have not succeeded in forming for the non-conservative case."(Edwin Schrödinger, "Quantisation as a Problem of Proper Values" , Annalen der Physik Vol. 81 (4), 1926)

"The solution of the difficulty is that the two mental pictures which experiment lead us to form - the one of the particles, the other of the waves - are both incomplete and have only the validity of analogies which are accurate only in limiting cases." (Werner Heisenberg,"On Quantum Mechanics", 1930)

"In his desire to consider at any cost the propagation phenomenon of the waves ψ as something real in the classical sense of the word, the author had refused to acknowledge that the whole development of the theory increasingly tended to highlight the essential complex nature of the wave function." (Edwin Schrödinger. "Mémoires sur la mécanique ondulatoire", 1933) [author‘s comment in the French translation] 

"Every object that we perceive appears in innumerable aspects. The concept of the object is the invariant of all these aspects. From this point of view, the present universally used system of concepts in which particles and waves appear simultaneously, can be completely justified. The latest research on nuclei and elementary particles has led us, however, to limits beyond which this system of concepts itself does not appear to suffice. The lesson to be learned from what I have told of the origin of quantum mechanics is that probable refinements of mathematical methods will not suffice to produce a satisfactory theory, but that somewhere in our doctrine is hidden a concept, unjustified by experience, which we must eliminate to open up the road." (Max Born, "The Statistical Interpretations of Quantum Mechanics", [Nobel lecture] 1954)

"The ultimate origin of the difficulty lies in the fact (or philosophical principle) that we are compelled to use the words of common language when we wish to describe a phenomenon, not by logical or mathematical analysis, but by a picture appealing to the imagination. Common language has grown by everyday experience and can never surpass these limits. Classical physics has restricted itself to the use of concepts of this kind; by analysing visible motions it has developed two ways of representing them by elementary processes; moving particles and waves. There is no other way of giving a pictorial description of motions - we have to apply it even in the region of atomic processes, where classical physics breaks down." (Max Born, "Atomic Physics", 1957)

"The mathematicians and physics men Have their mythology; they work alongside the truth, Never touching it; their equations are false But the things work. Or, when gross error appears, They invent new ones; they drop the theory of waves In universal ether and imagine curved space." (Robinson Jeffers," The Beginning and the End and Other Poems, The Great Wound", 1963)

"So much of science consists of things we can never see: light ‘waves’ and charged ‘particles’; magnetic ‘fields’ and gravitational ‘forces’; quantum ‘jumps’ and electron ‘orbits’. In fact, none of these phenomena is literally what we say it is. Light waves do not undulate through empty space in the same way that water waves ripple over a still pond; a field is only a mathematical description of the strength and direction of a force; an atom does not literally jump from one quantum state to another, and electrons do not really travel around the atomic nucleus in orbits. The words we use are merely metaphors." (K C Cole, "On Imagining the Unseeable", Discover Magazine, 1982)

"Mathematics is more than doing calculations, more than solving equations, more than proving theorems, more than doing algebra, geometry or calculus, more than a way of thinking. Mathematics is the design of a snowflake, the curve of a palm frond, the shape of a building, the joy of a game, the frustration of a puzzle, the crest of a wave, the spiral of a spider's web. It is ancient and yet new. Mathematics is linked to so many ideas and aspects of the universe." (Theoni Pappas, "More Joy of Mathematics: Exploring Mathematics All Around You", 1986)

"One reason nature pleases us is its endless use of a few simple principles: the cube-square law; fractals; spirals; the way that waves, wheels, trig functions, and harmonic oscillators are alike; the importance of ratios between small primes; bilateral symmetry; Fibonacci series, golden sections, quantization, strange attractors, path-dependency, all the things that show up in places where you don’t expect them [...] these rules work with and against each other ceaselessly at all levels, so that out of their intrinsic simplicity comes the rich complexity of the world around us. That tension - between the simple rules that describe the world and the complex world we see - is itself both simple in execution and immensely complex in effect. Thus exactly the levels, mixtures, and relations of complexity that seem to be hardwired into the pleasure centers of the human brain - or are they, perhaps, intrinsic to intelligence and perception, pleasant to anything that can see, think, create? - are the ones found in the world around us." (John Barnes, "Mother of Storms", 1994)

"So the ground state, or lowest energy state, of a pendulum does not have zero energy, as one might expect. Instead, even in its ground state a pendulum or any oscillating system must have a certain minimum amount of what are called zero point fluctuations. These mean that the pendulum won't necessarily be pointing straight down but will also have a probability of being found at a small angle to the vertical. Similarly, even in the vacuum or lowest energy state, the waves in the Maxwell field won't be exactly zero but can have small sizes. The higher the frequency (the number of swings per minute) of the pendulum or wave, the higher the energy of the ground state." (Stephen W Hawking, "The Universe in a Nutshell", 2001)

"In string theory one studies strings moving in a fixed classical spacetime. […] what we call a background-dependent approach. […] One of the fundamental discoveries of Einstein is that there is no fixed background. The very geometry of space and time is a dynamical system that evolves in time. The experimental observations that energy leaks from binary pulsars in the form of gravitational waves - at the rate predicted by general relativity to the […] accuracy of eleven decimal place - tell us that there is no more a fixed background of spacetime geometry than there are fixed crystal spheres holding the planets up." (Lee Smolin, "Loop Quantum Gravity", The New Humanists: Science at the Edge, 2003)

"Basis real and imaginary numbers have eternal and necessary reality. Complex numbers do not. They are temporal and contingent in the sense that for complex numbers to exist, we first have to carry out an operation: adding basis real and imaginary numbers together. Complex numbers therefore do not exist in their own right. They are constructed. They are derived. Symmetry breaking is exactly where constructed numbers come into existence. The very act of adding a sine wave to a cosine wave is the sufficient condition to create a broken symmetry: a complex number. The 'Big Bang', mathematically, is simply where a perfect array of basis sine and cosine waves start entering into linear combinations, creating a chain reaction, an 'explosion', of complex numbers - which corresponds to the “physical” universe." (Thomas Stark, "God Is Mathematics: The Proofs of the Eternal Existence of Mathematics", 2018)

"It is in fact mathematics itself that is simplest in hypothesis and also richest in phenomena (i.e. the simple source of all complexity). In ontological mathematics, all of existence comprises sinusoidal waves arranged into autonomous units called monads, and these are all that are required to explain everything." (Thomas Stark, "God Is Mathematics: The Proofs of the Eternal Existence of Mathematics", 2018)

"Because of the geometry of a circle, there’s always a quarter-cycle off set between any sine wave and the wave derived from it as its derivative, its rate of change. In this analogy, the point’s direction of travel is like its rate of change. It determines where the point will go next and hence how it changes its location. Moreover, this compass heading of the arrow itself rotates in a circular fashion at a constant speed as the point goes around the circle, so the compass heading of the arrow follows a sine-wave pattern in time. And since the compass heading is like the rate of change, voilà! The rate of change follows a sine-wave pattern too." (Steven H Strogatz, "Infinite Powers: The Story of Calculus - The Most Important Discovery in Mathematics", 2019)

On Limitations in Mathematics III

"Most scientific theories, however, are ephemeral. Exceptions will likely be found that invalidate a theory in one or more of its tenets. These can then stimulate a new round of research leading either to a more comprehensive theory or perhaps to a more restrictive (i.e., more precisely defined) theory. Nothing is ever completely finished in science; the search for better theories is endless. The interpretation of a scientific experiment should not be extended beyond the limits of the available data. In the building of theories, however, scientists propose general principles by extrapolation beyond available data. When former theories have been shown to be inadequate, scientists should be prepared to relinquish the old and embrace the new in their never-ending search for better solutions. It is unscientific, therefore, to claim to have 'proof of the truth' when all that scientific methodology can provide is evidence in support of a theory." (William D Stansfield, "The Science of Evolution", 1977)

"Needless to say that once the universal approach exists we can go from one field to another and use the results of one field to promote another field. However, we should never forget limitations of 'universal approaches'. It is hiqhly dangerous to apply such an approach, if it has worked in a certain domain, to other domains as a dogma. Using any universal approach you must again and again check whether the prepositions made are fulfilled by the objects to which these approaches are applied. Going to more and more abstractions where we must heavily rely on mathematics which, after all, is the Queen of science." (Hermann Haken, 1979)

"Probability does pervade the universe, and in this sense, the old chestnut about baseball imitating life really has validity. The statistics of streaks and slumps, properly understood, do teach an important lesson about epistemology, and life in general. The history of a species, or any natural phenomenon, that requires unbroken continuity in a world of trouble, works like a batting streak. All are games of a gambler playing with a limited stake against a house with infinite resources. The gambler must eventually go bust. His aim can only be to stick around as long as possible, to have some fun while he's at it, and, if he happens to be a moral agent as well, to worry about staying the course with honor!" (Stephen J Gould, 1991)

"To look at the development of physics since Newton is to observe a struggle to define the limits of science. Part of this process has been the intrusion of scientific methods and ideas into domains that have traditionally been the province of metaphysics or religion. In this conflict, Hawking’s phrase ‘to know the Mind of God’ is just one example of a border infringement. But by playing the God card, Hawking has cleverly fanned the flames of his own publicity, appealing directly to the popular allure of the scientist-as-priest."  (Peter Coles, "Hawking and the Mind of God", 2000)

"Why was progress in computing technology so fast compared with the lack of progress in space travel? The reason is very simple: computing technology is only now approaching scientific limits such as quantum uncertainty and the speed of light, while space technology has already run into its limits that derive from the basic principles of physics and chemistry." (Mordechai Ben-Ari, "Just a Theory: Exploring the Nature of Science", 2005)

"In mathematics, it’s the limitations of a reasoned argument with the tools you have available, and with magic it’s to use your tools and sleight of hand to bring about a certain effect without the audience knowing what you’re doing. [...] When you’re inventing a trick, it’s always possible to have an elephant walk on stage, and while the elephant is in front of you, sneak something under your coat, but that’s not a good trick. Similarly with mathematical proof, it is always possible to bring out the big guns, but then you lose elegance, or your conclusions aren’t very different from your hypotheses, and it’s not a very interesting theorem." (Persi Diaconis, 2008)

"[…] the usefulness of mathematics is by no means limited to finite objects or to those that can be represented with a computer. Mathematical concepts depending on the idea of infinity, like real numbers and differential calculus, are useful models for certain aspects of physical reality." (Alfred S Posamentier & Bernd Thaller, "Numbers: Their tales, types, and treasures", 2015) 

"The trig functions’ input consists of the sizes of angles inside right triangles. Their output consists of the ratios of the lengths of the triangles’ sides. Thus, they act as if they contained phone-directory-like groups of paired entries, one of which is an angle, and the other is a ratio of triangle-side lengths associated with the angle. That makes them very useful for figuring out the dimensions of triangles based on limited information." (David Stipp, "A Most Elegant Equation: Euler's Formula and the Beauty of Mathematics", 2017)

On Limitations in Mathematics I

"Things [...] are some of them continuous [...] which are properly and peculiarly called 'magnitudes'; others are discontinuous, in a side-by-side arrangement, and, as it were, in heaps, which are called 'multitudes,' a flock, for instance, a people, a heap, a chorus, and the like. Wisdom, then, must be considered to be the knowledge of these two forms. Since, however, all multitude and magnitude are by their own nature of necessity infinite - for multitude starts from a definite root and never ceases increasing; and magnitude, when division beginning with a limited whole is carried on, cannot bring the dividing process to an end [...] and since sciences are always sciences of limited things, and never of infinites, it is accordingly evident that a science dealing with magnitude [...] or with multitude [...] could never be formulated. […] A science, however, would arise to deal with something separated from each of them, with quantity, set off from multitude, and size, set off from magnitude." (Nicomachus, cca. 100 AD) 

"Even as the finite encloses an infinite series And in the unlimited limits appear, So the soul of immensity dwells in minutia And in narrowest limits no limit in here. What joy to discern the minute in infinity! The vast to perceive in the small, what divinity!"  (Jacques Bernoulli, "Ars Conjectandi", 1713)

"The great truths with which it [mathematics] deals, are clothed with austere grandeur, far above all purposes of immediate convenience or profit. It is in them that our limited understandings approach nearest to the conception of that   absolute and infinite, towards which in most other things they aspire in vain. In the pure mathematics we contemplate absolute truths, which existed in the divine mind before the morning stars sang together, and which will continue to exist there, when the last of their radiant host shall have fallen from heaven." (Edward Everett, "Orations and Speeches" Vol. 8, 1870)

"It is difficult to understand why statisticians commonly limit their inquiries to Averages, and do not revel in more comprehensive views. Their souls seem as dull to the charm of variety as that of the native of one of our flat English counties, whose retrospect of Switzerland was that, if its mountains could be thrown into its lakes, two nuisances would be got rid of at once. An Average is but a solitary fact, whereas if a single other fact be added to it, an entire Normal Scheme, which nearly corresponds to the observed one, starts potentially into existence." (Sir Francis Galton, "Natural Inheritance", 1889)

"Scientific facts are of little value in themselves. Their significance is their bearing upon other facts, enabling us to generalize and so to discover principles, just as the accurate measurement of the position of a star may be without value in itself, but in relation to other similar measurement of other stars may become the means of discovering their proper motions. We refine our instruments; we render more trustworthy our means of observation we extend our range of experimental inquiry, and thus lay the foundation for the future work, with the full knowledge that, although our researches can not extend beyond certain limits, the field itself is, even within those limits, inexhaustible." (Elihu Thompson, "The Field of Experimental Research", 1901)

"In the study of Nature conjecture must be entirely put aside, and vague hypothesis carefully guarded against. The study of Nature begins with facts, ascends to laws, and raises itself, as far as the limits of man’s intellect will permit, to the knowledge of causes, by the threefold means of observation, experiment and logical deduction." (Jean Baptiste-Andre Dumas)

"The mathematician is entirely free, within the limits of his imagination, to construct what worlds he pleases. What he is to imagine is a matter for his own caprice; he is not thereby discovering the fundamental principles of the universe nor becoming acquainted with the ideas of God." (John W N Sullivan)

On Limitations in Mathematics II

"The true mathematician is always a great deal of an artist, an architect, yes, of a poet. Beyond the real world, though perceptibly connected with it, mathematicians have created an ideal world which they attempt to develop into the most perfect of all worlds, and which is being explored in every direction. None has the faintest conception of this world except him who knows it; only presumptuous ignorance can assert that the mathematician moves in a narrow circle. The truth which he seeks is, to be sure, broadly considered, neither more nor less than consistency; but does not his mastership show, indeed, in this very limitation? To solve questions of this kind he passes unenviously over others." (Alfred Pringsheim, Jaresberichte der Deutschen Mathematiker Vereinigung Vol 13, 1904)

"No one can predict how far we shall be enabled by means of our limited intelligence to penetrate into the mysteries of a universe immeasurably vast and wonderful; nevertheless, each step in advance is certain to bring new blessings to humanity and new inspiration to greater endeavor." (Theodore W Richards, "The Fundamental Properties of the Elements", [Faraday lecture] 1911)

"Mathematics alone make us feel the limits of our intelligence. For we can always suppose in the case of an experiment that it is inexplicable because we don’t happen to have all the data. In mathematics we have all the data […] and yet we don’t understand. We always come back to the contemplation of our human wretchedness. What force is in relation to our will, the impenetrable opacity of mathematics is in relation to our intelligence." (Simone Weil, "The Notebooks of Simone Weil" Vol. 2, 1935)

"It is to be hoped that in the future more and more theoretical physicists will command a deep knowledge of mathematical principles; and also that mathematicians will no longer limit themselves so exclusively to the aesthetic development of mathematical abstractions." (George D Birkhoff, "Mathematical Nature of Physical Theories" American Scientific Vol. 31 (4), 1943)

"The ultimate origin of the difficulty lies in the fact (or philosophical principle) that we are compelled to use the words of common language when we wish to describe a phenomenon, not by logical or mathematical analysis, but by a picture appealing to the imagination. Common language has grown by everyday experience and can never surpass these limits. Classical physics has restricted itself to the use of concepts of this kind; by analysing visible motions it has developed two ways of representing them by elementary processes; moving particles and waves. There is no other way of giving a pictorial description of motions - we have to apply it even in the region of atomic processes, where classical physics breaks down." (Max Born, "Atomic Physics", 1957)

"We are terribly clever people, we moderns: we bend Nature to our will in countless ways. We move mountains, we make caves, fly at speeds no other organism can achieve and tap the power of the atom. We are terribly clever. The essentially religious feeling of subserviency to a power greater than ourselves comes hard to us clever people. But by our intelligence we are now beginning to make out the limits of our cleverness, the impotence principles that say what can and cannot be. In an operational sense, we are experiencing a return to a religious orientation toward the world." (Garrett Hardin, "Nature and Man’s Fate", 1959)

"The future offers very little hope for those who expect that our new mechanical slaves will offer us a world in which we may rest from thinking. Help us they may, but at the cost of supreme demands upon our honesty and intelligence. The world of the future will be an ever more demanding struggle against the limitations of our intelligence, not a comfortable hammock in which we can lay down to be waited upon by our robot slaves." (Norbert Wiener, "God and Golem, Inc.: A Comment on Certain Points Where Cybernetics Impinges on Religion", 1964)

"Our epoch is the epoch of increasing consciousness; in this field Mathematics has done its bit. It has made us conscious of the limits of its own capabilities." (Rózsa Péter, "Playing with Infinity: Mathematical Explorations and Excursions", 1976)