On Problem Solving XVIII: Practice

"The insights gained and garnered by the mind in its wanderings among basic concepts are benefits that theory can provide. Theory cannot equip the mind with formulas for solving problems, nor can it mark the narrow path on which the sole solution is supposed to lie by planting a hedge of principles on either side. But it can give the mind insight into the great mass of phenomena and of their relationships, then leave it free to rise into the higher realms of action." (Carl von Clausewitz, "On War", 1832)

"One of the most important tasks of the teacher is to help his students. This task is not quite easy; it demands time, practice, devotion, and sound principles." (George Pólya, "How to Solve It", 1945)

"We acquire any practical skill by imitation and practice. […] Trying to solve problems, you have to observe and to imitate what other people do when solving problems and, finally, you learn to do problems by doing them." (George Pólya, "How to Solve It", 1945)

"The trouble with mathematics is that, however diligent you are in going to lectures and in learning the theory, you may still not be able to do the problems." (Maurice Wilkes, "Memoirs of a Computer Pioneer", 1985)

"Design thinking taps into capacities we all have but that are overlooked by more conventional problem-solving practices. It is not only human-centered; it is deeply human in and of itself. Design thinking relies on our ability to be intuitive, to recognize patterns, to construct ideas that have emotional meaning as well as functionality, to express ourselves in media other than words or symbols." (Tim Brown, "Change by Design: How Design Thinking Transforms Organizations and Inspires Innovation", 2009)

"Solving problems is a practical skill like, let us say, swimming. We acquire any practical skill by imitation and practice." (George Pólya)

Previous <<||>> Next

On Problem Solving XX: Life

"The difficult problems in life always start off being simple. Great affairs always start off being small." (Lao Tzu, cca 400 BC)

"Man was not born to solve the problems of the universe, but rather to seek to lay bare the heart of the problem and then confine himself within the limits of what is amenable to understanding." (Johann Wolfgang von Goethe, 1825)

"The greatest art, both in teaching and in life itself, consists in transforming the problem into a postulate." (Johann Wolfgang von Goethe, 1928)

"The meaning and design of a problem seem not to lie in its solution, but in our working at it incessantly." (Carl G Jung, "Modern Man in Search of a Soul", 1933)

"The field of consciousness is tiny. It accepts only one problem at a time. Get into a fist fight, put your mind on the strategy of the fight, and you will not feel the other fellow's punches." (Antoine de Saint-Exupéry, "Flight to Arras", 1942)

"Life ultimately means taking the responsibility to find the right answer to its problems and to fulfill the tasks which it constantly sets for each individual." (Viktor E Frankl, "Man's Search for Meaning", 1946)

"We are built to conquer environment, solve problems, achieve goals, and we find no real satisfaction or happiness in life without obstacles to conquer and goals to achieve." (Maxwell Maltz, "Psycho-Cybernetics", 1960)

"The mystery of life isn't a problem to solve, but a reality to experience." (Frank Herbert, "Dune", 1965)

"Sometimes the situation is only a problem because it is looked at in a certain way. Looked at in another way, the right course of action may be so obvious that the problem no longer exists." (Edward de Bono, "The use of lateral thinking", 1967)

"There are problems in this universe for which there are no answers." (Frank Herbert, "Dune Messiah", 1969)

"The easiest way to solve a problem is to deny it exists." (Isaac Asimov, "The Gods Themselves", 1972)

"When a decision is made to cope with the symptoms of a problem, it is generally assumed that the corrective measures will solve the problem itself. They seldom do." (Masanobu Fukuoka, "The One-Straw Revolution", 1975)

"If you go through the world looking for excellence, you will find excellence. If you go through the world looking for problems you will find problems." (Joseph O'Connor & John Seymour, "Introducing Neuro-Linguistic Programming: Psychological Skills for Understanding and Influencing People", 1990)

"Every problem has a solution, although it may not be the outcome that was originally hoped for or expected."  (Alice Hoffman, "Practical Magic", 1995)

"Pain is a relatively objective, physical phenomenon; suffering is our psychological resistance to what happens. Events may create physical pain, but they do not in themselves create suffering. Resistance creates suffering. Stress happens when your mind resists what is... The only problem in your life is your mind's resistance to life as it unfolds." (Dan Millman, "Everyday Enlightenment: The Twelve Gateways to Personal Growth", 1998)

"[...] all problems can be reperceived as challenges, or 'opportunities' to change, grow or learn." (Robert B Dilts, "Sleight of Mouth: The Magic of Conversational Belief Change", 1999)

"All problems are illusions of the mind." (Eckhart Tolle, "Practicing the Power of Now: Essential Teachings, Meditations, and Exercises", 2001)

"We humans have two great problems: the first is knowing when to begin; the second is knowing when to stop." (Paulo Coelho, "The Zahir: A Novel of Obsession", 2005)

"Most problems we face in life, as I have said already, happen in our minds. Furthermore, problems generally exist in our concept of the past and the future. The past and the future don’t exist except in our minds." (Richard Bandler, "Get the Life You Want: The Secrets to Quick and Lasting Life Change with Neuro-Linguistic Programming", 2008)

"One of the most important aspects of what human beings do is build beliefs. Beliefs are what trap most people in their problems. Unless you believe you can get over something, get through something, or get to something, there is little likelihood you will be able to do it. Your beliefs refer to your sense of certainty on some of your thoughts." (Richard Bandler, "Get the Life You Want: The Secrets to Quick and Lasting Life Change with Neuro-Linguistic Programming", 2008)

"When you can take on board new, positive suggestions and disbelieve the old, limiting suggestions, you will be ready to tackle the rest of your problems, especially your fears." (Richard Bandler, "Get the Life You Want: The Secrets to Quick and Lasting Life Change with Neuro-Linguistic Programming", 2008)

"Our most important problems cannot be solved; they must be outgrown." (Wayne Dyer, "Excuses Begone!: How to Change Lifelong, Self-Defeating Thinking Habits", 2009)

"A problem is a difference between things as desired and things as perceived. […] Seen in this way, the problem could be solved either by changing desires or changing perceptions." (Donald C Gause & Gerald M Weinberg, "Are Your Lights On?", 2011)

"As a practical matter, it is impossible to define natural, day-to-day problems in a single, unique, totally unambiguous fashion. On the other hand, without some common understanding of the problem, a solution will almost invariably be to the wrong problem." (Donald C Gause & Gerald M Weinberg, "Are Your Lights On?", 2011)

"The really important thing in dealing with problems is to know that the question is never answered, but that it doesn't matter, as long as you keep asking. It's only when you fool yourself into thinking you have the final problem definition - the final, true answer - that you can be fooled into thinking you have the final solution. And if you think that, you're always wrong, because there is no such thing as a 'final solution'." (Donald C Gause & Gerald M Weinberg, "Are Your Lights On?", 2011)

On Problem Solving XII: Theories

"The insights gained and garnered by the mind in its wanderings among basic concepts are benefits that theory can provide. Theory cannot equip the mind with formulas for solving problems, nor can it mark the narrow path on which the sole solution is supposed to lie by planting a hedge of principles on either side. But it can give the mind insight into the great mass of phenomena and of their relationships, then leave it free to rise into the higher realms of action." (Carl von Clausewitz, "On War", 1832)

"No mathematical idea has ever been published in the way it was discovered. Techniques have been developed and are used, if a problem has been solved, to turn the solution procedure upside down, or if it is a larger complex of statements and theories, to turn definitions into propositions, and propositions into definitions, the hot invention into icy beauty. This then if it has affected teaching matter, is the didactical inversion, which as it happens may be anti-didactical." (Hans Freudenthal, "The Concept and the Role of the Model in Mathematics and Natural and Social Sciences", 1961)

"The final test of a theory is its capacity to solve the problems which originated it." (George Dantzig, "Linear Programming and Extensions", 1963)

"If we deal with our problem not knowing, or pretending not to know the general theory encompassing the concrete case before us, if we tackle the problem 'with bare hands', we have a better chance to understand the scientist's attitude in general, and especially the task of the applied mathematician." (George Pólya, "Mathematical Methods in Science", 1977)

"[...] two related deficiencies have prevented real progress in understanding insight and its role in problem solving. First, we do not yet have a system of classifying problems into those in which insight occurs versus those in which it does not. However, only if we can isolate problems in which insight occurs will we be able to set on a firm base our theories of the mechanisms underlying insight. Second, formulation of such a taxonomic system requires that we agree on a definition of insight." (Robert W Weisberg, "Prolegomena to theories of insight in problem solving: a taxonomy of problems", 1995)

"Mathematics is not a matter of 'anything goes', and every mathematician is guided by explicit or unspoken assumptions as to what counts as legitimate – whether we choose to view these assumptions as the product of birth, experience, indoctrination, tradition, or philosophy. At the same time, mathematicians are primarily problem solvers and theory builders, and answer first and foremost to the internal exigencies of their subject." (Jeremy Avigad, "Methodology and Metaphysics in the Development of Dedekind’s Theory of Ideals", 2006)

"Therefore, although the notion of insight as a distinct process has a long history in the psychological study of problem solving, it might be useful at this point to refrain from using analytic and insight as theoretical terms applied a priori to problems." (Jason M Chein et al, "Working memory and insight in the nine-dot problem", Memory & Cognition 38, 2010)

Previous <<||>> Next

On Problem Solving XIX: Insight

"The insights gained and garnered by the mind in its wanderings among basic concepts are benefits that theory can provide. Theory cannot equip the mind with formulas for solving problems, nor can it mark the narrow path on which the sole solution is supposed to lie by planting a hedge of principles on either side. But it can give the mind insight into the great mass of phenomena and of their relationships, then leave it free to rise into the higher realms of action." (Carl von Clausewitz, "On War", 1832)

"Some problems are just too complicated for rational logical solutions. They admit of insights, not answers." (Jerome B Wiesner, The New Yorker, 1963)

"[...] two related deficiencies have prevented real progress in understanding insight and its role in problem solving. First, we do not yet have a system of classifying problems into those in which insight occurs versus those in which it does not. However, only if we can isolate problems in which insight occurs will we be able to set on a firm base our theories of the mechanisms underlying insight. Second, formulation of such a taxonomic system requires that we agree on a definition of insight." (Robert W Weisberg, "Prolegomena to theories of insight in problem solving: a taxonomy of problems", 1995)

"Therefore, although the notion of insight as a distinct process has a long history in the psychological study of problem solving, it might be useful at this point to refrain from using analytic and insight as theoretical terms applied a priori to problems." (Jason M Chein et al, "Working memory and insight in the nine-dot problem", Memory & Cognition 38, 2010)

"A feeling of insight is a kind of 'Aha!' characterized by suddenness and obviousness (and often relief!) - like a revelation. You are relatively confident that your solution is correct without having to check it. In contrast, you experienced no Aha! if the solution occurs to you slowly and stepwise. As an example, imagine a light bulb that is switched on all at once in contrast to slowly dimming it up." (Amory H Danek et al, "Working wonders? Investigating insight with magic tricks", Cognition 130, 2014)

"If insight involves an abrupt change from a state of not knowing how to solve the problem to a state of knowing [...], with no conscious awareness of what caused the change, the person experiencing the insight can still be expected to be able to report that the change occurred." (Edward Bowden & Kristin Grunewald, "Whose insight is it anyway?", [in "Insight], 2018)

"In the problem-solving literature, the term 'insight' is used to designate the clear and sudden understanding of how to solve a problem. Insight is thought to arise as the result of a solver breaking free of unwarranted assumptions, or forming novel, task-related connections between existing concepts or skills. Although it would be preferable to have a single, clear, universally accepted definition of insight, this may not be possible – and almost certainly will not be possible until we start to assess on a regular basis the single feature that seems to be widely acknowledged as distinguishing insight solutions from other solutions: the 'Aha!'" (Edward Bowden & Kristin Grunewald, "Whose insight is it anyway?", [in "Insight], 2018)

"If insight involves an abrupt change from a state of not knowing how to solve the problem to a state of knowing how to solve the problem (or, in some cases, knowing the solution), with no conscious awareness of what caused the change, the person experiencing the insight can still be expected to be able to report that the change occurred." (Edward Bowden & Kristin Grunewald, "Whose insight is it anyway?", [in "Insight], 2018)

Previous <<||>> Next

On Problem Solving XVII: Heuristics

"Devising the plan of the solution, we should not be too afraid of merely plausible, heuristic reasoning. Anything is right that leads to the right idea. But we have to change this standpoint when we start carrying out the plan and then we should accept only conclusive, strict arguments." (George Pólya, "How to solve it", 1945)

"Heuristic reasoning is reasoning not regarded as final and strict but as provisional and plausible only, whose purpose is to discover the solution of the present problem. We are often obliged to use heuristic reasoning. We shall attain complete certainty when we shall have obtained the complete solution, but before obtaining certainty we must often be satisfied with a more or less plausible guess. We may need the provisional before we attain the final. We need heuristic reasoning when we construct a strict proof as we need scaffolding when we erect a building." (George Pólya, "How to solve it", 1945)

"Fast and frugal heuristics employ a minimum of time, knowledge, and computation to make adaptive choices in real environments. They can be used to solve problems of sequential search through objects or options, as in satisficing. They can also be used to make choices between simultaneously available objects, where the search for information (in the form of cues, features, consequences, etc.) about the possible options must be limited, rather than the search for the options themselves. Fast and frugal heuristics limit their search of objects or information using easily computable stopping rules, and they make their choices with easily computable decision rules." (Gerd Gigerenzer & Peter M Todd, "Fast and Frugal Heuristics: The Adaptive Toolbox" [in "Simple Heuristics That Make Us Smart"], 1999)

"Heuristics are rules of thumb that help constrain the problem in certain ways (in other words they help you to avoid falling back on blind trial and error), but they don't guarantee that you will find a solution. Heuristics are often contrasted with algorithms that will guarantee that you find a solution - it may take forever, but if the problem is algorithmic you will get there. However, heuristics are also algorithms." (S Ian Robertson, "Problem Solving", 2001)

"Heuristics are needed in situations where the world does not permit optimization. For many real-world problems (as opposed to optimization-tuned textbook problems), optimal solutions are unknown because the problems are computationally intractable or poorly defined." (Christoph Engel & Gerd Gigerenzer, "Law and Heuristics: An interdisciplinary venture" [in "Heuristics and the Law", 2006)

"Less apparent is that qualitatively different problem-solving techniques are required at high levels of complexity than at low ones. Purely analytical techniques, powerful for the lower levels, can be overwhelmed at the higher ones. At higher levels, architecting methods, experience-based heuristics, abstraction, and integrated modeling must be called into play."  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

Previous <<||>> Next

Donald C Gause - Collected Quotes

"A problem is a difference between things as desired and things as perceived. […] Seen in this way, the problem could be solved either by changing desires or changing perceptions." (Donald C Gause & Gerald M Weinberg, "Are Your Lights On?", 2011)

"As a practical matter, it is impossible to define natural, day-to-day problems in a single, unique, totally unambiguous fashion. On the other hand, without some common understanding of the problem, a solution will almost invariably be to the wrong problem." (Donald C Gause & Gerald M Weinberg, "Are Your Lights On?", 2011)

"Don't mistake a solution method for a problem definition - especially if it’s your own solution method." (Donald C Gause & Gerald M Weinberg, "Are Your Lights On?", 2011)

"Each solution is the source of next problem. We never get rid of problems. Problems, solutions, and new problems weave an endless chain. The best we can hope for is that the problems we substitute are less troublesome than the ones we 'solve'." (Donald C Gause & Gerald M Weinberg, "Are Your Lights On?", 2011)

"If you can’t think of at least three things that might be wrong with your understanding of the problem, you don’t understand the problem." (Donald C Gause & Gerald M Weinberg, "Are Your Lights On?", 2011)

"Much of what has traditionally been called 'problem-solving' is, in reality, puzzle solving. A puzzle is difficult by design - but that difficulty implies a designer." (Donald C Gause & Gerald M Weinberg, "Are Your Lights On?", 2011)

"The fledgling problem solver invariably rushes in with solutions before taking time to define the problem being solved. Even experienced solvers, when subjected to social pressure, yield to this demand for haste. When they do, many solutions are found, but not necessarily to the problem at hand." (Donald C Gause & Gerald M Weinberg, "Are Your Lights On?", 2011)

"The really important thing in dealing with problems is to know that the question is never answered, but that it doesn't matter, as long as you keep asking. It's only when you fool yourself into thinking you have the final problem definition - the final, true answer - that you can be fooled into thinking you have the final solution. And if you think that, you're always wrong, because there is no such thing as a 'final solution'." (Donald C Gause & Gerald M Weinberg, "Are Your Lights On?", 2011)

"Wherever possible, we initially place the problem in the semantic level that lends us the most comfort." (Donald C Gause & Gerald M Weinberg, "Are Your Lights On?", 2011)

On Problem Solving XVI: Trivia

"The problems of different fields are much more alike than their practitioners think, much more alike than different." (John W Tukey, "Analyzing Data: Sanctification or Detective Work?", 1969)

"Design is really a special case of problem solving. One wants to bring about a desired state of affairs. Occasionally one wants to remedy some fault but more usually one wants to bring about something new. For that reason design is more open ended than problem solving. It requires more creativity. It is not so much a matter of linking up a clearly defined objective with a clearly defined starting position (as in problem solving) but more a matter of starting out from a general position in the direction of a general objective." (Edward de Bono, "Lateral Thinking: Creativity Step by Step", 1970)

"A system represents someone's solution to a problem. The system doesn't solve the problem." (John Gall, "General Systemantics: How systems work, and especially how they fail", 1975)

"[...] any hope that we are smart enough to find even transiently optimum solutions to our data analysis problems is doomed to failure, and, indeed, if taken seriously, will mislead us in the allocation of effort, thus wasting both intellectual and computational effort." (John W Tukey, "Choosing Techniques for the Analysis of Data", 1981)

"We will treat problem solving as a process of search through a state space. A problem is defined by an initial state, one or more goal states to be reached, a set of operators that can transform one state into another, and constraints that an acceptable solution must meet. Problem-solving methods are procedures for selecting an appropriate sequence of operators that will succeed in transforming the initial state into a goal state through a series of steps." (John H Holland et al, "Induction: Processes Of Inference, Learning, And Discovery", 1986)

"Delay time, the time between causes and their impacts, can highly influence systems. Yet the concept of delayed effect is often missed in our impatient society, and when it is recognized, it’s almost always underestimated. Such oversight and devaluation can lead to poor decision making as well as poor problem solving, for decisions often have consequences that don’t show up until years later. Fortunately, mind mapping, fishbone diagrams, and creativity/brainstorming tools can be quite useful here." (Stephen G Haines, "The Managers Pocket Guide to Systems Thinking & Learning", 1998)

"A problem is defined as a gap between where you are and where you want to be that is confronted with obstacles that make closing the gap difficult. It is actually the obstacles that make the gap a problem." (James P Lewis, "Project Planning, Scheduling, and Control" 3rd Ed., 2001)

"Complex problems are the problems that persist - the problems that bounce back and continue to haunt us. People often go through a series of stages in dealing with such problems - from believing they are beyond hope, to galvanizing collective efforts of many people and dollars to address the problem, to despair, retreat, and rationalization."(Yaneer Bar-Yam, "Making Things Work: Solving Complex Problems in a Complex World", 2004)

"Design thinking taps into capacities we all have but that are overlooked by more conventional problem-solving practices. It is not only human-centered; it is deeply human in and of itself. Design thinking relies on our ability to be intuitive, to recognize patterns, to construct ideas that have emotional meaning as well as functionality, to express ourselves in media other than words or symbols." (Tim Brown, "Change by Design: How Design Thinking Transforms Organizations and Inspires Innovation", 2009)

"Mostly we rely on stories to put our ideas into context and give them meaning. It should be no surprise, then, that the human capacity for storytelling plays an important role in the intrinsically human-centered approach to problem solving, design thinking." (Tim Brown, "Change by Design: How Design Thinking Transforms Organizations and Inspires Innovation", 2009)

"The basic idea behind all of these techniques is to simplify problem solving by concentrating on its essentials. Consolidate and simplify the objectives. Focus on the things with the highest impact, things that determine other things. Put to one side minor issues likely to be resolved by the resolution of major ones. Discard the nonessentials. Model (abstract) the system at as high a level as possible, then progressively reduce the level of abstraction. In short: Simplify!"  (Mark W Maier, "The Art Systems of Architecting" 3rd Ed., 2009)

"Teaching statistical thinking means giving people tools for problem solving in the real world. It should not be taught as pure mathematics. Instead of mechanically solving a dozen problems with the help of a particular formula, children and adolescents should be asked to find solutions to real-life problems. That’s what teaches them how to solve problems, and also shows that there may be more than one good answer in the first place. Equally important is encouraging curiosity, such as asking for answers to questions by doing experiments." (Gerd Gigerenzer, "Risk Savvy: How to make good decisions", 2014)

"Building a comprehensive problem-solving framework is about leveraging a structured methodology that allows you to frame problems systematically and solve problems creatively." (Pearl Zhu, "Problem Solving Master: Frame Problems Systematically and Solve Problem Creatively", 2017)

"[...] strategy is about determining the problems and opportunities in front of you, defining them properly, and shaping a course of action that will give your business the greatest advantage. Balancing problem solving with creating and exploiting new opportunities through imagination and analysis is the cornerstone of a great strategy." (Eben Hewitt, "Technology Strategy Patterns: Architecture as strategy" 2nd Ed., 2019)

"The most fruitful research grows out of practical problems."  (Ralph B Peck)

"Understand the reality of the problem first, rather than wondering what its cause may be. The first step of problem solving is to understand the existing conditions." (Kaoru Ishikawa)

Previous <<||>> Next

S Ian Robertson - Collected Quotes

"An internal model corresponds to a specific concrete situation in the external world and allows us to reason about the external situation. To do so you used information about the problem presented in the problem statement. The process of understanding, then, refers to constructing an initial mental representation of what the problem is, based on the information in the problem statement about the goal, the initial state, what you are not allowed to do, and what operator to apply, as well as your own personal past experience." (S Ian Robertson, "Problem Solving", 2001)

"Heuristics are rules of thumb that help constrain the problem in certain ways (in other words they help you to avoid falling back on blind trial and error), but they don't guarantee that you will find a solution. Heuristics are often contrasted with algorithms that will guarantee that you find a solution - it may take forever, but if the problem is algorithmic you will get there. However, heuristics are also algorithms." (S Ian Robertson, "Problem Solving", 2001)

"Lack of understanding of a problem (or a concept, or a system of relations) can lead to superficial answers to problems, as well as to blindly following a procedure." (S Ian Robertson, "Problem Solving", 2001)

"Problem solving starts off from an initial given situation or statement of a problem (known as the initial state of the problem). Based on the problem situation and your prior knowledge you have to work towards a solution. When you reach it you are in the goal state of the problem. On the way from the initial state to the goal state you pass through a number of intermediate problem states." (S Ian Robertson, "Problem Solving", 2001)

"The difficulty facing us when we have to make inferences is two-fold. First, we may build entirely the wrong mental model from the information we read or hear. […] The second difficulty facing us is that we may well build a reasonably correct initial representation of a problem, but this representation may be impoverished in some way because we have no idea what inferences are relevant […]" (S Ian Robertson, "Problem Solving", 2001)

"The process of abstracting out the common features of things in this way is known as induction. Applying what we have learned - generalising from one or a few examples to a whole range of new examples - is known as deduction. Induction is the process of moving from the particular to the general, and deduction is the process of going from the general to the particular." (S Ian Robertson, "Problem Solving", 2001)

"Thinking involves reasoning about a situation, and to do that we must have some kind of dynamic "model" of the situation in our heads. Any changes we make to this mental model of the world should ideally mirror changes in the real world." (S Ian Robertson, "Problem Solving", 2001)

"This does not mean that the representation has to be 'complete' before any problem solving can take place. If you had a 'complete' representation of a problem then you wouldn't have a problem, as you would know exactly how to get from where you are now to where you want to be. A problem only exists when it is not immediately obvious how to get from where you are now to your goal. An adequate representation should at least allow you to see what moves you can possibly make and allow you to start heading towards your goal." (S Ian Robertson, "Problem Solving", 2001)

"Understanding a problem means building some kind of representation of the problem in one's mind, based on what the situation is or what the problem statement says and on one's prior knowledge. It is then possible to reason about the problem within this mental representation. Generating a useful mental representation is therefore the most important single factor for successful problem solving." (S Ian Robertson, "Problem Solving", 2001)

"We are constantly using old knowledge in new situations. When a solver can successfully use a solution procedure used in the past to solve a target problem this is known as positive transfer. […] However, it is also the case that a procedure learned in the past can impede one's learning of a new procedure. This is known as negative transfer. In this case what you have learned prevents you from solving a new problem or at least prevents you from seeing an optimal solution." (S Ian Robertson, "Problem Solving", 2001)

"When we have difficulties solving a problem, insight into its solution can come about by restructuring the problem." (S Ian Robertson, "Problem Solving", 2001)

On Problem Solving XI: Problem Solvers

"The future mathematician should be a clever problem-solver; but to be a clever problem-solver is not enough. In due time, he should solve significant mathematical problems; and first he should find out for which kind of problems his native gift is particularly suited." (George Pólya, "How to solve it", 1945)

"The intelligent problem-solver tries first of all to understand the problem as fully and as clearly as he can. Yet understanding alone is not enough; he must concentrate upon the problem, he must desire earnestly to obtain its solution. If he cannot summon up real desire for solving the problem he would do better to leave it alone. The open secret of real success is to throw your whole personality into your problem." (George Pólya, "How to Solve It", 1945)

"The mathematical experience of the student is incomplete if he never had an opportunity to solve a problem invented by himself." (George Pólya, "How to Solve It", 1945)

"As long as we try and patiently do our best to solve the problem, although we may not get the answer we are looking for, we always get something - even if it is only the valuable experience." (Charles Kettering, "Short Stories of Science and Invention: A Collection of Radio Talks", 1954)

"An expert problem solver must be endowed with two incompatible qualities, a restless imagination and a patient pertinacity." (Howard W Eves, "In Mathematical Circles", 1969)

"Deep in the human nature there is an almost irresistible tendency to concentrate physical and mental energy on attempts at solving problems that seem to be unsolvable." (Ragnar Frisch, "From Utopian Theory to Practical Applications", [Nobel lecture] 1970)

"The definition of a problem and the action taken to solve it largely depend on the view which the individuals or groups that discovered the problem have of the system to which it refers. A problem may thus find itself defined as a badly interpreted output, or as a faulty output of a faulty output device, or as a faulty output due to a malfunction in an otherwise faultless system, or as a correct but undesired output from a faultless and thus undesirable system. All definitions but the last suggest corrective action; only the last definition suggests change, and so presents an unsolvable problem to anyone opposed to change." (Herbert Brün, "Technology and the Composer", 1971)

"As our ability to solve problems expands. the scale of the problems attacked themselves seems to expand at a similar rate. As a result there always exist over the horizon new categories of problems of greater size to tackle." (David M Himmelbau, "Decomposition Methods", 1977) 

"The problem solver needs to stand back and examine problem contexts in the light of different 'Ws' (Weltanschauungen). Perhaps he can then decide which 'W' seems to capture the essence of the particular problem context he is faced with. This whole process needs formalizing if it is to be carried out successfully. The problem solver needs to be aware of different paradigms in the social sciences, and he must be prepared to view the problem context through each of these paradigms." (Michael C Jackson, "Towards a System of Systems Methodologies", 1984)

"A problem exists when there is a discrepancy between an initial state and a goal state, and there is no ready-made solution for the problem solver." (John D Bransford & Barry S Stein, "A Guide for Improving Thinking, Learning, and Creativity" 2nd Ed., 1993)

"Our ability to solve problems is not simply equivalent to a set of general problem-solving skills. One implication of this conclusion is that the same individual may be both good and poor at problem solving, depending on the nature of the problem." (John D Bransford & Barry S Stein, "A Guide for Improving Thinking, Learning, and Creativity" 2nd Ed., 1993)

"The term mental model refers to knowledge structures utilized in the solving of problems. Mental models are causal and thus may be functionally defined in the sense that they allow a problem solver to engage in description, explanation, and prediction. Mental models may also be defined in a structural sense as consisting of objects, states that those objects exist in, and processes that are responsible for those objects’ changing states." (Robert Hafner & Jim Stewart, "Revising Explanatory Models to Accommodate Anomalous Genetic Phenomena: Problem Solving in the ‘Context of Discovery’", Science Education 79 (2), 1995)

"Often a successful problem-solver is one who creates a new context in which to view the problem. This can often be done by directing one's attention away from the distracting details of the difficulty. From a detached perspective, we may examine the situation in a new or different light and, after exploring information and options, choose an appropriate course of action." (John Templeton, "Wisdom From World Religions: Pathways Toward Heaven on Earth", 2002)

"We are constantly using old knowledge in new situations. When a solver can successfully use a solution procedure used in the past to solve a target problem this is known as positive transfer. […] However, it is also the case that a procedure learned in the past can impede one's learning of a new procedure. This is known as negative transfer. In this case what you have learned prevents you from solving a new problem or at least prevents you from seeing an optimal solution." (S Ian Robertson, "Problem Solving", 2001)

"Mathematics is not a matter of 'anything goes', and every mathematician is guided by explicit or unspoken assumptions as to what counts as legitimate – whether we choose to view these assumptions as the product of birth, experience, indoctrination, tradition, or philosophy. At the same time, mathematicians are primarily problem solvers and theory builders, and answer first and foremost to the internal exigencies of their subject." (Jeremy Avigad, "Methodology and Metaphysics in the Development of Dedekind’s Theory of Ideals", 2006)

"A good problem solver must also be a conceptual mathematician, with a good intuitive grasp of structures. But structures remain tools for the problem solver, instead of the main object of study." (David Ruelle, "The Mathematician's Brain", 2007)

"The fledgling problem solver invariably rushes in with solutions before taking time to define the problem being solved. Even experienced solvers, when subjected to social pressure, yield to this demand for haste. When they do, many solutions are found, but not necessarily to the problem at hand." (Donald C Gause & Gerald M Weinberg, "Are Your Lights On?", 2011)

"Under normal conditions the research scientist is not an innovator but a solver of puzzles, and the puzzles upon which he concentrates are just those which he believes can be both stated and solved within the existing scientific tradition." (Thomas S Kuhn, "The Essential Tension: Selected Studies in Scientific Tradition and Change", 2011)

"Calculus succeeds by breaking complicated problems down into simpler parts. That strategy, of course, is not unique to calculus. All good problem-solvers know that hard problems become easier when they’re split into chunks. The truly radical and distinctive move of calculus is that it takes this divide-and-conquer strategy to its utmost extreme - all the way out to infinity." (Steven H Strogatz, "Infinite Powers: The Story of Calculus - The Most Important Discovery in Mathematics", 2019)

"Diverse groups of problem solvers outperformed the groups of the best individuals at solving complex problems. The reason: the diverse groups got stuck less often than the smart individuals, who tended to think similarly." (Scott E Page)

Previous <<||>> Next

On Problem Solving XIV: Representation I

"This diagrammatic method has, however, serious inconveniences as a method for solving logical problems. It does not show how the data are exhibited by cancelling certain constituents, nor does it show how to combine the remaining constituents so as to obtain the consequences sought. In short, it serves only to exhibit one single step in the argument, namely the equation of the problem; it dispenses neither with the previous steps, i.e., 'throwing of the problem into an equation' and the transformation of the premises, nor with the subsequent steps, i.e., the combinations that lead to the various consequences. Hence it is of very little use, inasmuch as the constituents can be represented by algebraic symbols quite as well as by plane regions, and are much easier to deal with in this form." (Louis Couturat, "The Algebra of Logic", 1914)

"Graphics is the visual means of resolving logical problems." (Jacques Bertin, "Graphics and Graphic Information Processing", 2011)

"A mental model is a knowledge structure that incorporates both declarative knowledge (e.g., device models) and procedural knowledge (e.g., procedures for determining distributions of voltages within a circuit), and a control structure that determines how the procedural and declarative knowledge are used in solving problems (e.g., mentally simulating the behavior of a circuit)." (Barbara Y White & John R Frederiksen, "Causal Model Progressions as a Foundation for Intelligent Learning Environments", Artificial Intelligence 42, 1990)

"An important symptom of an emerging understanding is the capacity to represent a problem in a number of different ways and to approach its solution from varied vantage points; a single, rigid representation is unlikely to suffice." (Howard Gardner, “The Unschooled Mind”, 1991)

"The term mental model refers to knowledge structures utilized in the solving of problems. Mental models are causal and thus may be functionally defined in the sense that they allow a problem solver to engage in description, explanation, and prediction. Mental models may also be defined in a structural sense as consisting of objects, states that those objects exist in, and processes that are responsible for those objects’ changing states." (Robert Hafner & Jim Stewart, "Revising Explanatory Models to Accommodate Anomalous Genetic Phenomena: Problem Solving in the ‘Context of Discovery’", Science Education 79 (2), 1995)

"The purpose of a conceptual model is to provide a vocabulary of terms and concepts that can be used to describe problems and/or solutions of design. It is not the purpose of a model to address specific problems, and even less to propose solutions for them. Drawing an analogy with linguistics, a conceptual model is analogous to a language, while design patterns are analogous to rhetorical figures, which are predefined templates of language usages, suited particularly to specific problems." (Peter P Chen [Ed.], "Advances in Conceptual Modeling", 1999)

"What it means for a mental model to be a structural analog is that it embodies a representation of the spatial and temporal relations among, and the causal structures connecting the events and entities depicted and whatever other information that is relevant to the problem-solving talks. […] The essential points are that a mental model can be nonlinguistic in form and the mental mechanisms are such that they can satisfy the model-building and simulative constraints necessary for the activity of mental modeling." (Nancy J Nersessian, "Model-based reasoning in conceptual change", 1999)

"A model is an imitation of reality and a mathematical model is a particular form of representation. We should never forget this and get so distracted by the model that we forget the real application which is driving the modelling. In the process of model building we are translating our real world problem into an equivalent mathematical problem which we solve and then attempt to interpret. We do this to gain insight into the original real world situation or to use the model for control, optimization or possibly safety studies." (Ian T Cameron & Katalin Hangos, "Process Modelling and Model Analysis", 2001)

"Understanding a problem means building some kind of representation of the problem in one's mind, based on what the situation is or what the problem statement says and on one's prior knowledge. It is then possible to reason about the problem within this mental representation. Generating a useful mental representation is therefore the most important single factor for successful problem solving." (S Ian Robertson, "Problem Solving", 2001)

"The key role of representation in thinking is often downplayed because of an ideal of rationality that dictates that whenever two statements are mathematically or logically the same, representing them in different forms should not matter. Evidence that it does matter is regarded as a sign of human irrationality. This view ignores the fact that finding a good representation is an indispensable part of problem solving and that playing with different representations is a tool of creative thinking." (Gerd Gigerenzer, "Calculated Risks: How to know when numbers deceive you", 2002)

"Mathematical modeling is as much ‘art’ as ‘science’: it requires the practitioner to (i) identify a so-called ‘real world’ problem (whatever the context may be); (ii) formulate it in mathematical terms (the ‘word problem’ so beloved of undergraduates); (iii) solve the problem thus formulated (if possible; perhaps approximate solutions will suffice, especially if the complete problem is intractable); and (iv) interpret the solution in the context of the original problem." (John A Adam, "Mathematics in Nature", 2003)

"What is a mathematical model? One basic answer is that it is the formulation in mathematical terms of the assumptions and their consequences believed to underlie a particular ‘real world’ problem. The aim of mathematical modeling is the practical application of mathematics to help unravel the underlying mechanisms involved in, for example, economic, physical, biological, or other systems and processes." (John A Adam, "Mathematics in Nature", 2003)

Previous <<||>> Next

Robert L Flood - Collected Quotes

"In general, we seem to associate complexity with anything we find difficult to understand." (Robert L Flood, "Complexity: a definition by construction of a conceptual framework", Systems Research and Behavioral Science, 1987)

"A systems description of a situation is: an assembly of elements related in an organized whole. An element may be anything that is discernible by a noun or a noun phrase that all informed observers would agree exists. An element must normally be capable of behavior such that it has some significant property(ies) that may change. A relationship can be said to exist between A and B if the behavior of either is influenced by the other." (Robert L Flood & Ewart R Carson, "Dealing with Complexity: An introduction to the theory and application of systems", 1988)

"Any subject area with 'science' in its title strongly implies a distinct branch of systematic and well-formulated knowledge and the pursuit of principles for achieving this, suggesting that a science should have a clearly recorded and coherent historical development." (Robert L Flood & Ewart R Carson, "Dealing with Complexity: An introduction to the theory and application of systems", 1988)

"Cybernetics, although not ignoring formal networks, suggests that an informal communications structure will also be present such that complex conversations at a number of levels between two or more individuals exist." (Robert L Flood & Ewart R Carson, "Dealing with Complexity: An introduction to the theory and application of systems", 1988)

"In cybernetics a system is normally described as a black box whereby the whole of a system's generative mechanisms are lumped into a single transfer function (TF). This acts on an input to produce an output. To ensure that the output is monitored, so that a system may remain homeostatic (the critical variables remain within acceptable limits) or attain a new steady state (according to input decisions, say), the output of the TF is brought back into its input where the difference between the desired and actual levels is identified. This is known as feedback." (Robert L Flood & Ewart R Carson, "Dealing with Complexity: An introduction to the theory and application of systems", 1988)

"Man's attempts to control, service, and/ or design very complex situations have, however, often been fraught with disaster. A major contributory factor has been the unwitting adoption of piecemeal thinking, which sees only parts of a situation and its generative mechanisms. Additionally, it has been suggested that nonrational thinking sees only the extremes (the simple 'solutions' ) of any range of problem solutions. The net result of these factors is that situations exhibit counterintuitive behavior; outcomes of situations are rarely as we expect, but this is not an intrinsic property of situations; rather, it is largely caused by neglect of, or lack of respect being paid to, the nature and complexity of  a situation under investigation." (Robert L Flood & Ewart R Carson, "Dealing with Complexity: An introduction to the theory and application of systems", 1988)

"Negative feedback is associated with seeking defined objectives via control parameters; and positive feedback is either contained replication and growth or uncontained and unstable growth, which may lead to structural changes or death." (Robert L Flood & Ewart R Carson, "Dealing with Complexity: An introduction to the theory and application of systems", 1988)

"Nonlinear systems (the graph of at least one relationship displays some curved feature) are notoriously more difficult to comprehend than linear systems, that is, they are more complex. Consequently they are also more difficult to control. This is exemplified by the volumes of elegant mathematics that have been developed in the search for optimal control of linear systems." (Robert L Flood & Ewart R Carson, "Dealing with Complexity: An introduction to the theory and application of systems", 1988)

"Organized simplicity occurs where a small number of significant factors and a large number of insignificant factors appear initially to be complex, but on investigation display hidden simplicity." (Robert L Flood & Ewart R Carson, "Dealing with Complexity: An introduction to the theory and application of systems", 1988)

"Synergy is a term that is also used to describe the emergence of unexpected and interesting properties." (Robert L Flood & Ewart R Carson, "Dealing with Complexity: An introduction to the theory and application of systems", 1988)

"Systems thinking is a framework of thought that helps us to deal with complex things in a holistic way. The formalization of (giving an explicit, definite, and conventional form to) this thinking is what we have termed systems theory." (Robert L Flood & Ewart R Carson, "Dealing with Complexity: An introduction to the theory and application of systems", 1988)

"The activities of a system are thought of as processes occurring in a structure. Structure defines the way in which the elements are related to each other, providing the supporting framework in which the processes occur." (Robert L Flood & Ewart R Carson, "Dealing with Complexity: An introduction to the theory and application of systems", 1988)

"The concept of entropy relates to the tendency of things to move toward greater disorder, or disorganization. […] The second law of thermodynamics expresses precisely the same concept. This states that heat dissipates from a central source and the energy becomes degraded, although total energy remains constant (the first law of thermodynamics). Entropy suggests that organisms, organizations, societies, machines, and so on, will rapidly deteriorate into disorder and "death." The reason they do not is because animate things can self-organize and inanimate things may be serviced by man. These are negentropic activities which require energy. Energy, however, can be made available only by further degradation. Ultimately, therefore, entropy wins the day and the attempts to create order can seem rather a daunting task in the entropic scheme of things. Holding back entropy, however, is another of the challenging tasks for the systems scientist." (Robert L Flood & Ewart R Carson, "Dealing with Complexity: An introduction to the theory and application of systems", 1988)

"The state of development of mathematical theory in relation to some attributes of complexity is a clear measure of our ability/inability to deal with that attribute […]" (Robert L Flood & Ewart R Carson, "Dealing with Complexity: An introduction to the theory and application of systems", 1988)

"Different methodologies express different rationalities stemming from alternative theoretical positions which they reflect. These alternative positions must be respected, and methodologies and their appropriate theoretical underpinnings developed in partnership. (Robert L Flood, "Creative Problem Solving", 1991) 

"In the modern systems approach, the concept "system" is used not to refer to things in the world but to a particular way of organising our thoughts about the world." (Robert L Flood, "Creative Problem Solving", 1991)

Herbert A Simon - Collected Quotes

"All behavior involves conscious or unconscious selection of particular actions out of all those which are physically possible to the actor and to those persons over whom he exercises influence and authority." (Herbert A Simon, "Administrative Behavior: A Study of Decision-making Processes in Administrative Organization", 1947)

"Decision making processes are aimed at finding courses of action that are feasible or satisfactory in the light of multiple goals and constraints." (Herbert A Simon, "Administrative Behavior: A Study of Decision-making Processes in Administrative Organization", 1947)

"In the process of decision those alternatives are chosen which are considered to be appropriate means of reaching desired ends. Ends themselves, however, are often merely instrumental to more final objectives. We are thus led to the conception of a series, or hierarchy, of ends. Rationality has to do with the construction of means-ends chains of this kind." (Herbert A Simon, "Administrative Behavior", 1947)

"It is impossible for the behavior of a single, isolated individual to reach a high degree of rationality. The number of alternatives he must explore is so great, the information he would need to evaluate them so vast that even an approximation to objective rationality is hard to conceive. Individual choice takes place in rationality is hard to conceive. [...] Actual behavior falls short in at least three ways, of objective rationality." (Herbert A Simon, "Administrative Behavior", 1947)

"Many individuals and organization units contribute to every large decision, and the very problem of centralization and decentralization is a problem of arranging the complex system into an effective scheme." (Herbert A Simon, "Administrative Behavior: A Study of Decision-making Processes in Administrative Organization", 1947)

"Rationality requires a choice among all possible alternative behaviors. In actual behavior, only a very few of all these possible alternatives come to mind." (Herbert A Simon, "Administrative Behavior", 1947)

"Rationality requires a complete knowledge and anticipation of the consequences that will follow on each choice. In fact, knowledge of consequences is always fragmentary." (Herbert A Simon, "Administrative Behavior", 1947)

"Roughly speaking, rationality is concerned with the selection of preferred behavior alternatives in terms of some system of values, whereby the consequences of behavior can be evaluated." (Herbert A Simon, "Administrative Behavior", 1947)

"The function of knowledge in the decision-making process is to determine which consequences follow upon which of the alternative strategies. It is the task of knowledge to select from the whole class of possible consequences a more limited subclass, or even (ideally) a single set of consequences correlated with each strategy." (Herbert A Simon, "Administrative Behavior: A Study of Decision-making Processes in Administrative Organization", 1947)

"The principle of bounded rationality [is] the capacity of the human mind for formulating and solving complex problems is very small compared with the size of the problems whose solution is required for objectively rational behavior in the real world - or even for a reasonable approximation to such objective rationality." (Herbert A Simon, "Administrative Behavior", 1947)

"The first consequence of the principle of bounded rationality is that the intended rationality of an actor requires him to construct a simplified model of the real situation in order to deal with it. He behaves rationally with respect to this model, and such behavior is not even approximately optimal with respect to the real world. To predict his behavior we must understand the way in which this simplified model is constructed, and its construction will certainly be related to his psychological properties as a perceiving, thinking, and learning animal." (Herbert A Simon, "Models of Man", 1957)

"The mathematical and computing techniques for making programmed decisions replace man but they do not generally simulate him." (Herbert A Simon, "Management and Corporations 1985", 1960)

"Programs do not merely substitute brute force for human cunning. Increasingly, they imitate-and in some cases improve upon-human cunning." (Herbert A Simon, "Management and Corporations 1985", 1960)

"Roughly, by a complex system I mean one made up of a large number of parts that interact in a nonsimple way. In such systems, the whole is more than the sum of the parts, not in an ultimate, metaphysical sense, but in the important pragmatic sense that, given the properties of the parts and the laws of their interaction, it is not a trivial matter to infer the properties of the whole." (Herbert A Simon, "The Architecture of Complexity", Proceedings of the American Philosophical Society, Vol. 106 (6), 1962)

"Thus, the central theme that runs through my remarks is that complexity frequently takes the form of hierarchy, and that hierarchic systems have some common properties that are independent of their specific content. Hierarchy, I shall argue, is one of the central structural schemes that the architect of complexity uses." (Herbert A Simon, "The Architecture of Complexity", Proceedings of the American Philosophical Society Vol. 106 (6), 1962)

"A mathematical proof, as usually written down, is a sequence of expressions in the state space. But we may also think of the proof as consisting of the sequence of justifications of consecutive proof steps - i.e., the references to axioms, previously-proved theorems, and rules of inference that legitimize the writing down of the proof steps. From this point of view, the proof is a sequence of actions (applications of rules of inference) that, operating initially on the axioms, transform them into the desired theorem." (Herbert A Simon, "The Logic of Heuristic Decision Making", [in "The Logic of Decision and Action"], 1966)

"[...] a problem of design exists when (1) there is a language for naming actions and a language for naming states of the world, (2) there is a need to find an action that will produce a specified state of the world or a specified change in the state of the world, and (3) there is no non-trivial process for translating changes in the state of the world into their corresponding actions." (Herbert A Simon, "The Logic of Heuristic Decision Making", [in "The Logic of Decision and Action"], 1966)

"A problem will be difficult if there are no procedures for generating possible solutions that are guaranteed (or at least likely) to generate the actual solution rather early in the game. But for such a procedure to exist, there must be some kind of structural relation, at least approximate, between the possible solutions as named by the solution-generating process and these same solutions as named in the language of the problem statement." (Herbert A Simon, "The Logic of Heuristic Decision Making", [in "The Logic of Decision and Action"], 1966)

"An adaptive organism is connected with its environment by two kinds of channels. Afferent channels give it information about the state of the environment; efferent channels cause action on the environment. Problem statements define solutions in terms of afferent information to the organism; the organism's task is to discover a set of efferent signals which, changing the state of the environment, will produce the appropriate afferent. But, ab initio, the mapping of efferents on afferents is entirely arbitrary; the relations can only be discovered by experiment, by acting and observing the consequences of action." (Herbert A Simon, "The Logic of Heuristic Decision Making", [in "The Logic of Decision and Action"], 1966)

"Design problems - generating or discovering alternatives - are complex largely because they involve two spaces, an action space and a state space, that generally have completely different structures. To find a design requires mapping the former of these on the latter. For many, if not most, design problems in the real world systematic algorithms are not known that guarantee solutions with reasonable amounts of computing effort. Design uses a wide range of heuristic devices - like means-end analysis, satisficing, and the other procedures that have been outlined - that have been found by experience to enhance the efficiency of search. Much remains to be learned about the nature and effectiveness of these devices." (Herbert A Simon, "The Logic of Heuristic Decision Making", [in "The Logic of Decision and Action"], 1966)

"Every problem-solving effort must begin with creating a representation for the problem - a problem space in which the search for the solution can take place. Of course, for most of the problems we encounter in our daily personal or professional lives, we simply retrieve from memory a representation that we have already stored and used on previous occasions. Sometimes, we have to adapt the representation a bit to the new situation, but that is usually a rather simple matter." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"Natural science is knowledge about natural objects and phenomena." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"Learning is any change in a system that produces a more or less permanent change in its capacity for adapting to its environment. Understanding systems, especially systems capable of understanding problems in new task domains, are learning systems." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"Making discoveries belongs to the class of ill-structured problem-solving tasks that have relatively ill-defined goals." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"Solving a problem simply means representing it so as to make the solution transparent." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"The central task of a natural science is to make the wonderful commonplace: to show that complexity, correctly viewed, is only a mask for simplicity; to find pattern hidden in apparent chaos. […] This is the task of natural science: to show that the wonderful is not incomprehensible, to show how it can be comprehended - but not to destroy wonder. For when we have explained the wonderful, unmasked the hidden pattern, a new wonder arises at how complexity was woven out of simplicity. The aesthetics of natural science and mathematics is at one with the aesthetics of music and painting - both inhere in the discovery of a partially concealed pattern." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"The more we are willing to abstract from the detail of a set of phenomena, the easier it becomes to simulate the phenomena. Moreover we do not have to know, or guess at, all the internal structure of the system but only that part of it that is crucial to the abstraction." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"[...] in an information-rich world, the wealth of information means a dearth of something else: a scarcity of whatever it is that information consumes. What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention and a need to allocate that attention efficiently among the overabundance of information sources that might consume it." (Herbert Simon, "Designing Organizations for an Information-Rich World", 1971)

"But the answers provided by the theory of games are sometimes very puzzling and ambiguous. In many situations, no single course of action dominates all the others; instead, a whole set of possible solutions are all equally consistent with the postulates of rationality." (Herbert A Simon et al, "Decision Making and Problem Solving", Interfaces Vol. 17 (5), 1987)

"[...] problem solving generally proceeds by selective search through large sets of possibilities, using rules of thumb (heuristics) to guide the search. Because the possibilities in realistic problem situations are generally multitudinous, trial-and-error search would simply not work; the search must be highly selective." (Herbert A Simon et al, "Decision Making and Problem Solving", Interfaces Vol. 17 (5), 1987)

"The way in which an uncertain possibility is presented may have a substantial effect on how people respond to it." (Herbert A Simon et al, "Decision Making and Problem Solving", Interfaces Vol. 17 (5), 1987)

On Conjecture (Unsourced)

"In the study of Nature conjecture must be entirely put aside, and vague hypothesis carefully guarded against. The study of Nature begins with facts, ascends to laws, and raises itself, as far as the limits of man’s intellect will permit, to the knowledge of causes, by the threefold means of observation, experiment and logical deduction." (Jean Baptiste-Andre Dumas)

"Indeed, when in the course of a mathematical investigation we encounter a problem or conjecture a theorem, our minds will not rest until the problem is exhaustively solved and the theorem rigorously proved; or else, until we have found the reasons which made success impossible and, hence, failure unavoidable. Thus, the proofs of the impossibility of certain solutions plays a predominant role in modern mathematics; the search for an answer to such questions has often led to the discovery of newer and more fruitful fields of endeavour." (David Hilbert)

"The conjectures of the scientific intelligence are genuine creative novelties, inherently unpredictable and not determined by the character of the scientist’s physical environment. The thinking mind is not a causal mechanism." (Anthony M Quinton)

"The only use of an hypothesis is, that it should lead to experiments; that it should be a guide to facts. In this application, conjectures are always of use. The destruction of an error hardly ever takes place without the discovery of truth. [...] Hypothesis should be considered merely an intellectual instrument of discovery, which at any time may be relinquished for a better instrument. It should never be spoken of as truth; its highest praise is verisimility. Knowledge can only be acquired by the senses; nature has an archetype in the human imagination; her empire is given only to industry and action, guided and governed by experience." (Sir Humphry Davy) 

"The purpose of life is to conjecture and prove." (Paul Erdős)

"The theory of numbers, more than any other branch of mathematics, began by being an experimental science. Its most famous theorems have all been conjectured, sometimes a hundred years or more before they were proved; and they have been suggested by the evidence of a mass of computations." (Godfrey H Hardy)

"What certainty can there be in a Philosophy which consists in as many Hypotheses as there are Phaenomena to be explained. To explain all nature is too difficult a task for any one man or even for any one age. 'Tis much better to do a little with certainty, & leave the rest for others that come after you, than to explain all things by conjecture without making sure of any thing." (Sir Isaac Newton)

On Creativity (Mathematics I)

"Creativity is the heart and soul of mathematics at all levels. The collection of special skills and techniques is only the raw material out of which the subject itself grows. To look at mathematics without the creative side of it, is to look at a black-and-white photograph of a Cezanne; outlines may be there, but everything that matters is missing." (Robert C Buck, "Teaching Machines and Mathematics Programs", American Mathematical Monthly 69, 1962)

"There are, roughly speaking, two kinds of mathematical creativity. One, akin to conquering a mountain peak, consists of solving a problem which has remained unsolved for a long time and has commanded the attention of many mathematicians. The other is exploring new territory." (Mark Kac, "Enigmas Of Chance", 1985)

"Music and higher mathematics share some obvious kinship. The practice of both requires a lengthy apprenticeship, talent, and no small amount of grace. Both seem to spring from some mysterious workings of the mind. Logic and system are essential for both, and yet each can reach a height of creativity beyond the merely mechanical." (Frederick Pratter, "How Music and Math Seek Truth in Beauty", Christian Science Monitor, 1995)

"Mathematics is a fascinating discipline that calls for creativity, imagination, and the mastery of rigorous standards of proof." (John Meier & Derek Smith, "Exploring Mathematics: An Engaging Introduction to Proof", 2017)

"Math is the beautiful, rich, joyful, playful, surprising, frustrating, humbling and creative art that speaks to something transcendental. It is worthy of much exploration and examination because it is intrinsically beautiful, nothing more to say. Why play the violin? Because it is beautiful! Why engage in math? Because it too is beautiful!" (James Tanton, "Thinking Mathematics")

"Mathematics is the summit of human thinking. It has all the creativity and imagination that you can find in all kinds of art, but unlike art-charlatans and all kinds of quacks will not succeed there." (Meir Shalev)

"No discovery has been made in mathematics, or anywhere else for that matter, by an effort of deductive logic; it results from the work of creative imagination which builds what seems to be truth, guided sometimes by analogies, sometimes by an esthetic ideal, but which does not hold at all on solid logical bases. Once a discovery is made, logic intervenes to act as a control; it is logic that ultimately decides whether the discovery is really true or is illusory; its role therefore, though considerable, is only secondary." (Henri Lebesgue)

"The essential feature of mathematical creativity is the exploration, under the pressure of powerful implosive forces, of difficult problems for whose validity and importance the explorer is eventually held accountable by reality." (Alfred Adler)

On Structure: Structure in Mathematics I

"The first step, whenever a practical problem is set before a mathematician, is to form the mathematical hypothesis. It is neither needful nor practical that we should take account of the details of the structure as it will exist. We have to reason about a skeleton diagram in which bearings are reduced to points, pieces to lines, etc. and [in] which it is supposed that certain relations between motions are absolutely constrained, irrespective of forces." (Charles S Peirce, "Report on Live Loads", cca. 1895)

"For thought raised on specialization the most potent objection to the possibility of a universal organizational science is precisely its universality. Is it ever possible that the same laws be applicable to the combination of astronomic worlds and those of biological cells, of living people and the waves of the ether, of scientific ideas and quanta of energy? [...] Mathematics provide a resolute and irrefutable answer: yes, it is undoubtedly possible, for such is indeed the case. Two and two homogenous separate elements amount to four such elements, be they astronomic systems or mental images, electrons or workers; numerical structures are indifferent to any element, there is no place here for specificity." (Alexander Bogdanov, "Tektology: The Universal Organizational Science" Vol. I, 1913)

"Once a statement is cast into mathematical form it may be manipulated in accordance with [mathematical] rules and every configuration of the symbols will represent facts in harmony with and dependent on those contained in the original statement. Now this comes very close to what we conceive the action of the brain structures to be in performing intellectual acts with the symbols of ordinary language. In a sense, therefore, the mathematician has been able to perfect a device through which a part of the labor of logical thought is carried on outside the central nervous system with only that supervision which is requisite to manipulate the symbols in accordance with the rules." (Horatio B Williams, "Mathematics and the Biological Sciences", Bulletin of the American Mathematical Society Vol. 38, 1927)

"Physics is the attempt at the conceptual construction of a model of the real world and its lawful structure." (Albert Einstein, [letter to Moritz Schlick] 1931)

"Today's scientists have substituted mathematics for experiments, and they wander off through equation after equation, and eventually build a structure which has no relation to reality." (Nikola Tesla, "Radio Power Will Revolutionize the World", Modern Mechanics and Inventions, 1934)

"Men of science belong to two different types - the logical and the intuitive. Science owes its progress to both forms of minds. Mathematics, although a purely logical structure, nevertheless makes use of intuition. " (Alexis Carrel, "Man the Unknown", 1935)

"Statistics is a scientific discipline concerned with collection, analysis, and interpretation of data obtained from observation or experiment. The subject has a coherent structure based on the theory of Probability and includes many different procedures which contribute to research and development throughout the whole of Science and Technology." (Egon Pearson, 1936)

Richard E Bellman - Collected Quotes

"Computers do not decrease the need for mathematical analysis, but rather greatly increase this need. They actually extend the use of analysis into the fields of computers and computation, the former area being almost unknown until recently, the latter never having been as intensively investigated as its importance warrants. Finally, it is up to the user of computational equipment to define his needs in terms of his problems, In any case, computers can never eliminate the need for problem-solving through human ingenuity and intelligence." (Richard E Bellman & Paul Brock, "On the Concepts of a Problem and Problem-Solving", American Mathematical Monthly 67, 1960)

"There is the very real danger that a number of problems which could profitably be subjected to analysis, and so treated by simpler and more revealing techniques. will instead be routinely shunted to the computing machines [...] The role of computing machines as a mathematical tool is not that of a panacea for all computational ills." (Richard E Bellman & Paul Brock, "On the Concepts of a Problem and Problem-Solving", American Mathematical Monthly 67, 1960)

"The theory of elliptic functions is the fairyland of mathematics. The mathematician who once gazes upon this enchanting and wondrous domain crowded with the most beautiful relations and concepts is forever captivated." (Richard E Bellman, "A Brief Introduction to Theta Functions", 1961)

"A final goal of any scientific theory must be the derivation of numbers. Theories stand or fall, ultimately, upon numbers." (Richard E Bellman, "Eye of the Hurricane: An Autobiography", 1984)

"Another direction of research is fuzzy systems. This will greatly increase the use of mathematics from the inanimate to the animate. In the past, mathematics has been used for the analysis of physical systems. With fuzzy systems and computer simulation we can study many processes in the social sciences." (Richard E Bellman, "Eye of the Hurricane: An Autobiography", 1984)

"Conversely, once it was realized that the concept of policy was fundamental in control theory, the mathematicization of the basic engineering concept of 'feedback control', then the emphasis upon a state variable formulation became natural. We see then a very interesting interaction between dynamic programming and control theory. This reinforces the point that when working in the field of analysis it is exceedingly helpful to have some underlying physical processes clearly in mind." (Richard E Bellman, "Eye of the Hurricane: An Autobiography", 1984)

"Mathematical model making is an art. If the model is too small, a great deal of analysis and numerical solution can be done, but the results, in general, can be meaningless. If the model is too large, neither analysis nor numerical solution can be carried out, the interpretation of the results is in any case very difficult, and there is great difficulty in obtaining the numerical values of the parameters needed for numerical results." (Richard E Bellman, "Eye of the Hurricane: An Autobiography", 1984)

"If there are paradoxes in mathematics, think of how many paradoxes there must be in ordinary speech. The existence of paradoxes involves logic. This led naturally to the question of local logics. It seems to me that it is no more natural to expect that a universal logic will hold than that a universal geometry holds." (Richard E Bellman, "Eye of the Hurricane: An Autobiography", 1984)

"In the real world, none of these assumptions are uniformly valid. Often people want to know why mathematics and computers cannot be used to handle the meaningful problems of society, as opposed, let us say, to the moon boondoggle and high energy-high cost physics. The answer lies in the fact that we don't know how to describe the complex systems of society involving people, we don't understand cause and effect, which is to say the consequences of decisions, and we don't even know how to make our objectives reasonably precise. None of the requirements of classical science are met. Gradually, a new methodology for dealing with these 'fuzzy' problems is being developed, but the path is not easy." (Richard E Bellman, "Eye of the Hurricane: An Autobiography", 1984)

"Stability theory is the study of systems under various perturbing influences. Since there are many systems, many types of influences, and many equations describing systems, this is an open-ended problem. A system is designed so that it will be stable under external influences. However, one cannot predict all external influences, nor predict the magnitude of those that occur. Consequently, we need control theory. If one is interested in stability theory, a natural result is a theory of control." (Richard E Bellman, "Eye of the Hurricane: An Autobiography", 1984)

"The trouble with an analog computer is that one begins to construct mathematical models which can be treated using an analog computer. In many cases this is not realistic." (Richard E Bellman, "Eye of the Hurricane: An Autobiography", 1984)

"There are many difficulties in application of [the games] theory to the real world. [...] In general, competitors are not in complete opposition. As a matter of fact often they don't even have the same objectives. This difficulty can often be circumvented by using a different objective, 'games of survival'. Secondly, a decision is seldom made once. This motivated the study of multistage games [...]. Finally, decisions are not usually made simultaneously. Recognition of this fact leads to 'games of protocol' [...]. Games of protocol can also be used to handle processes involving three or more people." (Richard E Bellman, "Eye of the Hurricane: An Autobiography", 1984)

"Mathematicians makes natural questions precise." (Richard E Bellman, "Eye of the Hurricane: An Autobiography", 1984)

On Heuristics II

"Models of bounded rationality describe how a judgement or decision is reached (that is, the heuristic processes or proximal mechanisms) rather than merely the outcome of the decision, and they describe the class of environments in which these heuristics will succeed or fail." (Gerd Gigerenzer & Reinhard Selten [Eds., "Bounded Rationality: The Adaptive Toolbox", 2001)

"A second class of metaphors - mathematical algorithms, heuristics, and models - brings us closer to the world of computer science programs, simulations, and approximations of the brain and its cognitive processes." (Diego Rasskin-Gutman, "Chess Metaphors: Artificial Intelligence and the Human Mind", 2009)

"In particular, the accurate intuitions of experts are better explained by the effects of prolonged practice than by heuristics. We can now draw a richer and more balanced picture, in which skill and heuristics are alternative sources of intuitive judgments and choices." (Daniel Kahneman, "Thinking, Fast and Slow", 2011)

"This is the essence of intuitive heuristics: when faced with a difficult question, we often answer an easier one instead, usually without noticing the substitution." (Daniel Kahneman, "Thinking, Fast and Slow", 2011)

"Heuristics are an evolutionary solution to an ongoing problem: we have limited mental resources. As such, they have a very long and thoroughly time-tested history of helping us - on average - make better decisions." (Peter H Diamandis, "Abundance: The Future is Better Than You Think", 2012)

"Heuristics are simplified rules of thumb that make things simple and easy to implement. But their main advantage is that the user knows that they are not perfect, just expedient, and is therefore less fooled by their powers. They become dangerous when we forget that." (Nassim N Taleb, "Antifragile: Things that gain from disorder", 2012)

"The art of reasoned persuasion is an iterative, recursive heuristic, meaning that we must go back and forth between the facts and the rules until we have a good fit. We cannot see the facts properly until we know what framework to place them into, and we cannot determine what framework to place them into until we see the basic contours of the facts." (Joel P Trachtman, "The Tools of Argument", 2013)

"Heuristic decision making is fast and frugal and is often based on the evaluation of one or two salient bits of information." (Amitav Chakravarti, "Why People (Don’t) Buy: The Go and Stop Signals", 2015)

"A heuristic is a strategy we derive from previous experience with a similar problem." (Darius Foroux, "Think Straight", 2017)

"The social world that humans have made for themselves is so complex that the mind simplifies the world by using heuristics, customs, and habits, and by making models or assumptions about how things generally work (the ‘causal structure of the world’). And because people rely upon (and are invested in) these mental models, they usually prefer that they remain uncontested." (Dr James Brennan, "Psychological  Adjustment to Illness and Injury", West of England Medical Journal Vol. 117 (2), 2018)

On Generalization (1930-1949)

"The steady progress of physics requires for its theoretical formulation a mathematics which get continually more advanced. […] it was expected that mathematics would get more and more complicated, but would rest on a permanent basis of axioms and definitions, while actually the modern physical developments have required a mathematics that continually shifts its foundation and gets more abstract. Non-Euclidean geometry and noncommutative algebra, which were at one time were considered to be purely fictions of the mind and pastimes of logical thinkers, have now been found to be very necessary for the description of general facts of the physical world. It seems likely that this process of increasing abstraction will continue in the future and the advance in physics is to be associated with continual modification and generalisation of the axioms at the base of mathematics rather than with a logical development of any one mathematical scheme on a fixed foundation." (Paul A M Dirac, "Quantities singularities in the electromagnetic field", Proceedings of the Royal Society of London, 1931)

"It is time, therefore, to abandon the superstition that natural science cannot be regarded as logically respectable until philosophers have solved the problem of induction. The problem of induction is, roughly speaking, the problem of finding a way to prove that certain empirical generalizations which are derived from past experience will hold good also in the future." (Alfred J Ayer, "Language, Truth and Logic", 1936)

"The problem of induction is, roughly speaking, the problem of finding a way to prove that certain empirical generalizations which are derived from past experience will hold good also in the future. There are only two ways of approaching this problem on the assumption that it is a genuine problem, and it is easy to see that neither of them can lead to its solution." (Alfred J Ayer, "Language, Truth, and Logic", 1936)

"The ethos of science involves the functionally necessary demand that theories or generalizations be evaluated in [terms of] their logical consistency and consonance with facts." (Robert K Merton, "Science and the Social Order", Philosophy of Science Vol 5 (3), 1938)

"The former distrust of specialization has been supplanted by its opposite, a distrust of generalization. Not only has man become a specialist in practice, he is being taught that special facts represent the highest form of knowledge." (Richard Weaver, "Ideas have Consequences", 1948)

On Technology I

"Unlike art, science is genuinely progressive. Achievement in the fields of research and technology is cumulative; each generation begins at the point where its predecessor left off." (Aldous Huxley, "Science, Liberty and Peace", 1946)

"Doing engineering is practicing the art of the organized forcing of technological change." (George Spencer-Brown, Electronics, Vol. 32 (47),  1959)

"Science is the reduction of the bewildering diversity of unique events to manageable uniformity within one of a number of symbol systems, and technology is the art of using these symbol systems so as to control and organize unique events. Scientific observation is always a viewing of things through the refracting medium of a symbol system, and technological praxis is always handling of things in ways that some symbol system has dictated. Education in science and technology is essentially education on the symbol level." (Aldous L Huxley, "Essay", Daedalus, 1962)

"Engineering is the art of skillful approximation; the practice of gamesmanship in the highest form. In the end it is a method broad enough to tame the unknown, a means of combing disciplined judgment with intuition, courage with responsibility, and scientific competence within the practical aspects of time, of cost, and of talent. This is the exciting view of modern-day engineering that a vigorous profession can insist be the theme for education and training of its youth. It is an outlook that generates its strength and its grandeur not in the discovery of facts but in their application; not in receiving, but in giving. It is an outlook that requires many tools of science and the ability to manipulate them intelligently In the end, it is a welding of theory and practice to build an early, strong, and useful result. Except as a valuable discipline of the mind, a formal education in technology is sterile until it is applied." (Ronald B Smith, "Professional Responsibility of Engineering", Mechanical Engineering Vol. 86 (1), 1964)

"It is a commonplace of modern technology that there is a high measure of certainty that problems have solutions before there is knowledge of how they are to be solved." (John K Galbraith, "The New Industrial State", 1967)

"Technological invention and innovation are the business of engineering. They are embodied in engineering change." (Daniel V DeSimone & Hardy Cross, "Education for Innovation", 1968)

"The future masters of technology will have to be lighthearted and intelligent. The machine easily masters the grim and the dumb." (Marshall McLuhan, "Counterblast", 1969)

"It follows from this that man's most urgent and pre-emptive need is maximally to utilize cybernetic science and computer technology within a general systems framework, to build a meta-systemic reality which is now only dimly envisaged. Intelligent and purposeful application of rapidly developing telecommunications and teleprocessing technology should make possible a degree of worldwide value consensus heretofore unrealizable." (Richard F Ericson, "Visions of Cybernetic Organizations", 1972)

"The march of science and technology does not imply growing intellectual complexity in the lives of most people. It often means the opposite." (Thomas Sowell, "Knowledge And Decisions", 1980)

"A chipped pebble is almost part of the hand it never leaves. A thrown spear declares a sort of independence the moment it is released. [...] The whole trend in technology has been to devise machines that are less and less under direct control and more and more seem to have the beginning of a will of their own." (Isaac Asimov, "Past, Present, and Future", 1987)

On Technology IV

"If you think technology can solve your security problems, then you don't understand the problems and you don't understand the technology." (Bruce Schneier, "Secrets and Lies: Digital Security in a Networked World", 2000)

"Ultimately, progress in applications is not deterministic, but opportunistic, exploiting for new applications whatever new science and technology happen to be coming along." (Herbert Kroemer, "Quasi-Electric Fields and Band Offsets: Teaching Electrons New Tricks", [Nobel Lecture], 2000)

"Technology can relieve the symptoms of a problem without affecting the underlying causes. Faith in technology as the ultimate solution to all problems can thus divert our attention from the most fundamental problem - the problem of growth in a finite system - and prevent us from taking effective action to solve it." (Donella H Meadows & Dennis L Meadows, "The Limits to Growth: The 30 Year Update", 2004)

"Although the Singularity has many faces, its most important implication is this: our technology will match and then vastly exceed the refinement and suppleness of what we regard as the best of human traits."  (Ray Kurzweil, "The Singularity is Near", 2005)

"The Singularity will represent the culmination of the merger of our biological thinking and existence with our technology, resulting in a world that is still human but that transcends our biological roots. There will be no distinction, post-Singularity, between human and machine or between physical and virtual reality. If you wonder what will remain unequivocally human in such a world, it’s simply this quality: ours is the species that inherently seeks to extend its physical and mental reach beyond current limitations." (Ray Kurzweil, "The Singularity is Near", 2005)

"Chance is just as real as causation; both are modes of becoming.  The way to model a random process is to enrich the mathematical theory of probability with a model of a random mechanism. In the sciences, probabilities are never made up or 'elicited' by observing the choices people make, or the bets they are willing to place.  The reason is that, in science and technology, interpreted probability exactifies objective chance, not gut feeling or intuition. No randomness, no probability." (Mario Bunge, "Chasing Reality: Strife over Realism", 2006)

"Synergy occurs when organizational parts interact to produce a joint effect that is greater than the sum of the parts acting alone. As a result the organization may attain a special advantage with respect to cost, market power, technology, or employee." (Richard L Daft, "The Leadership Experience" 4th Ed., 2008)

"What’s next for technology and design? A lot less thinking about technology for technology’s sake, and a lot more thinking about design. Art humanizes technology and makes it understandable. Design is needed to make sense of information overload. It is why art and design will rise in importance during this century as we try to make sense of all the possibilities that digital technology now affords." (John Maeda, "Why Apple Leads the Way in Design", 2010) 

"Today, technology has lowered the barrier for others to share their opinion about what we should be focusing on. It is not just information overload; it is opinion overload." (Greg McKeown, "Essentialism: The Disciplined Pursuit of Less", 2014)

"For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." (Richard Feynman)