James G Scott - Collected Quotes

"A model is a metaphor, a description of a system that helps us to reason more clearly. Like all metaphors, models are approximations, and will never account for every last detail. A useful mantra here is: all models are wrong, but some models are useful." (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)

"[...] always remember that the construction of an ANOVA table is inherently sequential. For example, first we add the clutter variable, which remains in the model at every subsequent step; then we add the distance variable, which remains in the model at every subsequent step; and so forth. Thus the actual question being answered at each stage of an analysis of variance is: how much variation in the response can this new variable predict, in the context of what has already been predicted by other variables in the model? This point - the importance of context in interpreting an ANOVA table - is subtle, but important." (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)

"An obvious question is: do bootstrapped confidence intervals satisfy the frequentist coverage property? If your sample is fairly representative of the population, then the answer is a qualified yes. That is, the bootstrapping procedure yields nominal X% intervals that cover the true value 'approximately' X% of the time. Moreover, as the size of the original sample gets bigger, the quality of the approximation gets better. Alas, it is necessary to appeal to some very advanced probability theory to put both of these claims on firm footing." (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)

"At the core of the resampling approach to statistical inference lies a simple idea. Most of the time, we can’t feasibly take repeated samples of size n from the population, to see how our estimate changes from one sample to the next. But we can repeatedly take samples of size n from the sample itself, and apply our estimator afresh to each notional sample. The idea is that the variability of the estimates across all these samples can be used to approximate our estimator’s true sampling distribution. This process - pretending that our sample is the whole population, and taking repeated samples of size n with replacement from our original sample of size n - is called bootstrap resampling, or just bootstrapping" (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)

"By themselves, sums of squares are hard to interpret, because they are measured in squared units of the Y variable. But their ratios are highly meaningful. In fact, the ratio of PV to TV - or what fraction of the total variation has been predicted by the model - is one of the most frequently quoted summary measures in all of statistical modeling. This ratio is called the coefficient of determination, and is usually denoted by the symbol R2 [...] The correct interpretation of R2 sometimes trips people up, and is therefore worth repeating: it is the proportion of variance in the data that can be predicted using the statistical model in question." (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)

"Good estimators are those that usually yield estimates close to the truth, with minimal variation. Therefore, we typically summarize a sampling distribution using its standard deviation, which we refer to as the standard error. In quoting the standard error of an estimator’s sampling distribution, you are saying: 'If I were to take repeated samples from the population and use this estimatorfor every sample, my estimate is typically off from the truth by about this much.' Notice again that this is a claim about a procedure, not a particular estimate. The bigger the standard error, the less stable the estimator across different samples, and the less you can trust the estimate for any particular sample." (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)

"In fitting statistical models, we typically equate the trustworthiness of a procedure with its stability under the influence of luck, and we seek to measure the degree to which that procedure might have given a different answer if the forces of randomness had made the world look a bit different. Specifically, the question we seek to answer is: 'if our data set had been different merely due to chance, would our answer have been different, too?'" (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)

"Model-building requires much more than just technical knowledge of statistical ideas. It also requires care and judgment, and cannot be reduced to a flowchart, a table of formulas, or a tidy set of numerical summaries that wring every last drop of truth from a data set. There is almost never a single 'right' statistical model for some problem. But there are definitely such things as good models and bad models, and learning to tell the difference is important. Just remember: calling a model good or bad requires knowing both the tool and the task." (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)

"[...] complexity sometimes comes at the expense of explanatory power. We must avoid building models calibrated so perfectly to past experience that they do not generalize to future cases." (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)

"It is common to view a statistical model as nothing more than a recipe for calculating the fitted values, and to think that the residuals are just the errors made by this model. But we’ll have a richer picture if instead we view the residuals as part of the model. If you’ve ignored the variation in the residuals, then you really haven’t specified a complete forecast." (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)

"Resampling won’t yield the true sampling distribution of an estimator, but it is often good enough for approximating the standard error (which you’ll remember is just the standard deviation of the sampling distribution). We use the term bootstrapped standard error for the standard deviation of the bootstrapped sampling distribution. The bootstrapped standard error is an estimate of the true standard error." (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)

"Tables are almost always the best way to display categorical data sets with few classifying variables, for the simple reason that they convey a lot of information in a small space." (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)

"The residuals from a regression model are sometimes called 'errors'. This is especially true in experimental science, where measurements of some Y variable will be taken at different values of the X variable (called design points), and where noisy measurement instruments can introduce random errors into theobservations. But in many cases this interpretation of a residual as an error can be misleading. A regression model can still give a nonzero residual, even if there is no mistake in the measurement of the Y variable. It’s often far more illuminating to think of the residual as the part of the Y variable that it is left unpredicted by X." (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)

On Models: On Mathematical Models (1950-1959)

"Mathematical models for empirical phenomena aid the development of a science when a sufficient body of quantitative information has been accumulated. This accumulation can be used to point the direction in which models should be constructed and to test the adequacy of such models in their interim states. Models, in turn, frequently are useful in organizing and interpreting experimental data and in suggesting new directions for experimental research." (Robert R. Bush & Frederick Mosteller, "A Mathematical Model for Simple Learning", Psychological Review 58, 1951)

"There are at least four fundamental purposes that the study of mathematics should attain. First, it should serve as a functional tool in solving our individual everyday problems. [...] In the second place, mathematics serves as a handmaiden for the explanation of the quantitative situations in other subjects, such as economics, physics, navigation, finance, biology, and even the arts. [...] In the third place, mathematics, when properly conceived, becomes a model for thinking, for developing scientific structure, for drawing conclusions, and for solving problems. [...] In the fourth place, mathematics is the best describer of the universe about us." (Howard F Fehr,  "Reorientation in Mathematics Education", Teachers Record 54, 1953)

"The sciences do not try to explain, they hardly even try to interpret, they mainly make models. By a model is meant a mathematical construct which, with the addition of certain verbal interpretations, describes observed phenomena. The justification of such a mathematical construct is solely and precisely that it is expected to work" (John Von Neumann, "Method in the Physical Sciences", 1955)

"General Systems Theory is a name which has come into use to describe a level of theoretical model-building which lies somewhere between the highly generalized constructions of pure mathematics and the specific theories of the specialized disciplines. Mathematics attempts to organize highly general relationships into a coherent system, a system however which does not have any necessary connections with the 'real' world around us. It studies all thinkable relationships abstracted from any concrete situation or body of empirical knowledge." (Kenneth E Boulding, "General Systems Theory - The Skeleton of Science", Management Science Vol. 2 (3), 1956)

"Mathematical examination problems are usually considered unfair if insoluble or improperly described: whereas the mathematical problems of real life are almost invariably insoluble and badly stated, at least in the first balance. In real life, the mathematician's main task is to formulate problems by building an abstract mathematical model consisting of equations, which will be simple enough to solve without being so crude that they fail to mirror reality. Solving equations is a minor technical matter compared with this fascinating and sophisticated craft of model-building, which calls for both clear, keen common-sense and the highest qualities of artistic and creative imagination." (John Hammersley & Mina Rees, "Mathematics in the Market Place", The American Mathematical Monthly 65, 1958)

"There are two types of systems engineering - basis and applied. [...] Systems engineering is, obviously, the engineering of a system. It usually, but not always, includes dynamic analysis, mathematical models, simulation, linear programming, data logging, computing, optimating, etc., etc. It connotes an optimum method, realized by modern engineering techniques. Basic systems engineering includes not only the control system but also all equipment within the system, including all host equipment for the control system. Applications engineering is - and always has been - all the engineering required to apply the hardware of a hardware manufacturer to the needs of the customer. Such applications engineering may include, and always has included where needed, dynamic analysis, mathematical models, simulation, linear programming, data logging, computing, and any technique needed to meet the end purpose - the fitting of an existing line of production hardware to a customer's needs. This is applied systems engineering." (Instruments and Control Systems Vol. 31, 1958)

"If the system exhibits a structure which can be represented by a mathematical equivalent, called a mathematical model, and if the objective can be also so quantified, then some computational method may be evolved for choosing the best schedule of actions among alternatives. Such use of mathematical models is termed mathematical programming." (George Dantzig, "Linear Programming and Extensions", 1959)

On Models: On Mathematical Models (1980-1989)

"Let me start by posing what I tike to call 'the fundamental problem of equilibrium theory': how is economic equilibrium attained? A dual question more commonly raised is: why is economic equilibrium stable? Behind these questions lie the problem of modeling economic processes and introducing dynamics into equilibrium theory. A successful attack here would give greater validity to equilibrium theory. It may be however that a resolution of this fundamental problem will require a recasting of the foundations of equilibrium theory." (Steven Smale, "Some Dynamics Questions in Mathematical Economics", 1980)

"In physics it is usual to give alternative theoretical treatments of the same phenomenon. We construct different models for different purposes, with different equations to describe them. Which is the right model, which the 'true' set of equations? The question is a mistake. One model brings out some aspects of the phenomenon; a different model brings out others. Some equations give a rougher estimate for a quantity of interest, but are easier to solve. No single model serves all purposes best." (Nancy Cartwright, "How the Laws of Physics Lie", 1983)

"Physics is like that. It is important that the models we construct allow us to draw the right conclusions about the behaviour of the phenomena and their causes. But it is not essential that the models accurately describe everything that actually happens; and in general it will not be possible for them to do so, and for much the same reasons. The requirements of the theory constrain what can be literally represented. This does not mean that the right lessons cannot be drawn. Adjustments are made where literal correctness does not matter very much in order to get the correct effects where we want them; and very often, as in the staging example, one distortion is put right by another. That is why it often seems misleading to say that a particular aspect of a model is false to reality: given the other constraints that is just the way to restore the representation." (Nancy Cartwright, "How the Laws of Physics Lie", 1983)

"Cellular automata are mathematical models for complex natural systems containing large numbers of simple identical components with local interactions. They consist of a lattice of sites, each with a finite set of possible values. The value of the sites evolve synchronously in discrete time steps according to identical rules. The value of a particular site is determined by the previous values of a neighbourhood of sites around it." (Stephen Wolfram, "Nonlinear Phenomena, Universality and complexity in cellular automata", Physica 10D, 1984)

"Mathematical model making is an art. If the model is too small, a great deal of analysis and numerical solution can be done, but the results, in general, can be meaningless. If the model is too large, neither analysis nor numerical solution can be carried out, the interpretation of the results is in any case very difficult, and there is great difficulty in obtaining the numerical values of the parameters needed for numerical results." (Richard E Bellman, "Eye of the Hurricane: An Autobiography", 1984)

“Theoretical scientists, inching away from the safe and known, skirting the point of no return, confront nature with a free invention of the intellect. They strip the discovery down and wire it into place in the form of mathematical models or other abstractions that define the perceived relation exactly. The now-naked idea is scrutinized with as much coldness and outward lack of pity as the naturally warm human heart can muster. They try to put it to use, devising experiments or field observations to test its claims. By the rules of scientific procedure it is then either discarded or temporarily sustained. Either way, the central theory encompassing it grows. If the abstractions survive they generate new knowledge from which further exploratory trips of the mind can be planned. Through the repeated alternation between flights of the imagination and the accretion of hard data, a mutual agreement on the workings of the world is written, in the form of natural law.” (Edward O Wilson, “Biophilia”, 1984)

"The large-scale computer models of systems ecology do not fit under the heading of holism at all. Rather they are forms of large-scale reductionism: the objects of study are the naively given 'parts' -abundances or biomasses of populations. No new objects of study arise at the community level. The research is usually conducted on a single system - a lake, forest, or prairie - and the results are measurements of and projections for that lake, forest, or prairie, with no attempts to find the properties of lakes, forests, or prairies in general. Such modeling requires vast amounts of data for its simulations, and much of the scientific effort goes into problems of estimation." (Richard Levins & Richard C Lewontin, "The Dialectical Biologist", 1985)

"One of the features that distinguishes applied mathematics is its interest in framing important questions about the observed world in a mathematical way. This process of translation into a mathematical form can give a better handle for certain problems than would be otherwise possible. We call this the modeling process. It combines formal reasoning with intuitive insights. Understanding the models devised by others is a first step in learning some of the skills involved, and that is how we proceed in this text, which is an informal introduction to the mathematics of dynamical systems." (Edward Beltrami, "Mathematics for Dynamic Modeling", 1987)

"The essence of modeling, as we see it, is that one begins with a nontrivial word problem about the world around us. We then grapple with the not always obvious problem of how it can be posed as a mathematical question. Emphasis is on the evolution of a roughly conceived idea into a more abstract but manageable form in which inessentials have been eliminated. One of the lessons learned is that there is no best model, only better ones." (Edward Beltrami,"Mathematics for Dynamic Modeling", 1987)

"Even if there is only one possible unified theory, it is just a set of rules and equations. What is it that breathes fire into the equations and makes a universe for them to describe? The usual approach of science of constructing a mathematical model cannot answer the questions of why there should be a universe for the model to describe. Why does the universe go to all the bother of existing?" (Stephen W Hawking, "A Brief History of Time: From the Big Bang to Black Holes", 1988)

"Symmetries abound in nature, in technology, and - especially - in the simplified mathematical models we study so assiduously. Symmetries complicate things and simplify them. They complicate them by introducing exceptional types of behavior, increasing the number of variables involved, and making vanish things that usually do not vanish. They simplify them by introducing exceptional types of behavior, increasing the number of variables involved, and making vanish things that usually do not vanish. They violate all the hypotheses of our favorite theorems, yet lead to natural generalizations of those theorems. It is now standard to study the 'generic' behavior of dynamical systems. Symmetry is not generic. The answer is to work within the world of symmetric systems and to examine a suitably restricted idea of genericity." (Ian Stewart, "Bifurcation with symmetry", 1988)

“The usual approach of science of constructing a mathematical model cannot answer the questions of why there should be a universe for the model to describe. Why does the universe go to all the bother of existing?” (Stephen Hawking, "A Brief History of Time", 1988)

"In a real experiment the noise present in a signal is usually considered to be the result of the interplay of a large number of degrees of freedom over which one has no control. This type of noise can be reduced by improving the experimental apparatus. But we have seen that another type of noise, which is not removable by any refinement of technique, can be present. This is what we have called the deterministic noise. Despite its intractability it provides us with a way to describe noisy signals by simple mathematical models, making possible a dynamical system approach to the problem of turbulence." (David Ruelle, "Chaotic Evolution and Strange Attractors: The statistical analysis of time series for deterministic nonlinear systems", 1989)

On Models: On Mathematical Models (1960-1969)

"In fact, the construction of mathematical models for various fragments of the real world, which is the most essential business of the applied mathematician, is nothing but an exercise in axiomatics." (Marshall Stone, cca 1960)

"[...] sciences do not try to explain, they hardly even try to interpret, they mainly make models. By a model is meant a mathematical construct which, with the addition of certain verbal interpretations, describes observed phenomena. The justification of such a mathematical construct is solely and precisely that it is expected to work - that is, correctly to describe phenomena from a reasonably wide area. Furthermore, it must satisfy certain aesthetic criteria - that is, in relation to how much it describes, it must be rather simple.” (John von Neumann, “Method in the physical sciences”, 1961)

"If the system exhibits a structure which can be represented by a mathematical equivalent, called a mathematical model, and if the objective can be also so quantified, then some computational method may be evolved for choosing the best schedule of actions among alternatives. Such use of mathematical models is termed mathematical programming." (George B Dantzig, "Linear Programming and Extensions", 1963)

"Mathematical statistics provides an exceptionally clear example of the relationship between mathematics and the external world. The external world provides the experimentally measured distribution curve; mathematics provides the equation (the mathematical model) that corresponds to the empirical curve. The statistician may be guided by a thought experiment in finding the corresponding equation." (Marshall J Walker, "The Nature of Scientific Thought", 1963)

"Statistics provides a quantitative example of the scientific process usually described qualitatively by saying that scientists observe nature, study the measurements, postulate models to predict new measurements, and validate the model by the success of prediction." (Marshall J Walker, "The Nature of Scientific Thought", 1963)

"After all, without the experiment - either a real one or a mathematical model - there would be no reason for a theory of probability." (Thornton C Fry,"Probability and Its Engineering Uses", 1965)

"Pedantry and sectarianism aside, the aim of theoretical physics is to construct mathematical models such as to enable us, from the use of knowledge gathered in a few observations, to predict by logical processes the outcomes in many other circumstances. Any logically sound theory satisfying this condition is a good theory, whether or not it be derived from 'ultimate' or 'fundamental' truth. It is as ridiculous to deride continuum physics because it is not obtained from nuclear physics as it would be to reproach it with lack of foundation in the Bible." (Clifford Truesdell & Walter Noll, "The Non-Linear Field Theories of Mechanics", 1965)

"A mathematical model is neither an hypothesis nor a theory. Unlike the scientific hypothesis, a model is not verifiable directly by experiment. For all models are both true and false. Almost any plausible proposed relation among aspects of nature is likely to be true in the sense that it occurs (although rarely and slightly). Yet all models leave out a lot and are in that sense false, incomplete, inadequate. The validation of a model is not that it is 'true' but that it generates good testable hypotheses relevant to important problems. A model may be discarded in favor of a more powerful one, but it usually is simply outgrown when the live issues are not any longer those for which it was designed." (Richard Levins, "The Strategy of Model Building in Population Biology", American Scientist 54(4), 1966)

"The most natural way to give an independence proof is to establish a model with the required properties. This is not the only way to proceed since one can attempt to deal directly and analyze the structure of proofs. However, such an approach to set theoretic questions is unnatural since all our intuition come from our belief in the natural, almost physical model of the mathematical universe." (Paul J Cohen, "Set Theory and the Continuum Hypothesis", 1966)

"[…] mathematics is not portraying laws inherent in the design of the universe but is merely providing man-made schemes or models which we can use to deduce conclusions about our world only to the extent that the model is a good idealization." (Morris Kline, "Mathematics for the Nonmathematician", 1967)

"Mathematics is a self-contained microcosm, but it also has the potentiality of mirroring and modeling all the processes of thought and perhaps all of science. It has always had, and continues to an ever increasing degree to have, great usefulness. One could even go so far as to say that mathematics was necessary for man's conquest of nature and for the development of the human race through the shaping of its modes of thinking." (Mark Kac & Stanislaw M Ulam, "Mathematics and Logic", 1968)

"The mathematical models for many physical systems have manifolds as the basic objects of study, upon which further structure may be defined to obtain whatever system is in question. The concept generalizes and includes the special cases of the cartesian line, plane, space, and the surfaces which are studied in advanced calculus. The theory of these spaces which generalizes to manifolds includes the ideas of differentiable functions, smooth curves, tangent vectors, and vector fields. However, the notions of distance between points and straight lines (or shortest paths) are not part of the idea of a manifold but arise as consequences of additional structure, which may or may not be assumed and in any case is not unique." (Richard L Bishop & Samuel I Goldberg, "Tensor Analysis on Manifolds", 1968)

"The laws of nature 'discovered' by science are merely mathematical or mechanical models that describe how nature behaves, not why, nor what nature 'actually' is. Science strives to find representations that accurately describe nature, not absolute truths. This fact distinguishes science from religion." (George O Abell, "Exploration of the Universe", 1969)

On Models: On Mathematical Models (2010-2019)

"In mathematical models, a bifurcation occurs when a small change made to a parameter value of a system causes a sudden qualitative or topological change in its behavior." (Dmitriy Laschov & Michael Margaliot, "Mathematical Modeling of the λ Switch: A Fuzzy Logic Approach", 2010)

"System dynamics is an approach to understanding the behaviour of over time. It deals with internal feedback loops and time delays that affect the behaviour of the entire system. It also helps the decision maker untangle the complexity of the connections between various policy variables by providing a new language and set of tools to describe. Then it does this by modeling the cause and effect relationships among these variables." (Raed M Al-Qirem & Saad G Yaseen, "Modelling a Small Firm in Jordan Using System Dynamics", 2010)

"There are actually two sides to the success of mathematics in explaining the world around us (a success that Wigner dubbed ‘the unreasonable effectiveness of mathematics’), one more astonishing than the other. First, there is an aspect one might call ‘active’. When physicists wander through nature’s labyrinth, they light their way by mathematics - the tools they use and develop, the models they construct, and the explanations they conjure are all mathematical in nature. This, on the face of it, is a miracle in itself. […] But there is also a ‘passive’ side to the mysterious effectiveness of mathematics, and it is so surprising that the 'active' aspect pales by comparison. Concepts and relations explored by mathematicians only for pure reasons - with absolutely no application in mind - turn out decades (or sometimes centuries) later to be the unexpected solutions to problems grounded in physical reality!" (Mario Livio, "Is God a Mathematician?", 2011)

"A catastrophe is a universal unfolding of a singular function germ. The singular function germs are called organization centers of the catastrophes. [...] Catastrophe theory is concerned with the mathematical modeling of sudden changes - so called 'catastrophes' - in the behavior of natural systems, which can appear as a consequence of continuous changes of the system parameters. While in common speech the word catastrophe has a negative connotation, in mathematics it is neutral." (Werner Sanns, "Catastrophe Theory" [Mathematics of Complexity and Dynamical Systems, 2012])

"An important aspect of the global theory of dynamical systems is the stability of the orbit structure as a whole. The motivation for the corresponding theory comes from applied mathematics. Mathematical models always contain simplifying assumptions. Dominant features are modeled; supposed small disturbing forces are ignored. Thus, it is natural to ask if the qualitative structure of the set of solutions - the phase portrait - of a model would remain the same if small perturbations were included in the model. The corresponding mathematical theory is called structural stability." (Carmen Chicone, "Stability Theory of Ordinary Differential Equations" [Mathematics of Complexity and Dynamical Systems, 2012])

"Descriptive statistics are built on the assumption that we can use a single value to characterize a single property for a single universe. […] Probability theory is focused on what happens to samples drawn from a known universe. If the data happen to come from different sources, then there are multiple universes with different probability models. If you cannot answer the homogeneity question, then you will not know if you have one probability model or many. [...] Statistical inference assumes that you have a sample that is known to have come from one universe." (Donald J Wheeler, "Myths About Data Analysis", International Lean & Six Sigma Conference, 2012)

"Models do not and need not match reality in all of its aspects and details to be adequate. A mathematical model is usually developed for a specific class of target systems, and its validity is determined relative to its intended applications. A model is considered valid within its intended domain of applicability provided that its predictions in that domain fall within an acceptable range of error, specified prior to the model’s development or identification." (Zoltan Domotor, "Mathematical Models in Philosophy of Science" [Mathematics of Complexity and Dynamical Systems, 2012])

"Simplified description of a real world system in mathematical terms, e. g., by means of differential equations or other suitable mathematical structures." (Benedetto Piccoli, Andrea Tosin, "Vehicular Traffic: A Review of Continuum Mathematical Models" [Mathematics of Complexity and Dynamical Systems, 2012])

"Stated loosely, models are simplified, idealized and approximate representations of the structure, mechanism and behavior of real-world systems. From the standpoint of set-theoretic model theory, a mathematical model of a target system is specified by a nonempty set - called the model’s domain, endowed with some operations and relations, delineated by suitable axioms and intended empirical interpretation." (Zoltan Domotor, "Mathematical Models in Philosophy of Science" [Mathematics of Complexity and Dynamical Systems, 2012])

"The four questions of data analysis are the questions of description, probability, inference, and homogeneity. [...] Descriptive statistics are built on the assumption that we can use a single value to characterize a single property for a single universe. […] Probability theory is focused on what happens to samples drawn from a known universe. If the data happen to come from different sources, then there are multiple universes with different probability models.  [...] Statistical inference assumes that you have a sample that is known to have come from one universe." (Donald J Wheeler," Myths About Data Analysis", International Lean & Six Sigma Conference, 2012)

"The standard view among most theoretical physicists, engineers and economists is that mathematical models are syntactic (linguistic) items, identified with particular systems of equations or relational statements. From this perspective, the process of solving a designated system of (algebraic, difference, differential, stochastic, etc.) equations of the target system, and interpreting the particular solutions directly in the context of predictions and explanations are primary, while the mathematical structures of associated state and orbit spaces, and quantity algebras – although conceptually important, are secondary." (Zoltan Domotor, "Mathematical Models in Philosophy of Science" [Mathematics of Complexity and Dynamical Systems, 2012])

"Mathematical modeling is a mixed blessing for economics. Mathematical modeling provides real advantages in terms of precision of thought, in seeing how assumptions are linked to conclusions, in generating and communicating insights, in generalizing propositions, and in exporting knowledge from one context to another. In my opinion, these advantages are monumental, far outweighing the costs. But the costs are not zero. Mathematical modeling limits what can be tackled and what is considered legitimate inquiry. You may decide, with experience, that the sorts of models in this book do not help you understand the economic phenomena that you want to understand." (David M Kreps, "Microeconomic Foundations I: Choice and Competitive Markets", 2013) 

"Mathematical symmetry is an idealized model. However, slightly imperfect symmetry requires explanation; it’s not enough just to say ‘it’s asymmetric’." (Ian Stewart, "Symmetry: A Very Short Introduction", 2013)

"To put it simply, we communicate when we display a convincing pattern, and we discover when we observe deviations from our expectations. These may be explicit in terms of a mathematical model or implicit in terms of a conceptual model. How a reader interprets a graphic will depend on their expectations. If they have a lot of background knowledge, they will view the graphic differently than if they rely only on the graphic and its surrounding text." (Andrew Gelman & Antony Unwin, "Infovis and Statistical Graphics: Different Goals, Different Looks", Journal of Computational and Graphical Statistics Vol. 22(1), 2013)

"When confronted with multiple models, I find it revealing to pose the resulting uncertainty as a two-stage lottery. For the purposes of my discussion, there is no reason to distinguish unknown models from unknown parameters of a given model. I will view each parameter configuration as a distinct model. Thus a model, inclusive of its parameter values, assigns probabilities to all events or outcomes within the model’s domain. The probabilities are often expressed by shocks with known distributions and outcomes are functions of these shocks. This assignment of probabilities is what I will call risk. By contrast there may be many such potential models. Consider a two-stage lottery where in stage one we select a model and in stage two we draw an outcome using the model probabilities. Call stage one model ambiguity and stage two risk that is internal to a model." (Lars P Hansen, "Uncertainty Outside and Inside Economic Models", [Nobel lecture] 2013)

“Mathematical modeling is the application of mathematics to describe real-world problems and investigating important questions that arise from it.” (Sandip Banerjee, “Mathematical Modeling: Models, Analysis and Applications”, 2014)

"Models can be: formulations, abstractions, replicas, idealizations, metaphors - and combinations of these. [...] Some mathematical models have been blindly used - their presuppositions as little understood as any legal fine print one ‘agrees to’ but never reads - with faith in their trustworthiness. The very arcane nature of some of the formulations of these models might have contributed to their being given so much credence. If so, we mathematicians have an important mission to perform: to help people who wish to think through the fundamental assumptions underlying models that are couched in mathematical language, making these models intelligible, rather than" (merely) formidable Delphic oracles." (Barry Mazur, "The Authority of the Incomprehensible" , 2014)

"But we also have to know that every model has its limitations. The model of natural numbers and their sums is very successful to determine the number of objects in the union of two different groups of well-distinguished objects. But as a mathematical model, the arithmetic of numbers is not generally true but only validated and confirmed for certain well-controlled situations. […] If a model makes valid predictions in many concrete cases, if it already has been applied and tested successfully in many situations, we have some right to trust in that model. By now, we believe in the model 'natural numbers and their arithmetic' and in its predictions without having to check it every time. We do not expect that the result might be wrong; hence the verification step is not needed any longer for validating the model. If the model had a flaw, it would have been eliminated already in the past." (Alfred S Posamentier & Bernd Thaller, "Numbers: Their tales, types, and treasures", 2015)

"Design is the process of taking something that appears in the mind’s eye, modeling it in one or more of a number of ways, predicting how that thing will behave if it is made, and then making it, sometimes modifying the design as we make it. Design is what engineering is about. Furthermore, modeling is how engineering design is done. This includes mental models, mathematical models, computer models, plans and drawings, written language, and" (sometimes) physical models." (William M Bulleit, "The Engineering Way of Thinking: The Idea", Structure [magazine], 2015)

"[…] the usefulness of mathematics is by no means limited to finite objects or to those that can be represented with a computer. Mathematical concepts depending on the idea of infinity, like real numbers and differential calculus, are useful models for certain aspects of physical reality." (Alfred S Posamentier & Bernd Thaller, "Numbers: Their tales, types, and treasures", 2015)

"There are several reasons why reaction-diffusion systems have been a popular choice among mathematical modelers of spatio-temporal phenomena. First, their clear separation between non-spatial and spatial dynamics makes the modeling and simulation tasks really easy. Second, limiting the spatial movement to only diffusion makes it quite straightforward to expand any existing non-spatial dynamical models into spatially distributed ones. Third, the particular structure of reaction-diffusion equations provides aneasy shortcut in the stability analysis (to be discussed in the next chapter). And finally, despite the simplicity of their mathematical form, reaction-diffusion systems can show strikingly rich, complex spatio-temporal dynamics. Because of these properties, reaction-diffusion systems have been used extensively for modeling self-organization of spatial patterns." (Hiroki Sayama, "Introduction to the Modeling and Analysis of Complex Systems", 2015)

“A mathematical model is a mathematical description (often by means of a function or an equation) of a real-world phenomenon such as the size of a population, the demand for a product, the speed of a falling object, the concentration of a product in a chemical reaction, the life expectancy of a person at birth, or the cost of emission reductions. The purpose of the model is to understand the phenomenon and perhaps to make predictions about future behavior. [...] A mathematical model is never a completely accurate representation of a physical situation - it is an idealization." (James Stewart, “Calculus: Early Transcedentals” 8th Ed., 2016)

"Eventually, mechanical models failed too. They were duly abandoned, and replaced by much more abstract mathematical models. Compared to their predecessors, mathematical models are Spartan affairs. They consist of equations and formulas without the texture, the color, the visual detail - without the rich appeal - of their mechanical relatives. […] But what a mathematical model lacks in charm, it more than makes up for in generality and predictive power." (Hans C von Baeyer, "QBism: The future of quantum physics", 2016)

"The goal of physics is to explain the workings of the nonliving world. At first, philosophers described the properties of real objects: the wandering of planets across the night sky, the formation of ice, or the sound of a lyre. When attention turned to things that couldn’t be seen or measured so easily, physicists invented mechanical models to take the place of real things." (Hans C von Baeyer, "QBism: The future of quantum physics", 2016)

"A model may be defined as a substitute of any object or system. […] A mental image used in thinking is a model, and it is not the real system. A written description of a system is a model that presents one aspect of reality. The simulation model is logically complete and describes the dynamic behaviour of the system. Models can be broadly classified as (a) physical models and (b) abstract models [..] Mental models and mathematical models are examples of abstract models." (Bilash K Bala et al, "System Dynamics: Modelling and Simulation", 2017)

"Data almost always contain uncertainty. This uncertainty may arise from selection of the items to be measured, or it may arise from variability of the measurement process. Drawing general conclusions from data is the basis for increasing knowledge about the world, and is the basis for all rational scientific inquiry. Statistical inference gives us methods and tools for doing this despite the uncertainty in the data. The methods used for analysis depend on the way the data were gathered. It is vitally important that there is a probability model explaining how the uncertainty gets into the data." (William M Bolstad & James M Curran, "Introduction to Bayesian Statistics" 3rd Ed., 2017)

"Different models serve different purposes. Setting up a model involves focusing on features of the phenomenon that are compatible with the methodology being proposed, and neglecting features that are not compatible with it. A mathematical model in applied science explicitly refrains from attempting to be a complete picture of the phenomenon being modeled." (Reuben Hersh, ”Mathematics as an Empirical Phenomenon, Subject to Modeling”, 2017)

"Mathematical modeling is the modern version of both applied mathematics and theoretical physics. In earlier times, one proposed not a model but a theory. By talking today of a model rather than a theory, one acknowledges that the way one studies the phenomenon is not unique; it could also be studied other ways. One's model need not claim to be unique or final. It merits consideration if it provides an insight that isn't better provided by some other model." (Reuben Hersh, ”Mathematics as an Empirical Phenomenon, Subject to Modeling”, 2017)

"Model-building requires much more than just technical knowledge of statistical ideas. It also requires care and judgment, and cannot be reduced to a flowchart, a table of formulas, or a tidy set of numerical summaries that wring every last drop of truth from a data set. There is almost never a single 'right' statistical model for some problem. But there are definitely such things as good models and bad models, and learning to tell the difference is important. Just remember: calling a model good or bad requires knowing both the tool and the task." (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)

"The lack of direct control means the outside factors will be affecting the data. There is a danger that the wrong conclusions could be drawn from the experiment due to these uncontrolled outside factors. The important statistical idea of randomization has been developed to deal with this possibility. The unidentified outside factors can be 'averaged out' by randomly assigning each unit to either treatment or control group. This contributes variability to the data. Statistical conclusions always have some uncertainty or error due to variability in the data. We can develop a probability model of the data variability based on the randomization used. Randomization not only reduces this uncertainty due to outside factors, it also allows us to measure the amount of uncertainty that remains using the probability model. Randomization lets us control the outside factors statistically, by averaging out their effects." (William M Bolstad & James M Curran, "Introduction to Bayesian Statistics" 3rd Ed., 2017)

"The scientific method searches for cause-and-effect relationships between an experimental variable and an outcome variable. In other words, how changing the experimental variable results in a change to the outcome variable. Scientific modeling develops mathematical models of these relationships. Both of them need to isolate the experiment from outside factors that could affect the experimental results. All outside factors that can be identified as possibly affecting the results must be controlled." (William M Bolstad & James M Curran, "Introduction to Bayesian Statistics" 3rd Ed., 2017)

"When we use algebraic notation in statistical models, the problem becomes more complicated because we cannot 'observe' a probability and know its exact number. We can only estimate probabilities on the basis of observations." (David S Salsburg, "Errors, Blunders, and Lies: How to Tell the Difference", 2017)

"Some scientists (e.g., econometricians) like to work with mathematical equations; others (e.g., hard-core statisticians) prefer a list of assumptions that ostensibly summarizes the structure of the diagram. Regardless of language, the model should depict, however qualitatively, the process that generates the data - in other words, the cause-effect forces that operate in the environment and shape the data generated." (Judea Pearl & Dana Mackenzie, "The Book of Why: The new science of cause and effect", 2018)

"A neural-network algorithm is simply a statistical procedure for classifying inputs" (such as numbers, words, pixels, or sound waves) so that these data can mapped into outputs. The process of training a neural-network model is advertised as machine learning, suggesting that neural networks function like the human mind, but neural networks estimate coefficients like other data-mining algorithms, by finding the values for which the model’s predictions are closest to the observed values, with no consideration of what is being modeled or whether the coefficients are sensible." (Gary Smith & Jay Cordes,The 9 Pitfalls of Data Science", 2019)

"Mathematicians love math and many non-mathematicians are intimidated by math. This is a lethal combination that can lead to the creation of wildly unrealistic mathematical models. [...] A good mathematical model starts with plausible assumptions and then uses mathematics to derive the implications. A bad model focuses on the math and makes whatever assumptions are needed to facilitate the math." (Gary Smith & Jay Cordes, "The 9 Pitfalls of Data Science", 2019)

On Models: On Mathematical Models (2000-2009)

"The role of graphs in probabilistic and statistical modeling is threefold: (1) to provide convenient means of expressing substantive assumptions; (2) to facilitate economical representation of joint probability functions; and (3) to facilitate efficient inferences from observations." (Judea Pearl,Causality: Models, Reasoning, and Inference", 2000)

"A mathematical model uses mathematical symbols to describe and explain the represented system. Normally used to predict and control, these models provide a high degree of abstraction but also of precision in their application." (Lars Skyttner, "General Systems Theory: Ideas and Applications", 2001)

“A model is an imitation of reality and a mathematical model is a particular form of representation. We should never forget this and get so distracted by the model that we forget the real application which is driving the modelling. In the process of model building we are translating our real world problem into an equivalent mathematical problem which we solve and then attempt to interpret. We do this to gain insight into the original real world situation or to use the model for control, optimization or possibly safety studies." (Ian T Cameron & Katalin Hangos, “Process Modelling and Model Analysis”, 2001)

“Formulation of a mathematical model is the first step in the process of analyzing the behaviour of any real system. However, to produce a useful model, one must first adopt a set of simplifying assumptions which have to be relevant in relation to the physical features of the system to be modelled and to the specific information one is interested in. Thus, the aim of modelling is to produce an idealized description of reality, which is both expressible in a tractable mathematical form and sufficiently close to reality as far as the physical mechanisms of interest are concerned.” (Francois Axisa, “Discrete Systems” Vol. I, 2001)

"[…] interval mathematics and fuzzy logic together can provide a promising alternative to mathematical modeling for many physical systems that are too vague or too complicated to be described by simple and crisp mathematical formulas or equations. When interval mathematics and fuzzy logic are employed, the interval of confidence and the fuzzy membership functions are used as approximation measures, leading to the so-called fuzzy systems modeling." (Guanrong Chen & Trung Tat Pham, "Introduction to Fuzzy Sets, Fuzzy Logic, and Fuzzy Control Systems", 2001)

"Modeling, in a general sense, refers to the establishment of a description of a system (a plant, a process, etc.) in mathematical terms, which characterizes the input-output behavior of the underlying system. To describe a physical system […] we have to use a mathematical formula or equation that can represent the system both qualitatively and quantitatively. Such a formulation is a mathematical representation, called a mathematical model, of the physical system." (Guanrong Chen & Trung Tat Pham, "Introduction to Fuzzy Sets, Fuzzy Logic, and Fuzzy Control Systems", 2001)

"One might think this means that imaginary numbers are just a mathematical game having nothing to do with the real world. From the viewpoint of positivist philosophy, however, one cannot determine what is real. All one can do is find which mathematical models describe the universe we live in. It turns out that a mathematical model involving imaginary time predicts not only effects we have already observed but also effects we have not been able to measure yet nevertheless believe in for other reasons. So what is real and what is imaginary? Is the distinction just in our minds?" (Stephen W Hawking, "The Universe in a Nutshell", 2001)

“What is a mathematical model? One basic answer is that it is the formulation in mathematical terms of the assumptions and their consequences believed to underlie a particular ‘real world’ problem. The aim of mathematical modeling is the practical application of mathematics to help unravel the underlying mechanisms involved in, for example, economic, physical, biological, or other systems and processes.” (John A Adam, “Mathematics in Nature”, 2003)

“Mathematical modeling is as much ‘art’ as ‘science’: it requires the practitioner to (i) identify a so-called ‘real world’ problem (whatever the context may be); (ii) formulate it in mathematical terms (the ‘word problem’ so beloved of undergraduates); (iii) solve the problem thus formulated (if possible; perhaps approximate solutions will suffice, especially if the complete problem is intractable); and (iv) interpret the solution in the context of the original problem.” (John A Adam, “Mathematics in Nature”, 2003)

"Do I claim that everything that is not smooth is fractal? That fractals suffice to solve every problem of science? Not in the least. What I'm asserting very strongly is that, when some real thing is found to be un-smooth, the next mathematical model to try is fractal or multi-fractal. A complicated phenomenon need not be fractal, but finding that a phenomenon is 'not even fractal' is bad news, because so far nobody has invested anywhere near my effort in identifying and creating new techniques valid beyond fractals. Since roughness is everywhere, fractals - although they do not apply to everything - are present everywhere. And very often the same techniques apply in areas that, by every other account except geometric structure, are separate." (Benoît Mandelbrot, "A Theory of Roughness", 2004) 

"Fuzzy models can provide good numerical approximation of functions as well as linguistic information over the behavior of the functions. […] Fuzzy models with embedded linguistic interpretability are useful to extract knowledge from data. This knowledge is represented as a set of IF–THEN rules where the antecedents and the consequences are semantically meaningful." (Jairo Espinosa et al, "Fuzzy Logic, Identification and Predictive Control", 2005)

"Although fiction is not fact, paradoxically we need some fictions, particularly mathematical ideas and highly idealized models, to describe, explain, and predict facts.  This is not because the universe is mathematical, but because our brains invent or use refined and law-abiding fictions, not only for intellectual pleasure but also to construct conceptual models of reality." (Mario Bunge, "Chasing Reality: Strife over Realism", 2006)

"Chaotic system is a deterministic dynamical system exhibiting irregular, seemingly random behavior. Two trajectories of a chaotic system starting close to each other will diverge after some time (such an unstable behavior is often called 'sensitive dependence on initial conditions'). Mathematically, chaotic systems are characterized by local instability and global boundedness of the trajectories. Since local instability of a linear system implies unboundedness (infinite growth) of its solutions, chaotic system should be necessarily nonlinear, i.e., should be described by a nonlinear mathematical model." (Alexander L Fradkov, "Cybernetical Physics: From Control of Chaos to Quantum Control", 2007)

"In order to understand how mathematics is applied to understanding of the real world it is convenient to subdivide it into the following three modes of functioning: model, theory, metaphor. A mathematical model describes a certain range of phenomena qualitatively or quantitatively. […] A (mathematical) metaphor, when it aspires to be a cognitive tool, postulates that some complex range of phenomena might be compared to a mathematical construction." (Yuri I Manin," Mathematics as Metaphor: Selected Essays of Yuri I. Manin", 2007)

"The dichotomy of mathematical vs. statistical modeling says more about the culture of modeling and how different disciplines go about thinking about models than about how we should actually model ecological systems. A mathematician is more likely to produce a deterministic, dynamic process model without thinking very much about noise and uncertainty (e.g. the ordinary differential equations that make up the Lotka-Volterra predator prey model). A statistician, on the other hand, is more likely to produce a stochastic but static model, that treats noise and uncertainty carefully but focuses more on static patterns than on the dynamic processes that produce them (e.g. linear regression)." (Ben Bolker, "Ecological Models and Data in R", 2007)

"A science presents us with representations of the phenomena through artifacts, both abstract, such as theories and mathematical models, and concrete such as graphs, tables, charts, and ‘table-top’ models. These representations do not form a haphazard compilation though any unity, in the range of representations made available, is visible mainly at the more abstract levels." (Bas C van Fraassen, "Scientific Representation: Paradoxes of Perspective", 2008)

"It is impossible to construct a model that provides an entirely accurate picture of network behavior. Statistical models are almost always based on idealized assumptions, such as independent and identically distributed (i.i.d.) interarrival times, and it is often difficult to capture features such as machine breakdowns, disconnected links, scheduled repairs, or uncertainty in processing rates." (Sean Meyn, "Control Techniques for Complex Networks", 2008)

"Therefore, mathematical ecology does not deal directly with natural objects. Instead, it deals with the mathematical objects and operations we offer as analogs of nature and natural processes. These mathematical models do not contain all information about nature that we may know, but only what we think are the most pertinent for the problem at hand. In mathematical modeling, we have abstracted nature into simpler form so that we have some chance of understanding it. Mathematical ecology helps us understand the logic of our thinking about nature to help us avoid making plausible arguments that may not be true or only true under certain restrictions. It helps us avoid wishful thinking about how we would like nature to be in favor of rigorous thinking about how nature might actually work. (John Pastor, "Mathematical Ecology of Populations and Ecosystems", 2008)

"Much of the recorded knowledge of physics and engineering is written in the form of mathematical models. These mathematical models form the foundations of our understanding of the universe we live in. Furthermore, nearly all of the existing technology, in one way or another, rests on these models. To the extent that we are surrounded by evidence of the technology working and being reliable, human confidence in the validity of the underlying mathematical models is all but unshakable." (Jerzy A Filar, "Mathematical Models", 2009)

"To understand, how noise is related to scale-freeness, we have to do some mathematics again. Noise is usually characterized by a mathematical trick. The seemingly random fluctuation of the signal is regarded as a sum of sinusoidal waves. The components of the million waves giving the final noise structure are characterized by their frequency. To describe noise, we plot the contribution (called spectral density) of the various waves we use to model the noise as a function of their frequency. This transformation is called a Fourier transformation [...]" (Péter Csermely, "Weak Links: The Universal Key to the Stabilityof Networks and Complex Systems", 2009)

On Models in Analysis (1975-)

"We argue that in order to deal with the various inertial pressures the adaptation perspective must be supplemented with a selection orientation. We consider first two broad issues that are preliminary to ecological modelling. The first concerns appropriate units of analysis. Typical analyses of the relation of organizations to environments take the point of view of a single organization facing an environment." (Michael T Hannan, "The Population Ecology of Organizations", 1977)

"It is actually impossible in theory to determine exactly what the hidden mechanism is without opening the box, since there are always many different mechanisms with identical behavior. Quite apart from this, analysis is more difficult than invention in the sense in which, generally, induction takes more time to perform than deduction: in induction one has to search for the way, whereas in deduction one follows a straightforward path." (Valentino Braitenberg, "Vehicles: Experiments in Synthetic Psychology", 1984)

"Mathematical model making is an art. If the model is too small, a great deal of analysis and numerical solution can be done, but the results, in general, can be meaningless. If the model is too large, neither analysis nor numerical solution can be carried out, the interpretation of the results is in any case very difficult, and there is great difficulty in obtaining the numerical values of the parameters needed for numerical results." (Richard E Bellman, "Eye of the Hurricane: An Autobiography", 1984)

"The methods of science include controlled experiments, classification, pattern recognition, analysis, and deduction. In the humanities we apply analogy, metaphor, criticism, and (e)valuation. In design we devise alternatives, form patterns, synthesize, use conjecture, and model solutions." (Béla H Bánáthy, "Designing Social Systems in a Changing World", 1996)

"[…] it does not seem helpful just to say that all models are wrong. The very word model implies simplification and idealization. The idea that complex physical, biological or sociological systems can be exactly described by a few formulae is patently absurd. The construction of idealized representations that capture important stable aspects of such systems is, however, a vital part of general scientific analysis and statistical models, especially substantive ones, do not seem essentially different from other kinds of model." (David Cox, "Comment on ‘Model uncertainty, data mining and statistical inference’", Journal of the Royal Statistical Society, Series A 158, 1995)

"The methods of science include controlled experiments, classification, pattern recognition, analysis, and deduction. In the humanities we apply analogy, metaphor, criticism, and (e)valuation. In design we devise alternatives, form patterns, synthesize, use conjecture, and model solutions." (Béla H Bánáthy, "Designing Social Systems in a Changing World", 1996)

"Thematic analysis is a process for encoding quantitative information. The encoding requires an explicit 'code' . This may be a list of themes; a complex model with themes, indicators, and qualifications that are causally related; or something in between these two forms. A theme is a pattern found in the information that at minimum describes and organizes the possible observations and at maximum interprets aspects of the phenomenon. A theme may be identified at the manifest level" (directly observable in the information) or at the latent level" (underlying the phenomenon). The themes may be initially generated inductively from the raw information or generated deductively from theory or prior research." (Richard E Boyatzis, "Transforming qualitative information: Thematic analysis and code development", 1998)

"Theories of choice are at best approximate and incomplete. One reason for this pessimistic assessment is that choice is a constructive and contingent process. When faced with a complex problem, people employ a variety of heuristic procedures in order to simplify the representation and the evaluation of prospects. These procedures include computational shortcuts and editing operations, such as eliminating common components and discarding nonessential differences. The heuristics of choice do not readily lend themselves to formal analysis because their application depends on the formulation of the problem, the method of elicitation, and the context of choice." (Amos Tversky & Daniel Kahneman, "Advances in Prospect Theory: Cumulative Representation of Uncertainty" [in "Choices, Values, and Frames"], 2000)"

"A model is an imitation of reality and a mathematical model is a particular form of representation. We should never forget this and get so distracted by the model that we forget the real application which is driving the modelling. In the process of model building we are translating our real world problem into an equivalent mathematical problem which we solve and then attempt to interpret. We do this to gain insight into the original real world situation or to use the model for control, optimization or possibly safety studies." (Ian T Cameron & Katalin Hangos,"Process Modelling and Model Analysis", 2001

"Part of a meaningful quantitative analysis is to look at models and try to figure out their deficiencies and the ways in which they can be improved. A more subtle challenge for statistical methods is to explore systematically potential modeling errors in order to assess the quality of the model predictions. This kind of uncertainty about the adequacy of a model or model family is not only relevant for econometricians outside the model but potentially also for agents inside the models." (Lars P Hansen, "Uncertainty Outside and Inside Economic Models", [Nobel lecture] 2013)

On Models in Analysis (-1974)

"There are two types of systems engineering - basis and applied. [...] Systems engineering is, obviously, the engineering of a system. It usually, but not always, includes dynamic analysis, mathematical models, simulation, linear programming, data logging, computing, optimating, etc., etc. It connotes an optimum method, realized by modern engineering techniques. Basic systems engineering includes not only the control system but also all equipment within the system, including all host equipment for the control system. Applications engineering is - and always has been - all the engineering required to apply the hardware of a hardware manufacturer to the needs of the customer. Such applications engineering may include, and always has included where needed, dynamic analysis, mathematical models, simulation, linear programming, data logging, computing, and any technique needed to meet the end purpose - the fitting of an existing line of production hardware to a customer's needs. This is applied systems engineering." (Instruments and Control Systems Vol. 31, 1958)

"The principle of complementarity states that no single model is possible which could provide a precise and rational analysis of the connections between these phenomena [before and after measurement]. In such a case, we are not supposed, for example, to attempt to describe in detail how future phenomena arise out of past phenomena. Instead, we should simply accept without further analysis the fact that future phenomena do in fact somehow manage to be produced, in a way that is, however, necessarily beyond the possibility of a detailed description. The only aim of a mathematical theory is then to predict the statistical relations, if any, connecting the phenomena." (David Bohm, "A Suggested Interpretation of the Quantum Theory in Terms of ‘Hidden’ Variables", 1952)

"Because we can never be sure that a postulated model is entirely appropriate, we must proceed in such a manner that inadequacies can be taken account of and their implications considered as we go along. To do this we must regard statistical analysis, which is a step in the major iteration […] as itself an iteration. To be on firm ground we must do more than merely postulate a model; we must build and test a tentative model at each stage of the investigation.(George E P Box & George C Tjao, "Bayesian Inference in Statistical Analysis", 1973)

"Modeling is definitely the most important and critical problem. If the mathematical model is not valid, any subsequent analysis, estimation, or control study is meaningless. The development of the model in a convenient form can greatly reduce the complexity of the actual studies." (Fred C Scweppe, "Uncertain dynamic systems", 1973)

"The process [of statistical analysis] usually begins by the postulating of a model worthy to be tentatively entertained. The data analyst will have arrived at this tentative model in cooperation with the scientific investigator. They will choose it 'So that, in the light of the then available knowledge, it best takes account of relevant phenomena in the simplest way possible. it will usually contain unknown parameters. Given the data the analyst can now make statistical inferences about the parameters conditional on the correctness of this first tentative model. These inferences form part of the conditional analysis. If the model is correct, they provide all there is to know about the problem under study, given the data." (George E P Box & George C Tjao, "Bayesian Inference in Statistical Analysis", 1973)

"The pseudo approach to uncertainty modeling refers to the use of an uncertainty model instead of using a deterministic model which is actually (or at least theoretically) available. The uncertainty model may be desired because it results in a simpler analysis, because it is too difficult (expensive) to gather all the data necessary for an exact model, or because the exact model is too complex to be included in the computer." (Fred C Scweppe, "Uncertain dynamic systems", 1973)

On Models: Bad Models

"A bad model is a combination of assertions, some factual, others conjectural, and others plainly false but convenient. [..]  By definition, a bad model does not give power to see-accurately, deeply, or at all-into the actual situation, but only into the assertions embodied in the model. Thus, if the use of a bad model provides insight, it does so not by revealing truth about the world but by revealing its own assumptions and thereby causing its user to go learn something about the world." (James S Hodges, "Six (or So) Things You Can Do with a Bad Model", 1991)

"Just because a model is bad, however, does not mean it is useless. [...] A bad model can be used to construct correct paths from premises to conclusions, but because its relations to reality are questionable, it can only do so in a few ways-at least, ways that permit useful conclusions with respect to reality." (James S Hodges, "Six (or So) Things You Can Do with a Bad Model", 1991)

"Often, though, a policy or systems analyst is stuck with a bad model, that is, one that appeals to the analyst as adequately realistic but which is either: 1) contradicted by some data or is grossly implausible in some aspect it purports to represent, or 2) conjectural, that is, neither supported nor contradicted by data, either because data do not exist or because they are equivocal. [...] A model may have component parts that are not bad, but if, taken as a whole, it meets one of these criteria, it is a bad model." (James S Hodges, "Six (or So) Things You Can Do with a Bad Model", 1991)

"Some readers have argued that the criticism implied by the term 'bad models' is undeserved because they can be used appropriately in some cases. [...] If the logic works, the use is appropriate; if it fails, the use in inappropriate: (Cost effectiveness is a separate issue.) As for the pejorative connotation of the term bad model, perhaps we should admit that many useful models would be embarrassments to scientists, from whom we got the idea of a model, but whose job is to improve the match between models and reality." (James S Hodges, "Six (or So) Things You Can Do with a Bad Model", 1991)

"The definition of a ‘good model’ is when everything inside it is visible, inspectable and testable. It can be communicated effortlessly to others. A ‘bad model’ is a model that does not meet these standards, where parts are hidden, undefined or concealed and it cannot be inspected or tested; these are often labelled black box models." (Hördur V Haraldsson & Harald U Sverdrup,Finding Simplicity in Complexity in Biogeochemical Modelling" [inEnvironmental Modelling: Finding Simplicity in Complexity", Ed. by John Wainwright and Mark Mulligan, 2004])

"It is not always convenient to remember that the right model for a population can fit a sample of data worse than a wrong model - even a wrong model with fewer parameters. We cannot rely on statistical diagnostics to save us, especially with small samples. We must think about what our models mean, regardless of fit, or we will promulgate nonsense." (Leland Wilkinson,The Grammar of Graphics" 2nd Ed., 2005)

"Sometimes the most important fit statistic you can get is ‘convergence not met’ - it can tell you something is wrong with your model." (Oliver Schabenberger, "Applied Statistics in Agriculture Conference", 2006)

"An attempt to use the wrong model for a given data set is likely to provide poor results. Therefore, the core principle of discovering outliers is based on assumptions about the structure of the normal patterns in a given data set. Clearly, the choice of the 'normal' model depends highly upon the analyst’s understanding of the natural data patterns in that particular domain." (Charu C Aggarwal,Outlier Analysis", 2013)

"Things which ought to be expected can seem quite extraordinary if you’ve got the wrong model." (David Hand,Significance", 2014)

"Model-building requires much more than just technical knowledge of statistical ideas. It also requires care and judgment, and cannot be reduced to a flowchart, a table of formulas, or a tidy set of numerical summaries that wring every last drop of truth from a data set. There is almost never a single 'right' statistical model for some problem. But there are definitely such things as good models and bad models, and learning to tell the difference is important. Just remember: calling a model good or bad requires knowing both the tool and the task." (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)

"Overfitting and underfitting are two important factors that could impact the performance of machine-learning models. Overfitting occurs when the model performs well with training data and poorly with test data. Underfitting occurs when the model is so simple that it performs poorly with both training and test data. [...]  When the model does not capture and fit the data, it results in poor performance. We call this underfitting. Underfitting is the result of a poor model that typically does not perform well for any data." (Umesh R Hodeghatta & Umesha Nayak,Business Analytics Using R: A Practical Approach", 2017)

"Mathematicians love math and many non-mathematicians are intimidated by math. This is a lethal combination that can lead to the creation of wildly unrealistic mathematical models. [...] A good mathematical model starts with plausible assumptions and then uses mathematics to derive the implications. A bad model focuses on the math and makes whatever assumptions are needed to facilitate the math." (Gary Smith & Jay Cordes, "The 9 Pitfalls of Data Science", 2019)

"Bad data makes bad models. Bad models instruct people to make ineffective or harmful interventions. Those bad interventions produce more bad data, which is fed into more bad models." (Cory Doctorow,Machine Learning’s Crumbling Foundations", 2021)

Statistical Tools IV: Urns

"The early experts in probability theory were forever talking about drawing colored  balls out of 'urns' . This was not because people are really interested in jars or boxes full of a mixed-up lot of colored balls, but because those urns full of balls could often be designed so that they served as useful and illuminating models of important real situations. In fact, the urns and balls are not themselves supposed real. They are fictitious and idealized urns and balls, so that the probability of drawing out any one ball is just the same as for any other." (Warren Weaver, "Lady Luck: The Theory of Probability". 1963) 

"The urn model is to be the expression of three postulates: (1) the constancy of a probability distribution, ensured by the solidity of the vessel, (2) the random-character of the choice, ensured by the narrowness of the mouth, which is to prevent visibility of the contents and any consciously selective choice, (3) the independence of successive choices, whenever the drawn balls are put back into the urn. Of course in abstract probability and statistics the word 'choice' can be avoided and all can be done without any reference to such a model. But as soon as the abstract theory is to be applied, random choice plays an essential role." (Hans Freudenthal, "The Concept and the Role of the Model in Mathematics and Natural and Social Sciences", 1961)

"Specifically, it seems to me preferable to use, systematically: 'random' for that which is the object of the theory of probability […]; I will therefore say random process, not stochastic process. 'stochastic' for that which is valid 'in the sense of the calculus of probability': for instance; stochastic independence, stochastic convergence, stochastic integral; more generally, stochastic property, stochastic models, stochastic interpretation, stochastic laws; or also, stochastic matrix, stochastic distribution, etc. As for 'chance', it is perhaps better to reserve it for less technical use: in the familiar sense of'by chance', 'not for a known or imaginable reason', or (but in this case we should give notice of the fact) in the sense of, 'with equal probability' as in 'chance drawings from an urn', 'chance subdivision', and similar examples." (Bruno de Finetti, "Theory of Probability", 1974)

"Statisticians talk about populations. In probability books, the equivalent concept is an urn with numbered balls as a prototype for a population. In fact, when sampling from populations, it is customary to number the population and pretend the population is an urn from which we are drawing the sample." (Juana Sánchez, "Probability for Data Scientists", 2020)

"Many people mistakenly think that the defining property of a simple random sample is that every unit has an equal chance of being in the sample. However, this is not the case. A simple random sample of n units from a population of N means that every possible col‐lection of n of the N units has the same chance of being selected. A slight variant of this is the simple random sample with replacement, where the units/marbles are returned to the urn after each draw. This method also has the property that every sample of n units from a population of N is equally likely to be selected. The difference, though, is that there are more possible sets of n units because the same marble can appear more than once in the sample." (Sam Lau et al, "Learning Data Science: Data Wrangling, Exploration, Visualization, and Modeling with Python", 2023)

"Several key assumptions enter into this urn model, such as the assumption that the vaccine is ineffective. It’s important to keep track of the reliance on these assumptions because our simulation study gives us an approximation of the rarity of an outcome like the one observed only under these key assumptions." (Sam Lau et al, "Learning Data Science: Data Wrangling, Exploration, Visualization, and Modeling with Python", 2023)

"The urn model is a simple abstraction that can be helpful for understanding variation.This model sets up a container (an urn, which is like a vase or a bucket) full of identical marbles that have been labeled, and we use the simple action of drawing marbles from the urn to reason about sampling schemes, randomized controlled experiments, and measurement error. For each of these types of variation, the urn model helps us estimate the size of the variation using either probability or simulation." (Sam Lau et al, "Learning Data Science: Data Wrangling, Exploration, Visualization, and Modeling with Python", 2023)

Richard Levins - Collected Quotes

"A mathematical model is neither an hypothesis nor a theory. Unlike the scientific hypothesis, a model is not verifiable directly by experiment. For all models are both true and false. Almost any plausible proposed relation among aspects of nature is likely to be true in the sense that it occurs (although rarely and slightly). Yet all models leave out a lot and are in that sense false, incomplete, inadequate. The validation of a model is not that it is 'true' but that it generates good testable hypotheses relevant to important problems. A model may be discarded in favor of a more powerful one, but it usually is simply outgrown when the live issues are not any longer those for which it was designed." (Richard Levins, "The Strategy of Model Building in Population Biology", American Scientist 54(4), 1966)

"For population genetics, a population is specified by the frequencies of genotypes without reference to the age distribution, physiological state as a reflection of past history, or population density. A single population or species is treated at a time, and evolution is usually assumed to occur in a constant environment. Population ecology, on the other hand, recognizes multispecies systems, describes populations in terms of their age distributions, physiological states, and densities. The environment is allowed to vary but the species are treated as genetically homogeneous, so that evolution is ignored." (Richard Levins, "The Strategy of Model Building in Population Biology", American Scientist 54(4), 1966)

"It is of course desirable to work with manageable models which maximize generality, realism, and precision toward the overlapping but not identical goals of understanding, predicting, and modifying nature. But this cannot be done. Therefore, several alternative strategies have evolved: (1) Sacrifice generality to realism and precision. (2) Sacrifice realism to generality and precision. (3) Sacrifice precision to realism and generality." (Richard Levins, "The strategy of model building in population biology", American Scientist Vol. 54 (4), 1966) 

"The multiplicity of models is imposed by the contradictory demands of a complex, heterogeneous nature and a mind that can only cope with few variables at a time; by the contradictory desiderata of generality, realism, and precision; by the need to understand and also to control; even by the opposing esthetic standards which emphasize the stark simplicity and power of a general theorem as against the richness and the diversity of living nature. These conflicts are irreconcilable. Therefore, the alternative approaches even of contending schools are part of a larger mixed strategy. But the conflict is about method, not nature, for the individual models, while they are essential for understanding reality, should not be confused with that reality itself." (Richard Levins, "The Strategy of Model Building in Population Biology", American Scientist 54(4), 1966)

"The validation of a model is not that it is 'true' but that it generates good testable hypotheses relevant to important problems." (Richard Levins, "The Strategy of Model Building in Population Biology", American Scientist 54(4), 1966)

"[…] truth is the intersection of independent lies." (Richard Levins, "The Strategy of Model Building in Population Biology", 1966)

"Unlike the theory, models are restricted by technical considerations to a few components at a time, even in systems which are complex. Thus a satisfactory theory is usually a cluster of models. These models are related to each other in several ways : as coordinate alternative models for the same set of phenomena, they jointly produce robust theorems; as complementary models they can cope with different aspects of the same problem and give complementary as well as overlapping results; as hierarchically arranged 'nested' models, each provides an interpretation of the sufficient parameters of the next higher level where they are taken as given." (Richard Levins, "The Strategy of Model Building in Population Biology", American Scientist 54(4), 1966)

"All evolutionary theories, whether of physical, biological, or social phenomena, are theories of change. The present state of a system is seen as different from its past states, and its future states are predicted to again differ from the present. But the simple assertion that past, present, and future differ from one another is not in itself an evolutionary world view." (Richard Levins & Richard C Lewontin, "The Dialectical Biologist", 1985)

"Parts and wholes evolve in consequence of their relationship, and the relationship itself evolves. These are the properties of things that we call dialectical: that one thing cannot exist without the other, that one acquires its properties from its relation to the other, that the properties of both evolve as a consequence of their interpenetration." (Richard Levins & Richard C Lewontin, "The Dialectical Biologist", 1985)

"The concept of adaptation implies that there is a preexistent form, problem, or ideal to which organisms are fitted by a dynamical process. The process is adaptation and the end result is the state of being adapted. Thus a key may be adapted to fit a lock by cutting and filing it, or a part made for one model of a machine may be used in a different model by using an adaptor to alter its shape. There cannot be adaptation with out the ideal model according to which the adaptation is taking place. Thus the very notion of adaptation inevitably carried over into modern biology the theological view of a preformed physical world to which organisms were fitted." (Richard Levins & Richard C Lewontin, "The Dialectical Biologist", 1985)

"The large-scale computer models of systems ecology do not fit under the heading of holism at all. Rather they are forms of large-scale reductionism: the objects of study are the naively given 'parts' -abundances or biomasses of populations. No new objects of study arise at the community level. The research is usually conducted on a single system - a lake, forest, or prairie - and the results are measurements of and projections for that lake, forest, or prairie, with no attempts to find the properties of lakes, forests, or prairies in general. Such modeling requires vast amounts of data for its simulations, and much of the scientific effort goes into problems of estimation." (Richard Levins & Richard C Lewontin, "The Dialectical Biologist", 1985)

"The organism cannot be regarded as simply the passive object of autonomous internal and external forces; it is also the subject of its own evolution." (Richard Levins & Richard C Lewontin, "The Dialectical Biologist", 1985)

"Things are similar: this makes science possible. Things are different: this makes science necessary. At various times in the history of science important advances have been made either by abstracting away differences to reveal similarity or by emphasizing the richness of variation within a seeming uniformity. But either choice by itself is ultimately misleading. The general does not completely contain the particular as cases, but the empiricist refusal to group, generalize, and abstract reduces science to collecting - if not specimens, then examples." (Richard Levins & Richard C Lewontin, "The Dialectical Biologist", 1985)

"We believe that science, in all its sense, is a social process that both causes and is caused by social organisation." (Richard Levins & Richard C Lewontin, "The Dialectical Biologist", 1985)

"We can hardly have a serious discussion of a science without abstraction. What makes science materialist is that the process of abstraction is explicit and recognized as historically contingent within the science. Abstraction becomes destructive when the abstract is reified and when the historical process of abstraction is forgotten, so that the abstract descriptions are taken for descriptions of the actual objects. The level of abstraction appropriate in a given science at a given time is a historical issue." (Richard Levins & Richard C Lewontin, "The Dialectical Biologist", 1985)

On Idealization: Models

"A material model is the representation of a complex system by a system which is assumed simpler and which is also assumed to have some properties similar to those selected for study in the original complex system. A formal model is a symbolic assertion in logical terms of an idealised relatively simple situation sharing the structural properties of the original factual system." (Arturo Rosenblueth & Norbert Wiener, "The Role of Models in Science", Philosophy of Science Vol. 12 (4), 1945)

"This model will be a simplification and an idealization, and consequently a falsification. It is to be hoped that the features retained for discussion are those of greatest importance in the present state of knowledge." (Alain M Turing, "The Chemical Basis of Morphogenesis" , Philosophical Transactions of the Royal Society of London, Series B: Biological Sciences, Vol. 237 (641), 1952)

"The word model is used as a noun, adjective, and verb, and in each instance it has a slightly different connotation. As a noun 'model' is a representation in the sense in which an architect constructs a small-scale model of a building or a physicist a large-scale model of an atom. As an adjective 'model' implies a degree or perfection or idealization, as in reference to a model home, a model student, or a model husband. As a verb 'to model' means to demonstrate, to reveal, to show what a thing is like." (Russell L Ackoff, "Scientific method: optimizing applied research decisions", 1962)

"This other world is the so-called physical world image; it is merely an intellectual structure. To a certain extent it is arbitrary. It is a kind of model or idealization created in order to avoid the inaccuracy inherent in every measurement and to facilitate exact definition." (Max Planck, "The Philosophy of Physics", 1963)

"[…] mathematics is not portraying laws inherent in the design of the universe but is merely providing man-made schemes or models which we can use to deduce conclusions about our world only to the extent that the model is a good idealization." (Morris Kline, "Mathematics for the Nonmathematician", 1967)

"This distinction between regular and catastrophic points is obviously somewhat arbitrary because it depends on the fineness of the observation used. One might object, not without reason, that each point is catastrophic to sufficiently sensitive observational techniques. This is why the distinction is an idealization, to be made precise by a mathematical model, and to this end we summarize some ideas of qualitative dynamics." (René F Thom, "Structural Stability and Morphogenesis", 1972)

"To call a model an idealization is to suggest that the model is a simplification of what occurs in reality, usually a simplification which omits some relevant features, such as the extended mass of the planets or, in the example of the circuit model, the resistance in the bypass capacitor. Sometimes the omitted factors make only an insignificant contribution to the effect under study. But that does not seem to be essential to idealizations, especially to the idealizations that in the end are applied by engineers to study real things. In calling something an idealization it seems not so important that the contributions from omitted factors be small, but that they be ones for which we know how to correct. If the idealization is to be of use, when the time comes to apply it to a real system we had better know how to add back the contributions of the factors that have been left out. In that case the use of idealizations does not seem to counter realism: either the omitted factors do not matter much, or in principle we know how to treat them." (Nancy Cartwright, "How the Laws of Physics Lie", 1983)

"The assumption of rationality has a favored position in economics. It is accorded all the methodological privileges of a self-evident truth, a reasonable idealization, a tautology, and a null hypothesis. Each of these interpretations either puts the hypothesis of rational action beyond question or places the burden of proof squarely on any alternative analysis of belief and choice. The advantage of the rational model is compounded because no other theory of judgment and decision can ever match it in scope, power, and simplicity." (Amos Tversky & Daniel Kahneman, "Rational Choice and the Framing of Decisions", The Journal of Business Vol. 59 (4), 1986)

"[…] if a system is sufficiently complicated, the time it takes to return near a state already visited is huge (think of the hundred fleas on the checkerboard). Therefore if you look at the system for a moderate amount of time, eternal return is irrelevant, and you had better choose another idealization." (David Ruelle, "Chance and Chaos", 1991)

"[…] it does not seem helpful just to say that all models are wrong. The very word model implies simplification and idealization. The idea that complex physical, biological or sociological systems can be exactly described by a few formulae is patently absurd. The construction of idealized representations that capture important stable aspects of such systems is, however, a vital part of general scientific analysis and statistical models, especially substantive ones, do not seem essentially different from other kinds of model." (Sir David Cox, "Comment on ‘Model uncertainty, data mining and statistical inference’", Journal of the Royal Statistical Society, Series A 158, 1995)

"Through modeling, scientists manipulate symbols with meanings to represent an environment with structure. Such manipulations take place to fulfill a human need, solve a problem, or create a product. When constructing a model, one works in the cognitive space of ideas. Models are used to encapsulate, highlight, replicate or represent patterns of events and the structures of things. Of course, no model provides an exact duplication of the subject matter being modeled. Details are hidden, features are skewed, and certain properties are emphasized. Models are abstract and idealized. As an abstraction, a model omits some features of the subject matter, while retaining only significant properties. As an idealization, a model depicts a subject's properties in a more perfect form." (Daniel Rothbart [Ed.], "Modeling: Gateway to the Unknown", 2004)

"Abstract formulations of simply stated concrete ideas are often the result of efforts to create idealized models of complex systems. The models are 'idealized' in the sense that they retain only the most fundamental properties of the original systems. The vocabulary is chosen to be as inclusive as possible so that research into the model reveals facts about a wide variety of similar systems. Unfortunately, it is often the case that over time the connection between a model and the systems on which it was based is lost, and the interested reader is faced with something that looks as if it were created to be deliberately complicated - deliberately confusing - but the original intention was just the opposite. Often, the model was devised to be simpler and more transparent than any of the systems on which it was based." (John Tabak, "Beyond Geometry: A new mathematics of space and form", 2011)

"Models can be: formulations, abstractions, replicas, idealizations, metaphors - and combinations of these. [...] Some mathematical models have been blindly used - their presuppositions as little understood as any legal fine print one ‘agrees to’ but never reads - with faith in their trustworthiness. The very arcane nature of some of the formulations of these models might have contributed to their being given so much credence. If so, we mathematicians have an important mission to perform: to help people who wish to think through the fundamental assumptions underlying models that are couched in mathematical language, making these models intelligible, rather than (merely) formidable Delphic oracles." (Barry Mazur, "The Authority of the Incomprehensible" , 2014)

“A mathematical model is a mathematical description (often by means of a function or an equation) of a real-world phenomenon such as the size of a population, the demand for a product, the speed of a falling object, the concentration of a product in a chemical reaction, the life expectancy of a person at birth, or the cost of emission reductions. The purpose of the model is to understand the phenomenon and perhaps to make predictions about future behavior. [...] A mathematical model is never a completely accurate representation of a physical situation - it is an idealization." (James Stewart, “Calculus: Early Transcedentals” 8th Ed., 2016)