On Hierarchies I: Mathematics

"The world of ideas which it discloses or illuminates, the contemplation of divine beauty and order which it induces, the harmonious connexion of its parts, the infinite hierarchy and absolute evidence of the truths with which it is concerned, these, and such like, are the surest grounds of the title of mathematics to human regard, and would remain unimpeached and unimpaired were the plan of the universe unrolled like a map at our feet, and the mind of man qualified to take in the whole scheme of creation at a glance." (James J Sylvester, "The Study That Knows Nothing of Observation", 1869)

"It is a remarkable empirical fact that mathematics can be based on set theory. More precisely, all mathematical objects can be coded as sets (in the cumulative hierarchy built by transfinitely iterating the power set operation, starting with the empty set). And all their crucial properties can be proved from the axioms of set theory. (. . . ) At first sight, category theory seems to be an exception to this general phenomenon. It deals with objects, like the categories of sets, of groups etc. that are as big as the whole universe of sets and that therefore do not admit any evident coding as sets. Furthermore, category theory involves constructions, like the functor category, that lead from these large categories to even larger ones. Thus, category theory is not just another field whose set-theoretic foundation can be left as an exercise. An interaction between category theory and set theory arises because there is a real question: What is the appropriate set-theoretic foundation for category theory?" (Andreas Blass, "The interaction between category theory and set theory", 1983)

"A distinctive feature of mathematics, that feature in virtue of which it stands as a paradigmatically rational discipline, is that assertions are not accepted without proof. […] By proof is meant a deductively valid, rationally compelling argument which shows why this must be so, given what it is to be a triangle. But arguments always have premises so that if there are to be any proofs there must also be starting points, premises which are agreed to be necessarily true, self-evident, neither capable of, nor standing in need of, further justification. The conception of mathematics as a discipline in which proofs are required must therefore also be a conception of a discipline in which a systematic and hierarchical order is imposed on its various branches. Some propositions appear as first principles, accepted without proof, and others are ordered on the basis of how directly they can be proved from these first principle. Basic theorems, once proved, are then used to prove further results, and so on. Thus there is a sense in which, so long as mathematicians demand and provide proofs, they must necessarily organize their discipline along lines approximating to the pattern to be found in Euclid's Elements." (Mary Tiles,"Mathematics and the Image of Reason", 1991)

"For centuries the mind has dominated the eye in the hierarchy of mathematical practice; today the balance is being restored as mathematicians find new ways to see patterns, both with the eye and with the mind." (Lynn A Steen, "The Future of Mathematics Education", 1998)

"That is, the physicist likes to learn from particular illustrations of a general abstract concept. The mathematician, on the other hand, often eschews the particular in pursuit of the most abstract and general formulation possible. Although the mathematician may think from, or through, particular concrete examples in coming to appreciate the likely truth of very general statements, he will hide all those intuitive steps when he comes to present the conclusions of his thinking to outsiders. It presents the results of research as a hierarchy of definitions, theorems and proofs after the manner of Euclid; this minimizes unnecessary words but very effectively disguises the natural train of thought that led to the original results." (John D Barrow, "New Theories of Everything", 2007)

"Mathematics courses are hierarchical but every new course begins with the assumption that the student is at the level of conceptual development that would be implied by an optimal understanding of the previous course. Unfortunately many mathematical ideas are so subtle and logically complex that it may take students many years to develop an adequate conceptual understanding. As a result, in practice there is a lot of 'faking it' going on and not merely on the part of the students." (William Byers, "Deep Thinking: What Mathematics Can Teach Us About the Mind", 2015)

Robbie T Nakatsu - Collected Quotes

"Why do people use mental models? First, they are used as inference tools to predict the behavior of a system under novel conditions. They enable us to predict system outcomes from system parameters: We may run our mental model by modifying the system parameters and observing how the behavior of the system changes. Second, mental models can be used to produce explanations and justifications. Such explanations may give us confidence in using e system and enable us to more readily trust the results of the system. Third, mental models can be used as mnemonic devices to facilitate remembering and long-term retention of information. Here, a mental model may provide one with a "cover story" to make the understanding of the system more memorable and easier to recall." (Robbie Nakatsu, "Diagrammatic Reasoning in AI", 1994)

"A hierarchy is a diagram that shows how various components of a system are related, often with a downward direction (or alternatively a left-to-right direction) that moves from more general to more specific. One way to envision a hierarchy is as an inverted tree: We start with a single component (referred to as the root node or topmost node), typically denoted by a square, and then we draw one or more paths leading from it to other nodes. Each of these nodes, in turn, may subdivide into additional subpaths to other nodes. This process may be repeated any number of times to arrive at a multitiered, tree-like structure." (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)

"A mental model is the user's model of a target system; it is a model of a system that exists in a person's head. Through interaction with a complex system, it is a 'naturally evolving model. As a person develops more experience with a system, the model develops and becomes more refined. Hence, at any given point in time, the mental model, as seen through the eyes of the user, is a dynamic, usually incomplete specification of the target system. A conceptual model, on the other hand, is typically the designer's complete specification of a target system. As such, it is intended to be an accurate, consistent, and complete representation of a target system. Ideally, we would want the user's mental model to be the same as the system designer's conceptual model." (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)

"Diagrams are information graphics that are made up primarily of geometric shapes, such as rectangles, circles, diamonds, or triangles, that are typically (but not always) interconnected by lines or arrows. One of the major purposes of a diagram is to show how things, people, ideas, activities, etc. interrelate and interconnect. Unlike quantitative charts and graphs, diagrams are used to show interrelationships in a qualitative way." (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)

"Models are an important form of knowledge representation because expertise often lies in one's ability to reason about how the objects, or components, of a system are interconnected - whether physically, causally, relationally, or otherwise - in a domain of discourse." (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)

"On the one hand, a conceptual model seeks to faithfully represent the components, the connections, the relations, and the processes that act on the components. On the other hand, a mental model that employs analogical representations is chosen to invite comparisons between two dissimilar domains, never to faithfully and completely represent the target domain." (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)

"Prototyping is a method of developing systems rapidly by creating a quick-and-dirty mockup of a system, called a prototype. Once created, the prototype is given to end users so that they can provide their feedback and suggestions for improvement. Based on this feedback, you modify and enhance the prototype. It is an iterative process in that you can get feedback multiple times and enhance the prototype accordingly." (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)

"Rule-based expert systems require that you preprogram all the rules that represent the knowledge in the domain. To create a realistic and complete solution. the knowledge engineer must be well versed in the domain and have a clear sense of what the decision procedures are. This understanding must take place beforehand, before the system is created: misunderstandings about the domain can be very costly later on because rule-based expert systems can be extremely difficult to modify and extend into other areas. Even when this is possible. the new rules must be created manually - the expert system does not learn how to tine-tune the rules on its own. Case-based reasoning and neural networks are two Al approaches that are more suitable when you want to create a system that 'learns' how to solve problems on its own." (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)

"Semantic networks are used to illustrate how people organize information in their memories. Such representations have been used by cognitive psychologists to understand and theorize how one retrieves and processes information from long-term memory. In AI, semantic networks can also be used as a knowledge representation scheme that programs can use to retrieve information efficiently just like humans do." (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)

"Structure is the way that the individual components of a system are interconnected (as given by a system's topology); Behavior refers to what each of these components is supposed to do. From this definition, we may distinguish three levels of system description. First, diagrams are models, graphical in nature, that are used to illustrate structure (e.g., how components are physically interconnected); they do not capture functional behavior of a system. Second, heuristics describe relationships between inputs and outputs, based on the way that experts describe how inputs are transformed into outputs. (Heuristics may be represented as IF-THEN rules). Heuristic knowledge, however, does not attempt to create an explicit representation of system structure. Model-based reasoning is a more complete representation system in the sense that it describes both structure and behavior. From this, three levels of system description can be distinguished, based on whether they describe structure, behavior, or both."  (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)

"The development of a mental model, then, can be chronicled, much like the development of a cognitive skill. Three developmental processes8 seem to be at play when a mental model evolves. that mental model becomes more powerful because it works for a wider variety of situations. Second, discrimination means that a mental model is more sensitive to variations in a given situation so that a mental model may add an important new condition where previously it had been overlooked. Third, strengthening means that those aspects of a mental model that have been successfully applied in the past are strengthened and rendered more salient and significant." (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)

"This is always the case in analogical reasoning: Relations between two dissimilar domains never map completely to one another. In fact, it is often the salient similarities between the base and target domains that provoke thought and increase the usefulness of an analogy as a problem-solving tool." (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)

"Venn diagramming, it turns out, is a very effective technique for performing syllogistic reasoning. Its chief advantage (over the Euler graph in particular as we noted earlier) is the ability to incrementally add knowledge to the diagram. While an Euler graph has visual power in terms of representing the relations between sets very intuitively, it is impossible to combine more than one piece of information onto a Euler graph. A Venn diagram, on the other hand, easily lends itself to the representation of partial knowledge and can be manipulated to add successively more knowledge to the diagram. This means that when our knowledge of the relations between sets increases, we simply put in more symbols and shadings into the appropriate compartments of the Venn diagram. Thus we are able to accumulate knowledge in a Venn diagram. This capability turns out to be a powerful feature, one that endows Venn diagrams with a more dynamic quality that is sorely lacking in the Euler system." (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)

"What advantages do diagrams have over verbal descriptions in promoting system understanding? First, by providing a diagram, massive amounts of information can be presented more efficiently. A diagram can strip down informational complexity to its core - in this sense, it can result in a parsimonious, minimalist description of a system. Second, a diagram can help us see patterns in information and data that may appear disordered otherwise. For example, a diagram can help us see mechanisms of cause and effect or can illustrate sequence and flow in a complex system. Third, a diagram can result in a less ambiguous description than a verbal description because it forces one to come up with a more structured description." (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)

On Randomness XXIX (Networks)

"The first attempts to consider the behavior of so-called 'random neural nets' in a systematic way have led to a series of problems concerned with relations between the 'structure' and the 'function' of such nets. The 'structure' of a random net is not a clearly defined topological manifold such as could be used to describe a circuit with explicitly given connections. In a random neural net, one does not speak of 'this' neuron synapsing on 'that' one, but rather in terms of tendencies and probabilities associated with points or regions in the net." (Anatol Rapoport, "Cycle distributions in random nets", The Bulletin of Mathematical Biophysics 10(3), 1948)

"In a random network the peak of the distribution implies that the vast majority of nodes have the same number of links and that nodes deviating from the average are extremely rare. Therefore, a random network has a characteristic scale in its node connectivity, embodied by the average node and fixed by the peak of the degree distribution. In contrast, the absence of a peak in a power-law degree distribution implies that in a real network there is no such thing as a characteristic node. We see a continuous hierarchy of nodes, spanning from rare hubs to the numerous tiny nodes. The largest hub is closely fol - lowed by two or three somewhat smaller hubs, followed by dozens that are even smaller, and so on, eventually arriving at the numerous small nodes." (Albert-László Barabási, "Linked: How Everything Is Connected to Everything Else and What It Means for Business, Science, and Everyday Life", 2002)

"Networks are not en route from a random to an ordered state. Neither are they at the edge of randomness and chaos. Rather, the scale-free topology is evidence of organizing principles acting at each stage of the network formation process." (Albert-László Barabási, "Linked: How Everything Is Connected to Everything Else and What It Means for Business, Science, and Everyday Life", 2002)

"[…] real networks not only are connected but are well beyond the threshold of one. Random network theory tells us that as the average number of links per node increases beyond the critical one, the number of nodes left out of the giant cluster decreases exponentially. That is, the more links we add, the harder it is to find a node that remains isolated. Nature does not take risks by staying close to the threshold. It well surpasses it."  (Albert-László Barabási, "Linked: How Everything Is Connected to Everything Else and What It Means for Business, Science, and Everyday Life", 2002)

"Regular graphs are unique in that each node has exactly the same number of links. […] Such regularity is clearly absent from random graphs. The premise of the random network model is deeply egalitarian: We place the links completely randomly; thus all nodes have the same chance of getting one […] If the network is large, despite the links' completely random placement, almost all nodes will have approximately the same number of links." (Albert-László Barabási, "Linked: How Everything Is Connected to Everything Else and What It Means for Business, Science, and Everyday Life", 2002) 

"'There is an old debate', Erdos liked to say, 'about whether you create mathematics or just discover it. In other words, are the truths already there, even if we don't yet know them?' Erdos had a clear answer to this question: Mathematical truths are there among the list of absolute truths, and we just rediscover them. Random graph theory, so elegant and simple, seemed to him to belong to the eternal truths. Yet today we know that random networks played little role in assembling our universe. Instead, nature resorted to a few fundamental laws, which will be revealed in the coming chapters. Erdos himself created mathematical truths and an alternative view of our world by developing random graph theory." (Albert-László Barabási, "Linked: How Everything Is Connected to Everything Else and What It Means for Business, Science, and Everyday Life", 2002)

"In colloquial usage, chaos means a state of total disorder. In its technical sense, however, chaos refers to a state that only appears random, but is actually generated by nonrandom laws. As such, it occupies an unfamiliar middle ground between order and disorder. It looks erratic superficially, yet it contains cryptic patterns and is governed by rigid rules. It's predictable in the short run but unpredictable in the long run. And it never repeats itself: Its behavior is nonperiodic." (Steven Strogatz, "Sync: The Emerging Science of Spontaneous Order", 2003)

"Like regular networks, random ones are seductive idealizations. Theorists find them beguiling, not because of their verisimilitude, but because they're the easiest ones to analyze. [...] Random networks are small and poorly clustered; regular ones are big and highly clustered." (Steven Strogatz, "Sync: The Emerging Science of Spontaneous Order", 2003)

"In the telephone system a century ago, messages dispersed across the network in a pattern that mathematicians associate with randomness. But in the last decade, the flow of bits has become statistically more similar to the patterns found in self-organized systems. For one thing, the global network exhibits self-similarity, also known as a fractal pattern. We see this kind of fractal pattern in the way the jagged outline of tree branches look similar no matter whether we look at them up close or far away. Today messages disperse through the global telecommunications system in the fractal pattern of self-organization." (Kevin Kelly, "What Technology Wants", 2010)

"Although cascading failures may appear random and unpredictable, they follow reproducible laws that can be quantified and even predicted using the tools of network science. First, to avoid damaging cascades, we must understand the structure of the network on which the cascade propagates. Second, we must be able to model the dynamical processes taking place on these networks, like the flow of electricity. Finally, we need to uncover how the interplay between the network structure and dynamics affects the robustness of the whole system." (Albert-László Barabási, "Network Science", 2016)

On Events: Rare Events

"We must rather seek for a cause, for every event whether probable or improbable must have some cause." (Polybius, "The Histories", cca. 100 BC)

"There is nothing in the nature of a miracle that should render it incredible: its credibility depends upon the nature of the evidence by which it is supported. An event of extreme probability will not necessarily command our belief unless upon a sufficiency of proof; and so an event which we may regard as highly improbable may command our belief if it is sustained by sufficient evidence. So that the credibility or incredibility of an event does not rest upon the nature of the event itself, but depends upon the nature and sufficiency of the proof which sustains it." (Charles Babbage, "Passages from the Life of a Philosopher", 1864)

"Events with a sufficiently small probability never occur, or at least we must act, in all circumstances, as if they were impossible." (Émile Borel, "Probabilities and Life", 1962)

"Most accidents in well-designed systems involve two or more events of low probability occurring in the worst possible combination." (Robert E Machol, "Principles of Operations Research", 1975)

"[…] all human beings - professional mathematicians included - are easily muddled when it comes to estimating the probabilities of rare events. Even figuring out the right question to ask can be confusing." (Steven Strogatz, "Sync: The Emerging Science of Spontaneous Order", 2003)

"Bell curves don't differ that much in their bells. They differ in their tails. The tails describe how frequently rare events occur. They describe whether rare events really are so rare. This leads to the saying that the devil is in the tails." (Bart Kosko, "Noise", 2006)

"A Black Swan is a highly improbable event with three principal characteristics: It is unpredictable; it carries a massive impact; and, after the fact, we concoct an explanation that makes it appear less random, and more predictable, than it was. […] The Black Swan idea is based on the structure of randomness in empirical reality. [...] the Black Swan is what we leave out of simplification." (Nassim N Taleb, “The Black Swan”, 2007)

"A forecaster should almost never ignore data, especially when she is studying rare events […]. Ignoring data is often a tip-off that the forecaster is overconfident, or is overfitting her model - that she is interested in showing off rather than trying to be accurate."  (Nate Silver, "The Signal and the Noise: Why So Many Predictions Fail-but Some Don't", 2012)

"[…] according to the bell-shaped curve the likelihood of a very-large-deviation event (a major outlier) located in the striped region appears to be very unlikely, essentially zero. The same event, though, is several thousand times more likely if it comes from a set of events obeying a fat-tailed distribution instead of the bell-shaped one." (John L Casti, "X-Events: The Collapse of Everything", 2012)

"[…] both rarity and impact have to go into any meaningful characterization of how black any particular [black] swan happens to be." (John L Casti, "X-Events: The Collapse of Everything", 2012)

"Black Swans (capitalized) are large-scale unpredictable and irregular events of massive consequence - unpredicted by a certain observer, and such un - predictor is generally called the 'turkey' when he is both surprised and harmed by these events. [...] Black Swans hijack our brains, making us feel we 'sort of' or 'almost' predicted them, because they are retrospectively explainable. We don’t realize the role of these Swans in life because of this illusion of predictability. […] An annoying aspect of the Black Swan problem - in fact the central, and largely missed, point - is that the odds of rare events are simply not computable." (Nassim N Taleb, "Antifragile: Things that gain from disorder", 2012)

"Behavioral finance so far makes conclusions from statics not dynamics, hence misses the picture. It applies trade-offs out of context and develops the consensus that people irrationally overestimate tail risk (hence need to be 'nudged' into taking more of these exposures). But the catastrophic event is an absorbing barrier. No risky exposure can be analyzed in isolation: risks accumulate. If we ride a motorcycle, smoke, fly our own propeller plane, and join the mafia, these risks add up to a near-certain premature death. Tail risks are not a renewable resource." (Nassim N Taleb, "Statistical Consequences of Fat Tails: Real World Preasymptotics, Epistemology, and Applications" 2nd Ed., 2022)

"But note that any heavy tailed process, even a power law, can be described in sample (that is finite number of observations necessarily discretized) by a simple Gaussian process with changing variance, a regime switching process, or a combination of Gaussian plus a series of variable jumps (though not one where jumps are of equal size […])." (Nassim N Taleb, "Statistical Consequences of Fat Tails: Real World Preasymptotics, Epistemology, and Applications" 2nd Ed., 2022)

"[…] it is not merely that events in the tails of the distributions matter, happen, play a large role, etc. The point is that these events play the major role and their probabilities are not (easily) computable, not reliable for any effective use. The implication is that Black Swans do not necessarily come from fat tails; the problem can result from an incomplete assessment of tail events." (Nassim N Taleb, "Statistical Consequences of Fat Tails: Real World Preasymptotics, Epistemology, and Applications" 2nd Ed., 2022)

"Once we know something is fat-tailed, we can use heuristics to see how an exposure there reacts to random events: how much is a given unit harmed by them. It is vastly more effective to focus on being insulated from the harm of random events than try to figure them out in the required details (as we saw the inferential errors under thick tails are huge). So it is more solid, much wiser, more ethical, and more effective to focus on detection heuristics and policies rather than fabricate statistical properties." (Nassim N Taleb, "Statistical Consequences of Fat Tails: Real World Preasymptotics, Epistemology, and Applications" 2nd Ed., 2022)

On Synchronicity

"All the events in a man's life would accordingly stand in two fundamentally different kinds of connection: firstly, in the objective, causal connection of the natural process; secondly, in a subjective connection which exists only in relation to the individual who experiences it, and which is thus as subjective as his own dreams [...]" (Arthur Schopenhauer, "Parerga and Paralipomena", 1851)

“Synchronistic phenomena prove the simultaneous occurrence of meaningful equivalences in heterogeneous, causally unrelated processes; in other words, they prove that a content perceived by an observer can, at the same time, be represented by an outside event, without any causal connection. From this it follows either that the psyche cannot be localized in space, or that space is relative to the psyche. The same applies to the temporal determination of the psyche and the psychic relativity of time. I do not need to emphasize that the verification of these findings must have far-reaching consequences.” (Carl G Jung, Synchronicity: An Acausal Connecting Principle, 1952)

"Cellular automata are mathematical models for complex natural systems containing large numbers of simple identical components with local interactions. They consist of a lattice of sites, each with a finite set of possible values. The value of the sites evolve synchronously in discrete time steps according to identical rules. The value of a particular site is determined by the previous values of a neighbourhood of sites around it." (Stephen Wolfram, "Nonlinear Phenomena, Universality and complexity in cellular automata", Physica 10D, 1984)

"A depressing corollary of the butterfly effect (or so it was widely believed) was that two chaotic systems could never synchronize with each other. Even if you took great pains to start them the same way, there would always be some infinitesimal difference in their initial states. Normally that small discrepancy would remain small for a long time, but in a chaotic system, the error cascades and feeds on itself so swiftly that the systems diverge almost immediately, destroying the synchronization. Unfortunately, it seemed, two of the most vibrant branches of nonlinear science - chaos and sync - could never be married. They were fundamentally incompatible." (Steven Strogatz, "Sync: The Emerging Science of Spontaneous Order", 2003)

"The best case that can be made for human sync to the environment (outside of circadian entrainment) has to do with the possibility that electrical rhythms in our brains can be influenced by external signals." (Steven Strogatz, "Sync: The Emerging Science of Spontaneous Order", 2003)

"This synergistic character of nonlinear systems is precisely what makes them so difficult to analyze. They can't be taken apart. The whole system has to be examined all at once, as a coherent entity. As we've seen earlier, this necessity for global thinking is the greatest challenge in understanding how large systems of oscillators can spontaneously synchronize themselves. More generally, all problems about self-organization are fundamentally nonlinear. So the study of sync has always been entwined with the study of nonlinearity." (Steven Strogatz, "Sync: The Emerging Science of Spontaneous Order", 2003)

"There is no linear additive process that, if all the parts are taken together, can be understood to create the total system that occurs at the moment of self-organization; it is not a quantity that comes into being. It is not predictable in its shape or subsequent behavior or its subsequent qualities. There is a nonlinear quality that comes into being at the moment of synchronicity." (Stephen H Buhner, "Plant Intelligence and the Imaginal Realm: Beyond the Doors of Perception into the Dreaming of Earth", 2014)

"The most amazing thing about social insect colonies is that there's no individual in charge. If you look at a single ant, you may have the impression that it is behaving, if not randomly, at least not in synchrony with the rest of the colony. You feel that it is doing its own things without paying too much attention to what the others are doing." (Eric Bonabeau)

On Inferences (1975-1999)

"[Fuzzy logic is] a logic whose distinguishing features are (1) fuzzy truth-values expressed in linguistic terms, e. g., true, very true, more or less true, or somewhat true, false, nor very true and not very false, etc.; (2) imprecise truth tables; and (3) rules of inference whose validity is relative to a context rather than exact." (Lotfi A. Zadeh, "Fuzzy logic and approximate reasoning", 1975)

"Pencil and paper for construction of distributions, scatter diagrams, and run-charts to compare small groups and to detect trends are more efficient methods of estimation than statistical inference that depends on variances and standard errors, as the simple techniques preserve the information in the original data." (W Edwards Deming, "On Probability as Basis for Action", American Statistician, Volume 29, Number 4, November 1975)

"The treatment of the economy as a single system, to be controlled toward a consistent goal, allowed the efficient systematization of enormous information material, its deep analysis for valid decision-making. It is interesting that many inferences remain valid even in cases when this consistent goal could not be formulated, either for the reason that it was not quite clear or for the reason that it was made up of multiple goals, each of which to be taken into account." (Leonid V Kantorovich, "Mathematics in Economics: Achievements, Difficulties, Perspectives", [Nobel lecture] 1975)

"The advantage of semantic networks over standard logic is that some selected set of the possible inferences can be made in a specialized and efficient way. If these correspond to the inferences that people make naturally, then the system will be able to do a more natural sort of reasoning than can be easily achieved using formal logical deduction." (Avron Barr, Natural Language Understanding, AI Magazine Vol. 1 (1), 1980)

"Models are often used to decide issues in situations marked by uncertainty. However statistical differences from data depend on assumptions about the process which generated these data. If the assumptions do not hold, the inferences may not be reliable either. This limitation is often ignored by applied workers who fail to identify crucial assumptions or subject them to any kind of empirical testing. In such circumstances, using statistical procedures may only compound the uncertainty." (David A Greedman & William C Navidi, "Regression Models for Adjusting the 1980 Census", Statistical Science Vol. 1 (1), 1986)

"It is difficult to distinguish deduction from what in other circumstances is called problem-solving. And concept learning, inference, and reasoning by analogy are all instances of inductive reasoning. (Detectives typically induce, rather than deduce.) None of these things can be done separately from each other, or from anything else. They are pseudo-categories." (Frank Smith, "To Think: In Language, Learning and Education", 1990)

"[…] semantic nets fail to be distinctive in the way they (1) represent propositions, (2) cluster information for access, (3) handle property inheritance, and (4) handle general inference; in other words, they lack distinctive representational properties (i.e., 1) and distinctive computational properties (i.e., 2-4). Certain propagation mechanisms, notably 'spreading activation', 'intersection search', or 'inference propagation' have sometimes been regarded as earmarks of semantic nets, but since most extant semantic nets lack such mechanisms, they cannot be considered criterial in current usage." (Lenhart K Schubert, "Semantic Nets are in the Eye of the Beholder", 1990)

"This absolutist view of mathematical knowledge is based on two types of assumptions: those of mathematics, concerning the assumption of axioms and definitions, and those of logic concerning the assumption of axioms, rules of inference and the formal language and its syntax. These are local or micro-assumptions. There is also the possibility of global or macro-assumptions, such as whether logical deduction suffices to establish all mathematical truths." (Paul Ernest, "The Philosophy of Mathematics Education", 1991)

"The essential idea of semantic networks is that the graph-theoretic structure of relations and abstractions can be used for inference as well as understanding. […] A semantic network is a discrete structure as is any linguistic description. Representation of the continuous 'outside world' with such a structure is necessarily incomplete, and requires decisions as to which information is kept and which is lost." (Fritz Lehman, "Semantic Networks",  Computers & Mathematics with Applications Vol. 23 (2-5), 1992)

"Virtually all mathematical theorems are assertions about the existence or nonexistence of certain entities. For example, theorems assert the existence of a solution to a differential equation, a root of a polynomial, or the nonexistence of an algorithm for the Halting Problem. A platonist is one who believes that these objects enjoy a real existence in some mystical realm beyond space and time. To such a person, a mathematician is like an explorer who discovers already existing things. On the other hand, a formalist is one who feels we construct these objects by our rules of logical inference, and that until we actually produce a chain of reasoning leading to one of these objects they have no meaningful existence, at all." (John L Casti, "Reality Rules: Picturing the world in mathematics" Vol. II, 1992)

"When the distributions of two or more groups of univariate data are skewed, it is common to have the spread increase monotonically with location. This behavior is monotone spread. Strictly speaking, monotone spread includes the case where the spread decreases monotonically with location, but such a decrease is much less common for raw data. Monotone spread, as with skewness, adds to the difficulty of data analysis. For example, it means that we cannot fit just location estimates to produce homogeneous residuals; we must fit spread estimates as well. Furthermore, the distributions cannot be compared by a number of standard methods of probabilistic inference that are based on an assumption of equal spreads; the standard t-test is one example. Fortunately, remedies for skewness can cure monotone spread as well." (William S Cleveland, "Visualizing Data", 1993)

"The science of statistics may be described as exploring, analyzing and summarizing data; designing or choosing appropriate ways of collecting data and extracting information from them; and communicating that information. Statistics also involves constructing and testing models for describing chance phenomena. These models can be used as a basis for making inferences and drawing conclusions and, finally, perhaps for making decisions." (Fergus Daly et al, "Elements of Statistics", 1995)

"Fuzzy systems are excellent tools for representing heuristic, commonsense rules. Fuzzy inference methods apply these rules to data and infer a solution. Neural networks are very efficient at learning heuristics from data. They are 'good problem solvers' when past data are available. Both fuzzy systems and neural networks are universal approximators in a sense, that is, for a given continuous objective function there will be a fuzzy system and a neural network which approximate it to any degree of accuracy." (Nikola K Kasabov, "Foundations of Neural Networks, Fuzzy Systems, and Knowledge Engineering", 1996)

"Theories rarely arise as patient inferences forced by accumulated facts. Theories are mental constructs potentiated by complex external prods (including, in idealized cases, a commanding push from empirical reality)." (Stephen J Gould, "Leonardo's Mountain of Clams and the Diet of Worms", 1998)

"Whereas formal systems apply inference rules to logical variables, neural networks apply evolutive principles to numerical variables. Instead of calculating a solution, the network settles into a condition that satisfies the constraints imposed on it." (Paul Cilliers, "Complexity and Postmodernism: Understanding Complex Systems", 1998)

"Let us regard a proof of an assertion as a purely mechanical procedure using precise rules of inference starting with a few unassailable axioms. This means that an algorithm can be devised for testing the validity of an alleged proof simply by checking the successive steps of the argument; the rules of inference constitute an algorithm for generating all the statements that can be deduced in a finite number of steps from the axioms." (Edward Beltrami, "What is Random?: Chaos and Order in Mathematics and Life", 1999)

"[…] philosophical theories are structured by conceptual metaphors that constrain which inferences can be drawn within that philosophical theory. The (typically unconscious) conceptual metaphors that are constitutive of a philosophical theory have the causal effect of constraining how you can reason within that philosophical framework." (George Lakoff, "Philosophy in the Flesh: The Embodied Mind and its Challenge to Western Thought", 1999)

On Inference (2010-2019)

"A second approach to statistical inference is estimation, which focuses on finding the best point estimate of the population parameter that’s of greatest interest; it also gives an interval estimate of that parameter, to signal how close our point estimate is likely to be to the population value." (Geoff Cumming, "Understanding the New Statistics", 2012)

"Regression analysis, like all forms of statistical inference, is designed to offer us insights into the world around us. We seek patterns that will hold true for the larger population. However, our results are valid only for a population that is similar to the sample on which the analysis has been done." (Charles Wheelan, "Naked Statistics: Stripping the Dread from the Data", 2012)

"Statistical inference is the drawing of conclusions about the world (more specifically: about some population) from our sample data." (Geoff Cumming, "Understanding the New Statistics", 2012)

"The four questions of data analysis are the questions of description, probability, inference, and homogeneity. [...] Descriptive statistics are built on the assumption that we can use a single value to characterize a single property for a single universe. […] Probability theory is focused on what happens to samples drawn from a known universe. If the data happen to come from different sources, then there are multiple universes with different probability models.  [...] Statistical inference assumes that you have a sample that is known to have come from one universe." (Donald J Wheeler," Myths About Data Analysis", International Lean & Six Sigma Conference, 2012)

"Mental models represent possibilities, and the theory of mental models postulates three systems of mental processes underlying inference: (0) the construction of an intentional representation of a premise’s meaning – a process guided by a parser; (1) the building of an initial mental model from the intension, and the drawing of a conclusion based on heuristics and the model; and (2) on some occasions, the search for alternative models, such as a counterexample in which the conclusion is false. System 0 is linguistic, and it may be autonomous. System 1 is rapid and prone to systematic errors, because it makes no use of a working memory for intermediate results. System 2 has access to working memory, and so it can carry out recursive processes, such as the construction of alternative models." (Sangeet Khemlania & P.N. Johnson-Laird, "The processes of inference", Argument and Computation, 2012)

"The true foundations of mathematics do not lie in axioms, definitions, and logical inference, which are the foundational elements of formal mathematics. The true foundations of mathematics lie in the minds of mathematicians as they interact with and try to make sense of their world - in their ideas, their intuitions, and their aesthetic sensibility." (William Byers, "Deep Thinking: What Mathematics Can Teach Us About the Mind", 2015)

"Again, classical statistics only summarizes data, so it does not provide even a language for asking [a counterfactual] question. Causal inference provides a notation and, more importantly, offers a solution. As with predicting the effect of interventions [...], in many cases we can emulate human retrospective thinking with an algorithm that takes what we know about the observed world and produces an answer about the counterfactual world." (Judea Pearl & Dana Mackenzie, "The Book of Why: The new science of cause and effect", 2018)

On Inference (2000-2009)

"Even if our cognitive maps of causal structure were perfect, learning, especially double-loop learning, would still be difficult. To use a mental model to design a new strategy or organization we must make inferences about the consequences of decision rules that have never been tried and for which we have no data. To do so requires intuitive solution of high-order nonlinear differential equations, a task far exceeding human cognitive capabilities in all but the simplest systems."  (John D Sterman, "Business Dynamics: Systems thinking and modeling for a complex world", 2000)

"The difficulty facing us when we have to make inferences is two-fold. First, we may build entirely the wrong mental model from the information we read or hear. […] The second difficulty facing us is that we may well build a reasonably correct initial representation of a problem, but this representation may be impoverished in some way because we have no idea what inferences are relevant […]" (S Ian Robertson, "Problem Solving", 2001)

"Ignorance of relevant risks and miscommunication of those risks are two aspects of innumeracy. A third aspect of innumeracy concerns the problem of drawing incorrect inferences from statistics. This third type of innumeracy occurs when inferences go wrong because they are clouded by certain risk representations. Such clouded thinking becomes possible only once the risks have been communicated." (Gerd Gigerenzer, "Calculated Risks: How to know when numbers deceive you", 2002)

"Information needs representation. The idea that it is possible to communicate information in a 'pure' form is fiction. Successful risk communication requires intuitively clear representations. Playing with representations can help us not only to understand numbers (describe phenomena) but also to draw conclusions from numbers (make inferences). There is no single best representation, because what is needed always depends on the minds that are doing the communicating." (Gerd Gigerenzer, "Calculated Risks: How to know when numbers deceive you", 2002)

"Natural frequencies facilitate inferences made on the basis of numerical information. The representation does part of the reasoning, taking care of the multiplication the mind would have to perform if given probabilities. In this sense, insight can come from outside the mind." (Gerd Gigerenzer, "Calculated Risks: How to know when numbers deceive you", 2002)

"Overcoming innumeracy is like completing a three-step program to statistical literacy. The first step is to defeat the illusion of certainty. The second step is to learn about the actual risks of relevant events and actions. The third step is to communicate the risks in an understandable way and to draw inferences without falling prey to clouded thinking. The general point is this: Innumeracy does not simply reside in our minds but in the representations of risk that we choose." (Gerd Gigerenzer, "Calculated Risks: How to know when numbers deceive you", 2002)

"When natural frequencies are transformed into conditional probabilities, the base rate information is taken out (this is called normalization). The benefit of this normalization is that the resulting values fall within the uniform range of 0 and 1. The cost, however, is that when drawing inferences from probabilities (as opposed to natural frequencies), one has to put the base rates back in by multiplying the conditional probabilities by their respective base rates." (Gerd Gigerenzer, "Calculated Risks: How to know when numbers deceive you", 2002)

"Bayesian inference is a controversial approach because it inherently embraces a subjective notion of probability. In general, Bayesian methods provide no guarantees on long run performance." (Larry A Wasserman, "All of Statistics: A concise course in statistical inference", 2004)

"The Bayesian approach is based on the following postulates: (B1) Probability describes degree of belief, not limiting frequency. As such, we can make probability statements about lots of things, not just data which are subject to random variation. […] (B2) We can make probability statements about parameters, even though they are fixed constants. (B3) We make inferences about a parameter θ by producing a probability distribution for θ. Inferences, such as point estimates and interval estimates, may then be extracted from this distribution." (Larry A Wasserman, "All of Statistics: A concise course in statistical inference", 2004)

"Statistical inference, or 'learning' as it is called in computer science, is the process of using data to infer the distribution that generated the data." (Larry A Wasserman, "All of Statistics: A concise course in statistical inference", 2004)

"A mental model is conceived […] as a knowledge structure possessing slots that can be filled not only with empirically gained information but also with ‘default assumptions’ resulting from prior experience. These default assumptions can be substituted by updated information so that inferences based on the model can be corrected without abandoning the model as a whole. Information is assimilated to the slots of a mental model in the form of ‘frames’ which are understood here as ‘chunks’ of knowledge with a well-defined meaning anchored in a given body of shared knowledge." (Jürgen Renn, "Before the Riemann Tensor: The Emergence of Einstein’s Double Strategy", "The Universe of General Relativity" Ed. by A.J. Kox & Jean Eisenstaedt, 2005)

"Statistics is the branch of mathematics that uses observations and measurements called data to analyze, summarize, make inferences, and draw conclusions based on the data gathered." (Allan G Bluman, "Probability Demystified", 2005)

"The basic idea of going from an estimate to an inference is simple. Drawing the conclusion with confidence, and measuring the level of confidence, is where the hard work of professional statistics comes in." (Charles Livingston & Paul Voakes, "Working with Numbers and Statistics: A handbook for journalists", 2005)

"The dual meaning of the word significant brings into focus the distinction between drawing a mathematical inference and practical inference from statistical results." (Charles Livingston & Paul Voakes, "Working with Numbers and Statistics: A handbook for journalists", 2005)

"In specific cases, we think by applying mental rules, which are similar to rules in computer programs. In most of the cases, however, we reason by constructing, inspecting, and manipulating mental models. These models and the processes that manipulate them are the basis of our competence to reason. In general, it is believed that humans have the competence to perform such inferences error-free. Errors do occur, however, because reasoning performance is limited by capacities of the cognitive system, misunderstanding of the premises, ambiguity of problems, and motivational factors. Moreover, background knowledge can significantly influence our reasoning performance. This influence can either be facilitation or an impedance of the reasoning process." (Carsten Held et al, "Mental Models and the Mind", 2006)

"Mental models abide by the principle of parsimony: They represent only possibilities compatible with the premises, and they represent clauses in the premises only when they hold in a possibility. Fully explicit models represent clauses when they do not hold too. The advantage of mental models over fully explicit models is that they contain less information, and so they are easier to work with. But they can lead reasoners astray. The occurrence of these systematic and compelling fallacies is shocking. The model theory predicts them, and they are a 'litmus' test for mental models, because no other current theory predicts them. They have so far resisted explanation by theories of reasoning based on formal rules of inference, because these theories rely on valid rules." (Philip N Johnson-Laird, Mental Models, Sentential Reasoning, and Illusory Inferences, [in "Mental Models and the Mind"], 2006)

"In mathematics, the first principles are called axioms, and the rules are referred to as deduction/inference rules. A proof is a series of steps based on the (adopted) axioms and deduction rules which reaches a desired conclusion. Every step in a proof can be checked for correctness by examining it to ensure that it is logically sound." (Cristian S Calude et al, "Proving and Programming", 2007)

"[…] statistics is the key discipline for predicting the future or for making inferences about the unknown, or for producing convenient summaries of data." (David J Hand, "Statistics: A Very Short Introduction", 2008)

"Case-based reasoning is a paradigm in machine learning whose idea is that a new problem can be solved by noticing its similarity to a set of problems previously solved. Case-based reasoning regards the inference of some proper conclusions related to a new situation by the analysis of similar cases from a memory of previous cases. Very often similarity between two objects is expressed on a graded scale and this justifies application of fuzzy sets in this context." (Salvatore Greco et al, "Granular Computing and Data Mining for Ordered Data: The Dominance-Based Rough Set Approach", 2009)

"Causal inference is different, because a change in the system is contemplated - an intervention. Descriptive statistics tell you about the data that you happen to have. Causal models claim to tell you what will happen to some of the numbers if you intervene to change other numbers." (David A. Freedman, "Statistical Models: Theory and Practice", 2009)

"Traditional statistics is strong in devising ways of describing data and inferring distributional parameters from sample. Causal inference requires two additional ingredients: a science-friendly language for articulating causal knowledge, and a mathematical machinery for processing that knowledge, combining it with data and drawing new causal conclusions about a phenomenon." (Judea Pearl, "Causal inference in statistics: An overview", Statistics Surveys 3, 2009)

On Inferences (1950-1974)

"The study of inductive inference belongs to the theory of probability, since observational facts can make a theory only probable but will never make it absolutely certain." (Hans Reichenbach, "The Rise of Scientific Philosophy", 1951)

"The technical analysis of any large collection of data is a task for a highly trained and expensive man who knows the mathematical theory of statistics inside and out. Otherwise the outcome is likely to be a collection of drawings - quartered pies, cute little battleships, and tapering rows of sturdy soldiers in diversified uniforms - interesting enough in the colored Sunday supplement, but hardly the sort of thing from which to draw reliable inferences." (Eric T Bell, "Mathematics: Queen and Servant of Science", 1951)

"Statistics is the name for that science and art which deals with uncertain inferences - which uses numbers to find out something about nature and experience." (Warren Weaver, 1952)

"From the outset it was clear that the two kinds of reasoning have different tasks. From the outset. they appeared very different: demonstrative reasoning as definite, final, 'machinelike'; and plausible reasoning as vague, provisional, specifically 'human'. Now we may see the difference a little more distinctly. In opposition to demonstrative inference, plausible inference leaves indeterminate a highly relevant point: the 'strength' or the 'weight' of the conclusion. This weight may depend not only on clarified grounds such as those expressed in the premises, hut also on unclarified unexpressed grounds somewhere on the background of the person who draws the conclusion. A person has a background, a machine has not. Indeed, you can build a machine to draw demonstrative conclusions for you, but I think you can never build a machine that will draw plausible inferences." (George Pólya, "Mathematics and Plausible Reasoning", 1954)

"The result of the mathematician's creative work is demonstrative reasoning, a proof; but the proof is discovered by plausible reasoning, by guessing. If the learning of mathematics reflects to any degree the invention of mathematics, it must have a place for guessing, for plausible inference." (George Pólya, "Induction and Analogy in Mathematics", 1954)

"The heart of all major discoveries in the physical sciences is the discovery of novel methods of representation and so of fresh techniques by which inferences can be drawn - and drawn in ways which fit the phenomena under investigation." (Stephen Toulmin, "The Philosophy of Science", 1957)

"The first [principle], is that a mathematical theory can only he developed axiomatically in a fruitful way when the student has already acquired some familiarity with the corresponding material - a familiarity gained by working long enough with it on a kind of experimental, or semiexperimental basis, i.e. with constant appeal to intuition. The other principle [...]  is that when logical inference is introduced in some mathematical question, it should always he presented with absolute honesty - that is, without trying to hide gaps or flaws in the argument; any other way, in my opinion, is worse than giving no proof at all." (Jean Dieudonné, "Thinking in School Mathematics", 1961)

"Statistics is that branch of mathematics which deals with the accumulation and analysis of quantitative data. There are three principal subdivisions in the field of statistics but these overlap, more often than not, in actual practice. First, inference from samples to population by means of probability is called statistical inference. Second, descriptive statistics is defined as the characterization and summarization of a given set of data without direct reference to inference. And finally, sampling statistics deals with methods of obtaining samples for statistical inference." (David B MacNeil, "Modern Mathematics for the Practical Man", 1963)

"A mathematical proof, as usually written down, is a sequence of expressions in the state space. But we may also think of the proof as consisting of the sequence of justifications of consecutive proof steps - i.e., the references to axioms, previously-proved theorems, and rules of inference that legitimize the writing down of the proof steps. From this point of view, the proof is a sequence of actions (applications of rules of inference) that, operating initially on the axioms, transform them into the desired theorem." (Herbert A Simon, "The Logic of Heuristic Decision Making", [in "The Logic of Decision and Action"], 1966)

"Inductive inference is the only process known to us by which essential new knowledge comes into the world." (Sir Ronald A Fisher, "The Design of Experiments", 1971)

"Probability theory, for us, is not so much a part of mathematics as a part of logic, inductive logic, really. It provides a consistent framework for reasoning about statements whose correctness or incorrectness cannot be deduced from the hypothesis. The information available is sufficient only to make the inferences 'plausible' to a greater or lesser extent.". (Ralph Baierlein, "Atoms and Information Theory: An Introduction to Statistical Mechanics", 1971)

"The statistician cannot excuse himself from the duty of getting his head clear on the principles of scientific inference, but equally no other thinking man can avoid a like obligation." (Sir Ronald A Fisher, "The Design of Experiments", 1971)

"[...] we will adopt the broad view and will take 'probability', to be a quantitative relation, between a hypothesis and an inference, corresponding to the degree of rational belief in the correctness of the inference, given the hypothesis. The hypothesis is the information we possess, or assume for the sake of argument. The inference is a statement that, to a greater or lesser extent, is justified by the hypothesis. Thus 'the probability' of an inference, given a hypothesis, is the degree of rational belief in the correctness of the inference, given the hypothesis." (Ralph Baierlein, "Atoms and Information Theory: An Introduction to Statistical Mechanics", 1971)

"An analogy is a relationship between two entities, processes, or what you will, which allows inferences to be made about one of the things, usually that about which we know least, on the basis of what we know about the other. […] The art of using analogy is to balance up what we know of the likenesses against the unlikenesses between two things, and then on the basis of this balance make an inference as to what is called the neutral analogy, that about which we do not know." (Rom Harré," The Philosophies of Science", 1972)

"The process [of statistical analysis] usually begins by the postulating of a model worthy to be tentatively entertained. The data analyst will have arrived at this tentative model in cooperation with the scientific investigator. They will choose it 'So that, in the light of the then available knowledge, it best takes account of relevant phenomena in the simplest way possible. it will usually contain unknown parameters. Given the data the analyst can now make statistical inferences about the parameters conditional on the correctness of this first tentative model. These inferences form part of the conditional analysis. If the model is correct, they provide all there is to know about the problem under study, given the data." (George E P Box & George C Tjao, "Bayesian Inference in Statistical Analysis", 1973)

"[…] it is not enough to say: 'There's error in the data and therefore the study must be terribly dubious'. A good critic and data analyst must do more: he or she must also show how the error in the measurement or the analysis affects the inferences made on the basis of that data and analysis." (Edward R Tufte, "Data Analysis for Politics and Policy", 1974)

On Inferences (1900-1949)

"I may as well say at once that I do not distinguish between inference and deduction. What is called induction appears to me to be either disguised deduction or a mere method of making plausible guesses." (Bertrand Russell, "Principles of Mathematics", 1903)

"A theorem […] is an inference obtained by constructing a diagram according to a general precept, and after modifying it as ingenuity may dictate, observing in it certain relations, and showing that they must subsist in every case, retranslating the proposition into general terms." (Charles S Peirce, "New Elements" ["Kaina stoiceia"], 1904)

"The type of reasoning found in mathematics seems thus not only available but essentially interwoven with every inference in non-mathematical reasoning, being always used in one of its two steps ; facility in making the other step, the more difficult one, must be attained through other than purely mathematical training." (Jacob W A Young, "The Teaching of Mathematics", 1907)

"It is experience which has given us our first real knowledge of Nature and her laws. It is experience, in the shape of observation and experiment, which has given us the raw material out of which hypothesis and inference have slowly elaborated that richer conception of the material world which constitutes perhaps the chief, and certainly the most characteristic, glory of the modern mind." (Arthur J Balfour, "The Foundations of Belief", 1912)

"The ends to be attained [in mathematical teaching] are the knowledge of a body of geometrical truths to be used. In the discovery of new truths, the power to draw correct inferences from given premises, the power to use algebraic processes as a means of finding results in practical problems, and the awakening of interest In the science of mathematics." (J Craig, "A Course of Study for the Preparation of Rural School Teachers", 1912)

"Whenever possible, substitute constructions out of known entities for inferences to unknown entities." (Bertrand Russell, 1924)

"Hypothesis, however, is an inference based on knowledge which is insufficient to prove its high probability." (Frederick L Barry, "The Scientific Habit of Thought", 1927) 

"The development of mathematics toward greater precision has led, as is well known, to the formalization of large tracts of it, so that one can prove any theorem using nothing but a few mechanical rules. [...] One might therefore conjecture that these axioms and rules of inference are sufficient to decide any mathematical question that can at all be formally expressed in these systems. It will be shown below that this is not the case, that on the contrary there are in the two systems mentioned relatively simple problems in the theory of integers that cannot be decided on the basis of the axioms." (Kurt Gödel, "On Formally Undecidable Propositions of Principia Mathematica and Related Systems", 1931)

"An inference, if it is to have scientific value, must constitute a prediction concerning future data. If the inference is to be made purely with the help of the distribution theory of statistics, the experiments that constitute evidence for the inference must arise from a state of statistical control; until that state is reached, there is no universe, normal or otherwise, and the statistician’s calculations by themselves are an illusion if not a delusion. The fact is that when distribution theory is not applicable for lack of control, any inference, statistical or otherwise, is little better than a conjecture. The state of statistical control is therefore the goal of all experimentation." (William E Deming, "Statistical Method from the Viewpoint of Quality Control", 1939)

"An observation, strictly, is only a sensation. Nobody means that we should reject everything but sensations. But as soon as we go beyond sensations we are making inferences." (Sir Harold Jeffreys, "Theory of Probability", 1939)

"Inference by analogy appears to be the most common kind of conclusion, and it is possibly the most essential kind. It yields more or less plausible conjectures which may or may not be confirmed by experience and stricter reasoning." (George Pólya, "How to Solve It", 1945)

"If the chance of error alone were the sole basis for evaluating methods of inference, we would never reach a decision, but would merely keep increasing the sample size indefinitely." (C West Churchman, "Theory of Experimental Inference", 1948)

On Inferences (-1899)

"Analysis is the obtaining of the thing sought by assuming it and so reasoning up to an admitted truth; synthesis is the obtaining of the thing sought by reasoning up to the inference and proof of it." (Eudoxus, cca. 4th century BC)

"Every stage of science has its train of practical applications and systematic inferences, arising both from the demands of convenience and curiosity, and from the pleasure which, as we have already said, ingenious and active-minded men feel in exercising the process of deduction." (William Whewell, "The Philosophy of the Inductive Sciences Founded Upon Their History", 1840)

"There is in every step of an arithmetical or algebraical calculation a real induction, a real inference from facts to facts, and what disguises the induction is simply its comprehensive nature, and the consequent extreme generality of its language." (John S Mill, "A System of Logic, Ratiocinative and Inductive", 1843)

"A mere inference or theory must give way to a truth revealed; but a scientific truth must be maintained, however contradictory it may appear to the most cherished doctrines of religion." (David Brewster, "More Worlds Than One: The Creed of the Philosopher and the Hope of the Christian", 1856)

"Truths are known to us in two ways: some are known directly, and of themselves; some through the medium of other truths. The former are the subject of Intuition, or Consciousness; the latter, of Inference; the latter of Inference. The truths known by Intuition are the original premises, from which all others are inferred." (John S Mill, "A System of Logic, Ratiocinative and Inductive", 1858)

"And first, it is necessary to distinguish from true inductions, certain operations which are often improperly called by that name. A true induction is a process of inference - it proceeds from the known to the unknown; and whenever any operation contains no inference, it is not  really an induction. And yet most of the examples given in the common  works on logic, as examples of induction, are of this character." (George R Drysdale, "Logic and Utility: The tests of truth and falsehood, and of right and wrong", 1866)

"It must be the ground of all reasoning and inference that what is true of one thing will be true of its equivalent, and that under carefully ascertained conditions Nature repeats herself." (William S Jevons, "The Principles of Science: A Treatise on Logic and Scientific Method", 1874)

"Economic science is but the working of common sense aided by appliances of organized analysis and general reasoning, which facilitate the task of collecting, arranging, and drawing inferences from particular facts. Though its scope is always limited, though its work without the aid of common sense is vain, yet it enables common sense to go further in difficult problems than would otherwise be possible." (Alfred Marshall, "Principles of Economics", 1890)

Experience - Trivia

"It is frequently analogy which guides the experienced to what are called good guesses." (Francis W Newman, "Lectures on Logic", 1838)

"An experiment is an observation that can be repeated, isolated and varied. The more frequently you can repeat an observation, the more likely are you to see clearly what is there and to describe accurately what you have seen. The more strictly you can isolate an observation, the easier does your task of observation become, and the less danger is there of your being led astray by irrelevant circumstances, or of placing emphasis on the wrong point. The more widely you can vary an observation, the more clearly will be the uniformity of experience stand out, and the better is your chance of discovering laws." (Edward B Titchener, "A Text-Book of Psychology", 1909)

"Our system of philosophy is itself on trial; it must stand or fall according as it is broad enough to find room for this experience as an element of life." (Sir Arthur S Eddington, "Science and the Unseen World", 1929)

"One has to recognize that science is not metaphysics, and certainly not mysticism; it can never bring us the illumination and the satisfaction experienced by one enraptured in ecstasy. Science is sobriety and clarity of conception, not intoxicated vision." (Ludwig Von Mises, "Epistemological Problems of Economics", 1933)

"The great extension of our experience in recent years has brought light to the insufficiency of our simple mechanical conceptions and, as a consequence, has shaken the foundation on which the customary interpretation of observation was based." (Niels Bohr, "Atomic Physics and the Description of Nature", 1934)

"[T]he sudden inventions characteristic of the sixth stage [of infant development] are in reality the product of a long evolution of schemata and not only of an internal maturation of perceptive structures. [..] This is revealed by the existence of a fifth stage, characterized by experimental groping. […] What does this mean if not that the practice of actual experience is necessary in order to acquire the practice of mental experience and that invention does not arise entirely preformed despite appearances? (Jean Piaget, "The origin of intelligence in children" 1936)

"We can scarcely imagine a problem absolutely new, unlike and unrelated to any formerly solved problem; but if such a problem could exist, it would be insoluble. In fact, when solving a problem, we should always profit from previously solved problems, using their result or their method, or the experience acquired in solving them." (George Polya, 1945)

"We have here no esoteric theory of the ultimate nature of concepts, nor a philosophical championing of the primacy of the 'operation'. We have merely a pragmatic matter, namely that we have observed after much experience that if we want to do certain kinds of things with our concepts, our concepts had better be constructed in certain ways. In fact one can see that the situation here is no different from what we always find when we push our analysis to the limit; operations are not ultimately sharp or irreducible any more than any other sort of creature. We always run into a haze eventually, and all our concepts are describable only in spiralling approximation." (Percy W Bridgman, "Reflections of a Physicist", 1950)

"Modern scientific principle has been drawn from the investigation of natural laws, technology has developed from the experience of doing, and the two have been combined by means of mathematical system to form what we call engineering." (George S Emmerson, "Engineering Education: A Social History", 1973)

"Models are not intended to either reflect or construct a single objective reality. Rather, their purpose is to simulate some aspect of a possible reality. In NLP, for instance, it is not important whether or not a model is 'true' , but rather that it is 'useful' . In fact, all models can be perceived as symbolic or metaphoric, as opposed to reflective of reality. Whether the description being used is metaphorical or literal, the usefulness of a model depends on the degree to which it allows us to move effectively to the next step in the sequence of transformations connecting deeper structures and surface structures. Instead of 'constructing' reality, models establish a set of functions that serve as a tool or a bridge between deep structures and surface structures. It is this bridge that forms our 'understanding' of reality and allows us to generate new experiences and expressions of reality." (Richard Bandler & John Grinder, "The Structure of Magic", 1975)

"It is often the scientist’s experience that he senses the nearness of truth when such connections are envisioned. A connection is a step toward simplification, unification. Simplicity is indeed often the sign of truth and a criterion of beauty." (Mahlon B Hoagland, "Toward the Habit of Truth", 1990)

"Systems theory pursues the scientific exploration and understanding of systems that exist in the various realms of experience, in order to arrive at a general theory of systems: an organized expressing of sets of interrelated concepts and principles that apply to all systems." (Béla H Bánáthy, "Systems Design of Education", 1991)

"It is in the nature of exponential growth that events develop extremely slowly for extremely long periods of time, but as one glides through the knee of the curve, events erupt at an increasingly furious pace. And that is what we will experience as we enter the twenty-first century." (Ray Kurzweil, "The Age of Spiritual Machines: When Computers Exceed Human Intelligence", 1999)

"Cultural archetypes are the unconscious models that help us make sense of the world: they are the myths, narratives, images, symbols, and files into which we organize the data of our life experience" (Clotaire Rapaille, "Cultural Imprints", Executive Excellence Vol. 16 (10), 1999)

"Learning is a process of modifying or completely changing our mental models based on new experiences or evidence." (Edward D Hess, "Learn or Die: Using Science to Build a Leading-Edge Learning Organization", 2014)

"What is consciousness? Our brain simulates reality. So, our everyday experiences are a form of dreaming, which is to say, they are mental models, simulations, not the things they appear to be." (Stephen LaBerge, "Losi in Lucidity", 2014)

John McCarthy - Collected Quotes

"[…] the difference between creative thinking and unimaginative competent thinking lies in the injection of a some randomness. The randomness must be guided by intuition to be efficient." (John McCarthy et al, "A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence", 1955)

"The following are some aspects of the artificial intelligence problem: […] If a machine can do a job, then an automatic calculator can be programmed to simulate the machine. […] It may be speculated that a large part of human thought consists of manipulating words according to rules of reasoning and rules of conjecture. From this point of view, forming a generalization consists of admitting a new word and some rules whereby sentences containing it imply and are implied by others. This idea has never been very precisely formulated nor have examples been worked out. […] How can a set of (hypothetical) neurons be arranged so as to form concepts. […] to get a measure of the efficiency of a calculation it is necessary to have on hand a method of measuring the complexity of calculating devices which in turn can be done. […] Probably a truly intelligent machine will carry out activities which may best be described as self-improvement. […] A number of types of 'abstraction' can be distinctly defined and several others less distinctly. […] the difference between creative thinking and unimaginative competent thinking lies in the injection of a some randomness. The randomness must be guided by intuition to be efficient." (John McCarthy et al, "A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence", 1955)

"We call a problem well-defined if there is a test which can be applied to a proposed solution. In case the proposed solution is a solution, the test must confirm this in a finite number of steps." (John McCarthy, "The Inversion of Functions Denned by Turing Machines", 1956)

"We shall therefore say that a program has common sense if it automatically deduces for itself a sufficient wide class of immediate consequences of anything it is told and what it already knows. [...] Our ultimate objective is to make programs that learn from their experience as effectively as humans do. We shall [...] say that a program has common sense if it automatically deduces for itself a sufficient wide class of immediate consequences of anything it is told and what it already knows" (John McCarthy, "Programs with Common Sense", 1958)

"Intelligence has two parts, which we shall call the epistemological and the heuristic. The epistemological part is the representation of the world in such a form that the solution of problems follows from the facts expressed in the representation. The heuristic part is the mechanism that on the basis of the information solves the problem and decides what to do." (John McCarthy & Patrick J Hayes, "Some Philosophical Problems from the Standpoint of Artificial Intelligence", Machine Intelligence 4, 1969)

"The right way to think about the general problems of metaphysics and epistemology is not to attempt to clear one's own mind of all knowledge and start with 'Cogito ergo sum' and build up from there. Instead, we propose to use all of our knowledge to construct a computer program that knows. The correctness of our philosophical system will be tested by numerous comparisons between the beliefs of the program and our own observations and knowledge." (John McCarthy & Patrick J. Hayes, "Some Philosophical Problems from the Standpoint of Artificial Intelligence", 1969)

"[This] is or should be our main scientific activity - studying the structure of information and the structure of problem-solving processes independently of applications and independently of its realization in animals or humans." (John McCarthy, 1974)

"When we program a computer to make choices intelligently after determining its options, examining their consequences, and deciding which is most favorable or most moral or whatever, we must program it to take an attitude towards its freedom of choice essentially isomorphic to that which a human must take to his own." (John McCarthy, "Ascribing Mental Qualities to Machines", 1979) 

"It's difficult to be rigorous about whether a machine really 'knows', 'thinks', etc., because we're hard put to define these things. We understand human mental processes only slightly better than a fish understands swimming." (John McCarthy, "The Little Thoughts of Thinking Machines", Psychology Today, 1983)

"Program designers have a tendency to think of the users as idiots who need to be controlled. They should rather think of their program as a servant, whose master, the user, should be able to control it. If designers and programmers think about the apparent mental qualities that their programs will have, they'll create programs that are easier and pleasanter - more humane - to deal with." (John McCarthy, "The Little Thoughts of Thinking Machines", Psychology Today,1983)

"Whenever we write an axiom, a critic can say that the axiom is true only in a certain context. With a little ingenuity the critic can usually devise a more general context in which the precise form of the axiom doesn't hold. [...] There simply isn't a most general context." (John McCarthy, "Generality in Artificial Intelligence", 1987)

"I don't see that human intelligence is something that humans can never understand." (John McCarthy, 1989)

On Standards II: Mathematics II

"Perhaps the best way to approach the question of what mathematics is, is to start at the beginning. In the far distant prehistoric past, where we must look for the beginnings of mathematics, there were already four major faces of mathematics. First, there was the ability to carry on the long chains of close reasoning that to this day characterize much of mathematics. Second, there was geometry, leading through the concept of continuity to topology and beyond. Third, there was number, leading to arithmetic, algebra, and beyond. Finally there was artistic taste, which plays so large a role in modern mathematics. There are, of course, many different kinds of beauty in mathematics. In number theory it seems to be mainly the beauty of the almost infinite detail; in abstract algebra the beauty is mainly in the generality. Various areas of mathematics thus have various standards of aesthetics." (Richard Hamming, "The Unreasonable Effectiveness of Mathematics", The American Mathematical Monthly Vol. 87 (2), 1980)

"Thus statistics should generally be taught more as a practical subject with analyses of real data. Of course some theory and an appropriate range of statistical tools need to be learnt, but students should be taught that Statistics is much more than a collection of standard prescriptions." (Christopher Chatfield, "The Initial Examination of Data", Journal of the Royal Statistical Society A Vol. 148, 1985)

"[…] calling upon the needs of rigor to explain the development of mathematics constitutes a circular argument. In actual fact, new standards of rigor are formed when the old criteria no longer permit an adequate response to questions that arise in mathematical practice or to problems that are in a certain sense external to mathematics. When these are treated mathematically, they compel changes in the theoretical framework of mathematics. It is thus not by chance that mathematical physics and applied mathematics have generally been formidable stimuli to the development of pure mathematics." (Umberto Bottazzini, "The Higher Calculus: A History of Real and Complex Analysis from Euler to Weierstrass", 1986)

"As a practical matter, mathematics is a science of pattern and order. Its domain is not molecules or cells, but numbers, chance, form, algorithms, and change. As a science of abstract objects, mathematics relies on logic rather than observation as its standard of truth, yet employs observation, simulation, and even experimentation as a means of discovering truth." (National Research Council, "Everybody Counts", 1989)

"'Technique' is a term used equally by chess players and mathematicians to describe sequences of moves which are standard, familiar and unoriginal. Once upon a time, the particular technique was an invention, a new discovery, but no longer. The precise sequence of moves required may never have been played before in the history of the world, yet no new ideas, no originality and no imagination are demanded, at least of the experienced player. (To the learner, of course, the most mundane sequences will appear novel and require original thought.)" (David Wells, "You Are a Mathematician: A wise and witty introduction to the joy of numbers", 1995)

"Elegance and simplicity should remain important criteria in judging mathematics, but the applicability and consequences of a result are also important, and sometimes these criteria conflict. I believe that some fundamental theorems do not admit simple elegant treatments, and the proofs of such theorems may of necessity be long and complicated. Our standards of rigor and beauty must be sufficiently broad and realistic to allow us to accept and appreciate such results and their proofs. As mathematicians we will inevitably use such theorems when it is necessary in the practice our trade; our philosophy and aesthetics should reflect this reality." (Michael Aschbacher,"Highly complex proofs and implications", 2005)

“When mathematics is explained, formalized and written down, there is a strong tendency to favor symbolic modes of thought at the expense of everything else, because symbols are easier to write and more standardized than other modes of reasoning. But when mathematics loses its connection to our minds, it dissolves into a haze.” (William P Thurston, [preface to (John H Hubbard, “Teichmüller Theory and Applications”,Vol. 1, 2006)])

"Math is a way to describe reality and figure out how the world works, a universal language that has become the gold standard of truth. In our world, increasingly driven by science and technology, mathematics is becoming, ever more, the source of power, wealth, and progress. Hence those who are fluent in this new language will be on the cutting edge of progress." (Edward Frenkel, "Love and Math", 2014)

"Mathematics is a fascinating discipline that calls for creativity, imagination, and the mastery of rigorous standards of proof." (John Meier & Derek Smith, "Exploring Mathematics: An Engaging Introduction to Proof", 2017)

On Standards I: Mathematics I

"Measure consists in the superposition of the magnitudes to be compared; it therefore requires a means of using one magnitude as the standard for another. In the absence of this, two magnitudes can only be compared when one is a part of the other; in which case we can only determine the more or less and not the how much." (Bernhard Riemann, "On the hypotheses which lie at the foundation of geometry", 1854)

"It has long been a complaint against mathematicians that they are hard to convince: but it is a far greater disqualification both for philosophy, and for the affairs of life, to be too easily convinced; to have too low a standard of proof. The only sound intellects are those which, in the first instance, set their standards of proof high. Practice in concrete affairs soon teaches them to make the necessary abatement: but they retain the consciousness, without which there is no sound practical reasoning, that in accepting inferior evidence because there is no better to be had, they do not by that acceptance raise it to completeness." (John S Mill, "An Examination of Sir William Hamilton's Philosophy", 1865)

"Judged by the only standards which are admissible in a pure doctrine of numbers i is imaginary in the same sense as the negative, the fraction, and the irrational, but in no other sense; all are alike mere symbols devised for the sake of representing the results of operations even when these results are not numbers (positive integers)." (Henry B Fine, "The Number-System of Algebra", 1890)

"The main source of mathematical invention seems to be within man rather than outside of him: his own inveterate and insatiable curiosity, his constant itching for intellectual adventure; and likewise the main obstacles to mathematical progress seem to be also within himself; his scandalous inertia and laziness, his fear of adventure, his need of conformity to old standards, and his obsession by mathematical ghosts." (George Sarton, "The Study of the History of Mathematics", 1936)

"Mathematicians themselves set up standards of generality and elegance in their exposition which are a bar to understand." (Kenneth E Boulding, "Economic Analysis", 1941)

"As an Art, Mathematics has its own standard of beauty and elegance which can vie with the more decorative arts. In this it is diametrically opposed to a Baroque art which relies on a wealth of ornamental additions. Bereft of superfluous addenda, Mathematics may appear, on first acquaintance, austere and severe. But longer contemplation reveals the classic attributes of simplicity relative to its significance and depth of meaning." (Dudley E Littlewood,"The Skeleton Key of Mathematics", 1949)

"Demonstrative reasoning is safe, beyond controversy, and final. Plausible reasoning is hazardous, controversial, and provisional. Demonstrative reasoning penetrates the sciences just as far as mathematics does, but it is in itself (as mathematics is in itself) incapable of yielding essentially new knowledge about the world around us. Anything new that we learn about the world involves plausible reasoning, which is the only kind of reasoning, for which we care in everyday affairs. Demonstrative reasoning has rigid standards, codified and clarified by logic (formal or demonstrative logic), which is the theory of demonstrative reasoning. The standards of plausible reasoning are fluid, and there is no theory of such reasoning that could be compared to demonstrative logic in clarity or would command comparable consensus." (George Pólya, "Mathematics and Plausible Reasoning", 1954)

"It seems to be one of the fundamental features of nature that fundamental physical laws are described in terms of a mathematical theory of great beauty and power, needing quite a high standard of mathematics for one to understand it. You may wonder: Why is nature constructed along these lines? One can only answer that our present knowledge seems to show that nature is so constructed. We simply have to accept it. One could perhaps describe the situation by saying that God is a mathematician of a very high order, and He used very advanced mathematics in constructing the universe. Our feeble attempts at mathematics enable us to understand a bit of the universe, and as we proceed to develop higher and higher mathematics we can hope to understand the universe better." (Paul Dirac, "The Evolution of the Physicist's Picture of Nature", 1963)

"When we propose to apply mathematics we are stepping outside our own realm, and such a venture is not without dangers. For having stepped out, we must be prepared to be judged by standards not of our own making and to play games whose rules have been laid down with little or no consultation with us. Of course, we do not have to play, but if we do we have to abide by the rules and above all not try to change them merely because we find them uncomfortable or restrictive." (Mark Kac, "On Applying Mathematics: Reflections and Examples", Quarterly of Applied Mathematics, 1972)

"[...] despite an objectivity about mathematical results that has no parallel in the world of art, the motivation and standards of creative mathematics are more like those of art than of science. Aesthetic judgments transcend both logic and applicability in the ranking of mathematical theorems: beauty and elegance have more to do with the value of a mathematical idea than does either strict truth or possible utility." (Lynn A Steen, "Mathematics Today: Twelve Informal Essays", Mathematics Today, 1978)

Out of Context: On Thought (Definitions)

"It is probable that what we call thought is not an actual being, but no more than the relation between certain parts of that infinitely varied mass, of which the rest of the universe is composed, and which ceases to exist as soon as those parts change their position with regard to each other." (Percy B Shelley, "On a Future State", 1815)

"Thought is symbolical of Sensation as Algebra is of Arithmetic, and because it is symbolical, is very unlike what it symbolises." (George H Lewes "Problems of Life and Mind", 1873)

"[...] thought is the representative or cognitive apprehension of relations among notions; imagination is the affective or felt apprehension of relations among images." (James M Baldwin,"Handbook of Psychology: Senses and Intellect", 1890)

"Thought is existence. More than that, so far as we are concerned, existence is thought, all our conceptions of existence being some kind or other of thought." (Thomas H Huxley, "Method and Results", 1893)

"Consequently, all truly strict and exact thought is sustained by the symbolic and semiotics on which it is based." (Ernst Cassirer, "The Philosophy of Symbolic Forms", 1923)

"Thought is prior to language and consists in the simultaneous presentation to the mind of two different images." (Thomas E Hulme, "Notes on Language and Style", 1929)

"Analytic thought is based on detailed defined relations between two elements at a time. Intuitive thought is based on an emotional state associated with all the elements in the field of knowledge (overall impression). " (Tony Bastick, "Intuition: How we think and act", 1982) 

Mathematicians vs Scientists

"The man of science, who, forgetting the limits of philosophical inquiry, slides from these formulæ and symbols into what is commonly understood by materialism, seems to me to place himself on a level with the mathematician, who should mistake the x's and y's with which he works his problems for real entities - and with this further disadvantage, as compared with the mathematician, that the blunders of the latter are of no practical consequence, while the errors of systematic materialism may paralyse the energies and destroy the beauty of a life." (Thomas H Huxley, "Method and Results", 1893)

"Mathematicians and other scientists, however great they may be, do not know the future. Their genius may enable them to project their purpose ahead of them; it is as if they had a special lamp, unavailable to lesser men, illuminating their path; but even in the most favorable cases the lamp sends only a very small cone of light into the infinite darkness." (George Sarton, "The Study of the History of Mathematics", 1936)

"One of the difficulties which a mathematician has in describing his work to non-mathematicians is that the present day language of mathematics has become so esoteric that a well educated layman, or even a group of scientists, can comprehend essentially nothing of the discourse which mathematicians hold with each other, or of the accounts of their latest researches which are published in their professional journals." (Angus E Taylor," Some Aspects of Mathematical Research", American Scientist , Vol. 35, No. 2, 1947)

"A scientist worthy of the name, above all a mathematician, experiences in his work the same impression as an artist; his pleasure is as great and of the same nature. [...] we work not only to obtain the positive results which, according to the profane, constitute our one and only affection, as to experience this esthetic emotion and to convey it to others who are capable of experiencing it." (Henri Poincaré, "Notice sur Halphen", Journal de l'École Polytechnique, 1890)

"The entrepreneur's instinct is to exploit the natural world. The engineer's instinct is to change it. The scientist's instinct is to try to understand it - to work out what's really going on. The mathematician's instinct is to structure that process of understanding by seeking generalities that cut across the obvious subdivisions." (Ian Stewart, "Nature's Numbers", 1995)

"Obviously, the final goal of scientists and mathematicians is not simply the accumulation of facts and lists of formulas, but rather they seek to understand the patterns, organizing principles, and relationships between these facts to form theorems and entirely new branches of human thought." (Clifford A Pickover, "The Math Book", 2009)

"The reasoning of the mathematician and that of the scientist are similar to a point. Both make conjectures often prompted by particular observations. Both advance tentative generalizations and look for supporting evidence of their validity. Both consider specific implications of their generalizations and put those implications to the test. Both attempt to understand their generalizations in the sense of finding explanations for them in terms of concepts with which they are already familiar. Both notice fragmentary regularities and - through a process that may include false starts and blind alleys - attempt to put the scattered details together into what appears to be a meaningful whole. At some point, however, the mathematician’s quest and that of the scientist diverge. For scientists, observation is the highest authority, whereas what mathematicians seek ultimately for their conjectures is deductive proof." (Raymond S Nickerson, "Mathematical Reasoning: Patterns, Problems, Conjectures and Proofs", 2009)