📓On Literature: On Information (From Fiction to Science-Fiction

"What is called science today consists of a haphazard heap of information, united by nothing, often utterly unnecessary, and not only failing to present one unquestionable truth, but as often as not containing the grossest errors, today put forward as truths, and tomorrow overthrown." (Leo Tolstoy, "What Is Art?", 1897)

"The more complex a civilization, the more vital to its existence is the maintenance of the flow of information; hence the more vulnerable it becomes to any disturbance in that flow." (Stanislaw Lem, "Memoirs Found in a Bathtub", 1961)

"A scientist can not be measured quantitatively by the number of degrees or the accumulation of information. A true scientist should have a measure of courage to correct error and seek truth - no matter how painful. The alternative is more painful. To build error upon error is to drift into dogmas, metaphysics, science fiction, and mythology." (Alexander Wilf, "Origin and Destiny of the Moral Species", 1969)

"When action grows unprofitable, gather information; when information grows unprofitable, sleep". (Ursula K. Le Guin, "The Left Hand of Darkness", 1969)

"You can understand why a system would seek information - but why in hell does it offer information? Why do we strive to be understood? Why is a refusal to accept communication so painful?" (James Tiptree Jr, "Ten Thousand Light-Years From Home", 1973)

"Any information system of sufficient complexity will inevitably become infected with viruses - viruses generated from within itself." (Neal Stephenson, "Snow Crash", 1992)

"Universities are meant to pass the torch of civilization, not just download data into student skulls, and the values of the academic community are strongly at odds with those of all would-be information empires. Teachers at all levels, from kindergarten up, have proven to be shameless and persistent software and data pirates. Universities do not merely ‘leak information' but vigorously broadcast free thought." (Bruce Sterling, "The Hacker Crackdown", 1992)

"He had long since learned that when something unusual was going on, he would often find out more information faster by waiting than by asking." (Orson Scott Card, "First Meetings in Ender's Universe", 2002)

"Human evolution is driven by community needs […]. How is that possible, since genetic information is passed only by and to individuals?" (Orson Scott Card, "First Meetings in Ender's Universe", 2002)

"Well, all information looks like noise until you break the code." (Neal Stephenson, "Snow Crash", 2003)

"[…] that’s what science was - the sharing of information, the pooling of knowledge. (Orson Scott Card, "Ender in Exile", 2008)

"The absence of information is information.” (Orson Scott Card, "Ender in Exile", 2008)

"The process of thinking itself requires us to view the universe in the direction of entropy, since an abstraction always involves information loss, since symbols 'abstract' complexity from observed objects." (John C Wright, "Awake in the Night Land", 2014)

🪸On Systems Thinking: On Self-Organization (2010-)

"In the telephone system a century ago, messages dispersed across the network in a pattern that mathematicians associate with randomness. But in the last decade, the flow of bits has become statistically more similar to the patterns found in self-organized systems. For one thing, the global network exhibits self-similarity, also known as a fractal pattern. We see this kind of fractal pattern in the way the jagged outline of tree branches look similar no matter whether we look at them up close or far away. Today messages disperse through the global telecommunications system in the fractal pattern of self-organization." (Kevin Kelly, "What Technology Wants", 2010)

"Most systems in nature are inherently nonlinear and can only be described by nonlinear equations, which are difficult to solve in a closed form. Non-linear systems give rise to interesting phenomena such as chaos, complexity, emergence and self-organization. One of the characteristics of non-linear systems is that a small change in the initial conditions can give rise to complex and significant changes throughout the system. This property of a non-linear system such as the weather is known as the butterfly effect where it is purported that a butterfly flapping its wings in Japan can give rise to a tornado in Kansas. This unpredictable behaviour of nonlinear dynamical systems, i.e. its extreme sensitivity to initial conditions, seems to be random and is therefore referred to as chaos. This chaotic and seemingly random behaviour occurs for non-linear deterministic system in which effects can be linked to causes but cannot be predicted ahead of time." (Robert K Logan, "The Poetry of Physics and The Physics of Poetry", 2010)

"Nature is capable of building complex structures by processes of self-organization; simplicity begets complexity." (Victor J Stenger, God: "The Failed Hypothesis", 2010)

"But one of the major findings of quantum theory is the recognition that the universe is an open system. Open systems are neg-entropic (-S) and exhibit a tendency toward order. This means that according to quantum theory, the universe is a self-organizing system, continuously expanding and moving toward increasing order and complexity. In this context even subatomic particles show open system behavior and self-referencing tendencies." (Jamshid Gharajedaghi, "Systems Thinking: Managing Chaos and Complexity A Platform for Designing Business Architecture" 3rd Ed., 2011)

"A self–organizing system acts autonomously, as if the interconnecting components had a single mind. And as these components spontaneously march to the beat of their own drummer, they organize, adapt, and evolve toward a greater complexity than one would ever expect by just looking at the parts by themselves." (Lawrence K Samuels, "Defense of Chaos: The Chaology of Politics, Economics and Human Action", 2013)

"Complex systems are networks made of a number of components that interact with each other, typically in a nonlinear fashion. Complex systems may arise and evolve through self-organization, such that they are neither completely regular nor completely random, permitting the development of emergent behavior at macroscopic scales." (Hiroki Sayama, "Introduction to the Modeling and Analysis of Complex Systems", 2015)

"There are several reasons why reaction-diffusion systems have been a popular choice among mathematical modelers of spatio-temporal phenomena. First, their clear separation between non-spatial and spatial dynamics makes the modeling and simulation tasks really easy. Second, limiting the spatial movement to only diffusion makes it quite straightforward to expand any existing non-spatial dynamical models into spatially distributed ones. Third, the particular structure of reaction-diffusion equations provides aneasy shortcut in the stability analysis (to be discussed in the next chapter). And finally, despite the simplicity of their mathematical form, reaction-diffusion systems can show strikingly rich, complex spatio-temporal dynamics. Because of these properties, reaction-diffusion systems have been used extensively for modeling self-organization of spatial patterns." (Hiroki Sayama, "Introduction to the Modeling and Analysis of Complex Systems", 2015)

"Without chaos there would be no creation, no structure and no existence. After all, order is merely the repetition of patterns; chaos is the process that establishes those patterns. Without this creative self-organizing force, the universe would be devoid of biological life, the birth of stars and galaxies - everything we have come to know. (Lawrence K Samuels, "Chaos Gets a Bad Rap: Importance of Chaology to Liberty", 2015)

"Cybernetics studies the concepts of control and communication in living organisms, machines and organizations including self-organization. It focuses on how a (digital, mechanical or biological) system processes information, responds to it and changes or being changed for better functioning (including control and communication)." (Dmitry A Novikov, "Cybernetics 2.0", 2016)

On Information Theory II

"If quantum communication and quantum computation are to flourish, a new information theory will have to be developed." (Hans Christian von Baeyer, "Information, The New Language of Science", 2003)

"In fact, an information theory that leaves out the issue of noise turns out to have no content." (Hans Christian von Baeyer, "Information, The New Language of Science", 2003)

"But, really, mathematics is not religion; it cannot be founded on faith. And what was most important, the methods yielding such remarkable results in the hands of the great masters began to lead to errors and paradoxes when employed by their less talented students. The masters were kept from error by their perfect mathematical intuition, that subconscious feeling that often leads to the right answer more quickly than lengthy logical reasoning. But the students did not possess this intuition […]" (Naum Ya. Vilenkin, "Stories about Sets", 1968) 

"The mathematical theory of information [...] is solely concerned with the ‘surprise’ aspect of information. There are two reasons for this. Firstly, information theory was originally developed within the context of communication engineering, where it was only the surprise factor that was of relevance. Secondly, ‘meaning’ has so far proved too difficult a concept to develop mathematically. The consequence of this is that we should be aware that ‘information’ [...] has the restricted technical meaning of ‘measure of surprise’." (David Applebaum," Probability and Information: An Integrated Approach", 2008)

"Cognitive psychology has followed a different direction, considering intelligence as a set of mental representations and a series of processes that operate on these representations that allows the individual to adapt to the changing conditions of the environment. This type of approach is connected with information theory. The intelligent mind operates by processing information that it collects from the environment, and the better and faster this information is processed, the more intelligence is demonstrated." (Diego Rasskin-Gutman, "Chess Metaphors: Artificial Intelligence and the Human Mind", 2009)

"Information Theory is a mathematical treatment of what is left after the meanings have been removed from a Communication." (John Gall, "The Systems Bible: The Beginner's Guide to Systems Large and Small"[Systematics 3rd Ed.], 2011)

"In an information economy, entrepreneurs master the science of information in order to overcome the laws of the purely physical sciences. They can succeed because of the surprising power of the laws of information, which are conducive to human creativity. The central concept of information theory is a measure of freedom of choice. The principle of matter, on the other hand, is not liberty but limitation - it has weight and occupies space." (George Gilder, "Knowledge and Power: The Information Theory of Capitalism and How it is Revolutionizing our World", 2013)

"[…] the term 'information', as used in information theory, has nothing to do with meaning. It is a measure of the order, or nonrandomness, of a signal; and the main concern of information theory is the problem of how to get a message, coded as a signal, through a noisy channel." (Fritjof Capra, "The Systems View of Life: A Unifying Vision", 2014)

"Information theory leads to the quantification of the information content of the source, as denoted by entropy, the characterization of the information-bearing capacity of the communication channel, as related to its noise characteristics, and consequently the establishment of the relationship between the information content of the source and the capacity of the channel. In short, information theory provides a quantitative measure of the information contained in message signals and help determine the capacity of a communication system to transfer this information from source to sink over a noisy channel in a reliable fashion." (Ali Grami, "Information Theory", 2016)

"In information theory this notion, introduced by Claude Shannon, is used to express unpredictability of information content. For instance, if a data set containing n items was divided into k groups each comprising n i items, then the entropy of such a partition is H = p 1 log( p 1 ) + … + p k log( p k ), where p i = n i / n . In case of two alternative partitions, the mutual information is a measure of the mutual dependence between these partitions." (Slawomir T Wierzchon, "Ensemble Clustering Data Mining and Databases", 2018) 

John R Pierce - Collected Quotes

"A valid scientific theory seldom if ever offers the solution to the pressing problems which we repeatedly state. It seldom supplies a sensible answer to our multitudinous questions. Rather than rationalizing our ideas, it discards them entirely, or, rather, it leaves them as they were. It tells us in a fresh and new way what aspects of our experience can profitably be related and simply understood." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"Communication theory deals with certain important but abstract aspects of communication. Communication theory proceeds from clear and definite assumptions to theorems concerning information sources and communication channels. In this it is essentially mathematical, and in order to understand it we must understand the idea of a theorem as a statement which must be proved, that is, which must be shown to be the necessary consequence of a set of initial assumptions. This is an idea which is the very heart of mathematics as mathematicians understand it." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"Communication theory tells us how many bits of information can be sent per second over perfect and imperfect communication channels in terms of rather abstract descriptions of the properties of these channels. Communication theory tells us how to measure the rate at which a message source, such as a speaker or a writer, generates information. Communication theory tells us how to represent, or encode, messages from a particular message source efficiently for transmission over a particular sort of channel, such as an electrical circuit, and it tells us when we can avoid errors in transmission." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"However, it turns out that a one-to-one mapping of the points in a square into the points on a line cannot be continuous. As we move smoothly along a curve through the square, the points on the line which represent the successive points on the square necessarily jump around erratically, not only for the mapping described above but for any one-to-one mapping whatever. Any one-to-one mapping of the square onto the line is discontinuous." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"In communication theory we consider a message source, such as a writer or a speaker, which may produce on a given occasion any one of many possible messages. The amount of information conveyed by the message increases as the amount of uncertainty as to what message actually will be produced becomes greater. A message which is one out of ten possible messages conveys a smaller amount of information than a message which is one out of a million possible messages. The entropy of communication theory is a measure of this uncertainty and the uncertainty, or entropy, is taken as the measure of the amount of information conveyed by a message from a source. The more we know about what message the source will produce, the less uncertainty, the less the entropy, and the less the information." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"Mathematics is a way of finding out, step by step, facts which are inherent in the statement of the problem but which are not immediately obvious. Usually, in applying mathematics one must first hit on the facts and then verify them by proof. Here we come upon a knotty problem, for the proofs which satisfied mathematicians of an earlier day do not satisfy modem mathematicians." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"Mathematicians start out with certain assumptions and definitions, and then by means of mathematical arguments or proofs they are able to show that certain statements or theorems are true." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"One of these is that many of the most general and powerful discoveries of science have arisen, not through the study of phenomena as they occur in nature, but, rather, through the study of phenomena in man-made devices, in products of technology, if you will. This is because the phenomena in man’s machines are simplified and ordered in comparison with those occurring naturally, and it is these simplified phenomena that man understands most easily." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"Ordinarily, while mathematicians may suspect or conjecture the truth of certain statements, they have to prove theorems in order to be certain." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"The ideas and assumptions of a theory determine the generalityof the theory, that is, to how wide a range of phenomena the theory applies." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"The fact that network theory evolved from the study of idealized electrical systems rather than from the study of idealized mechanical systems is a matter of history, not of necessity." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"Theories are strongly physical when they describe very completely some range of physical phenomena, which in practice is always limited. Theories become more mathematical or abstract when they deal with an idealized class of phenomena or with only certain aspects of phenomena." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"Thus, in physics, entropy is associated with the possibility of converting thermal energy into mechanical energy. If the entropy does not change during a process, the process is reversible. If the entropy increases, the available energy decreases. Statistical mechanics interprets an increase of entropy as a decrease in order or, if we wish, as a decrease in our knowledge." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"Thus, information is sometimes associated with the idea of knowledge through its popular use rather than with uncertainty and the resolution of uncertainty, as it is in communication theory." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

Edwin T Jaynes - Collected Quotes

"In conventional statistical mechanics the energy plays a preferred role among all dynamical quantities because it is conserved both in the time development of isolated systems and in the interaction of different systems. Since, however, the principles of maximum-entropy inference are independent of any physical properties, it appears that in subjective statistical mechanics all measurable quantities may be treated on the same basis, subject to certain precautions." (Edwin T Jaynes, "Information Theory and Statistical Mechanics" I, 1956)

"Just as in applied statistics the crux of a problem is often the devising of some method of sampling that avoids bias, our problem is that of finding a probability assignment which avoids bias, while agreeing with whatever information is given. The great advance provided by information theory lies in the discovery that there is a unique, unambiguous criterion for the 'amount of uncertainty' represented by a discrete probability distribution, which agrees with our intuitive notions that a broad distribution represents more uncertainty than does a sharply peaked one, and satisfies all other conditions which make it reasonable." (Edwin T Jaynes, "Information Theory and Statistical Mechanics" I, 1956)

"On the other hand, the 'subjective' school of thought, regards probabilities as expressions of human ignorance; the probability of an event is merely a formal expression of our expectation that the event will or did occur, based on whatever information is available. To the subjectivist, the purpose of probability theory is to help us in forming plausible conclusions in cases where there is not enough information available to lead to certain conclusions; thus detailed verification is not expected. The test of a good subjective probability distribution is does it correctly represent our state of knowledge as to the value of x?" (Edwin T Jaynes, "Information Theory and Statistical Mechanics" I, 1956)

"The mere fact that the same mathematical expression -Σ pi log(pi) [i is index], occurs both in statistical mechanics and in information theory does not in itself establish any connection between these fields. This can be done only by finding new viewpoints from which thermodynamic entropy and information-theory entropy appear as the same concept." (Edwin T Jaynes, "Information Theory and Statistical Mechanics" I, 1956)

"[...] thermodynamics knows of no such notion as the 'entropy of a physical system'. Thermodynamics does have the concept of the entropy of a thermodynamic system; but a given physical system corresponds to many different thermodynamic systems." (Edwin T Jaynes, "Gibbs vs Boltzmann Entropies", 1964)

"In particular, the uncertainty principle has stood for a generation, barring the way to more detailed descriptions of nature; and yet, with the lesson of parity still fresh in our minds, how can anyone be quite so sure of its universal validity when we note that, to this day, it has never been subjected to even one direct experimental test?" (Edwin T Jaynes, "Foundations of Probability Theory and Statistical Mechanics", 1967)

"'You cannot base a general mathematical theory on imprecisely defined concepts. You can make some progress that way; but sooner or later the theory is bound to dissolve in ambiguities which prevent you from extending it further.' Failure to recognize this fact has another unfortunate consequence which is, in a practical sense, even more disastrous: 'Unless the conceptual problems of a field have been clearly resolved, you cannot say which mathematical problems are the relevant ones worth working on; and your efforts are more than likely to be wasted.'" (Edwin T Jaynes, "Foundations of Probability Theory and Statistical Mechanics", 1967)

"In decision theory, mathematical analysis shows that once the sampling distribution, loss function, and sample are specified, the only remaining basis for a choice among different admissible decisions lies in the prior probabilities. Therefore, the logical foundations of decision theory cannot be put in fully satisfactory form until the old problem of arbitrariness (sometimes called 'subjectiveness') in assigning prior probabilities is resolved." (Edwin T Jaynes, "Prior Probabilities", 1978)

"It appears to be a quite general principle that, whenever there is a randomized way of doing something, then there is a nonrandomized way that delivers better performance but requires more thought." (Edwin T Jaynes, "Probability Theory: The Logic of Science", 1979)

"The semiliterate on the next bar stool will tell you with absolute, arrogant assurance just how to solve the world's problems; while the scholar who has spent a lifetime studying their causes is not at all sure how to do this." (Edwin T Jaynes, "Probability Theory: The Logic of Science", 1979)

"The difference is that energy is a property of the microstates, and so all observers, whatever macroscopic variables they may choose to define their thermodynamic states, must ascribe the same energy to a system in a given microstate. But they will ascribe different entropies to that microstate, because entropy is not a property of the microstate, but rather of the reference class in which it is embedded. As we learned from Boltzmann, Planck, and Einstein, the entropy of a thermodynamic state is a measure of the number of microstates compatible with the macroscopic quantities that you or I use to define the thermodynamic state." (Edwin T Jaynes, "Papers on Probability, Statistics, and Statistical Physics", 1983)

"There is no end to this search for the ultimate ‘true’ entropy until we have reached the point where we control the location of each atom independently. But just at that point the notion of entropy collapses, and we are no longer talking thermodynamics." (Edwin T Jaynes, "Papers on Probability, Statistics, and Statistical Physics", 1983)

Claude E Shannon - Collected Quotes

"The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem. The significant aspect is that the actual message is one selected from a set of possible messages." (Claude E Shannon, "A mathematical theory of communication", Bell Systems Technical Journal 27, 1948)

"Almost every problem that you come across is befuddled with all kinds of extraneous data of one sort or another; and if you can bring this problem down into the main issues, you can see more clearly what you’re trying to do." (Claude E Shannon, "Creative Thinking", 1952)

"Another approach for a given problem is to try to restate it in just as many different forms as you can. Change the words. Change the viewpoint. Look at it from every possible angle. After you’ve done that, you can try to look at it from several angles at the same time and perhaps you can get an insight into the real basic issues of the problem, so that you can correlate the important factors and come out with the solution." (Claude E Shannon, "Creative Thinking", 1952)

"Many proofs in mathematics have been actually found by extremely roundabout processes. A man starts to prove this theorem and he finds that he wanders all over the map. He starts off and prove a good many results which don’t seem to be leading anywhere and then eventually ends up by the back door on the solution of the given problem." (Claude E Shannon, "Creative Thinking", 1952)

"Electronic computers are normally used for the solution of numerical problems arising in science or industry. The fundamental design of these computers, however, is so flexible and so universal in conception that they maybe programmed to perform many operations which do not involve numbers at all - operations such as the translation of language, the analysis of a logical situation or the playing of games. The same orders which are used in constructing a numerical program maybe used to symbolize operations on abstract entities such as the words of a language or the positions in a chess game." (Claude E Shannon, "Game Playing Machines, 1955) 

"This duality can be pursued further and is related to a duality between past and future and the notions of control and knowledge. Thus we may have knowledge of the past but cannot control it; we may control the future but have no knowledge of it." (Claude E Shannon, "Coding theorems for a discrete source with a fidelity criterion", IRE International Convention Records Vol. 7, 1959)

"It is very difficult to estimate how well a computer can be made to play with ideal programming. I tend to agree that it would be very difficult to reach the caliber of world champions or even most chess masters but I do not regard this as unthinkable. The machines do have certain very strong advantages of accuracy, speed, etc., and our present techniques of programming are bound to improve enormously in the future." (Claude E Shannon)

"It seems to be much easier to make two small jumps than the one big jump in any kind of mental thinking." (Claude E Shannon)

"Suppose that you are given a problem to solve, I don’t care what kind of a problem - a machine to design, or a physical theory to develop, or a mathematical theorem to prove, or something of that kind - probably a very powerful approach to this is to attempt to eliminate everything from the problem except the essentials; that is, cut it down to size." (Claude E Shannon)

"The chief weakness of the machine is that it will not learn by its mistakes. The only way to improve its play is by improving the program. Some thought has been given to designing a program that would develop its own improvements in strategy with increasing experience in play. Although it appears to be theoretically possible, the methods thought of so far do not seem to be very practical. One possibility is to devise a program that would change the terms and coefficients involved in the evaluation function on the basis of the results of games the machine had already played. Small variations might be introduced in these terms, and the values would be selected to give the greatest percentage of wins." (Claude E Shannon)

"The idea of a machine thinking is by no means repugnant to all of us. In fact, I find the converse idea, that the human brain may itself be a machine which could be possibly duplicated functionally with inanimate objects, quite attractive. Until clearly disproved, this hypothesis concerning the brain seems the natural scientific one in line with the principle of parsimony, etc., rather than hypothecating intangible and unreachable “vital forces,” “souls” and the like." (Claude E Shannon)

"The redundancy of a language is related to the existence of crossword puzzles. If the redundancy is zero any sequence of letters is a reasonable text in the language and any two dimensional array of letters forms a crossword puzzle. If the redundancy is too high the language imposes too many constraints for large crossword puzzles to be possible." (Claude E Shannon)

"There is a vast, explored sea of nature just waiting for things to be discovered in it, and the science and technology needed are progressing at an exponential rate. You see, it all feeds back into itself. Someone discovers a new principle or a new theory and it’s not only new knowledge but a new instrument for seeking more knowledge." (Claude E Shannon)

🧩Out of Context: On Information (Definitions)

"Information is a set of marks that have meaning." (Edmund C Berkeley & Lawrence Wainwright, Computers: Their Operation and Applications", 1956)

"Information is carried by physical entities, such as books or sound waves or brains, but it is not itself material. Information in a living system is a feature of the order and arrangement of its parts, which arrangement provides the signs that constitute a ‘code’ or ‘language’." (John Z Young, "Programs of the Brain", 1978)

"Information is recorded in vast interconnecting networks. Each idea or image has hundreds, perhaps thousands, of associations and is connected to numerous other points in the mental network." (Peter Russell, "The Brain Book: Know Your Own Mind and How to Use it", 1979)

"Neither noise nor information is predictable." (Ray Kurzweil, "The Age of Spiritual Machines: When Computers Exceed Human Intelligence", 1999)

"Most of the information is fuzzy and linguistic in form." (Timothy J Ross & W Jerry Parkinson, "Fuzzy Set Theory, Fuzzy Logic, and Fuzzy Systems", 2002)

"Information is assimilated to the slots of a mental model in the form of ‘frames’ which are understood here as ‘chunks’ of knowledge with a well-defined meaning anchored in a given body of shared knowledge." (Jürgen Renn, "Before the Riemann Tensor: The Emergence of Einstein’s Double Strategy", "The Universe of General Relativity" Ed. by A.J. Kox & Jean Eisenstaedt, 2005)

"One advantage of the use of fuzzy models is the fact that their complexity can be gradually increased as more information is gathered. This increase in complexity can be done automatically or manually by a careful commission of the new operating point." (Jairo Espinosa et al, "Fuzzy Logic, Identification and Predictive Control", 2005)

"In a physical system, information is the opposite of entropy, as it involves uncommon and highly correlated configurations that are difficult to arrive at." (César A Hidalgo, "Why Information Grows: The Evolution of Order, from Atoms to Economies", 2015)

John M Ziman - Collected Quotes

"Many philosophers have now sadly come to the conclusion that there is no ultimate procedure which will wring the last drops of uncertainty from what scientists call their knowledge." (John M Ziman, "Public Knowledge: An Essay Concerning the Social Dimension of Science", 1968)

"Although the best and most famous scientific discoveries seem to open whole new windows of the mind, a typical scientific paper has never pretended to be more than another piece in a large jig-saw not significant in itself but as an element in a grander scheme. This technique, of soliciting many modest contributions to the vast store of human knowledge, has been the secret of western science since the seventeenth century, for it achieves a corporate collective power that is far greater than any one individual can exert. Primary scientific papers are not meant to be final statements of indisputable truths; each is merely a tiny tentative step forward, through the jungle of ignorance." (John M Zimer, Vol. 224, 1969)

"It is not enough to observe, experiment, theorize, calculate and communicate; we must also argue, criticize, debate, expound, summarize, and otherwise transform the information that we have obtained individually into reliable, well established, public knowledge." (John M Ziman, "Information, Communication, Knowledge", Nature Vol. 224 (5217), 1969)

"The sooner we all face up to the fact that theory and practice are indissoluble, and that there is no contradiction between the qualities of usefulness and beauty, the better." (John M Ziman, "Growth and Spread of Science", Nature Vol. 221 (5180), 1969)

"A significant fraction of the ordinary scientific literature in any field is concerned with essentially irrational theories put forward by a few well-established scholars who have lost touch with reality." (John M Ziman, "Some Pathologies of the Scientific Life", Nature Vol. 227, 1970)

"The communication of modern science to the ordinary citizen, necessary, important, desirable as it is, cannot be considered an easy task. The prime obstacle is lack of education. [...] There is also the difficulty of making scientific discoveries interesting and exciting without completely degrading them intellectually. [...] It is a weakness of modern science that the scientist shrinks from this sort of publicity, and thus gives an impression of arrogant mystagoguery." (John M Ziman,"The Force of Knowledge: The Scientific Dimension of Society", 1976)

"Physics defines itself as the science devoted to discovering, developing and refining those aspects of reality that are amenable to mathematical analysis." (John M Ziman, "Reliable Knowledge: An Exploration of the Grounds for Belief in Science", 1978)

"The most astonishing achievements of science, intellectually and practically, have been in physics, which many people take to be the ideal type of scientific knowledge. In fact, physics is a very special type of science, in which the subject matter is deliberately chosen so as to be amenable to quantitative analysis." (John M Ziman, "Reliable Knowledge: An Exploration of the Grounds for Belief in Science", 1978)

"'Disorder' is not mere chaos; it implies defective order." (John M Ziman, "Models of Disorder", 1979)

"A philosopher is a person who knows less and less about more and more, until he knows nothing about everything. […] A scientist is a person who knows more and more about less and less, until he knows everything about nothing." (John M Ziman, "Knowing Everything about Nothing: Specialization and Change in Scientific Careers", 1987)

"Any research organization requires generous measures of the following: (1) Social space for personal initiative and creativity; (2) Time for ideas to grow to maturity; (3) Openness to debate and criticism; (4) Hospitality towards novelty; and (5) Respect for specialized expertise." (John M Ziman, "Prometheus Bound", 1994)

"Theoretical physicists are like pure mathematicians, in that they are often interested in the hypothetical behaviour of entirely imaginary objects, such as parallel universes, or particles traveling faster than light, whose actual existence is not being seriously proposed at all." (John M Ziman, "Real Science: What it Is, and what it Means", 2000)

On Algorithms I

"Mathematics is an aspect of culture as well as a collection of algorithms." (Carl B Boyer, "The History of the Calculus and Its Conceptual Development", 1959)

"An algorithm must be seen to be believed, and the best way to learn what an algorithm is all about is to try it." (Donald E Knuth, The Art of Computer Programming Vol. I, 1968)

"Scientific laws give algorithms, or procedures, for determining how systems behave. The computer program is a medium in which the algorithms can be expressed and applied. Physical objects and mathematical structures can be represented as numbers and symbols in a computer, and a program can be written to manipulate them according to the algorithms. When the computer program is executed, it causes the numbers and symbols to be modified in the way specified by the scientific laws. It thereby allows the consequences of the laws to be deduced." (Stephen Wolfram, "Computer Software in Science and Mathematics", 1984)

"Algorithmic complexity theory and nonlinear dynamics together establish the fact that determinism reigns only over a quite finite domain; outside this small haven of order lies a largely uncharted, vast wasteland of chaos." (Joseph Ford, "Progress in Chaotic Dynamics: Essays in Honor of Joseph Ford's 60th Birthday", 1988)

"On this view, we recognize science to be the search for algorithmic compressions. We list sequences of observed data. We try to formulate algorithms that compactly represent the information content of those sequences. Then we test the correctness of our hypothetical abbreviations by using them to predict the next terms in the string. These predictions can then be compared with the future direction of the data sequence. Without the development of algorithmic compressions of data all science would be replaced by mindless stamp collecting - the indiscriminate accumulation of every available fact. Science is predicated upon the belief that the Universe is algorithmically compressible and the modern search for a Theory of Everything is the ultimate expression of that belief, a belief that there is an abbreviated representation of the logic behind the Universe's properties that can be written down in finite form by human beings." (John D Barrow, New Theories of Everything", 1991)

"Algorithms are a set of procedures to generate the answer to a problem." (Stuart Kauffman, "At Home in the Universe: The Search for Laws of Complexity", 1995)

"Let us regard a proof of an assertion as a purely mechanical procedure using precise rules of inference starting with a few unassailable axioms. This means that an algorithm can be devised for testing the validity of an alleged proof simply by checking the successive steps of the argument; the rules of inference constitute an algorithm for generating all the statements that can be deduced in a finite number of steps from the axioms." (Edward Beltrami, "What is Random?: Chaos and Order in Mathematics and Life", 1999)

"Heuristics are rules of thumb that help constrain the problem in certain ways (in other words they help you to avoid falling back on blind trial and error), but they don't guarantee that you will find a solution. Heuristics are often contrasted with algorithms that will guarantee that you find a solution - it may take forever, but if the problem is algorithmic you will get there. However, heuristics are also algorithms." (S Ian Robertson, "Problem Solving", 2001)

"An algorithm is a simple rule, or elementary task, that is repeated over and over again. In this way algorithms can produce structures of astounding complexity." (F David Peat, "From Certainty to Uncertainty", 2002)

"Many people have strong intuitions about whether they would rather have a vital decision about them made by algorithms or humans. Some people are touchingly impressed by the capabilities of the algorithms; others have far too much faith in human judgment. The truth is that sometimes the algorithms will do better than the humans, and sometimes they won’t. If we want to avoid the problems and unlock the promise of big data, we’re going to need to assess the performance of the algorithms on a case-by-case basis. All too often, this is much harder than it should be. […] So the problem is not the algorithms, or the big datasets. The problem is a lack of scrutiny, transparency, and debate." (Tim Harford, "The Data Detective: Ten easy rules to make sense of statistics", 2020)

On Algorithms II

"The vast majority of information that we have on most processes tends to be nonnumeric and nonalgorithmic. Most of the information is fuzzy and linguistic in form." (Timothy J Ross & W Jerry Parkinson, "Fuzzy Set Theory, Fuzzy Logic, and Fuzzy Systems", 2002)

"Knowledge is encoded in models. Models are synthetic sets of rules, and pictures, and algorithms providing us with useful representations of the world of our perceptions and of their patterns." (Didier Sornette, "Why Stock Markets Crash - Critical Events in Complex Systems", 2003)

"Swarm Intelligence can be defined more precisely as: Any attempt to design algorithms or distributed problem-solving methods inspired by the collective behavior of the social insect colonies or other animal societies. The main properties of such systems are flexibility, robustness, decentralization and self-organization." ("Swarm Intelligence in Data Mining", Ed. Ajith Abraham et al, 2006)

"The burgeoning field of computer science has shifted our view of the physical world from that of a collection of interacting material particles to one of a seething network of information. In this way of looking at nature, the laws of physics are a form of software, or algorithm, while the material world - the hardware - plays the role of a gigantic computer." (Paul C W Davies, "Laying Down the Laws", New Scientist, 2007)

"An algorithm refers to a successive and finite procedure by which it is possible to solve a certain problem. Algorithms are the operational base for most computer programs. They consist of a series of instructions that, thanks to programmers’ prior knowledge about the essential characteristics of a problem that must be solved, allow a step-by-step path to the solution." (Diego Rasskin-Gutman, "Chess Metaphors: Artificial Intelligence and the Human Mind", 2009)

"Programming is a science dressed up as art, because most of us don’t understand the physics of software and it’s rarely, if ever, taught. The physics of software is not algorithms, data structures, languages, and abstractions. These are just tools we make, use, and throw away. The real physics of software is the physics of people. Specifically, it’s about our limitations when it comes to complexity and our desire to work together to solve large problems in pieces. This is the science of programming: make building blocks that people can understand and use easily, and people will work together to solve the very largest problems." (Pieter Hintjens, "ZeroMQ: Messaging for Many Applications", 2012)

"These nature-inspired algorithms gradually became more and more attractive and popular among the evolutionary computation research community, and together they were named swarm intelligence, which became the little brother of the major four evolutionary computation algorithms." (Yuhui Shi, "Emerging Research on Swarm Intelligence and Algorithm Optimization", Information Science Reference, 2014)

"Again, classical statistics only summarizes data, so it does not provide even a language for asking [a counterfactual] question. Causal inference provides a notation and, more importantly, offers a solution. As with predicting the effect of interventions [...], in many cases we can emulate human retrospective thinking with an algorithm that takes what we know about the observed world and produces an answer about the counterfactual world." (Judea Pearl & Dana Mackenzie, "The Book of Why: The new science of cause and effect", 2018)

"An algorithm, meanwhile, is a step-by-step recipe for performing a series of actions, and in most cases 'algorithm' means simply 'computer program'." (Tim Harford, "The Data Detective: Ten easy rules to make sense of statistics", 2020)

"Big data is revolutionizing the world around us, and it is easy to feel alienated by tales of computers handing down decisions made in ways we don’t understand. I think we’re right to be concerned. Modern data analytics can produce some miraculous results, but big data is often less trustworthy than small data. Small data can typically be scrutinized; big data tends to be locked away in the vaults of Silicon Valley. The simple statistical tools used to analyze small datasets are usually easy to check; pattern-recognizing algorithms can all too easily be mysterious and commercially sensitive black boxes." (Tim Harford, "The Data Detective: Ten easy rules to make sense of statistics", 2020)

"Each of us is sweating data, and those data are being mopped up and wrung out into oceans of information. Algorithms and large datasets are being used for everything from finding us love to deciding whether, if we are accused of a crime, we go to prison before the trial or are instead allowed to post bail. We all need to understand what these data are and how they can be exploited." (Tim Harford, "The Data Detective: Ten easy rules to make sense of statistics", 2020)

Herbert A Simon - Collected Quotes

"All behavior involves conscious or unconscious selection of particular actions out of all those which are physically possible to the actor and to those persons over whom he exercises influence and authority." (Herbert A Simon, "Administrative Behavior: A Study of Decision-making Processes in Administrative Organization", 1947)

"Decision making processes are aimed at finding courses of action that are feasible or satisfactory in the light of multiple goals and constraints." (Herbert A Simon, "Administrative Behavior: A Study of Decision-making Processes in Administrative Organization", 1947)

"In the process of decision those alternatives are chosen which are considered to be appropriate means of reaching desired ends. Ends themselves, however, are often merely instrumental to more final objectives. We are thus led to the conception of a series, or hierarchy, of ends. Rationality has to do with the construction of means-ends chains of this kind." (Herbert A Simon, "Administrative Behavior", 1947)

"It is impossible for the behavior of a single, isolated individual to reach a high degree of rationality. The number of alternatives he must explore is so great, the information he would need to evaluate them so vast that even an approximation to objective rationality is hard to conceive. Individual choice takes place in rationality is hard to conceive. [...] Actual behavior falls short in at least three ways, of objective rationality." (Herbert A Simon, "Administrative Behavior", 1947)

"Many individuals and organization units contribute to every large decision, and the very problem of centralization and decentralization is a problem of arranging the complex system into an effective scheme." (Herbert A Simon, "Administrative Behavior: A Study of Decision-making Processes in Administrative Organization", 1947)

"Rationality requires a choice among all possible alternative behaviors. In actual behavior, only a very few of all these possible alternatives come to mind." (Herbert A Simon, "Administrative Behavior", 1947)

"Rationality requires a complete knowledge and anticipation of the consequences that will follow on each choice. In fact, knowledge of consequences is always fragmentary." (Herbert A Simon, "Administrative Behavior", 1947)

"Roughly speaking, rationality is concerned with the selection of preferred behavior alternatives in terms of some system of values, whereby the consequences of behavior can be evaluated." (Herbert A Simon, "Administrative Behavior", 1947)

"The function of knowledge in the decision-making process is to determine which consequences follow upon which of the alternative strategies. It is the task of knowledge to select from the whole class of possible consequences a more limited subclass, or even (ideally) a single set of consequences correlated with each strategy." (Herbert A Simon, "Administrative Behavior: A Study of Decision-making Processes in Administrative Organization", 1947)

"The principle of bounded rationality [is] the capacity of the human mind for formulating and solving complex problems is very small compared with the size of the problems whose solution is required for objectively rational behavior in the real world - or even for a reasonable approximation to such objective rationality." (Herbert A Simon, "Administrative Behavior", 1947)

"The first consequence of the principle of bounded rationality is that the intended rationality of an actor requires him to construct a simplified model of the real situation in order to deal with it. He behaves rationally with respect to this model, and such behavior is not even approximately optimal with respect to the real world. To predict his behavior we must understand the way in which this simplified model is constructed, and its construction will certainly be related to his psychological properties as a perceiving, thinking, and learning animal." (Herbert A Simon, "Models of Man", 1957)

"The mathematical and computing techniques for making programmed decisions replace man but they do not generally simulate him." (Herbert A Simon, "Management and Corporations 1985", 1960)

"Programs do not merely substitute brute force for human cunning. Increasingly, they imitate-and in some cases improve upon-human cunning." (Herbert A Simon, "Management and Corporations 1985", 1960)

"Roughly, by a complex system I mean one made up of a large number of parts that interact in a nonsimple way. In such systems, the whole is more than the sum of the parts, not in an ultimate, metaphysical sense, but in the important pragmatic sense that, given the properties of the parts and the laws of their interaction, it is not a trivial matter to infer the properties of the whole." (Herbert A Simon, "The Architecture of Complexity", Proceedings of the American Philosophical Society, Vol. 106 (6), 1962)

"Thus, the central theme that runs through my remarks is that complexity frequently takes the form of hierarchy, and that hierarchic systems have some common properties that are independent of their specific content. Hierarchy, I shall argue, is one of the central structural schemes that the architect of complexity uses." (Herbert A Simon, "The Architecture of Complexity", Proceedings of the American Philosophical Society Vol. 106 (6), 1962)

"A mathematical proof, as usually written down, is a sequence of expressions in the state space. But we may also think of the proof as consisting of the sequence of justifications of consecutive proof steps - i.e., the references to axioms, previously-proved theorems, and rules of inference that legitimize the writing down of the proof steps. From this point of view, the proof is a sequence of actions (applications of rules of inference) that, operating initially on the axioms, transform them into the desired theorem." (Herbert A Simon, "The Logic of Heuristic Decision Making", [in "The Logic of Decision and Action"], 1966)

"[...] a problem of design exists when (1) there is a language for naming actions and a language for naming states of the world, (2) there is a need to find an action that will produce a specified state of the world or a specified change in the state of the world, and (3) there is no non-trivial process for translating changes in the state of the world into their corresponding actions." (Herbert A Simon, "The Logic of Heuristic Decision Making", [in "The Logic of Decision and Action"], 1966)

"A problem will be difficult if there are no procedures for generating possible solutions that are guaranteed (or at least likely) to generate the actual solution rather early in the game. But for such a procedure to exist, there must be some kind of structural relation, at least approximate, between the possible solutions as named by the solution-generating process and these same solutions as named in the language of the problem statement." (Herbert A Simon, "The Logic of Heuristic Decision Making", [in "The Logic of Decision and Action"], 1966)

"An adaptive organism is connected with its environment by two kinds of channels. Afferent channels give it information about the state of the environment; efferent channels cause action on the environment. Problem statements define solutions in terms of afferent information to the organism; the organism's task is to discover a set of efferent signals which, changing the state of the environment, will produce the appropriate afferent. But, ab initio, the mapping of efferents on afferents is entirely arbitrary; the relations can only be discovered by experiment, by acting and observing the consequences of action." (Herbert A Simon, "The Logic of Heuristic Decision Making", [in "The Logic of Decision and Action"], 1966)

"Design problems - generating or discovering alternatives - are complex largely because they involve two spaces, an action space and a state space, that generally have completely different structures. To find a design requires mapping the former of these on the latter. For many, if not most, design problems in the real world systematic algorithms are not known that guarantee solutions with reasonable amounts of computing effort. Design uses a wide range of heuristic devices - like means-end analysis, satisficing, and the other procedures that have been outlined - that have been found by experience to enhance the efficiency of search. Much remains to be learned about the nature and effectiveness of these devices." (Herbert A Simon, "The Logic of Heuristic Decision Making", [in "The Logic of Decision and Action"], 1966)

"Every problem-solving effort must begin with creating a representation for the problem - a problem space in which the search for the solution can take place. Of course, for most of the problems we encounter in our daily personal or professional lives, we simply retrieve from memory a representation that we have already stored and used on previous occasions. Sometimes, we have to adapt the representation a bit to the new situation, but that is usually a rather simple matter." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"Natural science is knowledge about natural objects and phenomena." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"Learning is any change in a system that produces a more or less permanent change in its capacity for adapting to its environment. Understanding systems, especially systems capable of understanding problems in new task domains, are learning systems." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"Making discoveries belongs to the class of ill-structured problem-solving tasks that have relatively ill-defined goals." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"Solving a problem simply means representing it so as to make the solution transparent." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"The central task of a natural science is to make the wonderful commonplace: to show that complexity, correctly viewed, is only a mask for simplicity; to find pattern hidden in apparent chaos. […] This is the task of natural science: to show that the wonderful is not incomprehensible, to show how it can be comprehended - but not to destroy wonder. For when we have explained the wonderful, unmasked the hidden pattern, a new wonder arises at how complexity was woven out of simplicity. The aesthetics of natural science and mathematics is at one with the aesthetics of music and painting - both inhere in the discovery of a partially concealed pattern." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"The more we are willing to abstract from the detail of a set of phenomena, the easier it becomes to simulate the phenomena. Moreover we do not have to know, or guess at, all the internal structure of the system but only that part of it that is crucial to the abstraction." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"[...] in an information-rich world, the wealth of information means a dearth of something else: a scarcity of whatever it is that information consumes. What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention and a need to allocate that attention efficiently among the overabundance of information sources that might consume it." (Herbert Simon, "Designing Organizations for an Information-Rich World", 1971)

"But the answers provided by the theory of games are sometimes very puzzling and ambiguous. In many situations, no single course of action dominates all the others; instead, a whole set of possible solutions are all equally consistent with the postulates of rationality." (Herbert A Simon et al, "Decision Making and Problem Solving", Interfaces Vol. 17 (5), 1987)

"[...] problem solving generally proceeds by selective search through large sets of possibilities, using rules of thumb (heuristics) to guide the search. Because the possibilities in realistic problem situations are generally multitudinous, trial-and-error search would simply not work; the search must be highly selective." (Herbert A Simon et al, "Decision Making and Problem Solving", Interfaces Vol. 17 (5), 1987)

"The way in which an uncertain possibility is presented may have a substantial effect on how people respond to it." (Herbert A Simon et al, "Decision Making and Problem Solving", Interfaces Vol. 17 (5), 1987)

📁 On DIKW: On Knowledge (1990-1999)

"[By understanding] I mean simply a sufficient grasp of concepts, principles, or skills so that one can bring them to bear on new problems and situations, deciding in which ways one’s present competencies can suffice and in which ways one may require new skills or knowledge." (Howard Gardner, "The Unschooled Mind", 1991)

"The worst, i.e., most dangerous, feature of 'accepting the null hypothesis' is the giving up of explicit uncertainty. [...] Mathematics can sometimes be put in such black-and-white terms, but our knowledge or belief about the external world never can." (John Tukey, "The Philosophy of Multiple Comparisons", Statistical Science Vol. 6 (1), 1991)

"We live on an island surrounded by a sea of ignorance. As our island of knowledge grows, so does the shore of our ignorance." (John A Wheeler, Scientific American Vol. 267, 1992)

"Indeed, knowledge that one will be judged on some criterion of ‘creativeness’ or ‘originality’ tends to narrow the scope of what one can produce (leading to products that are then judged as relatively conventional); in contrast, the absence of an evaluations seems to liberate creativity." (Howard Gardner, "Creating Minds", 1993)

"Knowledge is theory. We should be thankful if action of management is based on theory. Knowledge has temporal spread. Information is not knowledge. The world is drowning in information but is slow in acquisition of knowledge. There is no substitute for knowledge." (William E Deming, "The New Economics for Industry, Government, Education", 1993)

"Worldviews are social constructions, and they channel the search for facts. But facts are found and knowledge progresses, however fitfully. Fact and theory are intertwined, and all great scientists understand the interaction." (Stephen J Gould, "Shields of Expectation - and Actuality", 1993)

"At the very least (there is certainly more), cybernetics implies a new philosophy about (1) what we can know, (2) about what it means for something to exist, and (3) about how to get things done. Cybernetics implies that knowledge is to be built up through effective goal-seeking processes, and perhaps not necessarily in uncovering timeless, absolute, attributes of things, irrespective of our purposes and needs." (Jeff Dooley, "Thoughts on the Question: What is Cybernetics", 1995)

"Crude complexity is ‘the length of the shortest message that will describe a system, at a given level of coarse graining, to someone at a distance, employing language, knowledge, and understanding that both parties share (and know they share) beforehand." (Murray Gell-Mann, "What is Complexity?" Complexity Vol. 1 (1), 1995)

"Humans may crave absolute certainty; they may aspire to it; they may pretend, as partisans of certain religions do, to have attained it. But the history of science - by far the most successful claim to knowledge accessible to humans - teaches that the most we can hope for is successive improvement in our understanding, learning from our mistakes, an asymptotic approach to the Universe, but with the proviso that absolute certainty will always elude us. We will always be mired in error. The most each generation can hope for is to reduce the error bars a little, and to add to the body of data to which error bars apply." (Carl Sagan, "The Demon-Haunted World: Science as a Candle in the Dark", 1995)

"The amount of understanding produced by a theory is determined by how well it meets the criteria of adequacy - testability, fruitfulness, scope, simplicity, conservatism - because these criteria indicate the extent to which a theory systematizes and unifies our knowledge." (Theodore Schick Jr., "How to Think about Weird Things: Critical Thinking for a New Age", 1995)

"The representational nature of maps, however, is often ignored - what we see when looking at a map is not the word, but an abstract representation that we find convenient to use in place of the world. When we build these abstract representations we are not revealing knowledge as much as are creating it." (Alan MacEachren, "How Maps Work: Representation, Visualization, and Design", 1995)

"The term mental model refers to knowledge structures utilized in the solving of problems. Mental models are causal and thus may be functionally defined in the sense that they allow a problem solver to engage in description, explanation, and prediction. Mental models may also be defined in a structural sense as consisting of objects, states that those objects exist in, and processes that are responsible for those objects’ changing states." (Robert Hafner & Jim Stewart, "Revising Explanatory Models to Accommodate Anomalous Genetic Phenomena: Problem Solving in the ‘Context of Discovery’", Science Education 79 (2), 1995) 

"Generalization is the process of matching new, unknown input data with the problem knowledge in order to obtain the best possible solution, or one close to it. Generalization means reacting properly to new situations, for example, recognizing new images, or classifying new objects and situations. Generalization can also be described as a transition from a particular object description to a general concept description. This is a major characteristic of all intelligent systems." (Nikola K Kasabov, "Foundations of Neural Networks, Fuzzy Systems, and Knowledge Engineering", 1996)

"In the new systems thinking, the metaphor of knowledge as a building is being replaced by that of the network. As we perceive reality as a network of relationships, our descriptions, too, form an interconnected network of concepts and models in which there are no foundations. For most scientists such a view of knowledge as a network with no firm foundations is extremely unsettling, and today it is by no means generally accepted. But as the network approach expands throughout the scientific community, the idea of knowledge as a network will undoubtedly find increasing acceptance." (Fritjof Capra," The Web of Life: a new scientific understanding of living systems", 1996)

"Discourses are ways of referring to or constructing knowledge about a particular topic of practice: a cluster (or formation) of ideas, images and practices, which provide ways of talking about, forms of knowledge and conduct associated with, a particular topic, social activity or institutional site in society. These discursive formations, as they are known, define what is and is not appropriate in our formulation of, and our practices in relation to, a particular subject or site of social activity." (Stuart Hall, "Representation: Cultural Representations and Signifying Practices", 1997)

"Data is discrimination between physical states of things (black, white, etc.) that may convey or not convey information to an agent. Whether it does so or not depends on the agent's prior stock of knowledge." (Max Boisot, "Knowledge Assets", 1998)

"The social constructivist thesis is that mathematics is a social construction, a cultural product, fallible like any other branch of knowledge." (Paul Ernest, "Social Constructivism as a Philosophy of Mathematics", 1998)

"An individual understands a concept, skill, theory, or domain of knowledge to the extent that he or she can apply it appropriately in a new situation." (Howard Gardner, "The Disciplined Mind", 1999)

"Analysis of a system reveals its structure and how it works. It provides the knowledge required to make it work efficiently and to repair it when it stops working. Its product is know-how, knowledge, not understanding. To enable a system to perform effectively we must understand it - we must be able to explain its behavior—and this requires being aware of its functions in the larger systems of which it is a part." (Russell L Ackoff, "Re-Creating the Corporation", 1999)

📁 On DIKW: On Knowledge (1980-1989)

"Definitions, like questions and metaphors, are instruments for thinking. Their authority rests entirely on their usefulness, not their correctness. We use definitions in order to delineate problems we wish to investigate, or to further interests we wish to promote. In other words, we invent definitions and discard them as suits our purposes." (Neil Postman, "Language Education in a Knowledge Context", 1980)

"A schema, then is a data structure for representing the generic concepts stored in memory. There are schemata representing our knowledge about all concepts; those underlying objects, situations, events, sequences of events, actions and sequences of actions. A schema contains, as part of its specification, the network of interrelations that is believed to normally hold among the constituents of the concept in question. A schema theory embodies a prototype theory of meaning. That is, inasmuch as a schema underlying a concept stored in memory corresponds to the meaning of that concept, meanings are encoded in terms of the typical or normal situations or events that instantiate that concept." (David E Rumelhart, "Schemata: The building blocks of cognition", 1980)

“Analogies, metaphors, and emblems are the threads by which the mind holds on to the world even when, absentmindedly, it has lost direct contact with it, and they guarantee the unity of human experience. Moreover, in the thinking process itself they serve as models to give us our bearings lest we stagger blindly among experiences that our bodily senses with their relative certainty of knowledge cannot guide us through.” (Hannah Arendt, “The Life of the Mind”, 1981)

"Knowledge specialists may ascribe a degree of certainty to their models of the world that baffles and offends managers. Often the complexity of the world cannot be reduced to mathematical abstractions that make sense to a manager. Managers who expect complete, one-to-one correspondence between the real world and each element in a model are disappointed and skeptical." (Dale E Zand, "Information, Organization, and Power", 1981)

"The thinking person goes over the same ground many times. He looks at it from varying points of view - his own, his arch-enemy’s, others’. He diagrams it, verbalizes it, formulates equations, constructs visual images of the whole problem, or of troublesome parts, or of what is clearly known. But he does not keep a detailed record of all this mental work, indeed could not. […] Deep understanding of a domain of knowledge requires knowing it in various ways. This multiplicity of perspectives grows slowly through hard work and sets the state for the re-cognition we experience as a new insight." (Howard E Gruber, "Darwin on Man", 1981)

"Definitions are temporary verbalizations of concepts, and concepts - particularly difficult concepts - are usually revised repeatedly as our knowledge and understanding grows." (Ernst Mayr, "The Growth of Biological Thought", 1982)

"We are drowning in information but starved for knowledge." (John Naisbitt, "Megatrends: Ten New Directions Transforming Our Lives", 1982)

"We define a semantic network as 'the collection of all the relationships that concepts have to other concepts, to percepts, to procedures, and to motor mechanisms' of the knowledge." (John F Sowa, "Conceptual Structures", 1984)

"Contrary to the impression students acquire in school, mathematics is not just a series of techniques. Mathematics tells us what we have never known or even suspected about notable phenomena and in some instances even contradicts perception. It is the essence of our knowledge of the physical world. It not only transcends perception but outclasses it." (Morris Kline, "Mathematics and the Search for Knowledge", 1985)

"Knowledge is the appropriate collection of information, such that it's intent is to be useful. Knowledge is a deterministic process. When someone 'memorizes' information (as less-aspiring test-bound students often do), then they have amassed knowledge. This knowledge has useful meaning to them, but it does not provide for, in and of itself, an integration such as would infer further knowledge." (Russell L Ackoff, "Towards a Systems Theory of Organization", 1985)

"Although science literally means ‘knowledge’, the scientific attitude is concerned much more with rational perception through the mind and with testing such perceptions against actual fact, in the form of experiments and observations." (David Bohm & F David Peat, "Science, Order, and Creativity", 1987)

"There is no coherent knowledge, i.e. no uniform comprehensive account of the world and the events in it. There is no comprehensive truth that goes beyond an enumeration of details, but there are many pieces of information, obtained in different ways from different sources and collected for the benefit of the curious. The best way of presenting such knowledge is the list - and the oldest scientific works were indeed lists of facts, parts, coincidences, problems in several specialized domains." (Paul K Feyerabend, "Farewell to Reason", 1987)

"We admit knowledge whenever we observe an effective (or adequate) behavior in a given context, i.e., in a realm or domain which we define by a question (explicit or implicit)." (Humberto Maturana & Francisco J Varela, "The Tree of Knowledge", 1987)

"In the Information Age, the first step to sanity is FILTERING. Filter the information: extract for knowledge. Filter first for substance. Filter second for significance. […] Filter third for reliability. […] Filter fourth for completeness." (Marc Stiegler, "David’s Sling", 1988)

"Probabilities are summaries of knowledge that is left behind when information is transferred to a higher level of abstraction." (Judea Pearl, "Probabilistic Reasoning in Intelligent Systems: Network of Plausible, Inference", 1988)

"Science doesn't purvey absolute truth. Science is a mechanism. It's a way of trying to improve your knowledge of nature. It's a system for testing your thoughts against the universe and seeing whether they match. And this works, not just for the ordinary aspects of science, but for all of life. I should think people would want to know that what they know is truly what the universe is like, or at least as close as they can get to it." (Isaac Asimov, [Interview by Bill Moyers] 1988)

"A discovery in science, or a new theory, even where it appears most unitary and most all-embracing, deals with some immediate element of novelty or paradox within the framework of far vaster, unanalyzed, unarticulated reserves of knowledge, experience, faith, and presupposition. Our progress is narrow: it takes a vast world unchallenged and for granted." (James R Oppenheimer, "Atom and Void", 1989)

On Randomness XIX (Chaos II)

"The chaos theory will require scientists in all fields to, develop sophisticated mathematical skills, so that they will be able to better recognize the meanings of results. Mathematics has expanded the field of fractals to help describe and explain the shapeless, asymmetrical find randomness of the natural environment." (Theoni Pappas, "More Joy of Mathematics: Exploring mathematical insights & concepts", 1991)

"Chaos demonstrates that deterministic causes can have random effects […] There's a similar surprise regarding symmetry: symmetric causes can have asymmetric effects. […] This paradox, that symmetry can get lost between cause and effect, is called symmetry-breaking. […] From the smallest scales to the largest, many of nature's patterns are a result of broken symmetry; […]" (Ian Stewart & Martin Golubitsky, "Fearful Symmetry: Is God a Geometer?", 1992)

"Systems, acting dynamically, produce (and incidentally, reproduce) their own boundaries, as structures which are complementary (necessarily so) to their motion and dynamics. They are liable, for all that, to instabilities chaos, as commonly interpreted of chaotic form, where nowadays, is remote from the random. Chaos is a peculiar situation in which the trajectories of a system, taken in the traditional sense, fail to converge as they approach their limit cycles or 'attractors' or 'equilibria'. Instead, they diverge, due to an increase, of indefinite magnitude, in amplification or gain." (Gordon Pask, "Different Kinds of Cybernetics", 1992)

"Often, we use the word random loosely to describe something that is disordered, irregular, patternless, or unpredictable. We link it with chance, probability, luck, and coincidence. However, when we examine what we mean by random in various contexts, ambiguities and uncertainties inevitably arise. Tackling the subtleties of randomness allows us to go to the root of what we can understand of the universe we inhabit and helps us to define the limits of what we can know with certainty." (Ivars Peterson, "The Jungles of Randomness: A Mathematical Safari", 1998)

"Randomness, chaos, uncertainty, and chance are all a part of our lives. They reside at the ill-defined boundaries between what we know, what we can know, and what is beyond our knowing. They make life interesting." (Ivars Peterson, "The Jungles of Randomness: A Mathematical Safari", 1998)

"There are only patterns, patterns on top of patterns, patterns that affect other patterns. Patterns hidden by patterns. Patterns within patterns. If you watch close, history does nothing but repeat itself. What we call chaos is just patterns we haven't recognized. What we call random is just patterns we can't decipher. what we can't understand we call nonsense. What we can't read we call gibberish. There is no free will. There are no variables." (Chuck Palahniuk, "Survivor", 1999)

"Heat is the energy of random chaotic motion, and entropy is the amount of hidden microscopic information." (Leonard Susskind, "The Black Hole War", 2008)

"Chaos is impatient. It's random. And above all it's selfish. It tears down everything just for the sake of change, feeding on itself in constant hunger. But Chaos can also be appealing. It tempts you to believe that nothing matters except what you want." (Rick Riordan, "The Throne of Fire", 2011)

"A system in which a few things interacting produce tremendously divergent behavior; deterministic chaos; it looks random but its not." (Chris Langton)

On Fuzzy Logic IV

"Fuzzy systems are excellent tools for representing heuristic, commonsense rules. Fuzzy inference methods apply these rules to data and infer a solution. Neural networks are very efficient at learning heuristics from data. They are 'good problem solvers' when past data are available. Both fuzzy systems and neural networks are universal approximators in a sense, that is, for a given continuous objective function there will be a fuzzy system and a neural network which approximate it to any degree of accuracy." (Nikola K Kasabov, "Foundations of Neural Networks, Fuzzy Systems, and Knowledge Engineering", 1996)

"Fuzzy set theory [...] is primarily concerned with quantifying and reasoning using natural language in which words can have ambiguous meanings. It is widely used in a variety of fields because of its simplicity and similarity to human reasoning." (Tzung-Pei Hong et al, "Genetic-Fuzzy Data Mining Techniques", 2009)

"Fuzzy systems are rule-based expert systems based on fuzzy rules and fuzzy inference. Fuzzy rules represent in a straightforward way 'commonsense' knowledge and skills, or knowledge that is subjective, ambiguous, vague, or contradictory." (Nikola K Kasabov, "Foundations of Neural Networks, Fuzzy Systems, and Knowledge Engineering", 1996)

"A concept which has a position of centrality in fuzzy logic is that of a fuzzy set. Informally, a fuzzy set is a class with a fuzzy boundary, implying a gradual transition from membership to nonmembership. A fuzzy set is precisiated through graduation, that is, through association with a scale of grades of membership. Thus, membership in a fuzzy set is a matter of degree. Importantly, in fuzzy logic everything is or is allowed to be graduated, that is, be a matter of degree. Furthermore, in fuzzy logic everything is or is allowed to be granulated, with a granule being a clump of attribute-values drawn together by indistinguishability, equivalence, similarity, proximity or functionality. Graduation and granulation form the core of fuzzy logic. Graduated granulation is the basis for the concept of a linguistic variable – a variable whose values are words rather than numbers. The concept of a linguistic variable is employed in almost all applications of fuzzy logic." (Lofti A Zadeh, "Fuzzy Logic", 2009)

"[Fuzzy logic p]rovides formal means for the representation of, and inference based on imprecisely specified premises and rules of inference; can be understood in different ways, basically as fuzzy logic in a narrow sense, being some type of multivalued logic, and fuzzy logic in a broad sense, being a way to formalize inference based on imprecisely specified premises and rules of inference. (Janusz Kacprzyk, "Foundations of Fuzzy Sets Theory", 2009)

"Granular computing is a general computation theory for using granules such as subsets, classes, objects, clusters, and elements of a universe to build an efficient computational model for complex applications with huge amounts of data, information, and knowledge. Granulation of an object a leads to a collection of granules, with a granule being a clump of points (objects) drawn together by indiscernibility, similarity, proximity, or functionality. In human reasoning and concept formulation, the granules and the values of their attributes are fuzzy rather than crisp. In this perspective, fuzzy information granulation may be viewed as a mode of generalization, which can be applied to any concept, method, or theory." (Salvatore Greco et al, "Granular Computing and Data Mining for Ordered Data: The Dominance-Based Rough Set Approach", 2009)

"In essence, logic is concerned with formalization of reasoning. Correspondently, fuzzy logic is concerned with formalization of fuzzy reasoning, with the understanding that precise reasoning is a special case of fuzzy reasoning." (Lofti A Zadeh, "Fuzzy Logic", 2009)

"Science deals not with reality but with models of reality. In large measure, scientific progress is driven by a quest for better models of reality. In the real world, imprecision, uncertainty and complexity have a pervasive presence. In this setting, construction of better models of reality requires a better understanding of how to deal effectively with imprecision, uncertainty and complexity. To a significant degree, development of fuzzy logic has been, and continues to be, motivated by this need." (Lofti A Zadeh, "Fuzzy Logic", 2009)

"Unlike a conventional set, in a fuzzy set, a fuzzy membership function is used to define the degree of an element belonging to the set. Fuzzy logic is a superset of conventional (Boolean) logic that has been extended to handle the concept of partial truth as defined by membership functions. Fuzzy logic contributes to the machinery of granular computing." (Zhengxin Chen, "Philosophical Foundation for Granular Computing", 2009)

"Unlike the classic set theory where a set is represented as an indicator function to specify if an object belongs or not to it, a fuzzy set is an extension of a classic set where a subset is represented as a membership function to characterize the degree that an object belongs to it. The indicator function of a classic set takes value of 1 or 0, whereas the membership function of a fuzzy set takes value between 1 and 0." (Jianchao Han & Nick Cerone, Principles and Perspectives of Granular Computing, 2009) 

On Heuristics II

"Models of bounded rationality describe how a judgement or decision is reached (that is, the heuristic processes or proximal mechanisms) rather than merely the outcome of the decision, and they describe the class of environments in which these heuristics will succeed or fail." (Gerd Gigerenzer & Reinhard Selten [Eds., "Bounded Rationality: The Adaptive Toolbox", 2001)

"A second class of metaphors - mathematical algorithms, heuristics, and models - brings us closer to the world of computer science programs, simulations, and approximations of the brain and its cognitive processes." (Diego Rasskin-Gutman, "Chess Metaphors: Artificial Intelligence and the Human Mind", 2009)

"In particular, the accurate intuitions of experts are better explained by the effects of prolonged practice than by heuristics. We can now draw a richer and more balanced picture, in which skill and heuristics are alternative sources of intuitive judgments and choices." (Daniel Kahneman, "Thinking, Fast and Slow", 2011)

"This is the essence of intuitive heuristics: when faced with a difficult question, we often answer an easier one instead, usually without noticing the substitution." (Daniel Kahneman, "Thinking, Fast and Slow", 2011)

"Heuristics are an evolutionary solution to an ongoing problem: we have limited mental resources. As such, they have a very long and thoroughly time-tested history of helping us - on average - make better decisions." (Peter H Diamandis, "Abundance: The Future is Better Than You Think", 2012)

"Heuristics are simplified rules of thumb that make things simple and easy to implement. But their main advantage is that the user knows that they are not perfect, just expedient, and is therefore less fooled by their powers. They become dangerous when we forget that." (Nassim N Taleb, "Antifragile: Things that gain from disorder", 2012)

"The art of reasoned persuasion is an iterative, recursive heuristic, meaning that we must go back and forth between the facts and the rules until we have a good fit. We cannot see the facts properly until we know what framework to place them into, and we cannot determine what framework to place them into until we see the basic contours of the facts." (Joel P Trachtman, "The Tools of Argument", 2013)

"Heuristic decision making is fast and frugal and is often based on the evaluation of one or two salient bits of information." (Amitav Chakravarti, "Why People (Don’t) Buy: The Go and Stop Signals", 2015)

"A heuristic is a strategy we derive from previous experience with a similar problem." (Darius Foroux, "Think Straight", 2017)

"The social world that humans have made for themselves is so complex that the mind simplifies the world by using heuristics, customs, and habits, and by making models or assumptions about how things generally work (the ‘causal structure of the world’). And because people rely upon (and are invested in) these mental models, they usually prefer that they remain uncontested." (Dr James Brennan, "Psychological  Adjustment to Illness and Injury", West of England Medical Journal Vol. 117 (2), 2018)

📊On Statistics: On Randomness XI

"The postulate of randomness thus resolves itself into the question, 'of what population is this a random sample?' which must frequently be asked by every practical statistician." (Ronald Fisher, "On the Mathematical Foundation of Theoretical Statistics", Philosophical Transactions of the Royal Society of London Vol. A222, 1922)

"The definition of random in terms of a physical operation is notoriously without effect on the mathematical operations of statistical theory because so far as these mathematical operations are concerned random is purely and simply an undefined term." (Walter A Shewhart & W. Edwards "Deming, Statistical Method from the Viewpoint of Quality Control", 1939)

"The purely random sample is the only kind that can be examined with entire confidence by means of statistical theory, but there is one thing wrong with it. It is so difficult and expensive to obtain for many uses that sheer cost eliminates it." (Harold Hotelling, "How to Lie with Statistics", 1954)

"The point is that every experiment involves an error, the magnitude of which is not known beforehand and it varies from one experiment to another. For this reason, no matter what finite number of experiments have been carried out, the arithmetic mean of the values obtained will contain an error. Of course, if the experiments are conducted under identical conditions and the errors are random errors, then the error of the mean will diminish as the number of experiments is increased, but it cannot be reduced to zero for a finite number of experiments. […] The choice of entities for an experiment must be perfectly random, so that even an apparently inessential cause could not lead to erroneous conclusions." (Yakov Khurgin, "Did You Say Mathematics?", 1974)

"Statistics has been likened to a telescope. The latter enables one to see further and to make clear objects which were diminished or obscured by distance. The former enables one to discern structure and relationships which were distorted by other factors or obscured by random variation." (David J Hand, "The Role of Statistics in Psychiatry", Psychological Medicine Vol. 15, 1985)

"We will use the convenient expression 'chosen at random' to mean that the probabilities of the events in the sample space are all the same unless some modifying words are near to the words 'at random'. Usually we will compute the probability of the outcome based on the uniform probability model since that is very common in modeling simple situations. However, a uniform distribution does not imply that it comes from a random source; […]" (Richard W Hamming, "The Art of Probability for Scientists and Engineers", 1991)

"If you perceive the world as some place where things happen at random - random events over which you have sometimes very little control, sometimes fairly good control, but still random events - well, one has to be able to have some idea of how these things behave. […] People who are not used to statistics tend to see things in data - there are random fluctuations which can sometimes delude them - so you have to understand what can happen randomly and try to control whatever can be controlled. You have to expect that you are not going to get a clean-cut answer. So how do you interpret what you get? You do it by statistics." (Lucien LeCam, [interview] 1988)

"When looking at the end result of any statistical analysis, one must be very cautious not to over interpret the data. Care must be taken to know the size of the sample, and to be certain the method for gathering information is consistent with other samples gathered. […] No one should ever base conclusions without knowing the size of the sample and how random a sample it was. But all too often such data is not mentioned when the statistics are given - perhaps it is overlooked or even intentionally omitted." (Theoni Pappas, "More Joy of Mathematics: Exploring mathematical insights & concepts", 1994)

"We use mathematics and statistics to describe the diverse realms of randomness. From these descriptions, we attempt to glean insights into the workings of chance and to search for hidden causes. With such tools in hand, we seek patterns and relationships and propose predictions that help us make sense of the world." (Ivars Peterson, "The Jungles of Randomness: A Mathematical Safari", 1998)

"[In statistics] you have the fact that the concepts are not very clean. The idea of probability, of randomness, is not a clean mathematical idea. You cannot produce random numbers mathematically. They can only be produced by things like tossing dice or spinning a roulette wheel. With a formula, any formula, the number you get would be predictable and therefore not random. So as a statistician you have to rely on some conception of a world where things happen in some way at random, a conception which mathematicians don’t have." (Lucien LeCam, [interview] 1988)

"[...] in the statistical world, what we see and measure around us can be considered as the sum of a systematic mathematical idealized form plus some random contribution that cannot yet be explained. This is the classic idea of the signal and the noise." (David Spiegelhalter, "The Art of Statistics: Learning from Data", 2019)

"Too little attention is given to the need for statistical control, or to put it more pertinently, since statistical control (randomness) is so rarely found, too little attention is given to the interpretation of data that arise from conditions not in statistical control." (William E Deming)

📁 On DIKW: On Facts (Unsourced)

"A fact is not novel if it has an analogue which could have some interest. A fact which does not fi t in with a series of known facts is a fact which deserves particular attention. If the mind had to retain all individual facts, it could not manage and science would not exist; but when these facts can be connected by general laws and by theories, when a large number of these facts can be represented by a single one, one can remember them more easily, one can generalise one’s ideas, one can compare one general fact with another general fact and discoveries can succeed each other. It is only when laws can be introduced into a science that it assumes the true character of science." (Joseph L Gay-Lussac)

"All the pictures which science draws of Nature, and which alone seem capable of according with observational facts, are mathematical pictures." (Sir James Jeans)

"By observation, facts are distinctly and minutely impressed in the mind; by analogy, similar facts are connected ; by experiment, new facts are discovered ; and, in the progression of knowledge, observation, guided by analogy, leads to experiment, and analogy, confirmed by experiment, becomes scientific truth." (Sir Humphry Davy)

"Cognitive psychology has shown that the mind best understands facts when they are woven into a conceptual fabric, such as a narrative, mental map, or intuitive theory. Disconnected facts in the mind are like unlinked pages on the Web: They might as well not exist." (Steven Pinker) 

"Education is not the piling on of learning, information, data, facts, skills, or abilities - that's training or instruction - but is rather making visible what is hidden as a seed." (Thomas W Moore)

"Facts and values are entangled in science. It's not because scientists are biased, not because they are partial or influenced by other kinds of interests, but because of a commitment to reason, consistency, coherence, plausibility and replicability. These are value commitments." (Alva Noë)

"Facts are stubborn things, but statistics are pliable." (Mark Twain)

"[…] facts by themselves are silent. Observation discovers nothing directly of the actions of causes, but only of sequences in time." (Alfred Marshall)

"First accumulate a mass of Facts: and then construct a Theory." (Lewis Carroll)

[Maier’s Law:] "If the facts do not conform to the theory, they must be disposed of." (Norman R F Maier)

"Imagination, as well as reason, is necessary to perfection in the philosophical mind. A rapidity of combination, a power of perceiving analogies, and of comparing them by facts, is the creative source of discovery." (Sir Humphry Davy)

"In physical science the discovery of new facts is open to every blockhead with patience, manual dexterity, and acute senses; it is less effectually promoted by genius than by co-operation, and more frequently the result of accident than of design." (John Thomson)

"In the study of Nature conjecture must be entirely put aside, and vague hypothesis carefully guarded against. The study of Nature begins with facts, ascends to laws, and raises itself, as far as the limits of man’s intellect will permit, to the knowledge of causes, by the threefold means of observation, experiment and logical deduction." (Jean Baptiste-Andre Dumas)

"No good model ever accounted for all the facts, since some data was bound to be misleading if not plain wrong." (James Dewey Watson)

"Nothing in education is so astonishing as the amount of ignorance it accumulates in the form of inert facts." (Henry B Adams)

"One might describe the mathematical quality in Nature by saying that the universe is so constituted that mathematics is a useful tool in its description. However, recent advances in physical science show that this statement of the case is too trivial. The connection between mathematics and the description of the universe goes far deeper than this, and one can get an appreciation of it only from a thorough examination of the various facts that make it up." (Paul A M Dirac)

"Science does more than collect facts; it makes sense of them. Great scientists are virtuosi of the art of discovering the meaning of what otherwise might seem barren observations." (Theodosius Dobzhansky)

"Science is not a technique or a body of knowledge, though it uses both. It is rather an attitude of inquiry, or observation and reasoning, with respect to the world. It can be developed, not by memorizing facts or juggling formulas to get an answer, but only by actual practice of scientific observation and reasoning." (Karl T Compton)

"Science is opposed to theological dogmas because science is founded on fact. To me, the universe is simply a great machine which never came into being and never will end. The human being is no exception to the natural order. Man, like the universe, is a machine." (Nikola Tesla)

"Some facts can be seen more clearly by example than by proof." (Leonard Euler)

"The aim of education is the knowledge not of facts but of values." (William R Inge)

"The arguments […] by which you support my theories, are most ingenious, but not founded on demonstrated facts; analogy is no proof." (Louis Pasteur)

"The art of observation and that of experimentation are very distinct. In the first case, the fact may either proceed from logical reasons or be mere good fortune; it is sufficient to have some penetration and a sense of truth in order to profit by it. But the art of experimentation leads from the first to the last link of the chain, without hesitation and without a blank, making successive use of Reason, which suggests an alternative, and of Experience, which decides on it, until, starting from a faint glimmer, the full blaze of light is reached." (Jean Baptiste-Andre Dumas)

"The disclosure of a new fact, the leap forward, the conquest over yesterday’s ignorance, is an act not of reason but of imagination, of intuition." (Charles Nicolle)

"The experiment serves two purposes, often independent one from the other: it allows the observation of new facts, hitherto either unsuspected, or not yet well defined; and it determines whether." (René J Dubos)

"[…] the mathematician learns early to accept no fact, to believe no statement, however apparently reasonable or obvious or trivial, until it has been proved, rigorously and totally by a series of steps proceeding from universally accepted first principles." (Alfred Adler)

"The object of education is not only to produce a man who knows, but one who does; who makes his mark in the straggle of life and succeeds well in whatever he undertakes: who can solve the problems of nature and of humanity as they arise, and who, when he knows he is right, can boldly convince the world of the fact." (Henry A Rowland)

"The present system of education is all wrong. The mind is crammed with facts before it knows how to think. Control of the mind should be taught first. It takes people a long time to learn things because they can't concentrate their minds at will." (Swami Vivekananda)

"Theory helps us to bear our ignorance of facts." (George Santayana)

"There is no drawing the line between physics and metaphysics. If you examine every day facts at all closely, you are a physicist; but if you press your physics at all home, you become a metaphysician; if you press your metaphysics at all home, you are in a fog." (Samuel Butler)

"There is nothing more deceptive than an obvious fact." (Sir Arthur C Doyle)