Communication Theory II

"Communication theory deals with certain important but abstract aspects of communication. Communication theory proceeds from clear and definite assumptions to theorems concerning information sources and communication channels. In this it is essentially mathematical, and in order to understand it we must understand the idea of a theorem as a statement which must be proved, that is, which must be shown to be the necessary consequence of a set of initial assumptions. This is an idea which is the very heart of mathematics as mathematicians understand it." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"Communication theory tells us how many bits of information can be sent per second over perfect and imperfect communication channels in terms of rather abstract descriptions of the properties of these channels. Communication theory tells us how to measure the rate at which a message source, such as a speaker or a writer, generates information. Communication theory tells us how to represent, or encode, messages from a particular message source efficiently for transmission over a particular sort of channel, such as an electrical circuit, and it tells us when we can avoid errors in transmission." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"In communication theory we consider a message source, such as a writer or a speaker, which may produce on a given occasion any one of many possible messages. The amount of information conveyed by the message increases as the amount of uncertainty as to what message actually will be produced becomes greater. A message which is one out of ten possible messages conveys a smaller amount of information than a message which is one out of a million possible messages. The entropy of communication theory is a measure of this uncertainty and the uncertainty, or entropy, is taken as the measure of the amount of information conveyed by a message from a source. The more we know about what message the source will produce, the less uncertainty, the less the entropy, and the less the information." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"The amount of information conveyed by the message increases as the amount of uncertainty as to what message actually will be produced becomes greater. A message which is one out of ten possible messages conveys a smaller amount of information than a message which is one out of a million possible messages. The entropy of communication theory is a measure of this uncertainty and the uncertainty, or entropy, is taken as the measure of the amount of information conveyed by a message from a source. The more we know about what message the source will produce, the less uncertainty, the less the entropy, and the less the information." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals and Noise", 1980)

"Thus, information is sometimes associated with the idea of knowledge through its popular use rather than with uncertainty and the resolution of uncertainty, as it is in communication theory." (John R Pierce, "An Introduction to Information Theory: Symbols, Signals & Noise" 2nd Ed., 1980)

"Cybernetics is concerned with scientific investigation of systemic processes of a highly varied nature, including such phenomena as regulation, information processing, information storage, adaptation, self-organization, self-reproduction, and strategic behavior. Within the general cybernetic approach, the following theoretical fields have developed: systems theory (system), communication theory, game theory, and decision theory." (Fritz B Simon et al, "Language of Family Therapy: A Systemic Vocabulary and Source Book", 1985)

"General evolution theory, based on the integration of the relevant tenets of general system theory, cybernetics, information and communication theory, chaos theory, dynamical systems theory, and nonequilibrium thermodynamics, can convey a sound understanding of the laws and dynamics that govern the evolution of complex systems in the various realms of investigation [...]. The basic notions of this new discipline can be developed to give an adequate account of the dynamical evolution of human societies as well. Such an account could furnish the basis of a system of knowledge better able to orient human beings and societies in their rapidly changing milieu." (Ervin László et al, "The Evolution of Cognitive Maps: New Paradigms for the Twenty-first Century", 1993)

"Communication theory is enormously rich in the range of ideas that fall within its nominal scope, and new theoretical work on communication has recently been flourishing. Nevertheless, despite the ancient roots and growing profusion of theories about communication, I argue that communication theory as an identifiable field of study does not yet exist." (Robert T Craig "Communication Theory as a Field", 1999)

The very essence of mass communication theory is a simple but all-embracing expression of technological determinism, since the essential features depend on what certain technologies have made possible, certain technologies have made possible, especially the following: communication at a distance, the multiplication and simultaneous distribution of diverse ‘messages’, the enormous capacity and speed of carriers, and the limitations on response. There is no escaping the implication that public communication as practised in modern societies is profoundly shaped by these general features." (Denis McQuail, "McQuail's Reader in Mass Communication Theory", 2002)

"Without an understanding of causality there can be no theory of communication. What passes as information theory today is not communication at all, but merely transportation." (Marshall McLuhan, "The Book of Probes : Marshall McLuhan", 2011) 

"Cybernetics is an interdisciplinary science. It originated ‘at the junction’ of mathematics, logic, semiotics, physiology, biology and sociology. Among its inherent features, we mention analysis and revelation of general principles and approaches in scientific cognition. Control theory, communication theory, operations research and others represent most weighty theories within cybernetics 1.0." (Dmitry A Novikov, "Cybernetics 2.0", 2016)

Communication Theory I

"We have decided to call the entire field of control and communication theory, whether in the machine or in the animal, by the name Cybernetics, which we form from the Greek [...] for steersman. In choosing this term, we wish to recognize that the first significant paper on feedback mechanisms is an article on governors, which was published by Clerk Maxwell in 1868, and that governor is derived from a Latin corruption [...] We also wish to refer to the fact that the steering engines of a ship are indeed one of the earliest and best-developed forms of feedback mechanisms." (Norbert Wiener, "Cybernetics", 1948)

"Incomplete knowledge of the future, and also of the past of the transmitter from which the future might be constructed, is at the very basis of the concept of information. On the other hand, complete ignorance also precludes communication; a common language is required, that is to say an agreement between the transmitter and the receiver regarding the elements used in the communication process [... The information of a message can] be defined as the 'minimum number of binary decisions which enable the receiver to construct the message, on the basis of the data already available to him.' These data comprise both the convention regarding the symbols and the language used, and the knowledge available at the moment when the message started." (Dennis Gabor, "Optical transmission" [in: "Information Theory: Papers Read at a Symposium on Information Theory"], 1952) 

"The theory of communication is partly concerned with the measurement of information content of signals, as their essential property in the establishment of communication links. But the information content of signals is not to be regarded as a commodity; it is more a property or potential of the signals, and as a concept it is closely related to the idea of selection, or discrimination. This mathematical theory first arose in telegraphy and telephony, being developed for the purpose of measuring the information content of telecommunication signals. It concerned only the signals themselves as transmitted along wires, or broadcast through the aether, and is quite abstracted from all questions of 'meaning'. Nor does it concern the importance, the value, or truth to any particular person. As a theory, it lies at the syntactic level of sign theory and is abstracted from the semantic and pragmatic levels. We shall argue [...] that, though the theory does not directly involve biological elements, it is nevertheless quite basic to the study of human communication - basic but insufficient." (Colin Cherry, "On Human Communication", 1957)

"A more viable model, one much more faithful to the kind of system that society is more and more recognized to be, is in process of developing out of, or is in keeping with, the modern systems perspective (which we use loosely here to refer to general systems research, cybernetics, information and communication theory, and related fields). Society, or the sociocultural system, is not, then, principally an equilibrium system or a homeostatic system, but what we shall simply refer to as a complex adaptive system." (Walter F Buckley, "Society as a complex adaptive system", 1968)

"The general notion in communication theory is that of information. In many cases, the flow of information corresponds to a flow of energy, e. g. if light waves emitted by some objects reach the eye or a photoelectric cell, elicit some reaction of the organism or some machinery, and thus convey information." (Ludwig von Bertalanffy, "General System Theory", 1968) 

"The 'flow of information' through human communication channels is enormous. So far no theory exists, to our knowledge, which attributes any sort of unambiguous measure to this 'flow'." (Anatol Rapoport, "Modern Systems Research for the Behavioral Scientist", 1969)

"The term hypothesis testing arises because the choice as to which process is observed is based on hypothesized models. Thus hypothesis testing could also be called model testing. Hypothesis testing is sometimes called decision theory. The detection theory of communication theory is a special case." (Fred C Scweppe, "Uncertain dynamic systems", 1973)

Claude E Shannon - Collected Quotes

"The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem. The significant aspect is that the actual message is one selected from a set of possible messages." (Claude E Shannon, "A mathematical theory of communication", Bell Systems Technical Journal 27, 1948)

"Almost every problem that you come across is befuddled with all kinds of extraneous data of one sort or another; and if you can bring this problem down into the main issues, you can see more clearly what you’re trying to do." (Claude E Shannon, "Creative Thinking", 1952)

"Another approach for a given problem is to try to restate it in just as many different forms as you can. Change the words. Change the viewpoint. Look at it from every possible angle. After you’ve done that, you can try to look at it from several angles at the same time and perhaps you can get an insight into the real basic issues of the problem, so that you can correlate the important factors and come out with the solution." (Claude E Shannon, "Creative Thinking", 1952)

"Many proofs in mathematics have been actually found by extremely roundabout processes. A man starts to prove this theorem and he finds that he wanders all over the map. He starts off and prove a good many results which don’t seem to be leading anywhere and then eventually ends up by the back door on the solution of the given problem." (Claude E Shannon, "Creative Thinking", 1952)

"Electronic computers are normally used for the solution of numerical problems arising in science or industry. The fundamental design of these computers, however, is so flexible and so universal in conception that they maybe programmed to perform many operations which do not involve numbers at all - operations such as the translation of language, the analysis of a logical situation or the playing of games. The same orders which are used in constructing a numerical program maybe used to symbolize operations on abstract entities such as the words of a language or the positions in a chess game." (Claude E Shannon, "Game Playing Machines, 1955) 

"This duality can be pursued further and is related to a duality between past and future and the notions of control and knowledge. Thus we may have knowledge of the past but cannot control it; we may control the future but have no knowledge of it." (Claude E Shannon, "Coding theorems for a discrete source with a fidelity criterion", IRE International Convention Records Vol. 7, 1959)

"It is very difficult to estimate how well a computer can be made to play with ideal programming. I tend to agree that it would be very difficult to reach the caliber of world champions or even most chess masters but I do not regard this as unthinkable. The machines do have certain very strong advantages of accuracy, speed, etc., and our present techniques of programming are bound to improve enormously in the future." (Claude E Shannon)

"It seems to be much easier to make two small jumps than the one big jump in any kind of mental thinking." (Claude E Shannon)

"Suppose that you are given a problem to solve, I don’t care what kind of a problem - a machine to design, or a physical theory to develop, or a mathematical theorem to prove, or something of that kind - probably a very powerful approach to this is to attempt to eliminate everything from the problem except the essentials; that is, cut it down to size." (Claude E Shannon)

"The chief weakness of the machine is that it will not learn by its mistakes. The only way to improve its play is by improving the program. Some thought has been given to designing a program that would develop its own improvements in strategy with increasing experience in play. Although it appears to be theoretically possible, the methods thought of so far do not seem to be very practical. One possibility is to devise a program that would change the terms and coefficients involved in the evaluation function on the basis of the results of games the machine had already played. Small variations might be introduced in these terms, and the values would be selected to give the greatest percentage of wins." (Claude E Shannon)

"The idea of a machine thinking is by no means repugnant to all of us. In fact, I find the converse idea, that the human brain may itself be a machine which could be possibly duplicated functionally with inanimate objects, quite attractive. Until clearly disproved, this hypothesis concerning the brain seems the natural scientific one in line with the principle of parsimony, etc., rather than hypothecating intangible and unreachable “vital forces,” “souls” and the like." (Claude E Shannon)

"The redundancy of a language is related to the existence of crossword puzzles. If the redundancy is zero any sequence of letters is a reasonable text in the language and any two dimensional array of letters forms a crossword puzzle. If the redundancy is too high the language imposes too many constraints for large crossword puzzles to be possible." (Claude E Shannon)

"There is a vast, explored sea of nature just waiting for things to be discovered in it, and the science and technology needed are progressing at an exponential rate. You see, it all feeds back into itself. Someone discovers a new principle or a new theory and it’s not only new knowledge but a new instrument for seeking more knowledge." (Claude E Shannon)

John M Ziman - Collected Quotes

"Many philosophers have now sadly come to the conclusion that there is no ultimate procedure which will wring the last drops of uncertainty from what scientists call their knowledge." (John M Ziman, "Public Knowledge: An Essay Concerning the Social Dimension of Science", 1968)

"Although the best and most famous scientific discoveries seem to open whole new windows of the mind, a typical scientific paper has never pretended to be more than another piece in a large jig-saw not significant in itself but as an element in a grander scheme. This technique, of soliciting many modest contributions to the vast store of human knowledge, has been the secret of western science since the seventeenth century, for it achieves a corporate collective power that is far greater than any one individual can exert. Primary scientific papers are not meant to be final statements of indisputable truths; each is merely a tiny tentative step forward, through the jungle of ignorance." (John M Zimer, Vol. 224, 1969)

"It is not enough to observe, experiment, theorize, calculate and communicate; we must also argue, criticize, debate, expound, summarize, and otherwise transform the information that we have obtained individually into reliable, well established, public knowledge." (John M Ziman, "Information, Communication, Knowledge", Nature Vol. 224 (5217), 1969)

"The sooner we all face up to the fact that theory and practice are indissoluble, and that there is no contradiction between the qualities of usefulness and beauty, the better." (John M Ziman, "Growth and Spread of Science", Nature Vol. 221 (5180), 1969)

"A significant fraction of the ordinary scientific literature in any field is concerned with essentially irrational theories put forward by a few well-established scholars who have lost touch with reality." (John M Ziman, "Some Pathologies of the Scientific Life", Nature Vol. 227, 1970)

"The communication of modern science to the ordinary citizen, necessary, important, desirable as it is, cannot be considered an easy task. The prime obstacle is lack of education. [...] There is also the difficulty of making scientific discoveries interesting and exciting without completely degrading them intellectually. [...] It is a weakness of modern science that the scientist shrinks from this sort of publicity, and thus gives an impression of arrogant mystagoguery." (John M Ziman,"The Force of Knowledge: The Scientific Dimension of Society", 1976)

"Physics defines itself as the science devoted to discovering, developing and refining those aspects of reality that are amenable to mathematical analysis." (John M Ziman, "Reliable Knowledge: An Exploration of the Grounds for Belief in Science", 1978)

"The most astonishing achievements of science, intellectually and practically, have been in physics, which many people take to be the ideal type of scientific knowledge. In fact, physics is a very special type of science, in which the subject matter is deliberately chosen so as to be amenable to quantitative analysis." (John M Ziman, "Reliable Knowledge: An Exploration of the Grounds for Belief in Science", 1978)

"'Disorder' is not mere chaos; it implies defective order." (John M Ziman, "Models of Disorder", 1979)

"A philosopher is a person who knows less and less about more and more, until he knows nothing about everything. […] A scientist is a person who knows more and more about less and less, until he knows everything about nothing." (John M Ziman, "Knowing Everything about Nothing: Specialization and Change in Scientific Careers", 1987)

"Any research organization requires generous measures of the following: (1) Social space for personal initiative and creativity; (2) Time for ideas to grow to maturity; (3) Openness to debate and criticism; (4) Hospitality towards novelty; and (5) Respect for specialized expertise." (John M Ziman, "Prometheus Bound", 1994)

"Theoretical physicists are like pure mathematicians, in that they are often interested in the hypothetical behaviour of entirely imaginary objects, such as parallel universes, or particles traveling faster than light, whose actual existence is not being seriously proposed at all." (John M Ziman, "Real Science: What it Is, and what it Means", 2000)

Information Overload III

"Every person seems to have a limited capacity to assimilate information, and if it is presented to him too rapidly and without adequate repetition, this capacity will be exceeded and communication will break down." (R Duncan Luce, "Developments in Mathematical Psychology", 1960)

"People today are in danger of drowning in information; but, because they have been taught that information is useful, they are more willing to drown than they need be. If they could handle information, they would not have to drown at all." (Idries Shah, "Reflections", 1968)

"Everyone spoke of an information overload, but what there was in fact was a non-information overload." (Richard S Wurman, "What-If, Could-Be", 1976)

"The greater the uncertainty, the greater the amount of decision making and information processing. It is hypothesized that organizations have limited capacities to process information and adopt different organizing modes to deal with task uncertainty. Therefore, variations in organizing modes are actually variations in the capacity of organizations to process information and make decisions about events which cannot be anticipated in advance." (John K Galbraith, "Organization Design", 1977)

"We are drowning in information but starved for knowledge." (John Naisbitt, "Megatrends: Ten New Directions Transforming Our Lives", 1982)

"In the Information Age, the first step to sanity is FILTERING. Filter the information: extract for knowledge. Filter first for substance. Filter second for significance. […] Filter third for reliability. […] Filter fourth for completeness." (Marc Stiegler, "David’s Sling", 1988)

"It has become evident time and again that when events become too complex and move too rapidly as appears to be the case today, human beings become demonstrably less able to cope." (Alan Greenspan, "The Structure of the International Financial System", 1998)

"Specialization, once a maneuver methodically to collect information, now is a manifestation of information overloads. The role of information has changed. Once justified as a means of comprehending the world, it now generates a conflicting and contradictory, fleeting and fragmentation field of disconnected and undigested data." (Stelarc, From Psycho-Body to Cyber-Systems: Images as Post-human Entities, 1998)

"Our needs going forward will be best served by how we make use of not just this data but all data. We live in an era of Big Data. The world has seen an explosion of information in the past decades, so much so that people and institutions now struggle to keep pace. In fact, one of the reasons for the attachment to the simplicity of our indicators may be an inverse reaction to the sheer and bewildering volume of information most of us are bombarded by on a daily basis. […] The lesson for a world of Big Data is that in an environment with excessive information, people may gravitate toward answers that simplify reality rather than embrace the sheer complexity of it." (Zachary Karabell, "The Leading Indicators: A short history of the numbers that rule our world", 2014)

"Today, technology has lowered the barrier for others to share their opinion about what we should be focusing on. It is not just information overload; it is opinion overload." (Greg McKeown, "Essentialism: The Disciplined Pursuit of Less", 2014)

🪷On Mind: Mental Models LXII

"A mental model is a collection of 'connected' autonomous objects. Running  a mental model corresponds to modifying the parameters of the model by propagating information using the internal rules and specified topology. Running a mental model can also occur when autonomous objects change state. For us the definition of state is distinct from the current parameter values of an object. A state change consists of the replacement of one set of behavior rules with another." (Michael D Williams et al, "Human Reasoning About a Simple Physical System", [in "Mental Models", Ed(s). Dedre Gentner & Albert L Stevens], 1983)

"Central to this conception of mental models is the notion of autonomous objects. An autonomous object is a mental object with an explicit representation of state, an explicit representation of its topological connections to other objects, and a set of internal parameters. Associated with each autonomous object is a set of rules which modify its parameters and thus specify its behavior." (Michael D Williams et al, "Human Reasoning About a Simple Physical System", [in "Mental Models", Ed(s). Dedre Gentner & Albert L Stevens], 1983)

"In the consideration of mental models we need really consider four different things: the target system, the conceputal model of that target system, the user’s mental model of the target system, and the scientist's conceptualization of that mental model. The system that the person is learning or using is, by definition, the target system. A conceptual model is invented to provide an appropriate representation of the target system, appropriate in the sense of being accurate, consistent, and complete." (Donald A Norman, "Some Observations on Mental Models" [in "Mental Models", Ed(s). Dedre Gentner & Albert L Stevens], 1983)

"The purpose of a mental model is to allow the person to understand and to anticipate the behavior of a physical system. This means that the model must have predictive power, either by applying rules of inference or by procedural derivation (in whatever manner these properties may be realized in a person); in other words, it should be possible for people to ' run' their models mentally. This means that the conceptual mental model must also include a model of the relevant human information processing and knowledge structures that make it possible for the person to use a mental model to predict and understand the physical system." (Donald A Norman, "Some Observations on Mental Models" [in "Mental Models"], Ed(s). Dedre Gentner & Albert L Stevens], 1983)

"From a functional point of view, mental models can be described as symbolic structures which permit people: to generate descriptions of the purpose of a system, to generate descriptions of the architecture of a system, to provide explanations of the state of a system, to provide explanations of the functioning of a system, to make predictions of future states of a system." (Gert Rickheit & Lorenz Sichelschmidt, "Mental Models: Some Answers, Some Questions, Some Suggestions", 1999)

"Under the label 'cognitive maps', mental models have been conceived of as the mental representation of spatial aspects of the environment. A mental model, in this sense, comprises the topology of an area, including relevant districts, landmarks, and paths. [...] Under the label 'naive physics', mental models have been conceived of as the mental representation of natural or technical systems. A mental model, in this sense, comprises the effective determinants, true or not, of the functioning of a physical system. [...] Under the label 'model based reasoning', the mental models notion is featured in yet another area of cognitive science - deductive reasoning. In contrast to the commonly held view that logical competence depends on formal rules of deduction, it has been argued that reasoning is a semantic process based on the manipulation of mental models. [...] Finally, under terms like 'discourse model', 'situation model', or 'scenario', mental models have been conceived of as the mental representation of a verbal description of some real or fictional state of affairs. The role of mental models in the comprehension of discourse is discussed in more detail below." (Gert Rickheit & Lorenz Sichelschmidt, "Mental Models: Some Answers, Some Questions, Some Suggestions", 1999)

"A mental model is an internal representation with analogical relations to its referential object, so that local and temporal aspects of the object are preserved. It comes somewhat close to the mental images people report having in their minds whilst processing information. The great advantage of the notion of mental models, however, is its ability to include the notion of a partner model and the notion of a situation model. Thus, mental models can build a bridge to the other two dimensions of communication, namely interaction and situation." (Gert Rickheit et al, "The concept of communicative competence" [in "Handbook of Communication Competence"], 2008)

"Because all mental models or mindsets are incomplete, we can engage in second-order studies, evaluations, judgments, and assessments about our own and other operative mental models. Of course this is highly complex since the act of reflection is itself a further of framing or reframing." (Patricia H Werhane et al, "Obstacles to Ethical: Decision-Making Mental Models, Milgram and the Problem of Obedience", 2013)

"Mental models bind our awareness within a particular scaffold and then selectively can filter the content we subsequently receive. Through recalibration using revised mental models, we argue, we cultivate strategies anew, creating new habits, and galvanizing more intentional and evolved mental models. This recalibration often entails developing a strong sense of self and self-worth, realizing that each of us has a range of moral choices that may deviate from those in authority, and moral imagination." (Patricia H Werhane et al, "Obstacles to Ethical: Decision-Making Mental Models, Milgram and the Problem of Obedience", 2013)

"These framing perspectives or mental models construe the data of our experiences, and it is the construed data that we call 'facts'. What we often call reality, or the world, is constructed or socially construed in certain ways such that one cannot get at the source of the data except through these construals." (Patricia H Werhane et al, "Obstacles to Ethical: Decision-Making Mental Models, Milgram and the Problem of Obedience", 2013)

Gert Rickheit - Collected Quotes

"According to the representational format claim, images constitute a class of mental models that is particularly suited to represent visually perceptible information. More specifically, an image can be conceived of as a viewer centered projection of an underlying mental model which, in turn, represents spatiotemporal aspects of external objects or events. Unlike propositional or network representations, images depict, rather than describe, a particular state of affairs." (Gert Rickheit & Lorenz Sichelschmidt, „Mental Models: Some Answers, Some Questions, Some Suggestions", 1999)

"From a functional point of view, mental models can be described as symbolic structures which permit people: to generate descriptions of the purpose of a system, to generate descriptions of the architecture of a system, to provide explanations of the state of a system, to provide explanations of the functioning of a system, to make predictions of future states of a system." (Gert Rickheit & Lorenz Sichelschmidt, "Mental Models: Some Answers, Some Questions, Some Suggestions", 1999)

"In broad terms, a mental model is to be understood as a dynamic symbolic representation of external objects or events on the par. t of some natural or artificial cognitive system. Mental models are thought to have certain properties which make them stand out against other forms of symbolic representations." (Gert Rickheit & Lorenz Sichelschmidt, "Mental Models: Some Answers, Some Questions, Some Suggestions", 1999)

"In order to be able to embrace a multitude of external situations, mental models must cope with a variety of representational formats - from quasi-verbal propositions to quasi-pictorial images."  (Gert Rickheit & Lorenz Sichelschmidt, "Mental Models: Some Answers, Some Questions, Some Suggestions", 1999)

"In the end, structural analogy may turn out to be the defining characteristic of mental models. Provided that the modeling function is specified with respect to the aspects figured and the aspects disregarded, and provided that there is sufficient circumstantial evidence for assuming a correspondence in structure between an external situation and its internal representation, regarding mental models as a unique form of symbolic representation may be justified." (Gert Rickheit & Lorenz Sichelschmidt, "Mental Models: Some Answers, Some Questions, Some Suggestions", 1999)

"The major problem of the mental model approach lies in the fact that the external world is to be represented in a highly specific way. Representing indeterminacy in terms of mental models thus poses difficulties, casting some doubt on the contention that mental models can do without variables." (Gert Rickheit & Lorenz Sichelschmidt, "Mental Models: Some Answers, Some Questions, Some Suggestions", 1999)

"Under the label 'cognitive maps', mental models have been conceived of as the mental representation of spatial aspects of the environment. A mental model, in this sense, comprises the topology of an area, including relevant districts, landmarks, and paths. [...] Under the label 'naive physics', mental models have been conceived of as the mental representation of natural or technical systems. A mental model, in this sense, comprises the effective determinants, true or not, of the functioning of a physical system. [...] Under the label 'model based reasoning', the mental models notion is featured in yet another area of cognitive science - deductive reasoning. In contrast to the commonly held view that logical competence depends on formal rules of deduction, it has been argued that reasoning is a semantic process based on the manipulation of mental models. [...] Finally, under terms like 'discourse model', 'situation model', or 'scenario', mental models have been conceived of as the mental representation of a verbal description of some real or fictional state of affairs. The role of mental models in the comprehension of discourse is discussed in more detail below." (Gert Rickheit & Lorenz Sichelschmidt, "Mental Models: Some Answers, Some Questions, Some Suggestions", 1999)

"A mental model is an internal representation with analogical relations to its referential object, so that local and temporal aspects of the object are preserved. It comes somewhat close to the mental images people report having in their minds whilst processing information. The great advantage of the notion of mental models, however, is its ability to include the notion of a partner model and the notion of a situation model. Thus, mental models can build a bridge to the other two dimensions of communication, namely interaction and situation." (Gert Rickheit et al, "The concept of communicative competence" [in "Handbook of Communication Competence"], 2008)

On Nonlinearity II

"Indeed, except for the very simplest physical systems, virtually everything and everybody in the world is caught up in a vast, nonlinear web of incentives and constraints and connections. The slightest change in one place causes tremors everywhere else. We can't help but disturb the universe, as T.S. Eliot almost said. The whole is almost always equal to a good deal more than the sum of its parts. And the mathematical expression of that property - to the extent that such systems can be described by mathematics at all - is a nonlinear equation: one whose graph is curvy." (M Mitchell Waldrop, "Complexity: The Emerging Science at the Edge of Order and Chaos", 1992)

"Today the network of relationships linking the human race to itself and to the rest of the biosphere is so complex that all aspects affect all others to an extraordinary degree. Someone should be studying the whole system, however crudely that has to be done, because no gluing together of partial studies of a complex nonlinear system can give a good idea of the behaviour of the whole." (Murray Gell-Mann, 1997)

"Much of the art of system dynamics modeling is discovering and representing the feedback processes, which, along with stock and flow structures, time delays, and nonlinearities, determine the dynamics of a system. […] the most complex behaviors usually arise from the interactions (feedbacks) among the components of the system, not from the complexity of the components themselves." (John D Sterman, "Business Dynamics: Systems thinking and modeling for a complex world", 2000)

"The mental models people use to guide their decisions are dynamically deficient. […] people generally adopt an event-based, open-loop view of causality, ignore feedback processes, fail to appreciate time delays between action and response and in the reporting of information, do not understand stocks and flows and are insensitive to nonlinearities that may alter the strengths of different feedback loops as a system evolves." (John D Sterman, "Business Dynamics: Systems thinking and modeling for a complex world", 2000)

"Most physical processes in the real world are nonlinear. It is our abstraction of the real world that leads us to the use of linear systems in modeling these processes. These linear systems are simple, understandable, and, in many situations, provide acceptable simulations of the actual processes. Unfortunately, only the simplest of linear processes and only a very small fraction of the nonlinear having verifiable solutions can be modeled with linear systems theory. The bulk of the physical processes that we must address are, unfortunately, too complex to reduce to algorithmic form - linear or nonlinear. Most observable processes have only a small amount of information available with which to develop an algorithmic understanding. The vast majority of information that we have on most processes tends to be nonnumeric and nonalgorithmic. Most of the information is fuzzy and linguistic in form." (Timothy J Ross & W Jerry Parkinson, "Fuzzy Set Theory, Fuzzy Logic, and Fuzzy Systems", 2002)

"Swarm intelligence can be effective when applied to highly complicated problems with many nonlinear factors, although it is often less effective than the genetic algorithm approach [...]. Swarm intelligence is related to swarm optimization […]. As with swarm intelligence, there is some evidence that at least some of the time swarm optimization can produce solutions that are more robust than genetic algorithms. Robustness here is defined as a solution’s resistance to performance degradation when the underlying variables are changed. (Michael J North & Charles M Macal, Managing Business Complexity: Discovering Strategic Solutions with Agent-Based Modeling and Simulation, 2007)

"[…] our mental models fail to take into account the complications of the real world - at least those ways that one can see from a systems perspective. It is a warning list. Here is where hidden snags lie. You can’t navigate well in an interconnected, feedback-dominated world unless you take your eyes off short-term events and look for long-term behavior and structure; unless you are aware of false boundaries and bounded rationality; unless you take into account limiting factors, nonlinearities and delays. You are likely to mistreat, misdesign, or misread systems if you don’t respect their properties of resilience, self-organization, and hierarchy." (Donella H Meadows, "Thinking in Systems: A Primer", 2008)

"You can’t navigate well in an interconnected, feedback-dominated world unless you take your eyes off short-term events and look for long term behavior and structure; unless you are aware of false boundaries and bounded rationality; unless you take into account limiting factors, nonlinearities and delays." (Donella H Meadow, "Thinking in Systems: A Primer", 2008)

"A network of many simple processors ('units' or 'neurons') that imitates a biological neural network. The units are connected by unidirectional communication channels, which carry numeric data. Neural networks can be trained to find nonlinear relationships in data, and are used in various applications such as robotics, speech recognition, signal processing, medical diagnosis, or power systems." (Adnan Khashman et al, "Voltage Instability Detection Using Neural Networks", 2009)

"Linearity is a reductionist’s dream, and nonlinearity can sometimes be a reductionist’s nightmare. Understanding the distinction between linearity and nonlinearity is very important and worthwhile." (Melanie Mitchell, "Complexity: A Guided Tour", 2009)

On Information Theory I

"[…] information theory is characterised essentially by its dealing always with a set of possibilities; both its primary data and its final statements are almost always about the set as such, and not about some individual element in the set." (W Ross Ashby, "An Introduction to Cybernetics", 1956)

"Just as in applied statistics the crux of a problem is often the devising of some method of sampling that avoids bias, our problem is that of finding a probability assignment which avoids bias, while agreeing with whatever information is given. The great advance provided by information theory lies in the discovery that there is a unique, unambiguous criterion for the 'amount of uncertainty' represented by a discrete probability distribution, which agrees with our intuitive notions that a broad distribution represents more uncertainty than does a sharply peaked one, and satisfies all other conditions which make it reasonable." (Edwin T Jaynes, "Information Theory and Statistical Mechanics" I, 1956

"Cybernetics is concerned primarily with the construction of theories and models in science, without making a hard and fast distinction between the physical and the biological sciences. The theories and models occur both in symbols and in hardware, and by 'hardware’ we shall mean a machine or computer built in terms of physical or chemical, or indeed any handleable parts. Most usually we shall think of hardware as meaning electronic parts such as valves and relays. Cybernetics insists, also, on a further and rather special condition that distinguishes it from ordinary scientific theorizing: it demands a certain standard of effectiveness. In this respect it has acquired some of the same motive power that has driven research on modern logic, and this is especially true in the construction and application of artificial languages and the use of operational definitions. Always the search is for precision and effectiveness, and we must now discuss the question of effectiveness in some detail. It should be noted that when we talk in these terms we are giving pride of place to the theory of automata at the expense, at least to some extent, of feedback and information theory." (Frank H George, "The Brain As A Computer", 1962)

"Now we are looking for another basic outlook on the world - the world as organization. Such a conception - if it can be substantiated - would indeed change the basic categories upon which scientific thought rests, and profoundly influence practical attitudes. This trend is marked by the emergence of a bundle of new disciplines such as cybernetics, information theory, general system theory, theories of games, of decisions, of queuing and others; in practical applications, systems analysis, systems engineering, operations research, etc. They are different in basic assumptions, mathematical techniques and aims, and they are often unsatisfactory and sometimes contradictory. They agree, however, in being concerned, in one way or another, with ‘systems’, ‘wholes’ or ‘organizations’; and in their totality, they herald a new approach." (Ludwig von Bertalanffy, "General System Theory", 1968)

"The general notion in communication theory is that of information. In many cases, the flow of information corresponds to a flow of energy, e. g. if light waves emitted by some objects reach the eye or a photoelectric cell, elicit some reaction of the organism or some machinery, and thus convey information." (Ludwig von Bertalanffy, "General System Theory", 1968) 

"My analysis of living systems uses concepts of thermodynamics, information theory, cybernetics, and systems engineering, as well as the classical concepts appropriate to each level. The purpose is to produce a description of living structure and process in terms of input and output, flows through systems, steady states, and feedbacks, which will clarify and unify the facts of life." (James G Miller, "Living Systems: Basic Concepts", 1969)

"The 'flow of information' through human communication channels is enormous. So far no theory exists, to our knowledge, which attributes any sort of unambiguous measure to this 'flow'." (Anatol Rapoport, "Modern Systems Research for the Behavioral Scientist", 1969)

"Probability plays a central role in many fields, from quantum mechanics to information theory, and even older fields use probability now that the presence of "noise" is officially admitted. The newer aspects of many fields start with the admission of uncertainty." (Richard Hamming, "Methods of Mathematics Applied to Calculus, Probability, and Statistics", 1985)

"The field of 'information theory' began by using the old hardware paradigm of transportation of data from point to point." (Marshall McLuhan & Eric McLuhan, Laws of Media: The New Science, 1988)

"Without an understanding of causality there can be no theory of communication. What passes as information theory today is not communication at all, but merely transportation." (Marshall McLuhan & Eric McLuhan, "Laws of Media: The New Science", 1988)

"The cybernetics phase of cognitive science produced an amazing array of concrete results, in addition to its long-term (often underground) influence: the use of mathematical logic to understand the operation of the nervous system; the invention of information processing machines (as digital computers), thus laying the basis for artificial intelligence; the establishment of the metadiscipline of system theory, which has had an imprint in many branches of science, such as engineering (systems analysis, control theory), biology (regulatory physiology, ecology), social sciences (family therapy, structural anthropology, management, urban studies), and economics (game theory); information theory as a statistical theory of signal and communication channels; the first examples of self-organizing systems. This list is impressive: we tend to consider many of these notions and tools an integrative part of our life […]" (Francisco J Varela, "The Embodied Mind", 1991)

"[...] the mean information of a message is defined as the amount of chance (or randomness) present in a set of possible messages. To see that this is a natural definition, note that by choosing a message, one destroys the randomness present in the variety of possible messages. Information theory is thus concerned, as is statistical mechanics, with measuring amounts of randomness. The two theories are therefore closely related." (David Ruelle, "Chance and Chaos", 1991)

On Symbols (1920-1929)

"Language is a purely human and non-instinctive method of communicating ideas, emotions and desires by means of a system of voluntarily produced symbols." (Edward Sapir, "Language", 1921) 

"The essence of language lies, not in the use of this or that special means of communication, but in the employment of fixed associations (however these may have originated) in order that something now sensible - a spoken word, a picture, a gesture, or what not — may call up the 'idea' of something else. Whenever this is done, what is now sensible may be called a 'sign' or 'symbol', and that of which it is intended to call up the 'idea' may be called its 'meaning'." This is a rough outline of what constitutes 'meaning'." (Bertrand Russell, "Analysis of Mind", 1921)

"Mere deductive logic, whether you clothe it in mathematical symbols and phraseology or whether you enlarge its scope into a more general symbolic technique, can never take the place of clear relevant initial concepts of the meaning of your symbols, and among symbols I include words. If you are dealing with nature, your meanings must directly relate to the immediate facts of observation. We have to analyse first the most general characteristics of things observed, and then the more casual contingent occurrences. There can be no true physical science which looks first to mathematics for the provision of a conceptual model. Such a procedure is to repeat the errors of the logicians of the middle-ages." (Alfred N Whitehead, "Principle of Relativity", 1922)

"The fundamental concepts of each science, the instruments with which it pro pounds its questions and formulates its solutions, are regarded no longer as passive images of something but as symbols created by the intellect itself." (Ernst Cassirer, "The Philosophy of Symbolic Forms", 1923)

"The logic of things, i.e., of the material concepts and relations on which the structure of a science rests, cannot be separated by the logic of signs. For the sign is no mere accidental cloak of the idea, but its necessary and essential organ. It serves not merely to communicate a complete and given thought content, but is an instrument, by means of which this content develops and fully defines itself. […] Consequently, all truly strict and exact thought is sustained by the symbolic and semiotics on which it is based." (Ernst Cassirer, "The Philosophy of Symbolic Forms", 1923)

"These symbols are so constituted that the necessary logical consequences of the image are always images of the necessary natural consequences of the imagined objects." (Ernst Cassirer, "The Philosophy of Symbolic Forms", 1923)

"All traditional logic habitually assumes that precise symbols are being employed. It is therefore not applicable to this terrestrial life but only to an imagined celestial existence." (Bertrand Russell, 1923)

"A poem therefore is to be defined as a structure of words whose sound constitutes a rhythmical unity, complete in itself, irrefragable, unanalyzable, completing its symbolic references within the ambit of its sound effects." (Herbert Read, "What is a Poem", 1926) 

"Once a statement is cast into mathematical form it may be manipulated in accordance with [mathematical] rules and every configuration of the symbols will represent facts in harmony with and dependent on those contained in the original statement. Now this comes very close to what we conceive the action of the brain structures to be in performing intellectual acts with the symbols of ordinary language. In a sense, therefore, the mathematician has been able to perfect a device through which a part of the labor of logical thought is carried on outside the central nervous system with only that supervision which is requisite to manipulate the symbols in accordance with the rules." (Horatio B Williams, "Mathematics and the Biological Sciences", Bulletin of the American Mathematical Society Vol. 38, 1927)

"The self-organisation of society depends on commonly diffused symbols evoking commonly diffused ideas, and at the same time indicating commonly understood action." (Alfred N Whitehead, "Symbolism: Its Meaning and Effect", 1927)

"Science aims at constructing a world which shall be symbolic of the world of commonplace experience." (Sir Arthur S Eddington, "The Nature of the Physical World", 1928)

"If to-day you ask a physicist what he has finally made out the æther or the electron to be, the answer will not be a description in terms of billiard balls or fly-wheels or anything concrete; he will point instead to a number of symbols and a set of mathematical equations which they satisfy. What do the symbols stand for? The mysterious reply is given that physics is indifferent to that; it has no means of probing beneath the symbolism. To understand the phenomena of the physical world it is necessary to know the equations which the symbols obey but not the nature of that which is being symbolised [...]" (Arthur S Eddington, "Science and the Unseen World", 1929)

"Natural law is not applicable to the unseen world behind the symbols, because it is unadapted to anything except symbols, and its perfection is a perfection of symbolic linkage. You cannot apply such a scheme to the parts of our personality which are not measurable by symbols any more than you can extract the square root of a sonnet." (Arthur S Eddington, "Science and the Unseen World", 1929)

"Our environment may and should mean something towards us which is not to be measured with the tools of the physicist or described by the metrical symbols of the mathematician." (Arthur S Eddington, "Science and the Unseen World", 1929)

"The exploration of the external world by the methods of physical science leads not to a concrete reality but to a shadow world of symbols, beneath which those methods are unadapted for penetrating." (Arthur S Eddington, "Science and the Unseen World", 1929)

"Language is the communicative process par excellence in every known society, and it is exceedingly important to observe that whatever may be the shortcomings of a primitive society judged from the vantage point of civilization its language inevitably forms as sure, complete and potentially creative an apparatus of referential symbolism as the most sophisticated language that we know of." (Edward Sapir, "Communication", 1931)

On Symbols (2000-2009)

"Precision is greatly enhanced by the human capacity to symbolize. Symbols can be devised to stand for mathematical ideas, entities, operations, and relations. Symbols also permit precise and repeatable calculation." (George Lakoff & Rafael E Nuñez, "Where Mathematics Come From: How the Embodied Mind Brings Mathematics into Being, 2000)

"The motion of the mind is conveyed along a cloud of meaning. There is this paradox that we get to meaning only when we strip the meaning from symbols." (David Berlinski, "The Advent of the Algorithm: The Idea that Rules the World", 2000)

"A symbol is a mental representation regarding the internal reality referring to its object by a convention and produced by the conscious interpretation of a sign. In contrast to signals, symbols may be used every time if the receiver has the corresponding representation. Symbols also relate to feelings and thus give access not only to information but also to the communicator’s motivational and emotional state. The use of symbols makes it possible for the organism using it to evoke in the receiver the same response it evokes in himself. To communicate with symbols is to use a language." (Lars Skyttner, "General Systems Theory: Ideas and Applications", 2001)

"In the definition of meaning, it is assumed that both the source and receiver have previously coded (and stored) signals of the same or similar referents, such that the messages may have meaning and relate to behaviour. That is, the used symbols must have the same signification for both sender and receiver. If not, the receiver will create a different mental picture than intended by the transmitter. Meaning is generated by individuals in a process of social interaction with a more or less common environment. It is a relation subsisting within a field of experience and appears as an emergent property of a symbolic representation when used in culturally accepted interaction. The relation between the symbolic representation and its meaning is random. Of this, however, the mathematical theory has nothing to say. If human links in the chain of communication are missing, of course no questions of meaning will arise." (Lars Skyttner, "General Systems Theory: Ideas and Applications", 2001)

"A person thinking in the nonverbal mode is actually thinking with the meaning of the language in the form of mental pictures of the concepts and ideas it contains. Nonverbal thought doesn't require literacy. An illiterate person can communicate without knowing what the symbols look like. [...] Literacy, then, is established as the person learns how the symbols look and becomes able to recognize them as representing certain things or concepts." (Ronald D Davis & Eldon M Braun, "The Gift of Learning", 2003)

"Science does not speak of the world in the language of words alone, and in many cases it simply cannot do so. The natural language of science is a synergistic integration of words, diagrams, pictures, graphs, maps, equations, tables, charts, and other forms of visual and mathematical expression. […] [Science thus consists of] the languages of visual representation, the languages of mathematical symbolism, and the languages of experimental operations." (Jay Lemke, "Teaching all the languages of science: Words, symbols, images and actions", 2003)

"I often told the fanatics of realism that there is no such thing as realism in art: it only exists in the mind of the observer. Art is a symbol, a thing conjuring up reality in our mental image. That is why I don't see any contradiction between abstract and figurative art either." (Antoni Tàpies, "Tàpies, Werke auf Papier 1943 – 2003", 2004)

"A symbol is an object, act, or event that conveys meaning to others. Symbols can be considered a rich, non-verbal language that vibrantly conveys the organization’s important values concerning how people relate to one another and interact with the environment" (Richard L Daft & Dorothy Marcic, "Understanding Management" 5th Ed., 2006)

"But because of the way in which depictions represent, there is a correspondence between parts and spatial relations of the representation and those of the object; this structural mapping, which confers a type of resemblance, underlies the way images convey specific content. In this respect images are like pictures. Unlike words and symbols, depictions are not arbitrarily paired with what they represent." (Stephen Kosslyn et al," The Case for Mental Imagery", 2006)

"Imagination has the creative task of making symbols, joining things together in such a way that they throw new light on each other and on everything around them. The imagination is a discovering faculty, a faculty for seeing relationships, for seeing meanings that are special and even quite new." (Thomas Merton, "Angelic Mistakes: The Art of Thomas Merton", 2006)

"[...] the scientific models of concrete things are symbolic rather than iconic: they are systems of propositions, not pictures. Besides, such models are seldom if ever completely accurate, if only because they involve more or less brutal simplifications, such as pretending that a metallic surface is smooth, a crystal has no impurities, a biopopulation has a single predator, or a market is in equilibrium.  These are all fictions. However, they are stylizations rather than wild fantasies. Hence, introducing and using them to account for real existents does not commit us to fictionism, just as defending the role of experience need not make us empiricists, nor is admitting the role of intuition enough to qualify as intuitionist." (Mario Bunge, "Chasing Reality: Strife over Realism", 2006)

"But notice, a subatomic particle is itself a holon [hole/parts]. And so is a cell. And so is a symbol, and an image, and a concept. What all of those entities are, before they are anything else, is a holon. So the world is not composed of atoms or symbols or cells or concepts. It is composed of holons." (Ken Wilber, "A Brief History of Everything", 2007)

"Language use is a curious behavior, but once the transition to language is made, literature is a likely consequence, since it is linked to the dynamic of the linguistic symbol through the functioning of the imagination." (Russell Berman, "Fiction Sets You Free: Literature, Liberty and Western Culture", 2007)

"Images and pictures […] have played a key role in shaping our scientific picture of the world. […] Carefully constructed families of pictures can act as a calculus all their own. Like any successful systems of symbols, with an appropriate grammar they enlarge the number of things that we can do without consciously thinking." (John D Barrow, "Cosmic Imagery: Key Images in the History of Science", 2008)

"How are we to explain the contrast between the matter-of-fact way in which √-1 and other imaginary numbers are accepted today and the great difficulty they posed for learned mathematicians when they first appeared on the scene? One possibility is that mathematical intuitions have evolved over the centuries and people are generally more willing to see mathematics as a matter of manipulating symbols according to rules and are less insistent on interpreting all symbols as representative of one or another aspect of physical reality. Another, less self-congratulatory possibility is that most of us are content to follow the computational rules we are taught and do not give a lot of thought to rationales." (Raymond S Nickerson, "Mathematical Reasoning: Patterns, Problems, Conjectures, and Proofs", 2009)

"Mathematicians are sometimes described as living in an ideal world of beauty and harmony. Instead, our world is torn apart by inconsistencies, plagued by non sequiturs and, worst of all, made desolate and empty by missing links between words, and between symbols and their referents; we spend our lives patching and repairing it. Only when the last crack disappears are we rewarded by brief moments of harmony and joy." (Alexandre V Borovik, "Mathematics under the Microscope: Notes on Cognitive Aspects of Mathematical Practice", 2009)

"Mathematical ideas like number can only be really 'seen' with the 'eyes of the mind' because that is how one 'sees' ideas. Think of a sheet of music which is important and useful but it is nowhere near as interesting, beautiful or powerful as the music it represents. One can appreciate music without reading the sheet of music. Similarly, mathematical notation and symbols on a blackboard are just like the sheet of music; they are important and useful but they are nowhere near as interesting, beautiful or powerful as the actual mathematics (ideas) they represent." (Fiacre O Cairbre, "The Importance of Being Beautiful in Mathematics", IMTA Newsletter 109, 2009)