On Information Theory II

"If quantum communication and quantum computation are to flourish, a new information theory will have to be developed." (Hans Christian von Baeyer, "Information, The New Language of Science", 2003)

"In fact, an information theory that leaves out the issue of noise turns out to have no content." (Hans Christian von Baeyer, "Information, The New Language of Science", 2003)

"In an information economy, entrepreneurs master the science of information in order to overcome the laws of the purely physical sciences. They can succeed because of the surprising power of the laws of information, which are conducive to human creativity. The central concept of information theory is a measure of freedom of choice. The principle of matter, on the other hand, is not liberty but limitation - it has weight and occupies space." (George Gilder, "Knowledge and Power: The Information Theory of Capitalism and How it is Revolutionizing our World", 2013)

"Information theory leads to the quantification of the information content of the source, as denoted by entropy, the characterization of the information-bearing capacity of the communication channel, as related to its noise characteristics, and consequently the establishment of the relationship between the information content of the source and the capacity of the channel. In short, information theory provides a quantitative measure of the information contained in message signals and help determine the capacity of a communication system to transfer this information from source to sink over a noisy channel in a reliable fashion." (Ali Grami, "Information Theory", 2016)

"The mathematical theory of information [...] is solely concerned with the ‘surprise’ aspect of information. There are two reasons for this. Firstly, information theory was originally developed within the context of communication engineering, where it was only the surprise factor that was of relevance. Secondly, ‘meaning’ has so far proved too difficult a concept to develop mathematically. The consequence of this is that we should be aware that ‘information’ [...] has the restricted technical meaning of ‘measure of surprise’." (David Applebaum," Probability and Information: An Integrated Approach", 2008)

"Cognitive psychology has followed a different direction, considering intelligence as a set of mental representations and a series of processes that operate on these representations that allows the individual to adapt to the changing conditions of the environment. This type of approach is connected with information theory. The intelligent mind operates by processing information that it collects from the environment, and the better and faster this information is processed, the more intelligence is demonstrated." (Diego Rasskin-Gutman, "Chess Metaphors: Artificial Intelligence and the Human Mind", 2009)

"Information Theory is a mathematical treatment of what is left after the meanings have been removed from a Communication." (John Gall, "The Systems Bible: The Beginner's Guide to Systems Large and Small"[Systematics 3rd Ed.], 2011)

"In an information economy, entrepreneurs master the science of information in order to overcome the laws of the purely physical sciences. They can succeed because of the surprising power of the laws of information, which are conducive to human creativity. The central concept of information theory is a measure of freedom of choice. The principle of matter, on the other hand, is not liberty but limitation - it has weight and occupies space." (George Gilder, "Knowledge and Power: The Information Theory of Capitalism and How it is Revolutionizing our World", 2013)

"[…] the term 'information', as used in information theory, has nothing to do with meaning. It is a measure of the order, or nonrandomness, of a signal; and the main concern of information theory is the problem of how to get a message, coded as a signal, through a noisy channel." (Fritjof Capra, "The Systems View of Life: A Unifying Vision", 2014)

"Information theory leads to the quantification of the information content of the source, as denoted by entropy, the characterization of the information-bearing capacity of the communication channel, as related to its noise characteristics, and consequently the establishment of the relationship between the information content of the source and the capacity of the channel. In short, information theory provides a quantitative measure of the information contained in message signals and help determine the capacity of a communication system to transfer this information from source to sink over a noisy channel in a reliable fashion." (Ali Grami, "Information Theory", 2016)

"In information theory this notion, introduced by Claude Shannon, is used to express unpredictability of information content. For instance, if a data set containing n items was divided into k groups each comprising n i items, then the entropy of such a partition is H = p 1 log( p 1 ) + … + p k log( p k ), where p i = n i / n . In case of two alternative partitions, the mutual information is a measure of the mutual dependence between these partitions." (Slawomir T Wierzchon, "Ensemble Clustering Data Mining and Databases", 2018) 

On Probability Theory (-1899)

"To understand the theory of chance thoroughly, requires a great knowledge of numbers, and a pretty competent one of Algebra." (John Arbuthnot, "An Essay on the Usefulness of Mathematical Learning", 1700)

"The theory of probabilities is at bottom nothing but common sense reduced to calculus; it enables us to appreciate with exactness that which accurate minds feel with a sort of instinct for which of times they are unable to account." (Pierre-Simon Laplace, "Analytical Theory of Probability, 1812)

“One may even say, strictly speaking, that almost all our knowledge is only probable; and in the small number of things that we are able to know with certainty, in the mathematical sciences themselves, the principal means of arriving at the truth - induction and analogy - are based on probabilities, so that the whole system of human knowledge is tied up with the theory set out in this essay.” (Pierre-Simon Laplace, “Philosophical Essay on Probabilities”, 1814)

"Probability has reference partly to our ignorance, partly to our knowledge [..] The theory of chance consists in reducing all the events of the same kind to a certain number of cases equally possible, that is to say, to such as we may be equally undecided about in regard to their existence, and in determining the number of cases favorable to the event whose probability is sought. The ratio of this number to that of all cases possible is the measure of this probability, which is thus simply a fraction whose number is the number of favorable cases and whose denominator is the number of all cases possible." (Pierre-Simon Laplace, "Philosophical Essay on Probabilities", 1814)

“[…] probability, in its mathematical acceptation, has reference to the state of our knowledge of the circumstances under which an event may happen or fail. With the degree of information which we possess concerning the circumstances of an event, the reason we have to think that it will occur, or, to use a single term, our expectation of it, will vary. Probability is expectation founded upon partial knowledge. A perfect acquaintance with all the circumstances affecting the occurrence of an event would change expectation into certainty, and leave neither room nor demand for a theory of probabilities.” (George Boole, “The Laws of Thought”, 1854)

"There is no more remarkable feature in the mathematical theory of probability than the manner in which it has been found to harmonize with, and justify, the conclusions to which mankind have been led, not by reasoning, but by instinct and experience, both of the individual and of the race. At the same time it has corrected, extended, and invested them with a definiteness and precision of which these crude, though sound, appreciations of common sense were till then devoid." (Morgan W Crofton, "Probability", Encyclopaedia Britannica 9th Ed,, 1885)

"I am convinced that it is impossible to expound the methods of induction in a sound manner, without resting them on the theory of probability. Perfect knowledge alone can give certainty, and in nature perfect knowledge would be infinite knowledge, which is clearly beyond our capacities. We have, therefore, to content ourselves with partial knowledge, - knowledge mingled with ignorance, producing doubt." (William S Jevons, "The Principles of Science: A Treatise on Logic and Scientific Method", 1887)

"In applying dynamical principles to the motion of immense numbers of atoms, the limitation of our faculties forces us to abandon the attempt to express the exact history of each atom, and to be content with estimating the average condition of a group of atoms large enough to be visible. This method [...] which I may call the statistical method, and which in the present state of our knowledge is the only available method of studying the properties of real bodies, involves an abandonment of strict dynamical principles, and an adoption of the mathematical methods belonging to the theory of probability. […] If the actual history of Science had been different, and if the scientific doctrines most familiar to us had been those which must be expressed in this way, it is possible that we might have considered the existence of a certain kind of contingency a self evident truth, and treated the doctrine of philosophical necessity as a mere sophism." (James C Maxwell, "Introductory Lecture on Experimental Physics", "The Scientific Papers of James Clerk Maxwell", 1890)

On Geometry (Unsourced)

"A child[’s …] first geometrical discoveries are topological [...] If you ask him to copy a square or a triangle, he draws a closed circle." (Jean Piaget)

"A geometrical theory in physical interpretation can never be validated with mathematical certainty […] like any other theory of empirical science, it can acquire only a more or less high degree of confirmation." (Carl G Hempel)

"An essential defect of previous presentations of geometry is that one usually returns to discrete numerical ratios in the treatment of similarity theory. This procedure, which at first seems simple, soon enough becomes entangled in complicated investigations concerning incommensurable magnitudes, as we have already hinted above; and the initial impression of simplicity is revenged upon problems of a purely geometrical procedure by the appearance of a set of difficult investigations of a completely heterogeneous type, which shed no light on the essence of spatial magnitudes. To be sure, one cannot eliminate the problem of measuring spatial magnitudes and expressing the results of these measurements numerically. But this problem cannot originate in geometry itself, but only arises when one, equipped on the one hand with the concept of number and on the other with spatial perceptions, applies them to that problem, and thus in a mixed branch that one can in a general sense call by the name 'theory of measurement' […] To relegate the theory of similarity, and even that of surface area, to this branch as has previously occurred" (not to the form but to the substance) is to steal the essential content from what is called" (pure) geometry." (Hermann G Grassmann)

"Architecture is the triumph of human imagination over materials, methods, and men, to put man into possession of his own Earth. It is at least the geometric pattern of things, of life, of the human and social world. It is at best that magic framework of reality that we sometimes touch upon when we use the word order." (Frank L Wright)

"Born at the same time with the Greek art, the mathematics kept in its canvas, in its intimate structure, a certain affinity with art. It comes to the same harmony in Euclid’s geometry as in the ancient temples. It is the same silence, the same balance in demonstrating a theorem as in the admirable columns of the Acropolis." (Gheorghe Ţiţeica)

"Geometrical truths are in a way asymptotes to physical truths, that is to say, the latter approach the former indefinitely near without ever reaching them exactly." (Jean le Rond D’Alembert)

"Geometry is the science which restores the situation that existed before the creation of the world and tries to fill the 'gap', relinquishing the help of matter." (Lucian Blaga)

"[It used to be that] geometry must, like logic, rely on formal reasoning in order to rebut the quibblers. But the tables have turned. All reasoning concerned with what common sense knows in advance, serves only to conceal the truth and to weary the reader and is today disregarded." (Alexis C Clairaut)

"It is very helpful to represent these things in this fashion since nothing enters the mind more readily than geometric figures." (René Descartes)

"More evidently still astronomy attains through arithmetic the investigations that pertain to it, not alone because it is later than geometry in origin - for motion naturally comes after rest - nor because the motions of the stars have a perfectly melodious harmony, but also because risings, settings, progressions, retrogressions, increases, and all sorts of phases are governed by numerical cycles and quantities. So then we have rightly undertaken first the systematic treatment of this, as the science naturally prior, more honorable, and more venerable, and, as it were, mother and nurse of the rest; and here we will take our start for the sake of clearness." (Nicomachus of Gerasa)

"Music is the arithmetic of sounds as optics is the geometry of light." (Claude Debussy)

"My aim is to show that the heavenly machine is not a kind of divine, live being, but a kind of clockwork, insofar as nearly all the manifold motions are caused by a most simple, magnetic, and material force, just as all motions of the clock are caused by a simple weight. And I also show how these physical causes are to be given numerical and geometrical expression." (Johannes Kepler)

"Not that the propositions of geometry are only approximately true, but that they remain absolutely true in regard to that Euclidean space which has been so long regarded as being the physical space of our experience." (Arthur Cayley)

"Poetry is as exact a science as geometry." (Gustave Flaubert)

"Projective geometry has opened up for us with the greatest facility new territories in our science, and has rightly been called the royal road to our particular field of knowledge." (Felix Klein)

"Show all these fanatics a little geometry, and they learn it quite easily. But, strangely enough, their minds are not thereby rectified. They perceive the truths of geometry, but it does not teach them to weigh probabilities. Their minds have set hard. They will reason in a topsy-turvy wall all their lives, and I am sorry for it." (Voltaire)

"The attempt to apply rational arithmetic to a problem in geometry resulted in the first crisis in the history of mathematics. The two relatively simple problems - the determination of the diagonal of a square and that of the circumference of a circle - revealed the existence of new mathematical beings for which no place could be found within the rational domain." (Tobias Dantzig)

"The axioms of geometry are - according to my way of thinking - not arbitrary, but sensible. statements, which are, in general, induced by space perception and are determined as to their precise content by expediency." (Felix Klein)

"[…] the feeling of mathematical beauty, of the harmony of numbers and of forms, of geometric elegance. It is a genuinely esthetic feeling, which all mathematicians know. And this is sensitivity." (Henri Poincaré)

"The geometry of Algebraic Topology is so pretty, it would seem a pity to slight it and miss all the intuition which it provides. At deeper levels, algebra becomes increasingly important, so for the sake of balance it seems only fair to emphasize geometry at the beginning." (Allen Hatcher) 

"The knowledge of which geometry aims is the knowledge of the eternal." (Plato)

"The theory of ramification is one of pure colligation, for it takes no account of magnitude or position; geometrical lines are used, but these have no more real bearing on the matter than those employed in genealogical tables have in explaining the laws of procreation." (James J Sylvester)

"The theory of ramification is one of pure colligation, for it takes no account of magnitude or position; geometrical lines are used, but these have no more real bearing on the matter than those employed in genealogical tables have in explaining the laws of procreation." (J J Sylvester)

"Therefore one has taken everywhere the opposite road, and each time one encounters manifolds of several dimensions in geometry, as in the doctrine of definite integrals in the theory of imaginary quantities, one takes spatial intuition as an aid. It is well known how one gets thus a real overview over the subject and how only thus are precisely the essential points emphasized." (Bernhard Riemann)

Geometrical truths are in a way asymptotes to physical truths, that is to say, the latter approach the former indefinitely near without ever reaching them exactly." (Jean le Rond d’Alembert)"

"To follow a geometrical argument purely logically without having the figure on which the argument bears constantly before me is for me impossible." (Felix Klein)

"Topology is an elastic version of geometry that retains the idea of continuity but relaxes rigid metric notions of distance." (Samuel Eilenberg)

"Two other sciences in the same way will accurately treat of 'size': geometry, the part that abides and is at rest, [and] astronomy, that which moves and revolves." (Nicomachus of Gerasa)

"We admit, in geometry, not only infinite magnitudes, that is to say, magnitudes greater than any assignable magnitude, but infinite magnitudes infinitely greater, the one than the other. This astonishes our dimension of brains, which is only about six inches long, five broad, and six in depth, in the largest heads." (Voltaire)

"We ought either to exclude all lines, beside the circle and right line, out of geometry, or admit them according to the simplicity of their descriptions, in which case the Conchoid yields to none except the circle. That is arithmetically more simple which is determined by the more simple equations, but that is geometrically more simple which is determined by the more simple drawing of lines." (Sir Isaac Newton)

"With the exception of the geometric series, there does not exist in all of mathematics a single infinite series whose sum has been determined rigorously." (Niels H Abel)

On Geometry (1600-1699)

"Where there is matter, there is geometry." (Johannes Kepler, "Concerning the More Certain Fundamentals of Astrology", 1601)

"Geometry is one and eternal shining in the mind of God." (Johannes Kepler, "Conversation with the Sidereal Messenger" [an open letter to Galileo Galilei] 1610)

"Thus the analysis of angular sections involves geometric and arithmetic secrets which hitherto have been penetrated by no one." (François Viète, cca 1615)

"Mathematic is either Pure or Mixed: To Pure Mathematic belong those sciences which handle Quantity entirely severed from matter and from axioms of natural philosophy. These are two, Geometry and Arithmetic; the one handling quantity continued, the other dissevered. [...] Mixed Mathematic has for its subject some axioms and parts of natural philosophy, and considers quantity in so far as it assists to explain, demonstrate and actuate these." (Francis Bacon, "De Augmentis", 1623)

"Philosophy is written in this grand book, the universe, which stands continually open to our gaze. But the book cannot be understood unless one first learns to comprehend the language and read the letters in which it is composed. It is written in the language of mathematics, and its characters are triangles, circles, and other geometric figures without which it is humanly impossible to understand a single word of it; without these, one wanders about in a dark labyrinth." (Galileo Galilei, “The Assayer”, 1623)

"And having thus passed the principles of arithmetic, geometry, astronomy, and geography, with a general compact of physics, they may descend in mathematics to the instrumental science of trigonometry, and from thence to fortification, architecture, engineering, or navigation. And in natural philosophy they may proceed leisurely from the history of meteors, minerals, plants, and living creatures, as far as anatomy. Then also in course might be read to them out of some not tedious writer the institution of physic. […] To set forward all these proceedings in nature and mathematics, what hinders but that they may procure, as oft as shall be needful, the helpful experiences of hunters, fowlers, fishermen, shepherds, gardeners, apothecaries; and in other sciences, architects, engineers, mariners, anatomists." (John Milton, "On Education", 1644)

"In geometry, to be sure, the accident of a fraction or an irrational does not usually prevent equations from being solved readily. nor does the imperfection of a negative, for the subject on which the geometer works is always certain. But multiplicity of affections8 is a hindrance, and the higher the power and the order of an affection, the more likely it is that a fraction or surd9 will appear in the solution of a problem." (François Viète, "On the Structure of Equations as Shown by Zetetics, Plasmatic Modification and Syncrisis", 1646)

"Indeed, many geometric things can be discovered or elucidated by algebraic principles, and yet it does not follow that algebra is geometrical, or even that it is based on geometric principles (as some would seem to think). This close affinity of arithmetic and geometry comes about, rather, because geometry is, as it were, subordinate to arithmetic, and applies universal principles of arithmetic to its special objects." (John Wallis, "Mathesis Universalis", 1657)

"For, Mathematical Demonstrations being built upon the impregnable Foundations of Geometry and Arithmetick, are the only Truths, that can sink into the Mind of Man, void of all Uncertainty; and all other Discourses participate more or less of Truth, according as their Subjects are more or less capable of Mathematical Demonstration." (Christopher Wren, [lecture at Gresham College] 1657)

"Only geometry can hand us the thread [which will lead us through] the labyrinth of the continuum's composition, the maximum and the minimum, the infinitesimal and the infinite; and no one will arrive at a truly solid metaphysics except he who has passed through this [labyrinth]." (Gottfried W Leibniz, "Dissertatio Exoterica De Statu Praesenti et Incrementis Novissimis Deque Usu Geometriae", 1676)

"After all the progress I have made in these matters, I am still not happy with Algebra, because it provides neither the shortest ways nor the most beautiful constructions of Geometry. This is why when it comes to that, I think that we need another analysis which is properly geometric or linear, which expresses to us directly situm, in the same way as algebra expresses magnitudinem. And I think that I have the tools for that, and that we might represent figures and even engines and motion in character, in the same way as algebra represents numbers in magnitude." (Gottfried W Leibniz, [letter to Christiaan Huygens] 1679)

"The Excellence of Modern Geometry is in nothing more evident, than in those full and adequate Solutions it gives to Problems; representing all possible Cases in one view, and in one general Theorem many times comprehending whole Sciences; which deduced at length into Propositions, and demonstrated after the manner of the Ancients, might well become the subjects of large Treatises: For whatsoever Theorem solves the most complicated Problem of the kind, does with a due Reduction reach all the subordinate Cases." (Edmund Halley, "An Instance of the Excellence of Modern Algebra in the resolution of the problem of finding the foci of optic glasses universally", Philosophical Transactions, 1694)

On Karl Weierstrass

"[Up to that time] one would have said that a continuous function is essentially capable of being represented by a curve, and that a curve has always a tangent. Such reasoning has no mathematical value whatever; it is founded on intuition, or rather on a visible representation. But such representation is crude and misleading. We think we can figure to ourselves a curve without thickness; but we only figure a stroke of small thickness. In like manner we see the tangent as a straight band of small thickness, and when we say that it touches the curve, we wish merely to say that these two bands coincide without crossing. If that is what we call a curve and a tangent, it is clear that every curve has a tangent; but this has nothing to do with the theory of functions. We see to what error we are led by a foolish confidence in what we take to be visual evidence. By the discovery of this striking example Weierstrass has accordingly given us a useful reminder, and has taught us better to appreciate the faultless and purely arithmetical methods with which he more than any one has enriched our science." Henri Poincaré, "L'oeuvre Mathématique de Weierstrass",Acta Mathematica Vol.22, 1901)

"It is not surprising, in view of the polydynamic constitution of the genuinely mathematical mind, that many of the major heros of the science, men like Desargues and Pascal, Descartes and Leibnitz, Newton, Gauss and Bolzano, Helmholtz and Clifford, Riemann and Salmon and Plücker and Poincaré, have attained to high distinction in other fields not only of science but of philosophy and letters too. And when we reflect that the very greatest mathematical achievements have been due, not alone to the peering, microscopic, histologic vision of men like Weierstrass, illuminating the hidden recesses, the minute and intimate structure of logical reality, but to the larger vision also of men like Klein who survey the kingdoms of geometry and analysis for the endless variety of things that flourish there, as the eye of Darwin ranged over the flora and fauna of the world, or as a commercial monarch contemplates its industry, or as a statesman beholds an empire; when we reflect not only that the Calculus of Probability is a creation of mathematics but that the master mathematician is constantly required to exercise judgment. - judgment, that is, in matters not admitting of certainty - balancing probabilities not yet reduced nor even reducible perhaps to calculation; when we reflect that he is called upon to exercise a function analogous to that of the comparative anatomist like Cuvier, comparing theories and doctrines of every degree of similarity and dissimilarity of structure; when, finally, we reflect that he seldom deals with a single idea at a tune, but is for the most part engaged in wielding organized hosts of them, as a general wields at once the division of an army or as a great civil administrator directs from his central office diverse and scattered but related groups of interests and operations; then, I say, the current opinion that devotion to mathematics unfits the devotee for practical affairs should be known for false on a priori grounds. And one should be thus prepared to find that as a fact Gaspard Monge, creator of descriptive geometry, author of the classic Applications de l’analyse à la géométrie; Lazare Carnot, author of the celebrated works, Géométrie de position, and Réflections sur la Métaphysique du Calcul infinitesimal; Fourier, immortal creator of the Théorie analytique de la chaleur; Arago, rightful inheritor of Monge’s chair of geometry; Poncelet, creator of pure projective geometry; one should not be surprised, I say, to find that these and other mathematicians in a land sagacious enough to invoke their aid, rendered, alike in peace and in war, eminent public service." (Cassius J Keyser, "Lectures on Science, Philosophy and Art", 1908)

"Who has studied the works of such men as Euler, Lagrange, Cauchy, Riemann, Sophus Lie, and Weierstrass, can doubt that a great mathematician is a great artist? The faculties possessed by such men, varying greatly in kind and degree with the individual, are analogous with those requisite for constructive art. Not every mathematician possesses in a specially high degree that critical faculty which finds its employment in the perfection of form, in conformity with the ideal of logical completeness; but every great mathematician possesses the rarer faculty of constructive imagination." (Ernest W Hobson, "Presidential Address British Association for the Advancement of Science", Nature, 1910)

"Inspiration plays no less a role in science than it does in the realm of art. It is a childish notion to think that a mathematician attains any scientifically valuable results by sitting at his desk with a ruler, calculating machines or other mechanical means. The mathematical imagination of a Weierstrass is naturally quite differently oriented in meaning and result than is the imagination of an artist, and differs basically in quality. But the psychological processes do not differ. Both are frenzy (in the sense of Plato’s 'mania') and 'inspiration'." (Max Weber, [speech (1918) presented at Munich University] 1919) 

"The arithmetization of mathematics […] which began with Weierstrass […] had for its object the separation of purely mathematical concepts, such as number and correspondence and aggregate, from intuitional ideas, which mathematics had acquired from long association with geometry and mechanics. These latter, in the opinion of the formalists, are so firmly entrenched in mathematical thought that in spite of the most careful circumspection in the choice of words, the meaning concealed behind these words, may influence our reasoning. For the trouble with human words is that they possess content, whereas the purpose of mathematics is to construct pure thought. But how can we avoid the use of human language? The […] symbol. Only by using a symbolic language not yet usurped by those vague ideas of space, time, continuity which have their origin in intuition and tend to obscure pure reason-only thus may we hope to build mathematics on the solid foundation of logic." (Tobias Danzig & Joseph Mazur (ed.), "Number: The Language of Science", 1930

"Men of science belong to two different types - the logical and the intuitive. Science owes its progress to both forms of minds. Mathematics, although a purely logical structure, nevertheless makes use of intuition. Among the mathematicians there are intuitives and logicians, analysts and geometricians. Hermite and Weierstrass were intuitives. Riemann and Bertrand, logicians. The discoveries of intuition have always to be developed by logic." (Alexis Carrel,"Man the Unknown", 1935)

"Objections […] inspired Kronecker and others to attack Weierstrass’ 'sequential' definition of irrationals. Nevertheless, right or wrong, Weierstrass and his school made the theory work. The most useful results they obtained have not yet been questioned, at least on the ground of their great utility in mathematical analysis and its implications, by any competent judge in his right mind. This does not mean that objections cannot be well taken: it merely calls attention to the fact that in mathematics, as in everything else, this earth is not yet to be confused with the Kingdom of Heaven, that perfection is a chimaera, and that, in the words of Crelle, we can only hope for closer and closer approximations to mathematical truth - whatever that may be, if anything - precisely as in the Weierstrassian theory of convergent sequences of rationals defining irrationals." (Eric T Bell, "Men of Mathematics", 1937) 

"The splendid creations of this theory have excited the admiration of mathematicians mainly because they have enriched our science in an almost unparalleled way with an abundance of new ideas and opened up heretofore wholly unknown fields to research. The Cauchy integral formula, the Riemann mapping theorem and the Weierstrass power series calculus not only laid the groundwork for a new branch of mathematics but at the same time they furnished the first and till now the most fruitful example of the intimate connections between analysis and algebra. But it isn't just the wealth of novel ideas and discoveries which the new theory furnishes; of equal importance on the other hand are the boldness and profundity of the methods by which the greatest of difficulties are overcome and the most recondite of truths, the mysteria functiorum, are exposed tothe brightest." (Richard Dedekind) 

On Problem Solving 30: On Solvability (2010-)

"Quantum theory may be formulated using Hilbert spaces over any of the three associative normed division algebras: the real numbers, the complex numbers and the quaternions. Indeed, these three choices appear naturally in a number of axiomatic approaches. However, there are internal problems with real or quaternionic quantum theory. Here we argue that these problems can be resolved if we treat real, complex and quaternionic quantum theory as part of a unified structure. Dyson called this structure the ‘three-fold way’ […] This three-fold classification sheds light on the physics of time reversal symmetry, and it already plays an important role in particle physics." (John C Baez, "Division Algebras and Quantum Theory", 2011)

"Under normal conditions the research scientist is not an innovator but a solver of puzzles, and the puzzles upon which he concentrates are just those which he believes can be both stated and solved within the existing scientific tradition." (Thomas S Kuhn, "The Essential Tension: Selected Studies in Scientific Tradition and Change", 2011)

"Mathematicians approach problems the way rock climbers do cliffs: the more difficult the pitch, the more exhilarating the ascent. After a climb has been solved, others look for new routes, or try equipment that no one else has used, simply for the joy of pioneering." (David Perkins, "Calculus and Its Origins", 2012)

"Complex systems defy intuitive solutions. Even a third-order, linear differential equation is unsolvable by inspection. Yet, important situations in management, economics, medicine, and social behavior usually lose reality if simplified to less than fifth-order nonlinear dynamic systems. Attempts to deal with nonlinear dynamic systems using ordinary processes of description and debate lead to internal inconsistencies. Underlying assumptions may have been left unclear and contradictory, and mental models are often logically incomplete. Resulting behavior is likely to be contrary to that implied by the assumptions being made about' underlying system structure and governing policies." (Jay W Forrester, "Modeling for What Purpose?", The Systems Thinker Vol. 24 (2), 2013)

"Systems always contain problematic elements in a sense that is usually not clear until the implications of the new system are sufficiently explored. Often problems can be stated in the language of the initial system but can only be resolved by creating a new system. [...] Problems that can be stated in the language of one system often cannot be solved within that system because the solution depends upon the development of a new system." (William Byers, "Deep Thinking: What Mathematics Can Teach Us About the Mind", 2015)

"The subject of computational complexity theory is focused on classifying problems by how hard they are. […] (1) P problems are those that can be solved by a Turing machine" (TM)" (deterministic) in polynomial time. (‘P’ stands for polynomial). P problems form a class of problems that can be solved efficiently. (2) NP problems are those that can be solved by non-deterministic TM in polynomial time. A problem is in NP if you can quickly" (in polynomial time) test whether a solution is correct" (without worrying about how hard it might be to find the solution). NP problems are a class of problems that cannot be solved efficiently. NP does not stand for 'non-polynomial'. There are many complexity classes that are much harder than NP. (3) Undecidable problems: For some problems, we can prove that there is no algorithm that always solves them, no matter how much time or space is allowed." (K V N Sunitha & N Kalyani, "Formal Languages and Automata Theory", 2015)

"When we extend the system of natural numbers and counting to embrace infinite cardinals, the larger system need not have all of the properties of the smaller one. However, familiarity with the smaller system leads us to expect certain properties, and we can become confused when the pieces don’t seem to fit. Insecurity arose when the square of a complex number violated the real number principle that all squares are positive. This was resolved when we realised that the complex numbers cannot be ordered in the same way as their subset of reals." (Ian Stewart & David Tall, "The Foundations of Mathematics" 2nd Ed., 2015)

"Judgments made in difficult circumstances can be based on a limited number of simple, rapidly-arrived-at rules ('heuristics'), rather than formal, extensive algorithmic calculus and programs. Often, even complex problems can be solved quickly and accurately using such 'quick and dirty' heuristics. However, equally often, such heuristics can be beset by systematic errors or biases." (Jérôme Boutang & Michel De Lara, "The Biased Mind", 2016)

On Problem Solving 29: On Solvability (2000-2009)

"Theorems are fun especially when you are the prover, but then the pleasure fades. What keeps us going are the unsolved problems." (Carl Pomerance, 2000)"

"The existence of the tipping point means it is theoretically possible to completely eradicate a disease. Eradication does not require a perfect vaccine and universal immunization but only the weaker condition that the reproduction rate of the disease fall and remain below one so that new cases arise at a lower rate than old cases are resolved." (John D Sterman, "Business Dynamics: Systems thinking and modeling for a complex world. ", 2000)

"Theorems are fun especially when you are the prover, but then the pleasure fades. What keeps us going are the unsolved problems." (Carl Pomerance, MAA, 2000)

"And although mathematical ideas and thought are constantly evolving, you will also see that the most basic fundamental problems never go away. Many of these problems go back to the ancient Greeks, and maybe even to ancient Sumer, although we may never know for sure. The fundamental philosophical questions like the continuous versus the discrete or the limits of knowledge are never definitively solved. Each generation formulates its own answer, strong personalities briefly impose their views, but the feeling of satisfaction is always temporary, and then the process continues, it continues forever." (Gregory Chaitin, "Meta Math: The Quest for Omega", 2005)

"A model does not always predict all the features of the data. Nature has no privileged tendency to present me with solvable challenges." (Eliezer S Yudkowsky, "A Technical Explanation of Technical Explanation", 2005)

"Solvable Lie algebras are close to both upper triangular matrices and commutative Lie algebras. In contrast to this, semisimple Lie algebras are as far as possible from being commutative. By Levi’s decomposition theorem, any Lie algebra is built out of a solvable and a semisimple one. The nontrivial prototype of a solvable Lie algebra is the Heisenberg algebra. [3](Eberhard Zeidler, "Quantum Field Theory I: Gauge Theory", 2006)

"Knowing a solution is at hand is a huge advantage; it’s like not having a 'none of the above' option. Anyone with reasonable competence and adequate resources can solve a puzzle when it is presented as something to be solved. We can skip the subtle evaluations and move directly to plugging in possible solutions until we hit upon a promising one. Uncertainty is far more challenging." (Garry Kasparov, "How Life Imitates Chess", 2007)

"Social ecology is based on the conviction that nearly all of our present ecological problems originate in deep-seated social problems. It follows, from this view, that these ecological problems cannot be understood, let alone solved, without a careful understanding of our existing society and the irrationalities that dominate it. To make this point more concrete: economic, ethnic, cultural, and gender conflicts, among many others, lie at the core of the most serious ecological dislocations we face today - apart, to be sure, from those that are produced by natural catastrophes." (Murray Bookchin, "Social Ecology and Communalism", 2007)

"When in the sciences or techniques one states that a certain problem is unsolvable, a rigorous demonstration of such unsolvability is required. And when a scientist submits an article to publication, the least that its referees demand is that it be intelligible. Why? Because rational beings long for understanding and because only clear statements are susceptible to be put to examination to verify whether they are true or false. In the Humanities it is the same, or it should be, but it is not always so." (Mario Bunge, "Xenius, Platón y Manolito", La Nación, 2008)

"An algorithm refers to a successive and finite procedure by which it is possible to solve a certain problem. Algorithms are the operational base for most computer programs. They consist of a series of instructions that, thanks to programmers’ prior knowledge about the essential characteristics of a problem that must be solved, allow a step-by-step path to the solution." (Diego Rasskin-Gutman, "Chess Metaphors: Artificial Intelligence and the Human Mind", 2009)

"Chess also offers a modality that includes an exercise of totally free creation - compositions. These artificial positions are created for didactic reasons to illustrate a certain subject or to propose a problem that has to be solved following a series of indications" (Diego Rasskin-Gutman, "Chess Metaphors: Artificial Intelligence and the Human Mind", 2009)

"Chess, as a game of zero sum and total information is, theoretically, a game that can be solved. The problem is the immensity of the search tree: the total number of positions surpasses the number of atoms in our galaxy. When there are few pieces on the board, the search space is greatly reduced, and the problem becomes trivial for computers’ calculation capacity." (Diego Rasskin-Gutman, "Chess Metaphors: Artificial Intelligence and the Human Mind", 2009)

"Mental acuity of any kind comes from solving problems yourself, not from being told how to solve them." (Paul Lockhart,"A Mathematician's Lament", 2009)

"What literally defines social ecology as ‘social’ is its recognition of the often overlooked fact that nearly all our present ecological problems arise from deep-seated social problems. Conversely, present ecological problems cannot be clearly understood, much less resolved, without resolutely dealing with problems within society." (Murray Bookchin, "What is Social Ecology", 2009)

On Problem Solving 28: On Solvability (1990-1999)

"Humans have a strong tendency to guard their opinion of their own competence in acting. To a certain extent this makes sense, as someone who considers himself to be incapable of acting will hardly act. Guarding one's opinion of one's competence is an important motivation. But it can lead to deformations in the thought process. To maintain a high opinion of one's own competence, people fail to take notice of data that show that their hypotheses are wrong. Or they act 'ballistically' and do not check the effects of their actions so as to maintain the illusion of having solved the corresponding problems by means of their action. The underlying reasons for dispensing with self-reflection may also lie in the tendency to avoid looking at one's own mistakes so as not to endanger one's estimation of one's own competence." (Dietrich Dörner, "The Logic of Failure", Philosophical Transactions of the Royal Society of London" (B), 1990)

"[...] mystery is an inescapable ingredient of mathematics. Mathematics is full of unanswered questions, which far outnumber known theorems and results. It’s the nature of mathematics to pose more problems than it can solve. Indeed, mathematics itself may be built on small islands of truth comprising the pieces of mathematics that can be validated by relatively short proofs. All else is speculation." (Ivars Peterson, „Islands of Truth: A Mathematical Mystery Cruise", 1990)

"Although a system may exhibit sensitive dependence on initial condition, this does not mean that everything is unpredictable about it. In fact, finding what is predictable in a background of chaos is a deep and important problem." (Which means that, regrettably, it is unsolved.) In dealing with this deep and important problem, and for want of a better approach, we shall use common sense." (David Ruelle, "Chance and Chaos", 1991)

"One of the lessons that the history of mathematics clearly teaches us is that the search for solutions to unsolved problems, whether solvable or unsolvable, invariably leads to important discoveries along the way." (Carl B Boyer & Uta C Merzbach, "A History of Mathematics", 1991)

"The most persuasive positive argument for mental images as objects is [that] whenever one thinks one is seeing something there must be something one is seeing. It might be an object directly, or it might be a mental picture. [This] point is so plausible that it is deniable only at the peril of becoming arbitrary. One should concede that the question whether mental images are entities of some sort is not resolvable by logical or linguistic analysis, and believe what makes sense of experience." (Eva T H Brann,"The World of Imagination", 1991)

"Throughout the evolution of mathematics, problems have acted as catalysts in the discovery and development of mathematical ideas. In fact, the history of mathematics can probably be traced by studying the problems that mathematicians have tried to solve over the centuries. It is almost disheartening when an old problem is finally solved, for it will no longer be around to challenge and stimulate mathematical thought." (Theoni Pappas, "More Joy of Mathematics: Exploring mathematical insights & concepts", 1991)

"Dynamical systems that vary in discrete steps […] are technically known as mappings. The mathematical tool for handling a mapping is the difference equation. A system of difference equations amounts to a set of formulas that together express the values of all of the variables at the next step in terms of the values at the current step. […] For mappings, the difference equations directly express future states in terms of present ones, and obtaining chronological sequences of points poses no problems. For flows, the differential equations must first be solved. General solutions of equations whose particular solutions are chaotic cannot ordinarily be found, and approximations to the latter are usually determined by numerical methods." (Edward N Lorenz, "The Essence of Chaos", 1993)

"Mathematicians typically do not feel that they have completely solved a system of differential equations until they have written down a general solution - a set of formulas giving the value of each variable at every time, in terms of the supposedly known values at some initial time." (Edward N Lorenz, "The Essence of Chaos", 1993)

"Perhaps the greatest strength of graph theory is the abundance of natural and beautiful problems waiting to be solved. […] Paradoxically, much of what is wrong with graph theory is due to this richness of problems. It is all too easy to find new problems based on no theory whatsoever, and to solve the first few cases by straightforward methods. Unfortunately, in some instances the problems are unlikely to lead anywhere […]" (Béla Bollobás, "The Future of Graph Theory", [in "Quo Vadis, Graph Theory?"] 1993)

"To understand what kinds of problems are solvable by the Monte Carlo method, it is important to note that the method enables simulation of any process whose development is influenced by random factors. Second, for many mathematical problems involving no chance, the method enables us to artificially construct a probabilistic model (or several such models), making possible the solution of the problems." (Ilya M Sobol, "A Primer for the Monte Carlo Method", 1994)

"It remains an unhappy fact that there is no best method for finding the solution to general nonlinear optimization problems. About the best general procedure yet devised is one that relies upon imbedding the original problem within a family of problems, and then developing relations linking one member of the family to another. If this can be done adroitly so that one family member is easily solvable, then these relations can be used to step forward from the solution of the easy problem to that of the original problem. This is the key idea underlying dynamic programming, the most flexible and powerful of all optimization methods." (John L Casti, "Five Golden Rules", 1995)

"General relativity, one of the most famous theories, is formulated in terms of a nonlinear equation. This makes us wonder if some of the phenomena described by general relativity, namely black holes, objects orbiting black holes, and even the universe itself, can become chaotic under certain circumstances. [...] The problem is the equation itself, namely the equation of general relativity; it is so complex that the most general solution has never been obtained. It has, of course, been solved for many simple systems; if the system has considerable symmetry (e.g., it is spherical) the equation reduces to a number of ordinary equations that can be solved, but chaos does not occur in these cases. In more realistic cases - situations that actually occur in nature - chaos may occur, but the equations are either unsolvable or very difficult to solve. This presents a dilemma. If we try to model the system using many simplifications it won't exhibit chaos, but if we model it realistically we can't solve it." (Barry R Parker, "Chaos in the Cosmos: The stunning complexity of the universe", 1996)

"Our sole responsibility is to produce something smarter than we are; any problems beyond that are not ours to solve." (Eliezer S Yudkowsky, "Staring into the Singularity", 1996)

On Problem Solving 27: On Solvability (1970-1979)

"A problem adequately stated is a problem well on its way to being solved." (R Buckminster Fuller, "I Seem to be a Verb", 1970)

"Deep in the human nature there is an almost irresistible tendency to concentrate physical and mental energy on attempts at solving problems that seem to be unsolvable." (Ragnar Frisch, "From Utopian Theory to Practical Applications", [Nobel lecture] 1970)

"The definition of a problem and the action taken to solve it largely depend on the view which the individuals or groups that discovered the problem have of the system to which it refers. A problem may thus find itself defined as a badly interpreted output, or as a faulty output of a faulty output device, or as a faulty output due to a malfunction in an otherwise faultless system, or as a correct but undesired output from a faultless and thus undesirable system. All definitions but the last suggest corrective action; only the last definition suggests change, and so presents an unsolvable problem to anyone opposed to change." (Herbert Brün, "Technology and the Composer", 1971)

"The matter of the normalcy or non-normalcy of π will never, of course, be resolved by electronic computers. We have here an example of a theoretical problem which requires profound mathematical talent and cannot be solved by computations alone. The existence of such problems ought to furnish at least a partial antidote to the disease of computeritis, which seems so rampant today." (Howard Eves, "Mathematical Circles Revisited", 1971)

"Extrema is the generic term for the concepts 'maximum' and 'minimum' , like 'parents' is the generic term for 'father' and 'mother'. Extremal problems have to do with finding maxima and minima. We encounter them everywhere. It is hardly an exaggeration to say that all problems solved by living organisms are those involving a search for extrema." (Yakov Khurgin, "Did You Say Mathematics?", 1974)

"The synthetic mode of thought, when applied to systems problems, is called the systems approach. In this approach a problem is not solved by taking it apart but by viewing it as a part of a larger problem." (Russell L Ackoff, "Redesigning the future", 1974)

"It is one of our most exciting discoveries that local discovery leads to a complex of further discoveries. Corollary to this we find that we no sooner get a problem solved than we are overwhelmed with a multiplicity of additional problems in a most beautiful payoff of heretofore unknown, previously unrecognized, and as-yet unsolved problems." (Buckminster Fuller, "Synergetics: Explorations in the Geometry of Thinking", 1975)"

"A solution of newly appearing economic problems, and in particular those connected with the scientific-technical revolution often cannot be based on existing methods but needs new ideas and approaches. Such one is the problem of the protection of nature. The problem of economic valuation of technical innovations efficiency and rates of their spreading cannot be solved only by the long-term estimation of direct outcomes and results without accounting peculiarities of new industrial technology, its total contribution to technical progress." (Leonid V Kantorovich, "Mathematics in Economics: Achievements, Difficulties, Perspectives", [Nobel lecture]1975)

"The philosophical emphasis here is: to solve a geometrical problem of a global nature, one first reduces it to a homotopy theory problem; this is in turn reduced to an algebraic problem and is solved as such. This path has historically been the most fruitful one in algebraic topology." (Brayton Gray, Homotopy Theory", Pure and Applied Mathematics Vol. 64, 1975)

"One of the lessons that the history of mathematics clearly teaches us is that the search for solutions to unsolved problems, whether solvable or unsolvable, invariably leads to important discoveries along the way." (Carl B Boyer & Uta C Merzbach, "A History of Mathematics", 1976)

"All mathematical problems are solved by reasoning within a deductive system in which basic laws of logic are embedded." (Martin Gardner,"Aha! Insight", 1978)

On Problem Solving 26: On Solvability (1980-1989)

"Metaphors deny distinctions between things: problems often arise from taking structural metaphors too literally. Because unexamined metaphors lead us to assume the identity of unidentical things, conflicts can arise which can only be resolved by understanding the metaphor (which requires its recognition as such), which means reconstructing the analogy on which it is based. […] The unexplained extension of concepts can too often result in the destruction rather than the expansion of meaning." (David Pimm,"Metaphor and Analogy in Mathematics", For the Learning of Mathematics Vol. 1" (3), 1981)

"In various fields of knowledge the problem of the relationship between cause and condition is solved in different ways, depending mainly on the complexity of the relationships that are being studied, their uniformity or, on the contrary, the distinctness and comparative importance of separate factors." (Alexander Spirkin, "Dialectical Materialism", 1983)

"There are, roughly speaking, two kinds of mathematical creativity. One, akin to conquering a mountain peak, consists of solving a problem which has remained unsolved for a long time and has commanded the attention of many mathematicians. The other is exploring new territory." (Mark Kac, "Enigmas Of Chance", 1985)

"To learn mathematics is to learn mathematical problem solving." (Patrik W Thompson, 1985)

"One never knows how hard a problem is until it has been solved. You don’t necessarily know that you will succeed if you work harder or longer." (Linus Pauling, [interview] 1986)

"Linear relationships are easy to think about: the more the merrier. Linear equations are solvable, which makes them suitable for textbooks. Linear systems have an important modular virtue: you can take them apart and put them together again - the pieces add up. Nonlinear systems generally cannot be solved and cannot be added together. [...] Nonlinearity means that the act of playing the game has a way of changing the rules. [...] That twisted changeability makes nonlinearity hard to calculate, but it also creates rich kinds of behavior that never occur in linear systems." (James Gleick, "Chaos: Making a New Science", 1987)

"A scientific problem can be illuminated by the discovery of a profound analogy, and a mundane problem can be solved in a similar way." (Philip Johnson-Laird, "The Computer and the Mind", 1988)

"The successes of the differential equation paradigm were impressive and extensive. Many problems, including basic and important ones, led to equations that could be solved. A process of self-selection set in, whereby equations that could not be solved were automatically of less interest than those that could." (Ian Stewart, "Does God Play Dice? The Mathematics of Chaos", 1989)