🪷On Mind: On Perception (1875-1899)

"For of an image one requires some kind of sameness with the pictured object, of a statue sameness of form, of a delineation sameness of perspective projection in the visual field, of a painting also sameness of color. " (Heinrich Hertz, "The Facts in Perception", 1878)

"As for everything else, so for a mathematical theory: beauty can be perceived but not explained." (Arthur Cayley, [president's address] 1883)

"It is difficult to give an idea of the vast extent of modern mathematics. The word 'extent' is not the right one: I mean extent crowded with beautiful details - not an extent of mere uniformity such as an objectless plain, but of a tract of beautiful country seen at first in the distance, but which will bear to be rambled through and studied in every detail of hillside and valley, stream, rock, wood and flower. But, as for everything else, so for a mathematical theory - beauty can be perceived but not explained." (Arthur Cayley, [address before the meeting of the British Association at Southport] 1883)

"While all that we have is a relation of phenomena, a mental image, as such, in juxtaposition with or soldered to a sensation, we can not as yet have assertion or denial, a truth or a falsehood. We have mere reality, which is, but does not stand for anything, and which exists, but by no possibility could be true. […] the image is not a symbol or idea. It is itself a fact, or else the facts eject it. The real, as it appears to us in perception, connects the ideal suggestion with itself, or simply expels it from the world of reality. […] you possess explicit symbols all of which are universal and on the other side you have a mind which consists of mere individual impressions and images, grouped by the laws of a mechanical attraction." (Francis H Bradley, "Principles of Logic", 1883)

"Mathematics stands forth as that which unites, mediates between Man and Nature, inner and outer world, thought and perception, as no other subject does." (Julius Froebel, 1893)

"Experience is the collecting of what is similar in different particular perceptions." (Heinrich Hertz, "The Principles of Mechanics Presented in a New Form", 1894)

"It is a mistake to reduce mater to the perception which we have of it, a mistake also to make of it a thing able to produce in us perceptions, but in itself is of another nature than they. Matter, in our view, is an aggregate of ‘images’. And by ‘image‘ we mean a certain existence which is more than that which the idealist calls a representation, but less than that which the realist calls a thing - an existence placed halfway between the ‘thing‘ and the ‘representation’. This conception of matter is simply that of common sense. [...] For common sense, then, the object exists in itself, and, on the other hand, the object is, in itself, pictorial, as we perceive it image it is, but a self-existing image." (Henri Bergson,"Matter and Memory", 1896)

"We call that psychical process, which is operative in the clear perception of a narrow region of the content of consciousness, attention." (Wilhelm M Wundt,"Outlines of Psychology", 1897)

🪷On Mind: On Perception (1850-1874)

"All perception of truth is the detection of an analogy [...]" (Henry D Thoreau, 1851)

"In the extension of space-construction to the infinitely great, we must distinguish between unboundedness and infinite extent; the former belongs to the extent relations, the latter to the measure-relations. That space is an unbounded threefold manifoldness, is an assumption which is developed by every conception of the outer world; according to which every instant the region of real perception is completed and the possible positions of a sought object are constructed, and which by these applications is forever confirming itself. The unboundedness of space possesses in this way a greater empirical certainty than any external experience. But its infinite extent by no means follows from this; on the other hand if we assume independence of bodies from position, and therefore ascribe to space constant curvature, it must necessarily be finite provided this curvature has ever so small a positive value. If we prolong all the geodesies starting in a given surface-element, we should obtain an unbounded surface of constant curvature, i.e., a surface which in a flat manifoldness of three dimensions would take the form of a sphere, and consequently be finite." (Bernhard Riemann, "On the hypotheses which lie at the foundation of geometry", 1854)

"As you are aware, no perceptions obtained by the senses are merely sensations impressed on our nervous systems. A peculiar intellectual activity is required to pass from a nervous sensation to the conception of an external object, which the sensation has aroused. The sensations of our nerves of sense are mere symbols indicating certain external objects, and it is usually only after considerable practice that we acquire the power of drawing correct conclusions from our sensations respecting the corresponding objects." (Hermann von Helmholtz, "On the Physiological Causes of Harmony", 1857)

"The term 'intellect' includes all those powers by which we acquire, retain, and extend our knowledge; as perception, memory, imagination, judgment, and the like." (William Fleming, "A vocabulary of the philosophical sciences", 1857)

"Observation is so wide awake, and facts are being so rapidly added to the sum of human experience, that it appears as if the theorizer would always be in arrears, and were doomed forever to arrive at imperfect conclusion; but the power to perceive a law is equally rare in all ages of the world, and depends but little on the number of facts observed." (Henry D Thoreau, "A Week on the Concord and Merrimack Rivers", 1862)

"If one would see how a science can be constructed and developed to its minutest details from a very small number of intuitively perceived axioms, postulates, and plain definitions, by means of rigorous, one would almost say chaste, syllogism, which nowhere makes use of surreptitious or foreign aids, if one would see how a science may thus be constructed one must turn to the elements of Euclid." (Hermann Hankel, "Theorie der Complexen Zahlensysteme", 1867)

"When a mental image, as such, is the object of my apprehension, there is no meaning in seeking to distinguish its existence in my consciousness" (in me) from its existence out of my consciousness" (in itself) ; for the object apprehended is, in this case, one which does not even exist, as the objects of external perception do, in itself outside of my consciousness. It exists only within me." (Friedrich Ueberweg,"System of Logic and History of Logical Doctrines", 1871)

"Even if a curve is not drawn nor it is assumed to be tracked by the eye, but we have a 'perception' of it, it has in any case a limited precision and does not therefore correspond to the exact concept of a function of precision mathematics but rather to the idea of a function stripe." (Felix Klein, 1873)

🪷On Mind: On Perception (1800-1849)

"If we seek a cause wherever we perceive symmetry, it is not that we regard a symmetrical event as less possible than the others, but, since this event ought to be the effect of a regular cause or that of chance, the first of these suppositions is more probable than the second.” (Pierre-Simon de Laplace, "Philosophical Essay on Probabilities”, 1812)

"The difficulty of bringing idea and experience into relation with one another makes itself very painfully felt in all investigation of nature. The idea is independent of space and time. Research is limited in space and time. Hence in the idea simultaneous and successive features are most intimately linked, whereas these are always separated in experience; and to think of a process of nature as simultaneous and successive at once, in accordance with the idea, makes our heads spin. The understanding is unable to conceive of those sense data as jointly present which experience transmitted to it one at a time. Thus, the contradiction between ideation and perception remains forever unresolved. " (Johann Wolfgang von Goethe, "Doubt and Resignation", 1820)

"Truth travels down from the heights of philosophy to the humblest walks of life, and up from the simplest perceptions of an awakened intellect to the discoveries which almost change the face of the world. At every stage of its progress it is genial, luminous, creative." (Edward Everett, [address] 1835)

"A line, as defined by geometers, is wholly unconceivable. We can reason about a line as if it had no breadth; because we have a power, which is the foundation of all the control we can exercise over the operations of our minds; the power, when a perception is present to our senses, or a conception to our intellects, of attending to a part only of that perception or conception, instead of the whole. But we cannot conceive a line without breadth: we can form no mental picture of such a line: all the lines which we have in our minds are lines possessing breadth." (John S Mill, "System of Logic ratiocinative and inductive, being a connected view of the principles of evidence and the methods of scientific investigation", 1843)

"All existing things upon this earth, which have knowledge of their own existence, possess, some in one degree and some in another, the power of thought, accompanied by perception, which is the awakening of thought by the effects of external objects upon the senses." (Augustus De Morgan, "Formal Logic: Or, The Calculus of Inference, Necessary and Probable", 1847)

🪷On Mind: On Perception (1600-1699)

"That faculty which perceives and recognizes the noble proportions in what is given to the senses, and in other things situated outside itself, must be ascribed to the soul. It lies very close to the faculty which supplies formal schemata to the senses, or deeper still, and thus adjacent to the purely vital power of the soul, which does not think discursively […] Now it might be asked how this faculty of the soul, which does not engage in conceptual thinking, and can therefore have no proper knowledge of harmonic relations, should be capable of recognizing what is given in the outside world. For to recognize is to compare the sense perception outside with the original pictures inside, and to judge that it conforms to them." (Johannes Kepler, "Harmonices Mundi" ["Harmony of the World"], 1619)

"The entire method consists in the order and arrangement of the things to which the mind's eye must turn so that we can discover some truth." (René Descartes, "Rules for the Direction of the Mind", 1628)

"There is nothing in us which we must attribute to our soul except our thoughts. These are of two principal kinds, some being actions of the soul and others its passions. Those I call its actions are all our volitions […] the various perceptions or modes of knowledge present in us may be called its passions in the general sense, for it is often not our soul which makes them such as they are, and the soul always receives them from the things that are represented by them." (René Descartes, "Les passions de l’âme" ["Passions of the Soul"], 1649)

"Although time, space, place, and motion are very familiar to everyone, it must be noted that these quantities are popularly conceived solely with reference to the objects of sense perception." (Isaac Newton, "The Principia: Mathematical Principles of Natural Philosophy", 1687)

"Abstraction involves perceiving something, relating it to other things, grasping some common trait of those things, and conceiv­ing of the common trait as to it can be related not only to those things but also to other similar things."  (John Locke, "An Essay Concerning Human Understanding", 1689)

“Probability is the appearance of agreement upon fallible proofs. As demonstration is the showing the agreement or disagreement of two ideas by the intervention of one or more proofs, which have a constant, immutable, and visible connexion one with another; so probability is nothing but the appearance of such an agreement or disagreement by the intervention of proofs, whose connexion is not constant and immutable, or at least is not perceived to be so, but is, or appears for the most part to be so, and is enough to induce the mind to judge the proposition to be true or false, rather than the contrary.” (John Locke, “An Essay Concerning Human Understanding”, 1689)

“Sometimes the mind perceives the agreement or disagreement of two ideas immediately by themselves, without the intervention of any other; and this, I think, we may call intuitive knowledge. [...] Intuitive knowledge needs no probation, nor can have any, this being the highest of all human certainty.” (John Locke, “An Essay Concerning Human Understanding”, 1689)

🪷On Mind: On Perception (1700-1799)

"Moreover, it must be confessed that perception and that which depends upon it are inexplicable on mechanical grounds, that is to say, by means of figures and motions. And supposing there were a machine, so constructed as to think, feel, and have perception, it might be conceived as increased in size, while keeping the same proportions, so that one might go into it as into a mill. That being so, we should, on examining its interior, find only parts which work one upon another, and never anything by which to explain a perception. Thus it is in a simple substance, and not in a compound or in a machine, that perception must be sought for." (Gottfried W Leibniz, "Monadology", 1714)

"[...] things which do not now exist in the mind itself, can only be perceived, remembered, or imagined, by means of ideas or images of them in the mind, which are the immediate objects of perception, remembrance, and imagination. This doctrine appears evidently to be borrowed from the old system; which taught, that external things make impressions upon the mind, like the impressions of a seal upon wax; that it is by means of those impressions that we perceive, remember) or imagine them; and that those impressions must resemble the things from which they are taken. When we form our notions of the operations of the mind by analogy, this way of conceiving them seems to be very natural, and offers itself to our thoughts: for as every thing which is felt must make some impression upon the body, we are apt to think, that every thing which is understood must make some impression upon the mind." (Thomas Reid, "An Inquiry into the Human Mind", 1734)

"All the perceptions of the human mind resolve themselves into two distinct kinds, which I shall call impressions and ideas. The difference betwixt these consists in the degrees of force and liveliness, with which they strike upon the mind, and make their way into our thought or consciousness. Those perceptions, which enter with most force and violence, we may name impressions; and under this name I comprehend all our sensations, passions and emotions, as they make their first appearance in the soul. By ideas I mean the faint images of these in thinking and reasoning. I believe it will not be very necessary to employ many words in explaining this distinction." (David Hume, "A Treatise of Human Nature", 1738)

"Nay farther, even with relation to that succession, we cou'd only admit of those perceptions, which are immediately present to our consciousness, nor cou'd those lively images, with which the memory presents us, be ever receiv'd as true pictures of past perceptions. The memory, senses, and understanding are, therefore, all of them founded on the imagination, or the vivacity of our ideas.(David Hume, "A Treatise of Human Nature A Treatise of Human Nature", 1739)

"Nay farther, even with relation to that succession, we cou'd only admit of those perceptions, which are immediately present to our consciousness, nor cou'd those lively images, with which the memory presents us, be ever receiv'd as true pictures of past perceptions. The memory, senses, and understanding are, therefore, all of them founded on the imagination, or the vivacity of our ideas.(David Hume, "A Treatise of Human Nature", 1739)

"It seems evident, that men are carried, by a natural instinct or prepossession, to repose faith in their senses; and that, without any reasoning, or even almost before the use of reason, we always suppose an external universe, which depends not on our perception, but would exist, though we and every sensible creature were absent or annihilated. […] It seems also evident, that, when men follow this blind and powerful instinct of nature, they always suppose the very images, presented by the senses, to be the external objects." (David Hume, "An Enquiry Concerning Human Understanding", 1748)

"Show all these fanatics a little geometry, and they learn it quite easily. But, strangely enough, their minds are not thereby rectified. They perceive the truths of geometry, but it does not teach them to weigh probabilities. Their minds have set hard. They will reason in a topsy-turvy wall all their lives, and I am sorry for it." (Voltaire, "False Minds" ["Esprits faux"], A Philosophical Dictionary],1764)

"Things which do not now exist in the mind itself, can only be perceived, remembered, or imagined, by means of the ideas or images in the mind, which are the immediate objects of perception, remembrance, and imagination." (Thomas Reid, "An Inquiry into the Human Mind on the Principles", 1764)

"Psychologists have hitherto failed to realize that imagination is a necessary ingredient of perception itself." (Immanuel Kant, "Critique of Pure Reason", 1781)

🪷On Mind: On Perception (600-1599)

"The faculty which grasps such concepts acquires intelligible forms from sense-perception by force of an inborn disposition, so that forms, which are in the form-bearing faculty and the memorizing faculty, are made present to [the rational soul] with the assistance of the imaginative and estimative [faculties]." (Avicenna Latinus [Ibn Sina], "A Compendium on the Soul", cca. 996-997)

"a [thing’s] likeness often appears and seems to those who perceive it as if the image itself were speaking, as if they heard the words that they held and read." (Avicenna Latinus [Ibn Sina], Liber de Anima, cca. 1014-1027)

"It seems that all perception is but the grasping of the form of the perceived object in some manner. If, then, it is a perception of some material object, it consists in an apprehension of its form by abstracting it from matter in some way. But the kinds of abstraction are different and their degrees various. This is because, owing to matter, the material form is subject to certain states and conditions which do not belong to [the form] by itself insofar as it is this form. So sometimes the abstraction from matter is effected with all or some of these attachments, and sometimes it is complete in that the concept is abstracted from matter and from the accidents it possesses on account of the matter."(Avicenna Latinus [Ibn Sina], "Liber De anima", cca. 1014-1027)

"The animal faculties assist the rational soul in various ways, one of them being that sense-perception brings to it particulars, from which four things result in [the rational soul]: One of them is that the mind extracts single universals from the particulars, by abstracting their concepts from matter and the appendages of matter and its accidents, by considering what is common in it and what different, and what in its existence is essential and what accidental. From this the principles of conceptualization come about [in] the soul: and this with the help of its employing imagination and estimation." (Avicenna Latinus [Ibn Sina], "Liber De Anima", cca. 1014-1027)

"Sometimes a thing is perceived [via sense-perception] when it is observed; then it is imagined, when it is absent [in reality] through the representation of its form inside, Sense-perception grasps [the concept] insofar as it is buried in these accidents that cling to it because of the matter out of which it is made without abstracting it from [matter], and it grasps it only by means of a connection through position [ that exists] between its perception and its matter. It is for this reason that the form of [the thing] is not represented in the external sense when [sensation] ceases. As to the internal [faculty of] imagination, it imagines [the concept] together with these accidents, without being able to entirely abstract it from them. Still, [imagination] abstracts it from the afore-mentioned connection [through position] on which sense-perception depends, so that [imagination] represents the form [of the thing] despite the absence of the form's [outside] carrier." (Avicenna Latinus [Ibn Sina], "Pointer and Reminders", cca. 1030)

"[Intuitive] Understanding is consequent upon deliberation, and firmly embraces the better part. For [intuitive] understanding concerns itself with divine truths, and the relish, love, and observance of the latter constitutes true wisdom. Rather than being the [mere] product of nature, these successive steps are the result of grace. The latter, according to its own free determination, derives the various rivulets of the sciences and wisdom from the fountainhead of sense perception. Grace reveals hidden divine truths by means of those things which have been made, and by that unity which belongs to love, communicates what it has made manifest, thus uniting man to God." (, "Metalogicon", 1159)

"Part of the functions of the imaginative faculty is, as you well know, to retain impressions by the senses, to combine them, and chiefly to form images." (Moses Maimonides,"Guide of the Perplexed", 1190 [translated by Michael Friedländer, 1904])

🪷On Mind: On Perception (-599)

"As regards the question, therefore, what memory or remembering is, it has now been shown that it is the having of an image, related as a likeness to that of which it is an image; and as to the question of which of the faculties within us memory is a function, it has been shown that it is a function of the primary faculty of sense-perception, i.e. of that faculty whereby we perceive time. " (Aristotle, "De Memoria et Reminiscentia" ["On Memory and Recollection"], 4th century BC)

"It is obvious then, that memory belongs to that part of the soul to which imagination belongs. […] Just as the picture painted on the panel is at once a picture and a portrait, and though one and the same, is both, yet the essence of the two is not the same, and it is possible to think of it both as a picture and as a portrait, so in the same way we must regard the mental picture within us both as an object of contemplation in itself and as a mental picture of something else […]. Insofar as we consider it in relation to something else, e.g. as a likeness, it is also an aid to memory." (Aristotle,"De Memoria et Reminiscentia" [On Memory and Recollection], 4th century BC)

"The intellectual capacity thinks the forms in the phantasmata" (mental images) […] And for the following reason, as without having perceptual awareness no one could either learn or understand anything, so when one engages in intellectual activity one must at that time do so by means of a phantasma. For, phantasmata are just as perceptual states" (aisthemata) are [in actual external perception] but without matter." (Aristotle, "De Anima" III, cca. 350 BC)

"The only possible way to conceive universal is by induction, since we come to know abstractions by induction. But unless we have sense experience, we cannot make inductions. Even though sense perception relates to particular things, scientific knowledge concerning such can only be constructed by the successive steps of sense perception, induction, and formulation of universals." (Aristotle, "Posterior Analytics", cca. 350 BC)

"Constantly regard the universe as one living being, having one substance and one soul; and observe how all things have reference to one perception, the perception of this one living being; and how all things act with one movement; and how all things are the cooperating causes of all things which exist; observe too the continuous spinning of the thread and the contexture of the web." (Marcus Aurelius, "Meditations". cca. 121–180 AD)

"In the same way as regards the soul, when that kind of thing in us which mirrors the images of thought and intellect is undisturbed, we see them and know them in a way parallel to sense-perception, along with the prior knowledge that it is intellect and thought that are active. But when this is broken because the harmony of the body is upset, thought and intellect operate without an image, and then intellectual activity takes place without a mind-picture." (Plotinus,"Enneads", cca. 270 AD)

"[...] so external sensation is the image of this perception of the soul, which is in its essence truer and is a contemplation of forms alone without being affected. From these forms, from which the soul alone receives its lordship over the living being, come reasonings, and opinions and noetic acts; and this is precisely where ‘we’ are." (Plotinus, "Enneads", cca. 270 AD)

"We both are, and know that we are, and delight in our being, and our knowledge of it. Moreover, in these three things no true-seeming illusion disturbs us; for we do not come into contact with these by some bodily sense, as we perceive the things outside of us of all which sensible objects it is the images resembling them, but not themselves which we perceive in the mind and hold in the memory, and which excite us to desire the objects. But, without any delusive representation of images or phantasms, I am most certain that I am, and that I know and delight in this." (Aurelius Augustinus, "The City of God", early 400s)

🕸️On Graph Theory: On Neural Networks (2010-2019)

"Neural networks can model very complex patterns and decision boundaries in the data and, as such, are very powerful. In fact, they are so powerful that they can even model the noise in the training data, which is something that definitely should be avoided. One way to avoid this overfitting is by using a validation set in a similar way as with decision trees.[...] Another scheme to prevent a neural network from overfitting is weight regularization, whereby the idea is to keep the weights small in absolute sense because otherwise they may be fitting the noise in the data. This is then implemented by adding a weight size term (e.g., Euclidean norm) to the objective function of the neural network." (Bart Baesens, "Analytics in a Big Data World: The Essential Guide to Data Science and Its Applications", 2014)

"An important property of neural networks is that their weights are set randomly before learning is started, and this random initialization affects the model that is learned. That means that even when using exactly the same parameters, we can obtain very different models when using different random seeds. If the networks are large, and their complexity is chosen properly, this should not affect accuracy too much, but it is worth keeping in mind (particularly for smaller networks)." (Andreas C Müller & Sarah Guido, "Introduction to Machine Learning with Python: A Guide for Data Scientists", 2017)

"Neural networks - particularly the large and powerful ones - often take a long time to train. They also require careful preprocessing of the data, as we saw here. Similarly to SVMs, they work best with 'homogeneous' data, where all the features have similar meanings. For data that has very different kinds of features, tree-based models might work better. Tuning neural network parameters is also an art unto itself. In our experiments, we barely scratched the surface of possible ways to adjust neural network models and how to train them."  (Andreas C Müller & Sarah Guido, "Introduction to Machine Learning with Python: A Guide for Data Scientists", 2017)

"A neural network consists of a set of neurons that are connected together. A neuron takes a set of numeric values as input and maps them to a single output value. At its core, a neuron is simply a multi-input linear-regression function. The only significant difference between the two is that in a neuron the output of the multi-input linear-regression function is passed through another function that is called the activation function." (John D Kelleher & Brendan Tierney, "Data Science", 2018)

"Just as they did thirty years ago, machine learning programs (including those with deep neural networks) operate almost entirely in an associational mode. They are driven by a stream of observations to which they attempt to fit a function, in much the same way that a statistician tries to fit a line to a collection of points. Deep neural networks have added many more layers to the complexity of the fitted function, but raw data still drives the fitting process. They continue to improve in accuracy as more data are fitted, but they do not benefit from the 'super-evolutionary speedup'."  (Judea Pearl & Dana Mackenzie, "The Book of Why: The new science of cause and effect", 2018)

"A neural-network algorithm is simply a statistical procedure for classifying inputs (such as numbers, words, pixels, or sound waves) so that these data can mapped into outputs. The process of training a neural-network model is advertised as machine learning, suggesting that neural networks function like the human mind, but neural networks estimate coefficients like other data-mining algorithms, by finding the values for which the model’s predictions are closest to the observed values, with no consideration of what is being modeled or whether the coefficients are sensible." (Gary Smith & Jay Cordes, "The 9 Pitfalls of Data Science", 2019)

"Deep neural networks have an input layer and an output layer. In between, are “hidden layers” that process the input data by adjusting various weights in order to make the output correspond closely to what is being predicted. [...] The mysterious part is not the fancy words, but that no one truly understands how the pattern recognition inside those hidden layers works. That’s why they’re called 'hidden'. They are an inscrutable black box - which is okay if you believe that computers are smarter than humans, but troubling otherwise." (Gary Smith & Jay Cordes, "The 9 Pitfalls of Data Science", 2019)

"Neural-network algorithms do not know what they are manipulating, do not understand their results, and have no way of knowing whether the patterns they uncover are meaningful or coincidental. Nor do the programmers who write the code know exactly how they work and whether the results should be trusted. Deep neural networks are also fragile, meaning that they are sensitive to small changes and can be fooled easily." (Gary Smith & Jay Cordes, "The 9 Pitfalls of Data Science", 2019)

"The label neural networks suggests that these algorithms replicate the neural networks in human brains that connect electrically excitable cells called neurons. They don’t. We have barely scratched the surface in trying to figure out how neurons receive, store, and process information, so we cannot conceivably mimic them with computers." (Gary Smith & Jay Cordes, "The 9 Pitfalls of Data Science", 2019)

🕸️On Graph Theory: On Neural Networks (2000-2009)

"The most familiar example of swarm intelligence is the human brain. Memory, perception and thought all arise out of the nett actions of billions of individual neurons. As we saw earlier, artificial neural networks (ANNs) try to mimic this idea. Signals from the outside world enter via an input layer of neurons. These pass the signal through a series of hidden layers, until the result emerges from an output layer. Each neuron modifies the signal in some simple way. It might, for instance, convert the inputs by plugging them into a polynomial, or some other simple function. Also, the network can learn by modifying the strength of the connections between neurons in different layers." (David G Green, "The Serendipity Machine: A voyage of discovery through the unexpected world of computers", 2004)

"A neural network is a particular kind of computer program, originally developed to try to mimic the way the human brain works. It is essentially a computer simulation of a complex circuit through which electric current flows." (Keith J Devlin & Gary Lorden, "The Numbers behind NUMB3RS: Solving crime with mathematics", 2007)

 "Neural networks are a popular model for learning, in part because of their basic similarity to neural assemblies in the human brain. They capture many useful effects, such as learning from complex data, robustness to noise or damage, and variations in the data set. " (Peter C R Lane, Order Out of Chaos: Order in Neural Networks, 2007)

"A network of many simple processors ('units' or 'neurons') that imitates a biological neural network. The units are connected by unidirectional communication channels, which carry numeric data. Neural networks can be trained to find nonlinear relationships in data, and are used in various applications such as robotics, speech recognition, signal processing, medical diagnosis, or power systems." (Adnan Khashman et al, "Voltage Instability Detection Using Neural Networks", 2009)

"An artificial neural network, often just called a 'neural network' (NN), is an interconnected group of artificial neurons that uses a mathematical model or computational model for information processing based on a connectionist approach to computation. Knowledge is acquired by the network from its environment through a learning process, and interneuron connection strengths (synaptic weighs) are used to store the acquired knowledge." (Larbi Esmahi et al, "Adaptive Neuro-Fuzzy Systems", 2009)

"ANN is a pattern matching technique that uses training data to build a model and uses the model to predict unknown samples. It consists of input, output, and hidden nodes and connections between nodes. The weights of the connections are iteratively adjusted in order to get an accurate model." (Indranil Bose, "Data Mining in Tourism", 2009)

"Generally, these programs fall within the techniques of reinforcement learning and the majority use an algorithm of temporal difference learning. In essence, this computer learning paradigm approximates the future state of the system as a function of the present state. To reach that future state, it uses a neural network that changes the weight of its parameters as it learns." (Diego Rasskin-Gutman, "Chess Metaphors: Artificial Intelligence and the Human Mind", 2009)

"The simplest basic architecture of an artificial neural network is composed of three layers of neurons - input, output, and intermediary (historically called perceptron). When the input layer is stimulated, each node responds in a particular way by sending information to the intermediary level nodes, which in turn distribute it to the output layer nodes and thereby generate a response. The key to artificial neural networks is in the ways that the nodes are connected and how each node reacts to the stimuli coming from the nodes it is connected to. Just as with the architecture of the brain, the nodes allow information to pass only if a specific stimulus threshold is passed. This threshold is governed by a mathematical equation that can take different forms. The response depends on the sum of the stimuli coming from the input node connections and is 'all or nothing'." (Diego Rasskin-Gutman, "Chess Metaphors: Artificial Intelligence and the Human Mind", 2009)

🕸️On Graph Theory: On Neural Networks (2020-2029)

"GNNs are capable of generating representations of nodes that depend on the structure of the graph as well as on any feature information we have. These features could be nodes’ properties, relationship types, and relationship properties. That’s why GNNs could drive the final tasks to better results. These embeddings represent the input for tasks such as node classification, link prediction, and graph classification." (Alessandro Negro, "Graph-Powered Machine Learning", 2021)

"Graphs, with multiple node types and different types of relationships, are far from being a Euclidean space. This task is where graph neural networks (GNNs) comes in. GNNs are deep learning-based methods that operate on a graph domain to perform complex tasks such as node classification (the bot example), link prediction (the disease example), and so on. Due to its convincing performance, GNN has become a widely applied graph analysis method." (Alessandro Negro, "Graph-Powered Machine Learning", 2021)

"A critical aspect of neural networks is their ability to learn from data. This learning occurs during the training phase, where the network is exposed to vast datasets, allowing it to adjust its internal parameters - the weights and biases associated with each neuron. The goal of this adjustment is to minimize the difference between the network's predictions and the actual outcomes, a process known as optimization. Through techniques such as gradient descent and backpropagation, neural networks iteratively refine their parameters, enhancing their ability to make accurate predictions or decisions based on new input."(Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

"Attention is a mechanism used in deep learning models (not just Transformers) that assigns different weights to different parts of the input, allowing the model to prioritize and emphasize the most important information while performing tasks like translation or summarization. Essentially, attention allows a model to 'focus' on different parts of the input dynamically, leading to improved performance and more accurate results. Before the popularization of attention, most neural networks processed all inputs equally and the models relied on a fixed representation of the input to make predictions. Modern LLMs that rely on attention can dynamically focus on different parts of input sequences, allowing them to weigh the importance of each part in making predictions." (Sinan Ozdemir, "Quick Start Guide to Large Language Models: Strategies and Best Practices for Using ChatGPT and Other LLMs", 2024)

"The beauty of neural networks lies in their ability to learn and improve. Through a process known as 'training', a neural network is fed large amounts of data along with feedback on its performance. This feedback guides the network in adjusting its internal parameters, known as weights, to minimize errors in its predictions. This iterative process of learning from mistakes closely mirrors the cognitive and learning processes of the human brain, making neural networks particularly adept at tasks that involve pattern recognition, such as image and speech recognition [...]" (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

"The power of neural networks lies in their flexibility and adaptability. They are not confined to a single type of problem or dataset but can be tailored to a wide range of applications, from voice recognition and image classification to forecasting financial market movements. This versatility stems from the network's ability to capture and model complex, non-linear relationships within the data it is trained on, making it a potent tool in the arsenal of data scientists and algorithmic traders alike." (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

"Generative AI for coding and language tools is based on the LLM concept. A large language model is a type of neural network that processes and generates text in a humanlike way. It does this by being trained on a massive dataset of text, which allows it to learn human language patterns, as described previously. It lets LLMs translate, write, and answer questions with text. LLMs can contain natural language, source code, and  more." (Jeremy C Morgan, "Coding with AI: Examples in Python", 2025)