On Averages II

"An average value is a single value within the range of the data that is used to represent all of the values in the series. Since an average is somewhere within the range of the data, it is sometimes called a measure of central value." (Frederick E Croxton & Dudley J Cowden, "Practical Business Statistics", 1937)

"An average is a single value selected from a group of values to represent them in some way, a value which is supposed to stand for whole group of which it is part, as typical of all the values in the group." (Albert E Waugh, "Elements of Statistical Methods" 3rd Ed., 1952)

"An average does not tell the full story. It is hardly fully representative of a mass unless we know the manner in which the individual items scatter around it. A further description of the series is necessary if we are to gauge how representative the average is." (George Simpson & Fritz Kafka, "Basic Statistics", 1952)

"An average is sometimes called a 'measure of central tendency' because individual values of the variable usually cluster around it. Averages are useful, however, for certain  types of data in which there is little or no central tendency." (William A Spirr & Charles P Bonini, "Statistical Analysis for Business Decisions" 3rd Ed., 1967)

"The most widely used mathematical tools in the social sciences are statistical, and the prevalence of statistical methods has given rise to theories so abstract and so hugely complicated that they seem a discipline in themselves, divorced from the world outside learned journals. Statistical theories usually assume that the behavior of large numbers of people is a smooth, average 'summing-up' of behavior over a long period of time. It is difficult for them to take into account the sudden, critical points of important qualitative change. The statistical approach leads to models that emphasize the quantitative conditions needed for equilibrium-a balance of wages and prices, say, or of imports and exports. These models are ill suited to describe qualitative change and social discontinuity, and it is here that catastrophe theory may be especially helpful." (Alexander Woodcock & Monte Davis, "Catastrophe Theory", 1978)

"The arithmetic mean has another familiar property that will be useful to remember. The sum of the deviations of the values from their mean is zero, and the sum of the squared deviations of the values about the mean is a minimum. That is to say, the sum of the squared deviations is less than the sum of the squared deviations about any other value." (Charles T Clark & Lawrence L Schkade, "Statistical Analysis for Administrative Decisions", 1979)

"Having NUMBERSENSE means: (•) Not taking published data at face value; (•) Knowing which questions to ask; (•) Having a nose for doctored statistics. [...] NUMBERSENSE is that bit of skepticism, urge to probe, and desire to verify. It’s having the truffle hog’s nose to hunt the delicacies. Developing NUMBERSENSE takes training and patience. It is essential to know a few basic statistical concepts. Understanding the nature of means, medians, and percentile ranks is important. Breaking down ratios into components facilitates clear thinking. Ratios can also be interpreted as weighted averages, with those weights arranged by rules of inclusion and exclusion. Missing data must be carefully vetted, especially when they are substituted with statistical estimates. Blatant fraud, while difficult to detect, is often exposed by inconsistency." (Kaiser Fung, "Numbersense: How To Use Big Data To Your Advantage", 2013)

"What is so unconventional about the statistical way of thinking? First, statisticians do not care much for the popular concept of the statistical average; instead, they fixate on any deviation from the average. They worry about how large these variations are, how frequently they occur, and why they exist. [...] Second, variability does not need to be explained by reasonable causes, despite our natural desire for a rational explanation of everything; statisticians are frequently just as happy to pore over patterns of correlation. [...] Third, statisticians are constantly looking out for missed nuances: a statistical average for all groups may well hide vital differences that exist between these groups. Ignoring group differences when they are present frequently portends inequitable treatment. [...] Fourth, decisions based on statistics can be calibrated to strike a balance between two types of errors. Predictably, decision makers have an incentive to focus exclusively on minimizing any mistake that could bring about public humiliation, but statisticians point out that because of this bias, their decisions will aggravate other errors, which are unnoticed but serious. [...] Finally, statisticians follow a specific protocol known as statistical testing when deciding whether the evidence fits the crime, so to speak. Unlike some of us, they don’t believe in miracles. In other words, if the most unusual coincidence must be contrived to explain the inexplicable, they prefer leaving the crime unsolved." (Kaiser Fung, "Numbers Rule the World", 2010) 

"A very different - and very incorrect - argument is that successes must be balanced by failures (and failures by successes) so that things average out. Every coin flip that lands heads makes tails more likely. Every red at roulette makes black more likely. […] These beliefs are all incorrect. Good luck will certainly not continue indefinitely, but do not assume that good luck makes bad luck more likely, or vice versa." (Gary Smith, "Standard Deviations", 2014)

"Average deviation is the average amount of scatter of the items in a distribution from either the mean or the median, ignoring the signs of the deviations. The average that is taken of the scatter is an arithmetic mean, which accounts for the fact that this measure is often called the mean deviation." (Charles T Clark & Lawrence L Schkade)

On Laws I: The Law of Averages

"Except under controlled conditions, or in circumstances where it is possible to ignore individuals and consider only large numbers and the law of averages, any kind of accurate foresight is impossible." (Aldous Huxley, "Time Must Have a Stop", 1944)

"A misunderstanding of Bernoulli’s theorem is responsible for one of the commonest fallacies in the estimation of probabilities, the fallacy of the maturity of chances. When a coin has come down heads twice in succession, gamblers sometimes say that it is more likely to come down tails next time because ‘by the law of averages’ (whatever that may mean) the proportion of tails must be brought right some time." (William Kneale, "Probability and Induction", 1949)

"Only when there is a substantial number of trials involved is the law of averages a useful description or prediction." (Darell Huff, "How to Lie with Statistics", 1954)

"The equanimity of your average tosser of coins depends upon a law, or rather a tendency, or let us say a probability, or at any rate a mathematically calculable chance, which ensures that he will not upset himself by losing too much nor upset his opponent by winning too often." (Tom Stoppard, "Rosencrantz and Guildenstern Are Dead", 1967)

"This faulty intuition as well as many modern applications of probability theory are under the strong influence of traditional misconceptions concerning the meaning of the law of large numbers and of a popular mystique concerning a so-called law of averages." (William Feller, "An Introduction to Probability Theory and Its Applications", 1968)

"I take the view that life is a nonspiritual, almost mathematical property that can emerge from networklike arrangements of matter. It is sort of like the laws of probability; if you get enough components together, the system will behave like this, because the law of averages dictates so. Life results when anything is organized according to laws only now being uncovered; it follows rules as strict as those that light obeys." (Kevin Kelly, "Out of Control: The New Biology of Machines, Social Systems and the Economic World", 1995)

"The slightly chaotic character of mind goes even deeper, to a degree our egos may find uncomfortable. It is very likely that intelligence, at bottom, is a probabilistic or statistical phenomenon — on par with the law of averages. The distributed mass of ricocheting impulses which form the foundation of intelligence forbid deterministic results for a given starting point. Instead of repeatable results, outcomes are merely probabilistic. Arriving at a particular thought, then, entails a bit of luck." (Kevin Kelly, "Out of Control: The New Biology of Machines, Social Systems and the Economic World", 1995)

"Losing streaks and winning streaks occur frequently in games of chance, as they do in real life. Gamblers respond to these events in asymmetric fashion: they appeal to the law of averages to bring losing streaks to a speedy end. And they appeal to that same law of averages to suspend itself so that winning streaks will go on and on. The law of averages hears neither appeal. The last sequence of throws of the dice conveys absolutely no information about what the next throw will bring. Cards, coins, dice, and roulette wheels have no memory." (Peter L Bernstein, "Against the Gods: The Remarkable Story of Risk", 1996)

"However, random walk theory also tells us that the chance that the balance never returns to zero - that is, that H stays in the lead for ever - is 0. This is the sense in which the 'law of averages' is true. If you wait long enough, then almost surely the numbers of heads and tails will even out. But this fact carries no implications about improving your chances of winning, if you're betting on whether H or T turns up. The probabilities are unchanged, and you don't know how long the 'long run' is going to be. Usually it is very long indeed." (Ian Stewart, The Magical Maze: Seeing the world through mathematical eyes", 1997)

"The basis of many misconceptions about probability is a belief in something usually referred to as 'the law of averages', which alleges that any unevenness in random events gets ironed out in the long run. For example, if a tossed coin keeps coming up heads, then it is widely believed that at some stage there will be a predominance of tails to balance things out." (Ian Stewart, The Magical Maze: Seeing the world through mathematical eyes", 1997)

"The 'law of averages' asserts itself not by removing imbalances, but by swamping them. Random walk theory tells us that if you wait long enough - on average, infinitely long - then eventually the numbers will balance out. If you stop at that very instant, then you may imagine that your intuition about a 'law of averages' is justified. But you're cheating: you stopped when you got the answer you wanted. Random walk theory also tells us that if you carry on for long enough, you will reach a situation where the number of H's is a billion more than the number of T's." (Ian Stewart, The Magical Maze: Seeing the world through mathematical eyes", 1997)

"People sometimes appeal to the 'law of averages' to justify their faith in the gambler’s fallacy. They may reason that, since all outcomes are equally likely, in the long run they will come out roughly equal in frequency. However, the next throw is very much in the short run and the coin, die or roulette wheel has no memory of what went before." (Alan Graham, "Developing Thinking in Statistics", 2006)

"Another kind of error possibly related to the use of the representativeness heuristic is the gambler’s fallacy, otherwise known as the law of averages. If you are playing roulette and the last four spins of the wheel have led to the ball’s landing on black, you may think that the next ball is more likely than otherwise to land on red. This cannot be. The roulette wheel has no memory. The chance of black is just what it always is. The reason people tend to think otherwise may be that they expect the sequence of events to be representative of random sequences, and the typical random sequence at roulette does not have five blacks in a row." (Jonathan Baron, "Thinking and Deciding" 4th Ed, 2008)

"The 'law of averages' asserts that an event is more likely if it has not occurred for a long time. Perhaps belief in this bit of folk wisdom is based on confusion of different types of experiments." (Glenn Ledder, "Mathematics for the Life Sciences: Calculus, Modeling, Probability, and Dynamical Systems", 2013)

"A very different - and very incorrect - argument is that successes must be balanced by failures (and failures by successes) so that things average out. Every coin flip that lands heads makes tails more likely. Every red at roulette makes black more likely. […] These beliefs are all incorrect. Good luck will certainly not continue indefinitely, but do not assume that good luck makes bad luck more likely, or vice versa." (Gary Smith, "Standard Deviations", 2014)

"[…] many gamblers believe in the fallacious law of averages because they are eager to find a profitable pattern in the chaos created by random chance." (Gary Smith, "Standard Deviations", 2014)

On Nonlinearity III

"Finite systems of deterministic ordinary nonlinear differential equations may be designed to represent forced dissipative hydrodynamic flow. Solutions of these equations can be identified with trajectories in phase space. For those systems with bounded solutions, it is found that nonperiodic solutions are ordinarily unstable with respect to small modifications, so that slightly differing initial states can evolve into considerably different states. Systems with bounded solutions are shown to possess bounded numerical solutions. (Edward N Lorenz, "Deterministic Nonperiodic Flow", Journal of the Atmospheric Science 20, 1963)

"We've seen that even in the simplest situations nonlinearities can interfere with a linear approach to aggregates. That point holds in general: nonlinear interactions almost always make the behavior of the aggregate more complicated than would be predicted by summing or averaging." (Lewis Mumford, "The Myth of the Machine" Vol 1, 1967)

"The structure of a complex system is not a simple feedback loop where one system state dominates the behavior. The complex system has a multiplicity of interacting feedback loops. Its internal rates of flow are controlled by non‐linear relationships. The complex system is of high order, meaning that there are many system states (or levels). It usually contains positive‐feedback loops describing growth processes as well as negative, goal‐seeking loops." (Jay F Forrester, "Urban Dynamics", 1969)

"I would therefore urge that people be introduced to [the logistic equation] early in their mathematical education. This equation can be studied phenomenologically by iterating it on a calculator, or even by hand. Its study does not involve as much conceptual sophistication as does elementary calculus. Such study would greatly enrich the student’s intuition about nonlinear systems. Not only in research but also in the everyday world of politics and economics, we would all be better off if more people realized that simple nonlinear systems do not necessarily possess simple dynamical properties." (Robert M May, "Simple Mathematical Models with Very Complicated Dynamics", Nature Vol. 261 (5560), 1976)

"Most physical systems, particularly those complex ones, are extremely difficult to model by an accurate and precise mathematical formula or equation due to the complexity of the system structure, nonlinearity, uncertainty, randomness, etc. Therefore, approximate modeling is often necessary and practical in real-world applications. Intuitively, approximate modeling is always possible. However, the key questions are what kind of approximation is good, where the sense of 'goodness' has to be first defined, of course, and how to formulate such a good approximation in modeling a system such that it is mathematically rigorous and can produce satisfactory results in both theory and applications." (Guanrong Chen & Trung Tat Pham, "Introduction to Fuzzy Sets, Fuzzy Logic, and Fuzzy Control Systems", 2001) 

"Thus, nonlinearity can be understood as the effect of a causal loop, where effects or outputs are fed back into the causes or inputs of the process. Complex systems are characterized by networks of such causal loops. In a complex, the interdependencies are such that a component A will affect a component B, but B will in general also affect A, directly or indirectly.  A single feedback loop can be positive or negative. A positive feedback will amplify any variation in A, making it grow exponentially. The result is that the tiniest, microscopic difference between initial states can grow into macroscopically observable distinctions." (Carlos Gershenson, "Design and Control of Self-organizing Systems", 2007)

"Where simplifications fail, causing the most damage, is when something nonlinear is simplified with the linear as a substitute. That is the most common Procrustean bed." (Nassim N Taleb, "Antifragile: Things that Gain from Disorder", 2012)

"Complex systems defy intuitive solutions. Even a third-order, linear differential equation is unsolvable by inspection. Yet, important situations in management, economics, medicine, and social behavior usually lose reality if simplified to less than fifth-order nonlinear dynamic systems. Attempts to deal with nonlinear dynamic systems using ordinary processes of description and debate lead to internal inconsistencies. Underlying assumptions may have been left unclear and contradictory, and mental models are often logically incomplete. Resulting behavior is likely to be contrary to that implied by the assumptions being made about' underlying system structure and governing policies." (Jay W Forrester, "Modeling for What Purpose?", The Systems Thinker Vol. 24 (2), 2013)

"There is no linear additive process that, if all the parts are taken together, can be understood to create the total system that occurs at the moment of self-organization; it is not a quantity that comes into being. It is not predictable in its shape or subsequent behavior or its subsequent qualities. There is a nonlinear quality that comes into being at the moment of synchronicity." (Stephen H Buhner, "Plant Intelligence and the Imaginal Realm: Beyond the Doors of Perception into the Dreaming of Earth", 2014)

"Exponentially growing systems are prevalent in nature, spanning all scales from biochemical reaction networks in single cells to food webs of ecosystems. How exponential growth emerges in nonlinear systems is mathematically unclear. […] The emergence of exponential growth from a multivariable nonlinear network is not mathematically intuitive. This indicates that the network structure and the flux functions of the modeled system must be subjected to constraints to result in long-term exponential dynamics." (Wei-Hsiang Lin et al, "Origin of exponential growth in nonlinear reaction networks", PNAS 117 (45), 2020)

Misquoted: Herbert G Wells on Mathematical Literacy

"Statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write."

The above quote on statistical literacy is often attributed to Herbert G Wells though it belongs to the statistician Samuel S Wilks, who in a 1951 presidential address was paraphrasing Wells:

"Perhaps H. G. Wells was right when he said ‘statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write’!" [4]

The original quote comes from “Mankind in the Making”, first published in 1903 (and not in 1911 as Wikipedia states):

"The great body of physical science, a great deal of the essential fact of financial science, and endless social and political problems are only accessible and only thinkable to those who have had a sound training in mathematical analysis, and the time may not be very remote when it will be understood that for complete initiation as an efficient citizen of one of the new great complex world-wide States that are now developing, it is as necessary to be able to compute, to think in averages and maxima and minima, as it is now to be able to read and write." [1]

Even if Wells mentions averages, maxima and minima, tools of statistics, the text refers to mathematical analysis and not statistics. Wilk’s paraphrasing makes sense in nowadays contexts, and seems somehow natural, even if statistical literacy is more about understanding and (critically) evaluating statements that involve rates and percentages.

Another paraphrasing of the same quote and probably closer to the essence of statistical literacy can be found in George A Lundberg paper published in 1940, however without giving credit to Wells:

"The time is perhaps at hand when it will be recognized that for intelligent living in modern society it is as necessary to be able to think in averages, percentages, and deviations as it is to be able to read and write." [2]

Previous Post <<||>> Next Post

References:
[1] “Mankind in the Making”, by Herbert G Wells, 1903 [Source]
[2] “Statistics in Modern Social Thought”, by George A Lundberg [in “Contemporary Social Theory”, Ed. by H. E. Barnes, H. Becker & F. Becker, 1940] [Source]
[3] “The H. G. Wells Quote on Statistics: A Question of Accuracy”, by James W Tankard Jr., Historia Mathematics 6, 1979 [Source]
[4] “Undergraduate Statistical Education”, by  Samuel S Wilks, Journal of the American Statistical Association, Vol. 46, 1951 [Source]

On Statistics: Twisted Statistics

"[…] the question ‘How many legs does a normal man have?’ should be answered by finding a statistical average. And since some men have only one leg, or none, this would lead inevitably to the conclusion that a ‘normal’ man is equipped with one and some fraction legs." (Joseph W Krutch, "Human Nature and the Human Condition", 1959)

"It is a well-known statistical paradox that the average age of women over forty is under forty…" (Morris J Slonim, "Sampling in a Nutshell, 1960)

"The statistics on insanity are that one out of every four people is suffering from some form of mental illness. Think of your three best friends. If they're okay, then it's got to be you." (Rita Mae Brown) [in: Susan Musgrave, "Musgrave Landing: Musings on the Writing Life", 1994)

"It is now proved beyond doubt that smoking is one of the leading causes of statistics." (Fletcher Knebel, 1961)

"Doing applied statistics is never easy, especially if you want to get it right." (Xiao-Li Meng,  [Joint Statistical Meetings], 2005) 

"A Bayesian is one who, vaguely expecting a horse, and catching a glimpse of a donkey, strongly believes he has seen a mule." (Stephen J Senn)

"According to the statistics, a man eats a prune every twenty seconds. I don't know who this fellow is, but I know where to find him." (Morey Amsterdam)

"If a man stands with his left foot on a hot stove and his right foot in a refrigerator, the statistician would say that, on the average, he’s comfortable." (Walter Heller)

"It is proven that the celebration of birthdays is healthy. Statistics show that those people who celebrate the most birthdays become the oldest." (S. den Hartog)

"If there is a 50-50 chance that something can go wrong, then 9 times out of ten it will." (Paul Harvey)

"Statistics show that 66% of clients are cured with psychotherapy; what statistics don't show is that 72% are cured without it." (Thomas Szasz)

"Statistics show that of those who contract the habit of eating, very few survive." (William W Irwin)

"Statistics show that teen pregnancy drops off significantly after age 2." (MaryAnne Tebedo)

"'Statistics' show that 66% of clients are cured with psychotherapy; what statistics don't show is that 72% are cured without it." (Thomas Szasz)

On Averages I

“It is difficult to understand why statisticians commonly limit their inquiries to Averages, and do not revel in more comprehensive views. Their souls seem as dull to the charm of variety as that of the native of one of our flat English counties, whose retrospect of Switzerland was that, if its mountains could be thrown into its lakes, two nuisances would be got rid of at once. An Average is but a solitary fact, whereas if a single other fact be added to it, an entire Normal Scheme, which nearly corresponds to the observed one, starts potentially into existence.” (Sir Francis Galton, “Natural Inheritance”, 1889)

“Statistics may rightly be called the science of averages. […] Great numbers and the averages resulting from them, such as we always obtain in measuring social phenomena, have great inertia. […] It is this constancy of great numbers that makes statistical measurement possible. It is to great numbers that statistical measurement chiefly applies.” (Sir Arthur L Bowley, “Elements of Statistics”, 1901)

“[…] the new mathematics is a sort of supplement to language, affording a means of thought about form and quantity and a means of expression, more exact, compact, and ready than ordinary language. The great body of physical science, a great deal of the essential facts of financial science, and endless social and political problems are only accessible and only thinkable to those who have had a sound training in mathematical analysis, and the time may not be very remote when it will be understood that for complete initiation as an efficient citizen of one of the new great complex world wide states that are now developing, it is as necessary to be able to compute, to think in averages and maxima and minima, as it is now to be able to read and to write.” (Herbert G Wells, “Mankind In the Making”, 1906)

“Of itself an arithmetic average is more likely to conceal than to disclose important facts; it is the nature of an abbreviation, and is often an excuse for laziness.” (Arthur L Bowley, “The Nature and Purpose of the Measurement of Social Phenomena”, 1915)

“Averages are like the economic man; they are inventions, not real. When applied to salaries they hide gaunt poverty at the lower end.” (Julia Lathrop, 1919)

“Scientific laws, when we have reason to think them accurate, are different in form from the common-sense rules which have exceptions: they are always, at least in physics, either differential equations, or statistical averages.” (Bertrand A Russell, “The Analysis of Matter”, 1927)

"An average is a single value which is taken to represent a group of values. Such a representative value may be obtained in several ways, for there are several types of averages. […] Probably the most commonly used average is the arithmetic average, or arithmetic mean." (John R Riggleman & Ira N Frisbee, "Business Statistics", 1938)

"Because they are determined mathematically instead of according to their position in the data, the arithmetic and geometric averages are not ascertained by graphic methods." (John R Riggleman & Ira N Frisbee, "Business Statistics", 1938)

“Myth is more individual and expresses life more precisely than does science. Science works with concepts of averages which are far too general to do justice to the subjective variety of an individual life.” (Carl G Jung, “Memories, Dreams, Reflections”, 1963)

“While the individual man is an insoluble puzzle, in the aggregate he becomes a mathematical certainty. You can, for example, never foretell what anyone man will be up to, but you can say with precision what an average number will be up to. Individuals vary, but percentages remain constant. So says the statistician.” (Sir Arthur C Doyle)

On Statistics: Some Historical Definitions (1901-1950)

"[…] statistics is the science of the measurement of the social organism, regarded as a whole, in all its manifestations." (Sir Arthur L Bowley, "Elements of Statistics", 1901)

"Statistics may rightly be called the science of averages. […] Great numbers and the averages resulting from them, such as we always obtain in measuring social phenomena, have great inertia. […] It is this constancy of great numbers that makes statistical measurement possible. It is to great numbers that statistical measurement chiefly applies." (Sir Arthur L Bowley, "Elements of Statistics", 1901)

"Statistics may, for instance, be called the science of counting. Counting appears at first sight to be a very simple operation, which any one can perform or which can be done automatically; but, as a matter of fact, when we come to large numbers, e.g., the population of the United Kingdom, counting is by no means easy, or within the power of an individual; limits of time and place alone prevent it being so carried out, and in no way can absolute accuracy be obtained when the numbers surpass certain limits." (Sir Arthur L Bowley, "Elements of Statistics", 1901)

"[...] statistics is the science that, through calculation, leads to an understanding of the characteristics of human societies, and its purpose is the study of masses through the enumeration of the units that compose them." (Armand Julin, "Summary for a Course of Statistics, General and Applied, 1910)

"The science of Statistics is the method of judging collective, natural or social phenomenon from the results obtained from the analysis or enumeration or collection of estimates." (Willford I King, "The Elements of Statistical Method", 1912)

"By Statistics we mean aggregate of facts affected to a marked extent by multiplicity of factors [...] and placed in relation to each other." (Horace Secrist, "An Introduction to Statistical Methods", 1917)

"Statistics may be defined as numerical statements of facts by means of which large aggregates are analyzed, the relations of individual units to their groups are ascertained, comparisons are made between groups, and continuous records are maintained for comparative purposes." (Melvin T Copeland. "Statistical Methods" [in: Harvard Business Studies, Vol. III, Ed. by Melvin T Copeland, 1917])

"Statistics may be regarded as (i) the study of populations, (ii) as the study of variation, and (iii) as the study of methods of the reduction of data." (Sir Ronald A Fisher, "Statistical Methods for Research Worker", 1925)

"Statistics is a scientific discipline concerned with collection, analysis, and interpretation of data obtained from observation or experiment. The subject has a coherent structure based on the theory of Probability and includes many different procedures which contribute to research and development throughout the whole of Science and Technology." (Egon Pearson, 1936)

"[Statistics] is both a science and an art. It is a science in that its methods are basically systematic and have general application; and an art in that their successful application depends to a considerable degree on the skill and special experience of the statistician, and on his knowledge of the field of application, e.g. economics." (Leonard H C Tippett, "Statistics", 1943)

"Statistics is the branch of scientific method which deals with the data obtained by counting or measuring the properties of populations of natural phenomena. In this definition 'natural phenomena' includes all the happenings of the external world, whether human or not " (Sir Maurice G Kendall, "Advanced Theory of Statistics", Vol. 1, 1943)

"To some people, statistics is ‘quartered pies, cute little battleships and tapering rows of sturdy soldiers in diversified uniforms’. To others, it is columns and columns of numerical facts. Many regard it as a branch of economics. The beginning student of the subject considers it to be largely mathematics." (The Editors, "Statistics, The Physical Sciences and Engineering", The American Statistician, Vol. 2, No. 4, 1948) [Link]

Further definitions:
1800-1900
1951-2000
2001- …

Statistics – Some Historical Definitions (1800-1900)

“Il y a, dans la statistique, deux choses qui se trouvent continuellement mélangées, und methode et une science. On emploie la statistique comme methode, toutes les fois que l’on compte on que l’on mesure quelque chose, par example, l’éndendue d’un district, le mobre de habitants d’un pays, la quantité ou le prix de certaines denrées, etc. […] Il y a, de plus, une science de la statistique. Elle consiste à savoir réunir les chiffres, les combiner et les calculer, de la manière la plus propre à conduire à des résultats certains. Mais ceci n’est, à propement parler qu’une branche de mathémetiques.“ (Alphonse P de Candolle, “Considérations sur le statistique des délits” 1833)

“There are two aspects of statistics that are continually mixed, the method and the science. Statistics are used as a method, whenever we measure something, for example, the size of a district, the number of inhabitants of a country, the quantity or price of certain commodities, etc. […] There is, moreover, a science of statistics. It consists of knowing how to gather numbers, combine them and calculate them, in the best way to lead to certain results. But this is, strictly speaking, a branch of mathematics." (Alphonse P de Candolle, “Considerations on Crime Statistics”, 1833)

"[statistics is] that department of political science which is concerned in collecting and arranging facts illustrative of the condition and resources of the state. To reason upon such facts and to draw conclusions from them is not within the province of statistics; but is the business of the statesman and of the political economist." (Penny Cyclopedia, 1842)

“Statistics has then for its object that of presenting a faithful representation of a state at a determined epoch.” (Adolphe Quetelet, 1849) 

"Observations and statistics agree in being quantities grouped about a Mean; they differ, in that the Mean of observations is real, of statistics is fictitious. The mean of observations is a cause, as it were the source from which diverging errors emanate. The mean of statistics is a description, a representative quantity put for a whole group, the best representative of the group, that quantity which, if we must in practice put one quantity for many, minimizes the error unavoidably attending such practice. Thus measurements by the reduction of which we ascertain a real time, number, distance are observations. Returns of prices, exports and imports, legitimate and illegitimate marriages or births and so forth, the averages of which constitute the premises of practical reasoning, are statistics. In short, observations are different copies of one original; statistics are different originals affording one ‘generic portrait’. Different measurements of the same man are observations; but measurements of different men, grouped round l’homme moyen, are prima facie at least statistics." (Francis Y Edgeworth, 1885) 

“[Statistics] are the only tools by which an opening can be cut through the formidable thicket of difficulties that bars the path of those who pursue the Science of man.” (Sir Francis Galton, “Natural Inheritance”, 1889)

Further definitions:
1901-1950
1951-2000
2001- …