Statistical Tools V: Roulette

"As an instrument for selecting at random, I have found nothing superior to dice. It is most tedious to shuffle cards thoroughly be- tween each successive draw, and the method of mixing and stirring up marked balls in a bag is more tedious still. A teetotum or some form of roulette is preferable to these, but dice are better than all. When they are shaken and tossed in a basket, they hurtle so variously against one another and against the ribs of the basket-work that they tumble wildly about, and their positions at the outset afford no perceptible clue to what they will be after even a single good shake and toss." (Francis Galton, Nature vol. 42, 1890)

"In no subject is there a rule, compliance with which will lead to new knowledge or better understanding. Skillful observations, ingenious ideas, cunning tricks, daring suggestions, laborious calculations, all these may be required to advance a subject. Occasionally the conventional approach in a subject has to be studiously followed; on other occasions it has to be ruthlessly disregarded. Which of these methods, or in what order they should be employed is generally unpredictable. Analogies drawn from the history of science are frequently claimed to be a guide; but, as with forecasting the next game of roulette, the existence of the best analogy to the present is no guide whatever to the future. The most valuable lesson to be learnt from the history of scientific progress is how misleading and strangling such analogies have been, and how success has come to those who ignored them." (Thomas Gold, "Cosmology", 1956) 

“[In statistics] you have the fact that the concepts are not very clean. The idea of probability, of randomness, is not a clean mathematical idea. You cannot produce random numbers mathematically. They can only be produced by things like tossing dice or spinning a roulette wheel. With a formula, any formula, the number you get would be predictable and therefore not random. So as a statistician you have to rely on some conception of a world where things happen in some way at random, a conception which mathematicians don’t have.” (Lucien LeCam, [interview] 1988)

"Losing streaks and winning streaks occur frequently in games of chance, as they do in real life. Gamblers respond to these events in asymmetric fashion: they appeal to the law of averages to bring losing streaks to a speedy end. And they appeal to that same law of averages to suspend itself so that winning streaks will go on and on. The law of averages hears neither appeal. The last sequence of throws of the dice conveys absolutely no information about what the next throw will bring. Cards, coins, dice, and roulette wheels have no memory." (Peter L Bernstein, "Against the Gods: The Remarkable Story of Risk", 1996)

"The dice and the roulette wheel, along with the stock market and the bond market, are natural laboratories for the study of risk because they lend themselves so readily to quantification; their language is the language of numbers." (Peter L Bernstein, "Against the Gods: The Remarkable Story of Risk", 1996)

"The theory of probability can define the probabilities at the gaming casino or in a lottery - there is no need to spin the roulette wheel or count the lottery tickets to estimate the nature of the outcome - but in real life relevant information is essential. And the bother is that we never have all the information we would like. Nature has established patterns, but only for the most part. Theory, which abstracts from nature, is kinder: we either have the information we need or else we have no need for information." (Peter L Bernstein, "Against the Gods: The Remarkable Story of Risk", 1996)

“Sequences of random numbers also inevitably display certain regularities. […] The trouble is, just as no real die, coin, or roulette wheel is ever likely to be perfectly fair, no numerical recipe produces truly random numbers. The mere existence of a formula suggests some sort of predictability or pattern.” (Ivars Peterson, “The Jungles of Randomness: A Mathematical Safari”, 1998)

"The chance events due to deterministic chaos, on the other hand, occur even within a closed system determined by immutable laws. Our most cherished examples of chance - dice, roulette, coin-tossing – seem closer to chaos than to the whims of outside events. So, in this revised sense, dice are a good metaphor for chance after all. It's just that we've refined our concept of randomness. Indeed, the deterministic but possibly chaotic stripes of phase space may be the true source of probability." (Ian Stewart, "Does God Play Dice: The New Mathematics of Chaos", 2002)

"People sometimes appeal to the ‘law of averages’ to justify their faith in the gambler’s fallacy. They may reason that, since all outcomes are equally likely, in the long run they will come out roughly equal in frequency. However, the next throw is very much in the short run and the coin, die or roulette wheel has no memory of what went before." (Alan Graham, "Developing Thinking in Statistics", 2006)

"Another kind of error possibly related to the use of the representativeness heuristic is the gambler’s fallacy, otherwise known as the law of averages. If you are playing roulette and the last four spins of the wheel have led to the ball’s landing on black, you may think that the next ball is more likely than otherwise to land on red. This cannot be. The roulette wheel has no memory. The chance of black is just what it always is. The reason people tend to think otherwise may be that they expect the sequence of events to be representative of random sequences, and the typical random sequence at roulette does not have five blacks in a row." (Jonathan Baron, "Thinking and Deciding" 4th Ed, 2008)

"A very different - and very incorrect - argument is that successes must be balanced by failures (and failures by successes) so that things average out. Every coin flip that lands heads makes tails more likely. Every red at roulette makes black more likely. […] These beliefs are all incorrect. Good luck will certainly not continue indefinitely, but do not assume that good luck makes bad luck more likely, or vice versa." (Gary Smith, "Standard Deviations", 2014)

Stephen M Stigler - Collected Quotes

"Beware of the problem of testing too many hypotheses; the more you torture the data, the more likely they are to confess, but confessions obtained under duress may not be admissible in the court of scientific opinion." (Stephen M Stigler, "Testing Hypotheses or fitting Models? Another Look at Mass Extinctions" [in "Neutral Models in Biology"], 1987)

"[…] good statistics re- quires a conversation between scientists and mathematical statisticians." (Stephen M Stigler, Statistics on the Table: the history of statistical concepts and methods, 1999) 

"[…] if statisticians are to be able to understand the limits and generality of their methodology, its worth in different circumstances and the means of adapting it to others, then it will need more than just mathematical statistics, but it will surely not need less. But neither should mathematical statisticians be complacent; above all it is the conversation between theory and applications that is crucially important." (Stephen M Stigler, Statistics on the Table: the history of statistical concepts and methods, 1999)

"No scientific discovery is named after its original discoverer." (Stephen M. Stigler, Statistics on the Table: the history of statistical concepts and methods, 1999) 

"The theory of errors held that a normal population distribution would be produced through the accumulation of a large number of small accidental deviations, and there seemed to be no other way to account for the ubiquitous appearance of that normal outline." (Stephen M. Stigler, Statistics on the Table: the history of statistical concepts and methods, 1999) 

"The recurrence of regression fallacies is testimony to its subtlety, deceptive simplicity, and, I speculate, to the wide use of the word regression to describe least squares fitting of curves, lines, and surfaces." (Stephen M Stigler, Statistics on the Table: the history of statistical concepts and methods, 1999) 

"The whole of the nineteenth- century theory of errors was keyed to this point: observation = truth + error. Without an objective truth, this sort of a split would be impossible, for where would error end and truth begin?" (Stephen M Stigler, Statistics on the Table: the history of statistical concepts and methods, 1999) 

"There was a fundamental difference between the application of statistical methods in astronomy, in experimental psychology, and in the social sciences, and this difference had a profound effect upon the spread of the methods and the pace of their adoption. Astronomy could exploit a theory exterior to the observations, a theory that defined an object for their inference. Truth was-or so they thought-well differentiated from error. Experimental psychologists could, through experimental design, create a baseline for measurement, and control the factors important for their investigation. For them the object of their inference-usually the difference between a treatment and a control group, or between two treatments-was created in the design of the experiment." (Stephen M Stigler, Statistics on the Table: the history of statistical concepts and methods, 1999)

Francis Y Edgeworth - Collected Quotes

"[…] in the Law of Errors we are concerned only with the objective quantities about which mathematical reasoning is ordinarily exercised; whereas in the Method of Least Squares, as in the moral sciences, we are concerned with a psychical quantity - the greatest possible quantity of advantage." (Francis Y Edgeworth, "The method of least squares", 1883)

"It may be replied that the principles of greatest advantage and greatest proba￾bility do not coincide in .qeneral; that here, as in other depart￾ments of action~ when there is a discrepancy between the principle of utility and any other rul% the former should have precedence."  (Francis Y Edgeworth, "The method of least squares", 1883)

"The probable error, the mean error, the mean square of error, are forms divined to resemble in an essential feature the real object of which they are the imperfect symbols - the quantity of evil, the diminution of pleasure, incurred by error. The proper symbol, it is submitted, for the quantity of evil incurred by a simple error is not any power of the error, nor any definite function at all, but an almost arbitrary function, restricted only by the conditions that it should vanish when the independent variable, the error, vanishes, and continually increase with the increase of the error." (Francis Y Edgeworth, "The method of least squares", 1883)

"Our reasoning appears to become more accurate as our ignorance becomes more complete; that when we have embarked upon chaos we seem to drop down into a cosmos."  (Francis Y Edgeworth, "The Philosophy of Chance", Mind Vol. 9, 1884) 

"Probability may be described, agreeably to general usage, as importing partial incomplete belief." (Francis Y Edgeworth, "The Philosophy of Chance", Mind Vol. 9, 1884)

"Observations and statistics agree in being quantities grouped about a Mean; they differ, in that the Mean of observations is real, of statistics is fictitious. The mean of observations is a cause, as it were the source from which diverging errors emanate. The mean of statistics is a description, a representative quantity put for a whole group, the best representative of the group, that quantity which, if we must in practice put one quantity for many, minimizes the error unavoidably attending such practice. Thus measurements by the reduction of which we ascertain a real time, number, distance are observations. Returns of prices, exports and imports, legitimate and illegitimate marriages or births and so forth, the averages of which constitute the premises of practical reasoning, are statistics. In short, observations are different copies of one original; statistics are different originals affording one ‘generic portrait’. Different measurements of the same man are observations; but measurements of different men, grouped round l’homme moyen, are prima facie at least statistics." (Francis Y Edgeworth, 1885)

"What is required for the elimination of chance is not that the raw material of our observations should fulfill the law of error; but that they should be constant to any law." (Francis Y Edgeworth, 1885)

"The Calculus of Probabilities is an instrument which requires the living hand to direct it" (Francis Y Edgeworth, 1887)

"The swarm of probabilities flying hither and thither, does not settle down on any particular point" (Francis Y Edgeworth, 1887)

"However we define error, the idea of calculating its extent may appear paradoxical. A science of errors seems a contradiction in terms." (Francis Y Edgeworth, "The Element of Chance in Competitive Examinations", Journal of the Royal Statistical Society Vol. 53, 1890) 

"What real and permanent tendencies there are lie hid beneath the shifting superfices of chance, as it were a desert in which the inexperienced traveller mistakes the temporary agglomerations of drifting sand for the real configuration of the ground" (Francis Y Edgeworth, 1898)

"[...] the great objection to the geometric mean is its cumbrousness." (Francis Y Edgeworth, 1906)

On Least Squares Method

"From the foregoing we see that the two justifications each leave something to be desired. The first depends entirely on the hypothetical form of the probability of the error; as soon as that form is rejected, the values of the unknowns produced by the method of least squares are no more the most probable values than is the arithmetic mean in the simplest case mentioned above. The second justification leaves us entirely in the dark about what to do when the number of observations is not large. In this case the method of least squares no longer has the status of a law ordained by the probability calculus but has only the simplicity of the operations it entails to recommend it." (Carl Friedrich Gauss, "Anzeige: Theoria combinationis observationum erroribus minimis obnoxiae: Pars prior", Göttingische gelehrte Anzeigen, 1821)

"[…] in the Law of Errors we are concerned only with the objective quantities about which mathematical reasoning is ordinarily exercised; whereas in the Method of Least Squares, as in the moral sciences, we are concerned with a psychical quantity - the greatest possible quantity of advantage." (Francis Y Edgeworth, "The method of least squares", 1883) 

"The method of least squares is used in the analysis of data from planned experiments and also in the analysis of data from unplanned happenings. The word 'regression' is most often used to describe analysis of unplanned data. It is the tacit assumption that the requirements for the validity of least squares analysis are satisfied for unplanned data that produces a great deal of trouble." (George E P Box, "Use and Abuse of Regression", 1966)

"At the heart of probabilistic statistical analysis is the assumption that a set of data arises as a sample from a distribution in some class of probability distributions. The reasons for making distributional assumptions about data are several. First, if we can describe a set of data as a sample from a certain theoretical distribution, say a normal distribution (also called a Gaussian distribution), then we can achieve a valuable compactness of description for the data. For example, in the normal case, the data can be succinctly described by giving the mean and standard deviation and stating that the empirical (sample) distribution of the data is well approximated by the normal distribution. A second reason for distributional assumptions is that they can lead to useful statistical procedures. For example, the assumption that data are generated by normal probability distributions leads to the analysis of variance and least squares. Similarly, much of the theory and technology of reliability assumes samples from the exponential, Weibull, or gamma distribution. A third reason is that the assumptions allow us to characterize the sampling distribution of statistics computed during the analysis and thereby make inferences and probabilistic statements about unknown aspects of the underlying distribution. For example, assuming the data are a sample from a normal distribution allows us to use the t-distribution to form confidence intervals for the mean of the theoretical distribution. A fourth reason for distributional assumptions is that understanding the distribution of a set of data can sometimes shed light on the physical mechanisms involved in generating the data." (John M Chambers et al, "Graphical Methods for Data Analysis", 1983)

"Least squares' means just what it says: you minimise the (suitably weighted) squared difference between a set of measurements and their predicted values. This is done by varying the parameters you want to estimate: the predicted values are adjusted so as to be close to the measurements; squaring the differences means that greater importance is placed on removing the large deviations." (Roger J Barlow, "Statistics: A guide to the use of statistical methods in the physical sciences", 1989)

"Principal components and principal factor analysis lack a well-developed theoretical framework like that of least squares regression. They consequently provide no systematic way to test hypotheses about the number of factors to retain, the size of factor loadings, or the correlations between factors, for example. Such tests are possible using a different approach, based on maximum-likelihood estimation." (Lawrence C Hamilton, "Regression with Graphics: A second course in applied statistics", 1991)

"Fuzzy models should make good predictions even when they are asked to predict on regions that were not excited during the construction of the model. The generalization capabilities can be controlled by an appropriate initialization of the consequences (prior knowledge) and the use of the recursive least squares to improve the prior choices. The prior knowledge can be obtained from the data." (Jairo Espinosa et al, "Fuzzy Logic, Identification and Predictive Control", 2005)

"Often when people relate essentially the same variable in two different groups, or at two different times, they see this same phenomenon - the tendency of the response variable to be closer to the mean than the predicted value. Unfortunately, people try to interpret this by thinking that the performance of those far from the mean is deteriorating, but it’s just a mathematical fact about the correlation. So, today we try to be less judgmental about this phenomenon and we call it regression to the mean. We managed to get rid of the term 'mediocrity', but the name regression stuck as a name for the whole least squares fitting procedure - and that’s where we get the term regression line." (Richard D De Veaux et al, "Stats: Data and Models", 2016)