"A bad model is a combination of assertions, some factual, others conjectural, and others plainly false but convenient. [..] By definition, a bad model does not give power to see-accurately, deeply, or at all-into the actual situa- tion, but only into the assertions embodied in the model. Thus, if the use of a bad model provides insight, it does so not by revealing truth about the world but by revealing its own assumptions and thereby causing its user to go learn something about the world." (James S Hodges, "Six (or So) Things You Can Do with a Bad Model", 1991)
"Just because a model is bad, however, does not mean it is useless. [...] A bad model can be used to construct correct paths from premises to conclusions, but because its relations to reality are questionable, it can only do so in a few ways-at least, ways that permit useful conclusions with respect to reality." (James S Hodges, "Six (or So) Things You Can Do with a Bad Model", 1991)
"Often, though, a policy or systems analyst is stuck with a bad model, that is, one that appeals to the analyst as adequately realistic but which is either: 1) contradicted by some data or is grossly implausible in some aspect it purports to represent, or 2) conjectural, that is, neither supported nor contradicted by data, either because data do not exist or because they are equivocal. [...] A model may have component parts that are not bad, but if, taken as a whole, it meets one of these criteria, it is a bad model." (James S Hodges, "Six (or So) Things You Can Do with a Bad Model", 1991)
"Some readers have argued that the criticism implied by the term "bad models" is undeserved because they can be used appropriately in some cases. [...] If the logic works, the use is appropriate; if it fails, the use in inappropriate: (Cost effectiveness is a separate issue.) As for the pejorative connotation of the term bad model, perhaps we should admit that many useful models would be embarrassments to scientists, from whom we got the idea of a model, but whose job is to improve the match between models and reality." (James S Hodges, "Six (or So) Things You Can Do with a Bad Model", 1991)
"The definition of a ‘good model’ is when everything inside it is visible, inspectable and testable. It can be communicated effortlessly to others. A ‘bad model’ is a model that does not meet these standards, where parts are hidden, undefined or concealed and it cannot be inspected or tested; these are often labelled black box models." (Hördur V Haraldsson & Harald U Sverdrup,Finding Simplicity in Complexity in Biogeochemical Modelling" [inEnvironmental Modelling: Finding Simplicity in Complexity", Ed. by John Wainwright and Mark Mulligan, 2004])
"It is not always convenient to remember that the right model for a population can fit a sample of data worse than a wrong model - even a wrong model with fewer parameters. We cannot rely on statistical diagnostics to save us, especially with small samples. We must think about what our models mean, regardless of fit, or we will promulgate nonsense." (Leland Wilkinson,The Grammar of Graphics" 2nd Ed., 2005)
"Sometimes the most important fit statistic you can get is ‘convergence not met’ - it can tell you something is wrong with your model." (Oliver Schabenberger, "Applied Statistics in Agriculture Conference", 2006)
"An attempt to use the wrong model for a given data set is likely to provide poor results. Therefore, the core principle of discovering outliers is based on assumptions about the structure of the normal patterns in a given data set. Clearly, the choice of the 'normal' model depends highly upon the analyst’s understanding of the natural data patterns in that particular domain." (Charu C Aggarwal,Outlier Analysis", 2013)
"Things which ought to be expected can seem quite extraordinary if you’ve got the wrong model." (David Hand,Significance", 2014)
"Model-building requires much more than just technical knowledge of statistical ideas. It also requires care and judgment, and cannot be reduced to a flowchart, a table of formulas, or a tidy set of numerical summaries that wring every last drop of truth from a data set. There is almost never a single 'right' statistical model for some problem. But there are definitely such things as good models and bad models, and learning to tell the difference is important. Just remember: calling a model good or bad requires knowing both the tool and the task." (James G Scott, "Statistical Modeling: A Gentle Introduction", 2017)
"Overfitting and underfitting are two important factors that could impact the performance of machine-learning models. Overfitting occurs when the model performs well with training data and poorly with test data. Underfitting occurs when the model is so simple that it performs poorly with both training and test data. [...] When the model does not capture and fit the data, it results in poor performance. We call this underfitting. Underfitting is the result of a poor model that typically does not perform well for any data." (Umesh R Hodeghatta & Umesha Nayak,Business Analytics Using R: A Practical Approach", 2017)
"Mathematicians love math and many non-mathematicians are intimidated by math. This is a lethal combination that can lead to the creation of wildly unrealistic mathematical models. [...] A good mathematical model starts with plausible assumptions and then uses mathematics to derive the implications. A bad model focuses on the math and makes whatever assumptions are needed to facilitate the math." (Gary Smith & Jay Cordes, "The 9 Pitfalls of Data Science", 2019)
"Bad data makes bad models. Bad models instruct people to make ineffective or harmful interventions. Those bad interventions produce more bad data, which is fed into more bad models." (Cory Doctorow,Machine Learning’s Crumbling Foundations", 2021)
