"A smaller model with fewer covariates has two advantages: it might give better predictions than a big model and it is more parsimonious (simpler). Generally, as you add more variables to a regression, the bias of the predictions decreases and the variance increases. Too few covariates yields high bias; this called underfitting. Too many covariates yields high variance; this called overfitting. Good predictions result from achieving a good balance between bias and variance. […] fiding a good model involves trading of fit and complexity."
"Mathematics is not about abstract entities alone but is about relation of abstract entities with real entities. […] Adequacy relations between abstract and real entities provide space or opportunity where mathematical and logical thought operates parsimoniously." (Navjyoti Singh, "Classical Indian Mathematical Thought", 2005)
"The model theory postulates that mental models are parsimonious. They represent what is possible, but not what is impossible, according to assertions. This principle of parsimony minimizes the load on working memory, and so it applies unless something exceptional occurs to overrule it." (Philip N Johnson-Laird, Mental Models, Sentential Reasoning, and Illusory Inferences, [in "Mental Models and the Mind"], 2006)
"Two systems concepts lie at the disposal of the architect to reflect the beauty of harmony: parsimony and variety. The law of parsimony states that given several explanations of a specific phenomenon, the simplest is probably the best. […] On the other hand, the law of requisite variety states that for a system to survive in its environment the variety of choice that the system is able to make must equal or exceed the variety of influences that the environment can impose on the system."
"What advantages do diagrams have over verbal descriptions in promoting system understanding? First, by providing a diagram, massive amounts of information can be presented more efficiently. A diagram can strip down informational complexity to its core - in this sense, it can result in a parsimonious, minimalist description of a system. Second, a diagram can help us see patterns in information and data that may appear disordered otherwise. For example, a diagram can help us see mechanisms of cause and effect or can illustrate sequence and flow in a complex system. Third, a diagram can result in a less ambiguous description than a verbal description because it forces one to come up with a more structured description." (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)
"In my view, the argument from parsimony is really no argument at all - it typically functions only to shut down more interesting discussion. If history is any guide, it's never a good idea to assume that a scientific problem is cornered." (David Eagleman, "Incognito: The Secret Lives of the Brain", 2011)
"Scientists often talk of parsimony (as in 'the simplest explanation is probably correct', also known as Occam’s razor), but we should not get seduced by the apparent elegance of argument from parsimony; this line of reasoning has failed in the past at least as many times as it has succeeded. For example, it is more parsimonious to assume that the sun goes around the Earth, that atoms at the smallest scale operate in accordance with the same rules that objects at larger scales follow, and that we perceive what is really out there. All of these positions were long defended by argument from parsimony, and they were all wrong. In my view, the argument from parsimony is really no argument at all - it typically functions only to shut down more interesting discussion. If history is any guide, it’s never a good idea to assume that a scientific problem is cornered." (David Eagleman, "Incognito: The Secret Lives of the Brain", 2011)
No comments:
Post a Comment