"A basic problem with MRA is that it typically assumes that the independent variables can be regarded as building blocks, with each variable taken by itself being logically independent of all the others. This is usually not the case, at least for behavioral data. […] Just as correlation doesn’t prove causation, absence of correlation fails to prove absence of causation. False-negative findings can occur using MRA just as false-positive findings do - because of the hidden web of causation that we’ve failed to identify."
"Deductive and inductive reasoning schemas essentially
regulate inferences. They tell us what kinds of inferences are valid and what
kinds are invalid. […] Dialectical reasoning isn’t formal or deductive and
usually doesn’t deal in abstractions. It’s concerned with reaching true and
useful conclusions rather than valid conclusions. In fact, conclusions based on
dialectical reasoning can actually be opposed to those based on formal logic."
"Dialectical thinking opposes formalism because of its separation
of form from content. We make errors by abstracting the elements of a problem
into a formal model and ignoring facts and contexts crucial to correct
analysis. Overemphasis on logical approaches leads to distortion, error, and
rigidity."
"Multiple regression analysis (MRA) examines the association between an independent variable and a dependent variable, controlling for the association between the independent variable and other variables, as well as the association of those other variables with the dependent variable. The method can tell us about causality only if all possible causal influences have been identified and measured reliably and validly. In practice, these conditions are rarely met." (Richard E Nisbett, "Mindware: Tools for Smart Thinking", 2015)
"One technique employing correlational analysis is multiple regression analysis (MRA), in which a number of independent variables are correlated simultaneously (or sometimes sequentially, but we won’t talk about that variant of MRA) with some dependent variable. The predictor variable of interest is examined along with other independent variables that are referred to as control variables. The goal is to show that variable A influences variable B 'net of' the effects of all the other variables. That is to say, the relationship holds even when the effects of the control variables on the dependent variable are taken into account."
"Science is often described as a 'seamless web'. What’s meant
by that is that the facts, methods, theories, and rules of inference discovered
in one field can be helpful for other fields. And philosophy and logic can
affect reasoning in literally every field of science."
"The closer that sample-selection procedures approach the
gold standard of random selection - for which the definition is that every
individual in the population has an equal chance of appearing in the sample - the
more we should trust them. If we don’t know whether a sample is random, any
statistical measure we conduct may be biased in some unknown way."
"The correlational technique known as multiple regression is used frequently in medical and social science research. This technique essentially correlates many independent (or predictor) variables simultaneously with a given dependent variable (outcome or output). It asks, 'Net of the effects of all the other variables, what is the effect of variable A on the dependent variable?' Despite its popularity, the technique is inherently weak and often yields misleading results. The problem is due to self-selection. If we don’t assign cases to a particular treatment, the cases may differ in any number of ways that could be causing them to differ along some dimension related to the dependent variable. We can know that the answer given by a multiple regression analysis is wrong because randomized control experiments, frequently referred to as the gold standard of research techniques, may give answers that are quite different from those obtained by multiple regression analysis." (Richard E Nisbett, "Mindware: Tools for Smart Thinking", 2015)
"The fundamental problem with MRA, as with all correlational methods, is self-selection. The investigator doesn’t choose the value for the independent variable for each subject (or case). This means that any number of variables correlated with the independent variable of interest have been dragged along with it. In most cases, we will fail to identify all these variables. In the case of behavioral research, it’s normally certain that we can’t be confident that we’ve identified all the plausibly relevant variables."
"The theory behind multiple regression analysis is that if you control for everything that is related to the independent variable and the dependent variable by pulling their correlations out of the mix, you can get at the true causal relation between the predictor variable and the outcome variable. That’s the theory. In practice, many things prevent this ideal case from being the norm."
"We are superb causal-hypothesis generators. Given an effect,
we are rarely at a loss for an explanation. Seeing a difference in observations
over time, we readily come up with a causal interpretation. Much of the time, no
causality at all is going on—just random variation. The compulsion to explain is
particularly strong when we habitually see that one event typically occurs in conjunction
with another event. Seeing such a correlation almost automatically provokes a
causal explanation. It’s tremendously useful to be on our toes looking for
causal relationships that explain our world. But there are two problems: (1) The
explanations come too easily. If we recognized how facile our causal hypotheses
were, we’d place less confidence in them. (2) Much of the time, no causal interpretation
at all is appropriate and wouldn’t even be made if we had a better understanding
of randomness."
"We don’t recognize how easy it is to generate hypotheses
about the world. If we did, we’d generate fewer of them, or at least hold them
more tentatively. We sprout causal theories in abundance when we learn of a
correlation, and we readily find causal explanations for the failure of the
world to confirm our hypotheses. We don’t realize how easy it is for us to explain
away evidence that would seem on the surface to contradict our hypotheses. And
we fail to generate tests of a hypothesis that could falsify the hypothesis if
in fact the hypothesis is wrong. This is one type of confirmation bias."
No comments:
Post a Comment