[reverett123]

An interesting article on understanding the limitations of scientific studies. It seems that neuroscience is one of the worst.

From http://www.guardian.co.uk/science/si...-power-matters

"When I started my PhD a few years ago, I thought that certain psychological findings were established fact. The next four years were an exercise in disillusionment. If the effects I was seeking to explore were so reliable, so established, why could I not detect them?

There is growing interest in the need to improve reliability in science. Many drugs show promise at the design and pre-clinical phases, only to fail (at great expense) in clinical trials. Many of the most hyped scientific discoveries eventually cannot be replicated.

Worryingly for science (but somewhat comforting for my self-esteem as a researcher) this may be because many of the conclusions drawn from published research findings are false.

A major factor that influences the reliability of science is statistical power. We cannot measure everyone or everything, so we take samples and use statistical inference to determine the probability that the results we observe in our sample reflect some underlying scientific truth. Statistical power determines whether we accurately conclude if there is an effect or not."

[soccertese]

statistics is one of the foundations of the industrial revolution, god forbid if you had to test every loaf of bread, jar of jam or pill that came off the assembly line, you'd pay a lot more for everything. statistics makes the world go around and gives you confidence that what you buy is somewhat safe.

if you give a new cancer treatment to 5 people that are terminal and they live, you don't need statistics. if you have a complete understanding of a biological process and a master of chemistry such that you can develop the perfect drug, you might need statistics. louis pasteur didn't need no stinkin statistics.

but with small differences, you absolutely need statistics to distinguish between the random variability in biological organisms - chance - and the real affect of the treatment you are testing. and knowing there is room for error, scientists alway give you the probability that their conclusion is wrong.
if you are developing a new variety of wheat that has a 1% improvement in yield on average over 4 years and 10 different locations, you need statistics.