Discussion World Forum  


Go Back   Discussion World Forum > Discussion Forums > Science

Science From your kid's science project to relativity, this is the place to discuss it.

Reply
 
Thread Tools Display Modes
  #1  
Old Aug 28th 2015, 08:28 AM
dilettante's Avatar
dilettante dilettante is offline
Moderator
Resident Historian
 
Join Date: Oct 2008
Location: Philadelphia
Posts: 3,082
Default Experimental reproducibility ...or not

Heard about this study on NPR this morning. Somewhat distressing!

Quote:
More than half of psychology papers are not reproducible

The Reproducibility Project: Psychology was launched in 2011 by the Center for Open Science, a US non-profit organisation, in the wake of a number of fraud scandals in psychology, such as that involving Diederik Stapel, who admitted in 2011 to faking more than 50 papers.

One hundred papers were chosen from 2008 issues of three important journals in psychology, and about 350 scientists were involved in meticulous attempts to reproduce them....

The results of the project are reported in a paper, “Estimating the reproducibility of psychological science”, published in the journal Science.

Based on five different measures of reproducibility, the paper concludes that reproduction was successful in less than half of cases. For instance, while 97 per cent of the original studies reported statistically significant results, only 36 per cent of replications did so.
You can see the structured abstract of the actual article in Science here:
http://www.sciencemag.org/content/34...c4716.abstract

From the results section:
Quote:
Here, we evaluated reproducibility using significance and P values, effect sizes, subjective assessments of replication teams, and meta-analysis of effect sizes. The mean effect size (r) of the replication effects (Mr = 0.197, SD = 0.257) was half the magnitude of the mean effect size of the original effects (Mr = 0.403, SD = 0.188), representing a substantial decline. Ninety-seven percent of original studies had significant results (P < .05). Thirty-six percent of replications had significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects.
__________________
kyrie eleison
Reply With Quote
  #2  
Old Aug 30th 2015, 06:55 PM
kowalskil's Avatar
kowalskil kowalskil is offline
Villager
 
Join Date: Aug 2010
Location: New Jersey, USA
Posts: 67
Default Re: Experimental reproducibility ...or not

You write: "More than half of psychology papers are not reproducible

The Reproducibility Project: Psychology was launched in 2011 by the Center for Open Science, a US non-profit organisation, in the wake of a number of fraud scandals in psychology, such as that involving Diederik Stapel, who admitted in 2011 to faking more than 50 papers.

One hundred papers were chosen from 2008 issues of three important journals in psychology, and about 350 scientists were involved in meticulous attempts to reproduce them...." I was not aware that the situation was so bad.
__________________
Ludwik Kowalski, author of a free ON-LINE book entitled “Diary of a Former Communist: Thoughts, Feelings, Reality.”

http://csam.montclair.edu/~kowalski/life/intro.html

It is a testimony based on a diary kept between 1946 and 2004 (in the USSR, Poland, France and the USA).
Reply With Quote
  #3  
Old Sep 6th 2015, 11:58 AM
Michael's Avatar
Michael Michael is offline
Administrator
Herder of Cats
 
Join Date: Oct 2008
Location: Toronto
Posts: 14,819
Default Re: Experimental reproducibility ...or not

Quote:
Originally Posted by dilettante View Post
Heard about this study on NPR this morning. Somewhat distressing!
Distressing yes, but not at all surprising. I used to read quite a bit about theory and methodology in social sciences and the related statistical inferrences drawn from the data (long time ago). Ugly subject. They usually failed most often by not having an actually random sample. Sometimes doing things right just costs too much, or is too inconvenient or is impossible, so alternative ways are used. It used to be a joke that college students were the most heavily studied people in the world - just because they were the most accessible participants for so many academic studies.
__________________
Remember what the dormouse said: Feed your head!
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -4. The time now is 06:33 PM.


Powered by vBulletin® Version 3.7.2
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
Copyright ©2008 - 2017, DiscussionWorldForum.com