The People vs. Academia: Crowdsourcing the analysis of sleep

Brainwaves during sleep are a window into cognition, early marker of mental disorders, and brain deterioration due to age. However, detection of brainwave patterns by highly trained experts is extremely time consuming and costly! We hypothesize, that through the power of crowdsourcing, the general public can match the performance of these learned experts and help us find these important patterns in the brain.

Ask the Scientists

Join The Discussion

What is the context of this research?

Polysomnography, which includes electroencephalography (EEG), is a common technique used to measure brainwaves during sleep. It has become increasingly apparent that features of this EEG signal such as sleep spindles, Rapid Eye Movements (REMs) and k-complexes play an important biological and psychological role in human behavior. Traditionally, human experts are used to detect these features, a process which is time consuming and costly. Critical for the advancement of sleep science is the development of automated feature detection tools. Improving the quality of these automated tools via techniques such as machine learning, requires massive, highly valid datasets, which are currently not available but can be collected through crowdsourcing.

What is the significance of this project?

We pit non-expert Mechanical Turk Workers against expert sleep technicians in the detection of sleep features from a large polysomnography dataset.

Questions:
Can we leverage the power of crowd-sourcing to detect sleep features?
Can the public performance match the quality of expert technicians with much lower cost and completion time?

Data collected will be made freely available and will serve as a 'gold standard', allowing for the development of advanced feature detection tools. These tools will help us understand how sleep plays a role in memory and aid the detection of mental disorders.

What are the goals of the project?

First we will use crowdsourcing to collect a large dataset of highly annotated sleep data (analysed by a very large number of experts and non-experts Mechanical Turk workers). Next, the high quality 'gold standard' data collected will be released for the public's use. This will allow engineers and others to develop exciting new tools for sleep science. Finally we will promote the quality of science by providing a third party validation service for new tools.

For more information please see our Open Science Project.

Data collection will start late May and initial results will be presented at the June meeting of the Associated Professional Sleep Societies in Denver, CO .

Budget

Please wait...

The majority of the budget will go towards paying Amazon Mechanical Turk Workers. These workers are members of the public from around the world who complete small online tasks (such as detecting patterns in brainwaves) for money. As this project is an investigation into the feasibility of crowdsourcing sleep science, a large number of workers is required.

Brainwaves need to be displayed to Mechanical Turk Workers. For this purpose we are developing an online sleep brainwave display tool. Development of this tool is time consuming and requires online hosting.

Endorsed by

These researchers are top notch ad really know what they are doing. I am very excited about this project. The results will be fascinating!

Gina Poe, Ph.D.

Professor, Neuroscience and Integrative Biology and Physiology

UCLA

The development of new methods for analyzing sleep oscillations is critical to progress in this field. I know that this team has the skills to accomplish their research goals.

Erin Wamsley

Assistant Professor

Furman University

As a fellow sleep scientist, I am excited for the contribution this could make for our field! If we can eventually automatically detect sleep stages - that will move not only sleep science forward but be an asset for clinical sleep medicine.

Sara C. Mednick

Associate Professor

University of California, RIverside

Meet the Team

Benjamin Yetton

Affiliates

University of Canterbury, Christchurch, New Zealand University of California, Riverside, CA, USA

View Profile

Simon Warby

Assistant Professor

Affiliates

University of Montreal

View Profile

Sara C Mednick

Associate Professor

Affiliates

UC Riverside

View Profile

Karine Lacourse

Research Assistant

Benjamin Yetton

I began my education in engineering, completing a Mechatronics Engineering Degree in 2010. Shortly after, I made the switch to Cognitive Psychology. I guess its processes that interest me, both natural and man made. The process that interests me the most however, is human memory and its relation to sleep. What exactly are our brains doing while we snooze, and how might that help us store and catalogue the myriad of information we experience daily?

Simon Warby

Simon Warby is an Assistant Professor at the Centre d'Études Avancées en Médecine du Sommeil at the University of Montréal.

His research is focused on the genetics of sleep and its relationship to neurological disease. He is interested in understanding how sleep disruption and disease can influence important EEG microarchitectural features, such as sleep spindles, that are linked to nervous system development, synaptic plasticity, and memory consolidation.

Specific EEG patterns are altered early in the progression of many neurodegenerative diseases, such as Huntington disease and Parkinson disease, as well as psychiatric conditions that are associated with sleep disorders, such as depression and schizophrenia. By refining techniques for the bioinformatic analysis of large polysomnography datasets, his laboratory examines how genetic variation influences the brain circuitry and neurochemistry that generates brain activity during sleep, and how changes in this activity can be used as biomarkers to track disease onset and progression.

Sara C Mednick

I am an Associate Professor in the Department of Psychology at the University of California, Riverside and author of the book, Take a Nap! Change your Life. In 2003, I received her PhD in psychology from Harvard University working with Ken Nakayama and Robert Stickgold. She moved to the Salk Institute for Biological Studies in La Jolla, CA with a three-year, National Institute of Health funded, National Research Service Award fellowship, where I was trained with Geoffrey Boynton and Sean Drummond. In fall 2007, I became faculty at University of California, San Diego and was awarded a five-year, National Institute of Mental Health funded, K01 Mentored Research Scientist Award. In 2011, I moved my lab to UC Riverside. I have conducted studies in conjunction with numerous academic institutions, the U.S. Navy, V.A. Medical Center and private businesses. My articles have been published in such leading scientific journals as Nature Neuroscience and The Proceedings from the National Academy of Science. My napping research has been covered by CNN, Reuters TV, NPR, The Economist, The Wall Street Journal, Consumer Reports Health Journal, Reader's Digest, The New York Times and many other major media outlets.

Karine Lacourse

After 5 years as an engineer in microelectronics (ASIC designer), I
reoriented my career in biomedical engineering. I completed in 2014 a master’s degree at École de technologie supérieure in Health technology. Since then, I worked as a research assistant at Centre d'Études Avancées en Médecine du Sommeil (CÉAMS). It's just more natural for me to directly analyze data from humans in order to understand them and to help them.