Experimenting to Improve Sleep Quality

comments on: Can a Humidifier Help You Sleep Better and Snore Less?

After doing some research I learned that humidifiers have helped folks snore less. So, after some more research, I picked up a slick little ultrasonic humidifier and gave it a try. Now, it’s been less than a week which I know isn’t enough to get too excited about statistically speaking. But one thing is becoming crystal clear…it’s most definitely helping me sleep better.

Interesting post, which includes control charts showing the impressive progress.

"I’m also still trying to figure out what caused the three special cause signals in January." One nice aspect of improvement is sometimes you can make a system improvement that even without knowing the causes of previous problems, the new improvement stops those from happening again. Maybe that won't be the case this time but maybe it will. Health related issues are so touchy that I could imagine it is something like a couple bad factors stacked on top just push things over the limit. So being a bit tired and say too low humidity and you didn't drink quite enough liquid and sleep quality is bad but just 1 or 2 of those and it might be a bit worse but not horrible.

Special cause signals will be more frequent if several factors together amplify each other (and they rarely happen together so those amplified results are rare). What happens is those rare amplified events will be special, outside of the system that generates that regular variation when they act alone but when all that variation lines up just right the result will be outside what is normal (due to the very large change in the result for that special case where the individual factors acting together (amplifying) create a very large change in the result.

Related: Gadgets to Mask Noise and Help You Sleep or Concentrate - Apply Management Improvement Principles to Your Situation - Zeo Personal Sleep Manager - Using Control Chart to Understand Free Throw Shooting Results

Remembering George E.P. Box

George Box passed away last week and a long (1919 - 2013), rewarding and productive life. His obituary ends with "a last message from George" (quoting Cole Porter's song - Experiment).

“Experiment! Make it your motto day and night. Experiment, And it will lead you to the light …Be Curious, …Get Furious… Experiment, And you’ll see!”

The full text of the song is quoted in Statistics for Experimenters (a book by George, my father and Stu Hunter on using design of experiments to improve). The song is included in the De-Lovely soundtrack.

If you want to honor the memory of George, contributions could be made to

  UW Foundation - George Box Endowment Fund (link to donate - include George Box Endowment Fund in the box for instructions) US Bank Lock Box 78807, Milwaukee, WI 53278.  This fund was started some years ago with the intention of assisting graduate students. It is a permanent endowment fund, so contributions to the fund are added to the principal and the annual earnings of the fund are used to support the fund purpose. The purpose of the fund is to support activities of the Statistics Department with a primary (but not exclusive) focus on activities of direct benefit to graduate students.  Recipients will be selected by the Department faculty (or their designates) with input from Departmental graduate students."

  Agrace HospiceCare (link for donating online), 5395 E. Cheryl Parkway Madison, WI 53711.

Examine the System, Don't Look to Blame a Person

Stress Solutions, Not Blame by Kevin Meyer

A couple years ago I told you how the organization had created a nonpunitive reporting system for air traffic controllers to report incidents. That led to a dramatic change, which might initially be seen as a scary negative but as most of us immediately realize is a huge positive:

New numbers released by the Federal Aviation Administration show reports of air-traffic errors have nearly doubled in three years. The number of reported incidents in 2007 was 1040, and that number rose to 1887 in 2010, an 81 percent increase. This cultural change in safety reporting has produced a wealth of information to help the FAA identify potential risks in the system and take swift action to address them.

The point of using in-process and process result measures on well functioning processes is often overlooked. You don't want to spend too many resources collecting data that has little value, but proper process measures are very useful and should be monitored. Also this helps when you decide to improve (or radically change something somewhat related) and can catch things (unintended consequences) very quickly. The point of understanding the data (in context) is critical. Brian Joiner did a very good job of emphasizing this idea I think. If you want to reduce complaints it is usually pretty easy to do so, by making it really hard to complain. When you really care about customer focus, understanding if complaints are up do to better processes to encourage complaints or because your service is lousy is critical. Related: Find the Root Cause Instead of the Person to Blame - Dr. Deming, 94% belongs to the system (responsibility of management) 6% special - European Blackout: Not Human Error (System Failures)

Experimenting to Discover

Causal Reasoning in Science: Don’t Dismiss Correlations (the broken link was removed)

Box, Hunter, and Hunter were/are theorists, in the sense that they don’t do experiments (or even collect data) themselves.
...
Science is about increasing certainty — about learning. You can learn from any observation, as distasteful as that may be to evidence snobs. By saying that experiments are “necessary” to find out something, Box et al. said the opposite of you can learn from any observation.

William Hunter was my father. He did many experiments. George Box did many experiments. You are entitled to your opinions obviously but the claim that they only dealt with other people's data is not accurate. It is true they were world renowned experts on experimenting and had many people consult them about their experiments, for help: designing them, analyzing them, what to do next, how to improve the process of experimentation in their organization, etc.. While it seems to be implied in the post that such consultation was a reason to distrust their thoughts on experimentation I hardly think that is a sensible conclusion to draw. Most of those they helped were running experiments in industry, to improve results (not to publish papers).

They were, and are, applied statisticians (and though I am obviously biased, I think many would agree, 2 of the most accomplished in that field in the 20th century). What experiments need to be done is critical for an applied statistician. What matters is making improvement in real world processes. If you don't run the right experiments, you won't learn things to help you improve.

They worked on the problem of where to focus, in order to learn, quite a bit. One significant part of there belief was to have those involved in the work do the thinking about what needed to be improved. This isn't tremendously radical today but in the past you had many people that thought "workers" should do what the college graduates in their office at headquarters tell them to do. Here is one of many such example, from Managing Our Way to Economic Success by William Hunter:

The key is that employees at all levels must have appropriate technical tools so that they can do the following things:

- recognize when a problem has arisen or an opportunity for improvement exists,
- collect relevant data,
- analyze the situation,
- determine whose responsibility it is to take further action,
- solve the problem or refer it to someone more appropriate...

I don't have the book in front of me, but doesn't it start with an example on learning where you can use inductive reasoning and from the facts that you see you can draw conclusions and construct a theory that fits the facts. If so, it seems to call into question the idea that they claimed "[the] opposite of you can learn from any observation." is not actually accurate. They understood you can use inductive reasoning to create theories. You then use experiments to test theories.

The books is called Statistics for Experimenters, right? Not statistics for drawing conclusions when not doing experiments. When you are experimenting you can test whether beliefs you have are accurate and you can learn about things you try. Smart people can make guesses what will happen and be right. I know the authors would believe those knowledgable about the system in question are well suited to determine what variables to test. It is that knowledge that will lead to experiments that are likely to be effective.

The authors of the book were trying to help those that often failed to learn as much from experiments as they could. Far too many people still don't use the most effective statistical tools when experimenting.

They emphasized, consistently, the need for those doing the work to involved in the experiments. The job of statisticians was to help in the cases where advanced statistical tools and knowledge would be useful. The reason for those who do the work (are familiar with the process) is because they have knowledge to bring to what should be tried in experiments.

When I read through The Scientific Context of Quality Improvement, 1987 by George Box and Soren Bisgaard it seems to me it discusses the types of issues you raise: how do we learn without experimenting? I am not sure if it is just me, or if it clearly addresses that issue. Here is another, Statistics as a Catalyst to Learning by Scientific Method by George E. P. Box. And another, Statistics for Discovery.

There are many other sources, I am sure. They understood the importance of learning as much as you could from available sources. They just also understood the importance of experiments and learning the most you could from experiments. And the book, Statistics for Experimenters, was focused on the most effective ways to improve using statistics to learn from experiments..

Here is what Box, said in his own words about the objective (and it isn't proving the hypothesis):

[too many people ]"can’t really get the fact that it’s not about proving a theorem, it’s about being curious about things. There aren’t enough people who will apply [DOE] as a way of finding things out"

Statistics for Experimenters: Design, Innovation, and Discovery shows that the goal of design of experiments is to learn and refine your experiment based on the knowledge you gain and experiment again. It is a process of discovery. That discovery is useful when it allows you to make improvement in real world outcomes. That is the objective.

Discrimination and Data

A Discriminatory Conundrum

The total American workforce has remained relatively constant over the last ten years – roughly 131M employees. The number of EEOC claims over the last ten years has increased by roughly 25% to almost 100K a year.

I would think these suits increase when there is a bad employment marketplace, but I don't have any data on that it just seems logical. When times are good all sorts of people get good jobs. When times get bad, jobs are hard to find, promotions are hard to find, people get demoted, bosses have to cut budgets and people. People are also very stressed out and it isn't surprising to me that lots of bad outcomes come from that including people feeling slighted, people getting into escalating cycles of bad words and actions, and more lawsuits.

In addition to the macro-effects of the economy data has quite a bit of variation. Often lawsuits over class differences get data that shows difference in result and claims this shows discrimination. While if we showed the same data for the first letter of the grade school the person attended, or eye color, or the state the person's Mother was born in we would see lots of variation in data that people would see as worthy of payment for the discriminated against class (though in those cases no-one would actually believe it). xkcd took a comic look at data analysis recently.

I am skeptical of short term variation having much meaning with this data. I would want to see it charted over time and then analyzed for having an indication it wasn't just random variation of a stable system.

If the data doesn't indication a special cause (for the variation in the data) that tells you that special cause problem solving is not the best way to improve. In that case you still want to improve but you improve using common cause problem solving techniques. This particular dataset would undoubtedly (at least in my opinion) benefit from stratifying the data to identify segments where it is more of an issue. My guess is that there would be wide variation but the data analysis would show if this is true or not.

SPC - Charting and Improving Results

Everett Clinic Video, Redux – The Need for SPC Thinking

Looking at 5.x% and comparing it against an arbitrary goal does little to tell us about the health of the work system. Is 5.x% the typical average performance? Is that much higher than usual?

This is a great opportunity to use the methods of Statistical Process Control. The main management decision is to decide "react" or "not react" to that daily data point. SPC helps us with this (again, Wheeler’s brilliant little book explains this far better than I can in a blog post).

If we choose “not react” because 5.x% is lower than the goal, we might be missing an opportunity for process improvement. Generally, it’s better to present more than one data point – even if you don’t do full-blown SPC, you should present a run chart.

Well put. A simple run chart can be very helpful. One of the uses is to identify special causes. And then to use special cause thinking in those cases. What is important about special cause thinking? That you want to identify what is special about the data point (instead of focusing on all the results as you normally would). What is important about doing that? You want to do it right away (not a week or a month later). Keeping the chart lets you identify when to use special cause thinking and react quickly (to fix problems or capture good special causes to try and replicate them).

You have to be careful as we tend to examine most everything as a special cause, when most likely it is just the expected result of the system (with normal variation in the data). Special cause thinking is not an effective strategy for common cause results.

Related: Quality, SPC and Your Career - Statistical Engineering Links Statistical Thinking, Methods and Tools