Looking at 5.x% and comparing it against an arbitrary goal does little to tell us about the health of the work system. Is 5.x% the typical average performance? Is that much higher than usual?Well put. A simple run chart can be very helpful. One of the uses is to identify special causes. And then to use special cause thinking in those cases. What is important about special cause thinking? That you want to identify what is special about the data point (instead of focusing on all the results as you normally would). What is important about doing that? You want to do it right away (not a week or a month later). Keeping the chart lets you identify when to use special cause thinking and react quickly (to fix problems or capture good special causes to try and replicate them).
This is a great opportunity to use the methods of Statistical Process Control. The main management decision is to decide "react" or "not react" to that daily data point. SPC helps us with this (again, Wheeler’s brilliant little book explains this far better than I can in a blog post).
If we choose “not react” because 5.x% is lower than the goal, we might be missing an opportunity for process improvement. Generally, it’s better to present more than one data point – even if you don’t do full-blown SPC, you should present a run chart.
You have to be careful as we tend to examine most everything as a special cause, when most likely it is just the expected result of the system (with normal variation in the data). Special cause thinking is not an effective strategy for common cause results.
Related: Quality, SPC and Your Career - Statistical Engineering Links Statistical Thinking, Methods and Tools