Sunday, August 29, 2004

Dangers of forgetting proxy nature of data

Topic: Management - Data
Written in relation to a thread (Staff Attitude) on the Deming Electronic Network (but the DEN archive seems to indicate I never posted this message).


We use data to act as a proxy for some results of the system. Often people forget that the end result is not for the number to be improved but for the situation to be improved. We hope, if the measure improves the situation will have improved. But there are many reasons this may not be the case (one number improving at the expense of other parts of the system, the failure of the number to accurately serve as a proxy, distorting numbers, etc.).

I find something I learned from Brian Joiner an excellent summary - which I remember as:

Data (measuring a system) can be improved by

1) distorting the system

2) distorting the data

or 3) improving the system (which tends to be more difficult though likely what is desired)
Brian Joiner's book, 4th Generation Management is a great book for managers.

It seems to me another danger to examine when looking at "results" is whether the proxy nature of the measure is deminishing. It seems to me the situation where turnover increases because people are "growing" (and therefore the assumed value as measure of employee dissatisfaction is diminished) is a situation where the system changes and reliance on the proxy of turnover to measure dissatisfaction breaks down.

This, to me, relates to the concept that all models are wrong but some are useful (which I learned from George Box) and leads me to believe that measures such as turnover and sick leave use can be valuable. And can be helpful in measuring the results of attempting to improve the system.

Another version of that concept states that all models are wrong but in a given world a model may be useful. In a certain state of equilibrium it may be that you could track increases in turnover (and use it as a proxy for employee satisfaction) even if that was not perfect measure for why all turnover happened. But then once the system shifted to a new state the turnover rate no longer served as an effective proxy. Reality is always more complex than the measures we use to try and understand it so even while the model is useful it is not a completely accurate representation of reality.

It seems to me that once we begin collecting data we often put very little thought into whether the data continue to be valuable (assuming we did so to begin with) and continue to serve the purpose they once did. And we often fail to explore whether changes in the numbers (which we call results) are representative of the "true results" of the system or they fail to do so.

Friday, August 27, 2004

Six Sigma and Deming Philosophies

Topic: Management - Six Sigma and Deming
Post of 23 Oct 2003 message to the Deming Electronic Network to this blog. View DEN thread

My oversimplified view is that the Deming philosophy aims to improve the system and Six Sigma is about improving within the current system. I think several Six Sigma efforts over the past 10 years have improved upon what many organizations tried with TQM, in large part by

1) putting substantial effort (significant training and application) into using statistical tools to improve

and 2) more focus on results (especially short term results)

Unfortunately I think even the best Six Sigma efforts don't lead to the organizational change that is needed (and while the focus on results can help it can also do damage - especially a short term focus). To be fair, I believe most proponents of Six Sigma would claim it aims to improve the system. I think most implementations of Six Sigma have too great a focus on money and too small a focus on the customer and systemic improvement. In addition, it has too great a focus on the measurable and too little focus on the unmeasurable.

However, at this point, I think the Deming philosophy has too few good examples of successful application. To me this is a negative aspect of the philosophy - I believe the value in improving management is in actual real world improvement. I realize, the lack of examples, does not prove the philosophy is poor or invalid. I do believe the relative sparsity of adoption is a weakness in the philosophy. Deming said "I never said it would be easy" (I am quoting by memory so feel free to correct me). The philosophy would be improved if people can find ways to make it easier to adopt. It may be just not be possible to make it significantly easier, or it may just needs to be continually improved for 50, 100 years or more before it will be adopted more completely in more organizations.

I think, for the last 20 years the Deming philosophy has had the most impact in the world through inspiring thought leaders and pioneering managers to adopt new ways of thinking and incorporate some of Deming's ideas in their organizations or their management philosophies. From them, pieces of Deming's philosophy has had a significant impact.

For selected online Six Sigma resources, articles, etc. see:

http://curiouscat.com/guides/sixsigma.cfm

and selected online Deming resources, articles, etc. see:

http://curiouscat.com/guides/deming.cfm


John Hunter
www.johnhunter.com