Tuesday, August 01, 2017

Wednesday, July 19, 2017

Code Software to be Robust and Easy to Update Quickly

Comment on: Correction vs Prevention in Software Development

I think both prevention and designing the software and management system so that rapid correction is possible are important. While rare events may be difficult to prevent when looking at each instance there are styles of coding that make more "edge case" failures more likely. Coding so that the system is as robust as possible is wise but you should also realize those efforts will likely not be perfect and so designing in visible notifications of failure and coding so rapid correction is possible is necessary.

In addition to the need to update quickly for bugs, software should be easy to update due to changing requirements and to aid in continual improvement efforts.

Related: Improving Software Development with Automated Tests - Software Supporting Processes Not the Other Way Around - Building a System to Reduce Interruptions for Software Developers
- Use Urls: Don’t Use Click x, Then Click y, Then Click z Instructions

Wednesday, July 12, 2017

Earning the Trust of Employees

My comments on: Why good employees (should not) leave (good companies)? – The employer perspective

One of the very challenging tasks as a manager is to get people to trust bringing up difficult topics. Often people are punished for doing so. Most people learn to keep quiet about management problems. Even when managers say they want to hear there are many instances when they then punish those who speak up.

I agree with you that for organizations to flourish management must know what needs to be improved. But few executives or managers put in the effort to earn people's trust. But building that trust is what organizations that want to flourish need to do.

Related: Ignoring Unpleasant Truths is Often Encouraged - How to Create a Continual Improvement Culture - Practical Ways to Respect People - The Problem is Likely Not the Person Pointing Out The Problem - Build an Environment Where Intrinsic Motivation Flourishes

Tuesday, July 04, 2017

Using Money to Motivate Creates More Problems Than Benefits

Comments on: Employee Motivation and Lean Implementation – Part 2: Money

Well said. Nearly all "motivation" efforts such as bonuses create far more problems than benefits. Paying people fairly is important, if they don't have enough to live they will be distracted and seeking new options.

And if they are substantially underpaid compared to the market (even if they have enough to live comfortably) they will be prone to seek new alternatives and be disgruntled because they feel they are being treated unfairly.

The best "motivation" managers can provide is to eliminate the de-motivation created by poor practices in the management system.

Sadly these are often common and managers can keep themselves very busy just doing this. If not, they are likely very lucky and probably have no motivation problems to worry about.

Related: Why Extrinsic Motivation Fails - The Defect Black Market - Dangers of Extrinsic Motivation - Extrinsic Incentives Kill Creativity - A "Demotivated" Workforce is a Symptom of the Culture of the Organization - The Potential Benefits, Risks and Folly of Stretch Goals

Wednesday, June 28, 2017

Systems Thinking and Management Improvement

This is a response to a comment on my own blog post: Peter Senge on Systems Thinking

Great video on systems. It seems systems thinkers believe it to be a cure all. Deming’s addition of psychology, epistemology, and statistics and interactions is so much more powerful. What do you think? Is it all Japan needed because they had the other elements? Do we need more?

Senge has very good ideas. I would add to his thoughts more of what Deming, Scholtes, Joiner... said. I think many people get into the idea that their areas of interest is nearly everything that is needed.

I don't think Japan just needed systems thinking (for one thing there organizations start with more of that understanding). For another, a real problem in Japan is going along and not speaking up about problems. That is an issue everywhere but is much worse in Japan than the West. Japan has an obsession with customer service that would be valuable for USA organizations to learn from.

A big part of what makes Deming's framework so useful is he was continually learning and adopting new ideas (Senge does a lot of this compared to most people but I can't think of anyone in the Management area that is close to as good as Deming was at this). I do think most Deming folks today would benefit greatly from much more thinking a about the organization as a system. It is often very superficial in my experience (repeating phrases like "we need to break down barriers between departments" or "it is a mistake to optimize the part because it sub-optimizes the whole"). Those ideas are great but you need to manage based on that concept not just say it and move on.


I can't remember if I have added a comment from my own blog here before :-/

Related: Why Do People Fail to Adopt Better Management Methods? - Leadership While Viewing the Organization as a System - A Good Management System is Robust and Continually Improving

Wednesday, June 21, 2017

Leadership: Finding the Right Path

Response to: The Simple Leader, Find Your Unique Path

Like the spiritual journeys of Buddhists, Gnostics, and Christians like Thomas Merton, when you are on your own journey, you must first seek to learn, understand, contemplate, and reflect on your circumstances and beliefs. Only then can you apply what makes sense to create your own path. Don’t simply accept what others say or copy what others do.

Very well said.

The value of being open to ideas that are not part of the dogma is overlooked. And as a leader highlighting those that challenge the dogma (such as Thomas Merton for the Catholic church) is wise for a leader to do. Of course, you need to highlight efforts that hold true to the spirit/core-values while challenging dogma. It isn't just different ideas that are needed but the right different ideas.


Bench in forest at the Abbey of Gethsemani (where Thomas Merton lived)

Wednesday, June 14, 2017

A Sign of Decline at Apple

Comments on: This Apple Store Sign Seems to be a Sign of Apple’s Broader Troubles

I agree that sign is more important than many people might think. For a company like Apple that spends likely hundreds of millions of dollars a year on design and conveying a message through that design (in Apple stores, with products, with presentations, with ads...) it is not acceptable. They have held themselves to high standards. When that starts to slip if they are not proactive it slips quickly.

For a normal business they would be at the mercy of the management company to fix the door and the manager of the store would pass the buck to them. I can say if I were the manager of that store for Apple, if it wasn't fixed immediately I would have it fixed myself (and then bill the management company). If it couldn't be fixed immediately I would have a decent sign put there and it would make sure it got fixed very quickly. That isn't the same action I would take if it were some small shop I was responsible for where I knew we could only afford a cheap place and things like broken doors take a while to be fixed. For an Apple store that is unacceptable.

My main complaint with Apple is the poor software quality over the last 5 to 10 years. Software quality started to slip and kept slipping and no-one at Apple that had the authority dealt with the decline. The Apple Maps fiasco was a symptom of this long term failure by Apple. Tim Cook responded to that symptom but I don't see Apple giving software quality nearly the attention it deserves. My MacBook Pro has had numerous software issues for years. I have looked at other hardware and it is very difficult to find hardware of the quality of Mac laptops. My next computer would likely be an Ubuntu laptop if I can find good enough hardware. Another option is installing Ubuntu on the MacBook and just using that most of the time (there are some reasons Mac software can be useful so having it as a fallback is a benefit). But it is sad that Apple has let software quality slide for so long.

Related: Practicing Mistake-Promoting Instead of Mistake-Proofing at Apple - Aligning Marketing Vision and Management - Human Proof Design - Vision can be a Powerful Driver but Most Often It is Just a Few Pretty Words

Tuesday, June 06, 2017

Priorities

Too often people chose "no" for the things they were planning on doing because they say "yes" to some new thing without comparing to what will have to be sacrificed in order to say "yes" to this new option.

Being open to new possibilities is good. But they must be considered within the context of what must be let go to accommodate them.

My comments on: The Simple Leader: Just Say No

Related: Manage Better by Managing Less - Carve Out Time to Think

Thursday, June 01, 2017

A "Demotivated" Workforce is a Symptom of the Culture of the Organization

Comment on: Are You "Perfectly Designed" for Morale Issues? (Yes) – and Employee Surveys Won't Fix It. (Demotivators: Part 1) [original post was removed so I removed the link].

Creating a system that gives people pride in their work will cause many motivation issues disappear. A significant part of that is eliminating the de-motivation that exists in the management system:

Motivate or Eliminate De-Motivation

Build an Environment Where Intrinsic Motivation Flourishes


A "demotivated" workforce is a symptom of the culture of the organization. That is how the issue needs to be looked at to improve results. Blaming people and attempting to motivate without fixing the causes of demotivation is not effective.

Related: How to Deal with Motivation Problems - Motivation, Rewards, Performance Appraisals and Your Career - Dangers of Extrinsic Motivation - Stop Demotivating Employees - How to Motivate Front Line Workers

Thursday, May 25, 2017

Interactions Among the Four Fields in Deming's System of Profound Knowledge

Question on Reddit:
I would like to see expansion of Deming's SoPK with more examples of interactions in the four fields of knowledge. Are you aware of any?

My response:

I think this is actually very common, but often it isn't explicitly mentioned. To explain this well would take a fair amount of time. Let me just give 2 quick examples

Distorting the System to meet a target

This certainly is about the interaction or understanding variation (in this case people not understanding data well enough and being mislead), psychology (how people respond to pressure to meet goals), theory of knowledge (not understanding the difference between the proxy value of data and the underlying truth) and systems thinking (how a system is likely to react to meet goals - distorting data and distorting the system, and using simple measures where those things work to get numbers).

Create a System That Lets People Take Pride in Their Work

Appreciating that results will be better when people are doing work they are proud of involves at least appreciation
for a system and psychology.

I think in reality nearly every example involves interactions. We can analytically separate out the one we want to discuss or the one that seems most influential in what we are looking at but in reality it isn't just one.
For example, data issues related to over-reacting to common cause variation is in the "understanding variation" realm. But it is also deeply ingrained in our psychology that we look for special causes. If our psychology was different it is very possible the mistake of "seeing" (and believing) special causes everywhere would not be a problem. But because those 2 area interact in the way they do it is an area of improvement for how we think and manage. By focusing on an understanding of variation we can limit the damage caused by are faulty psychology (seeing special causes where they don't exist - where it is just common causes). And that really integrates theory of knowledge and systems thinking (we chronically over-simplfy and ignore the large system).

Related: 94% Belongs to the System - Encourage Improvement Action by Everyone - Circle of Influence

Tuesday, May 16, 2017

Transforming a Management Culture

Thoughts on: Where Lean Went Wrong – A Historical Perspective

I believe that companies that say they are attempting to become lean fail to do so in the most important ways. I do believe most efforts result in improvement but usually are fairly limited by the existing management system and refusal to really change much.

More than "lean failing" I would say transforming to a different management culture fails. Saying lean fails makes it seem to me that what a lean management system was in use and failed which is not really the case it doesn't seem to me.

I wrote about these ideas on my blog: Why Use Lean if So Many Fail To Do So Effectively

and discussed them in this podcast on Building Organizational Capability.

Related: Why Do People Fail to Adopt Better Management Methods? - Transforming a Management System – A Case Study From the Madison Wisconsin Police Department - Culture Change Requires That Leaders Change Their Behavior - Change Management: Create a Culture Seeking Continual Improvement or Use Band-Aids?

Saturday, April 22, 2017

The Sociology of Organizational Change

Comments on: Researching Laggards

The Late Majority is the stabilizing force, the repository of institutional knowledge that slowly absorbs and productionizes the ideas proven to best serve the organization. They aren’t as eager for change as the Early Majority, but they’re happy to adopt proven practices.

The Laggards provide challenge the Instigators most directly, questioning or outright denying the value of a new idea, and provide the most vocal and active resistance. However, their direct criticism may inspire the Instigators to find unexpected common ground and more effective solutions than they otherwise might.

Yes, I think laggards really are common. The grey area between laggards and late majority may be pretty large. Many are swayed by the critical mass of opinion. At first they seem like laggards because they side with them, as the momentum grows they side with late majority...

True active laggards fighting well after the critical mass makes it obvious the culture expects the "new" behavior" isn't a huge group I don't believe. But getting the point where the those siding with laggards switch to siding with late majority is a very challenging point to reach for most significant changes.

How you help change the culture of an organization requires understanding the inertia against change in most organizations and the strategies that are useful in creating the critical mass to accept new ideas and cultural attributes as the new normal.

Related: Why Do People Fail to Adopt Better Management Methods? - Podcast: Building Organizational Capability - Culture Change Requires That Leaders Change Their Behavior - Transforming a Management System – A Case Study From the Madison Wisconsin Police Department - Change Management: Create a Culture Seeking Continual Improvement or Use Band-Aids? - Communicating Change - Building Adoption of Management Improvement Ideas in Your Organization - Grow Your Circle of Influence

Thursday, March 16, 2017

Iterate to Continually Improve

Thoughts on: The Challenge of PDSA: Feeling Like You’ve Fallen Short

For me, this snowball was the understanding of the continuous improvement cycle, the iterative process towards ideal state or what many call “true north.” I have seen and explained the well-known visual many times; the person climbing up towards target state and, ultimately, ideal state through PDSA, only seeing ahead of them as far as the flashlight reaches.

The relationship of PDSA iterations and ideal state never really dawned on me while I was working through PDSA cycles in problem solving. The visual depicts the learner stair-stepping up through PDSA cycles, each step up the flashlight seeing further, learning more and getting closer to ideal.
...
In absence of a clearly defined Target state, satisfaction with progress, pace, and incremental improvements may more times than not, leave you feeling as if you have fallen short.

Iteration and continual improvement are key. Understanding that "target state" is a temporary target is important. If a "ideal state" is too specific it can hamper innovation. This usually isn't so critical on fairly short term PDSA (except in those cases when we should look at innovation instead of improving the current process).

The PDSA process doesn't hamper innovation. But, when people set in their minds ideal states or targets that they move toward and don't see those as flexible based on new learning they can stunt innovation.

Related post: Resources for Using the PDSA Cycle to Improve Results - Continually Improving Using a Focus on Delighting Customers

Monday, February 20, 2017

The Problem Is Exacerbated by Fear of the Word Problem

Comments on What’s Another Word for “Problem”?

I think this is a wise recognition: "may need help in more areas than process improvement."

Fear is likely a part of the problem (yes problem). Such a desire to ignore problems and the word problem can also be greatly enhanced with performance appraisals systems that create a mindset that is focused on hiding potential issues that may reflect poorly on those appraisals...

The problem with the word problem is often not as simple as it may seem at first. Changing the word used may do a tiny bit of good but not much. The underlying issues that cause people to think problems are something to not acknowledge is not something solved by avoiding the word.

Related: If Your Staff Doesn’t Bring You Problems That is a Bad Sign - The Problem is Likely Not the Person Pointing Out The Problem - Is Using the Words Resources or Assets When Talking About People the Problem? - The Importance of Making Problems Visible

Thursday, February 09, 2017

Should I be in the Check Phase of PDCA Daily?

Below is my response on closed forum about whether doing the "check" phase of PDCA daily was too often. I expanded on my comments there a bit in this post.

The check/study phase should be reviewing the results of the experiment done in the Do the experiment phase. "Checking" how things are going during the experiment makes sense but that isn't the check/study phase of PDSA .

For example, you don't want to pay no attention during the experiment and then look at the data and discover the data shows obvious signs the operational definitions were not clear, or the process is providing very bad results. So you need to have those doing the experiment paying attention daily.

Remember one key to using the PDSA cycle is to turn through the whole cycle quickly. Daily would be exceptionally quick. Moving through the whole cycle in 2-6 weeks is more normal. Organizations successful using PDSA will quickly turn the cycle 4+ times for a specific effort (often the 2nd, 3rd... times through are much faster than the first time through).

More on how to use the PDSA well:

Monday, January 16, 2017

Intrinsic Motivation and the Danger of Overgeneralization

Comments on Motivation by Kurt Häusler

> You have to pay enough to keep the issue of money off the table

I agree with that sentiment. And I agree we do tend to overgeneralize and discuss management practices without enough attention to local conditions (at the country level, and even smaller geographic level and even very big differences between organizations).

But I strongly disagree with "so intrinsic motivation is of limited utility."

Creating and maintaining workplaces that let people take pride in their job is hugely important. We spend a huge amount of our time and energy at work. Even if we are paid less than we should be it is still important to have work we can be proud of doing. Yes, the issue of low pay also has to be addressed but it isn't an either-or choice.

In fact, by creating systems that let people take pride in their work we take advantage of more of their potential and thus create more value which can make it easier to pay more money. If we instead, decide to reduce the importance of intrinsic motivation in our management systems that is likely to be a mistake. Granted in some places the importance of intrinsic motivation may be so well understood and incorporated that focus should go elsewhere but I question how often organizations are really doing so well on that front they need to reduce that focus in order to focus elsewhere.

Related: Motivation, Rewards, Performance Appraisals and Your Career - Motivate or Eliminate De-Motivation - Two resources, largely untapped in American organizations, are potential information and employee creativity

Wednesday, December 28, 2016

Most Popular Posts on the Curious Cat Management Comments Blog in 2016


The most popular posts on this blog:



Breakdown of popular posts by year: 2016 - 2, 2015 - 2, 2014 - 2, 2013 - 1, 2012 - 1, 2006 - 1, 2005 - 1.

I started this blog over 10 years ago. After I figured out that I thought blogging would work for me I created a self hosted blog (the Curious Cat Management Improvement Blog) and moved the content to that blog. But I kept up the post here since web pages should live forever. For several years (about 2005 to 2011), I posted occasionally to this blog, sometimes the posts were comments made on other blogs.

In 2011 I started to use this blog a bit more consistently to collect the management and leadership related comments I made on other blogs here (when they seemed to say something useful or interesting that were worth posting on this blog - often things I wanted to be able to find later).

Related: Most Popular Posts on the Curious Cat Management Comments Blog (2015 edition) - Most Popular Posts on the Curious Cat Management Comments Blog (2014 edition)

Monday, December 26, 2016

Don't Claim Your Customer's Suffering from Your Management System Results are a "Learning Opportunity"

From, Microsoft finally admits that its malware-style Get Windows 10 upgrade campaign went too far":

It’s all well and good for a corporation to promise that its learning from mistakes, but it’s awful hard to believe such promises when the mistakes in question violate basic principles of software design and customer service

They are exactly right. This is one of the huge problems with the "learning from mistakes" excuse. Some mistakes are a sign of an extremely bad management system.

If you force the consequences of mistakes on your customers making up excuses about how this failure is a learning experience for you is only ok if you actually spell out how you are changing to assure you don't fail your customers due to this same management system failure again.

You need to design your systems to minimize consequences to customers when something goes wrong.

Acting as though a problem is due to some specific issue only with the exact circumstances that created the consequences is exactly the message you expect from businesses that have no respect for customers. It is exactly he cover your butt mentality of organizations you definitely do not want to be a customer of.

We need to stop accepting transparent excuses that indicate no acceptance of responsibility for mistreating customers. This wasn't a mistake about updating software. This was a mistake of a management system that allowed colossally customer hostile action to be taken and then continued and accepted meaningless excuses as if they were relevant. Microsoft manages to fail even the extremely low expectations we have for them over and over again.

I was foolish enough to continue to use Skype after Microsoft bought them. I added money to my account so that I would have access to Skype on my trip to China. 3 minutes into my first phone call they disconnected me. They then put up the most customer hostile form I have ever seen. I literally have over 30 questions that were required to be answered (things like what month and year did you sign up). I can't remember them all but at least 15 were insane to expect any customer to know. Needless to say they provided no way to contact them outside the ludicrous form. You can't have such repeated massive failures of basis common courtesy for decades without a horrible management system being in place.

It is so frustrating that such customer hostility is allowed to continue. Microsoft has a massive, decades long problem with treating customers horribly and making excuses for decades. This is just one more example of that pattern. Supposedly they are less horrible today than 20 years ago. Maybe that is true but they give me no reason to want to test out if that is true with their well publicized continuing of their customer hostile patterns.

Sure Apple's very poor software quality over the last 5+ years makes me frustrated with them. But Microsoft is much much worse so I have no desire to make from Macbook to any Microsoft software. Google has issues but if they would target users that don't have (or want to rely on) great internet connections to use their computer I would consider them. Ubuntu is the leading solution Apple has pushed me into strongly considering. The biggest issue I have not is the hardware for Ubuntu just isn't nearly as good as MacBooks. Granted the latest MacBook hardware choices Apple made are somewhat lame, but still it is much better hardware than others offer. Sadly it is stuck with their bad software and combine that with the sky high prices (the old MacBooks were expensive but well worth it) I just don't think I will buy another. While less than great I think one of the Dell laptops is in the lead for my next laptop.

You can't allow your business to treat customers horribly if you don't have a monopoly (or monopolistic position). Sadly for those stuck with Microsoft, they have close to that monopolistic position and rely on that. They have an extremely long way to go just to stop treating customers horribly. And treating an inexcusable failure as something they are learning from is yet another indication they are not learning at all.

Related: Practicing Mistake-Promoting Instead of Mistake-Proofing at Apple - Making Life Difficult for Customers - Incredibly Bad Customer Service from Discover Card

Sunday, December 11, 2016

Will the Government Adopt Better Management Methods This Time?

Reaction to, New Administration: Real Improvement This Time?


Sadly I don’t believe the odds of appreciable success are good. I think the odds are much lower than they were for previous attempts. I wrote about President Obama’s appointment of a Chief Performance Officer in 2009

it is dangerous if they believe their propaganda and don’t learn from all the previous essentially identical efforts: a claim of “first” is trying to convince people those past efforts do not exist. This self-delusional pattern is very common in the practice of management and a significant reason why the practice of management has not improved more rapidly over time. To achieve success you need to determine why the problem still exists and exploring the very similar past efforts is critical to such study.

in which I pointed out similar ideas as you state here about past efforts that amounted to very little.

I think there were some reasons to hope Gingrich might help apply some better management methods if he were in a position that gave him authority to do so in the 1990s, today I am very skeptical that he would help.

Related: Better Management in Government (2012) - Public Sector Continuous Improvement Site - Transformation and Redesign at the White House Communications Agency - Doing More with Less in the Public Sector (1986) - The Public Sector and Deming (2005)

Saturday, September 03, 2016

How to Improve at Understanding Variation and Using Data to Improve

My comments based on a question on, How to Use Data and Avoid Being Mislead by Data:

Thanks for this post John. This is the part of Deming’s teaching that I often struggle with (understanding variation). I read Wheeler’s book Understanding Variation and it helped me with the concept, but I am challenged trying to apply it where I work. I often am not sure what to measure and if I do, I’m not sure how to measure it. Folks appreciate my burn down charts showing trends, but this is about the best I’ve been able to do. Do you have any recommendations on where I can look to help me get better at this?

Getting better at using data is a bit tricky, so struggling is fairly common.
Probably the easiest thing to do is to stop reacting to normal variation (caused by the system) as if it were special. This isn’t super easy but it is the easiest step. And it does make a big difference even if it doesn’t seem very exciting.

The idea of actually using data properly provides big benefit but it much trickier. Don Wheeler’s book is a great start. Making predictions and evaluating how those predictions turn out is also valuable. And in doing so often (though not always) it will also spur you to collect data. This process of predicting, figuring out what data to use to help do so (and to evaluate the results) and considering the result of the prediction and how well the predictions overall are working can help.

You learn what data is often useful, you experiment with real data and real processes and you learn what needs to improve. If you are at least somewhat close to using data well then just doing it and learning from your experience is very useful. If you are really far off the experience might not help any 🙁
The links in the post above I think provide some useful tips (and the links within the posts they link to…).

More: Measurement and Data Collection


If you don’t have an answer for how you will use the data, once you get it, then you probably shouldn’t waste resources collecting it (and I find there is frequently no plan for using the results).

It isn’t uncommon that the measures you would like to have are just not realistically available or are hard to determine. How to get started in this is one of the tricker pieces in my experience. It is a place where consultants may be very helpful. If that isn’t an option another possibility is just to ask others at your workplace for ideas for metrics (there are issues with this and a big one is that many metrics will more likely to lead you astray than actually help).

This can also be an area where seeing what others are using can be helpful. Because it is hard to think up what are great metric seeing what others are doing may provide insight. Of course, the ideas must be evaluate for whether they would work for you (even if they are right for others they may not be right for you – and many are not really right for others it is just a thing they measure and while they have associated it with good things maybe they are wrong (correlation but not causation]).

Saturday, August 27, 2016

How to Help Instigate Change in an Organization

My comments on The W. Edwards Deming Institute blog in response to:

I’d like to see some posts about how to implement change in an organization. How does one get an organization to start looking at itself as a system? How does one get the organization to realize that the most important figures are unknown and unknowable? How does one convince an organization the importance of driving out fear? In short, how does one get an organization to listen to what Deming had to say?

Thanks for your comments. We will certainly address those topics in future posts.

We have explored some similar ideas in the past, here are some links that may be useful.

Dr. Deming "Statistical principles and techniques must be rooted and nourished with patience, support, and recognition from top management."

I don't think there are simple answers to your questions that take the form of do this simple thing and what concerns you is taken care of right away. You need to work with what you can and gain credibility so people are more and more willing to listen to you. Transforming the Organization – Deming Podcast with David Langford has some good ideas.

I have written about the questions you bring up on my Curious Cat Management blog: Habits and What to Do To Create a Continual Improvement Culture.

My basic philosophy is that the way to do what you are asking is to help people improve and while doing so explain how it relates to the points you mention (fear caused the problem we had to fix...). Few believe you at first. After you help numerous times more people start to believe maybe the overall philosophy actually is worth listening to since you seem to be able to make things better and you keep tying it back to view the organization as a system, understanding variation (and what data can and cannot tell you...), etc..