What causes agile to fail?
It's usually down to inertia. The inertia comes from centuries of working in a command and control way. Agile is a long-term commitment to change.
What happens, in reality, is that someone in a company makes a decision and we go along with it. Let's say the CEO comes along and says "We're doing an agile transformation. This is going to be super exciting." The person (who would have kept going the way they were before) is not yet committed to this change and may never be. In this instance, the CEO's role is to show why this person (and everyone else) should be committed.
Usually, the commitment has not been established. Agile begins to fail in these teams first. Then, once the CEO loses enthusiasm, it starts to unravel everywhere. Before you know it you are talking about agile, but you are not living it.
Agile needs full commitment to a new way of thinking, behaving, and working. It takes time to change mindsets. Once the company hits a bump in the road old habits creep back in. People want to micro-manage because they felt in control when they were doing that.
It's a bit of a catch-22 situation. Because the last few years have shown us that the world is unpredictable. In an unpredictable world agile is extremely beneficial. If your team is more nimble and able to react and change faster, you have a better chance to survive and even thrive. Yet teams need time to develop these skills and they need the right environment to do it. If at the first sign of trouble, the environment becomes unsafe. The teams will never become agile.