Critical Ingredients to High-Performing Teams in 2020: Part 3, Metrics​

Four agile coaches including MATRIX Agile Practice Director Jeremy Wood recently led a roundtable discussion on how to maintain high-performance teams while working remotely. Metrics is one popular topic they discussed.

Have you seen a trend where managers have a greater degree of reliance on tools to get metrics and KPIs, thus losing human interaction, and are now becoming more data driven?

Jeremy Wood: I think a lot of it depends on how the company is demonstrating that value. If they're doing scrum, or at least have some teams doing scrum. If their demos are very effective. If they have stakeholders show up and they're able to show working software, then they are probably demonstrating value.

On the other hand, if the right stakeholders don't show up for meetings, it’s difficult to measure value. In those types of scenarios, since teams can't track the output, they track some of the tasks. I have seen people counting story points or how many tasks somebody completed. Or how many hours they log into a tool. That's where I see the tool and template as the goal of how we're doing, versus the measure of success being the demos and interaction and working software.

Phil Ricci: I find that some of our managers don't know what tools they are using. They have to be translated for them. As a result, our organization must try to make that translation for managers. In some cases, to take information out and present it in an attractive and simple dashboard.

Bryan Agosto: What I've seen with metrics with some teams, at least in metric-driven organizations, now that they are remote, there's a lot of manipulation happening because they need to prove that they're being productive from home. It's given managers a way to say, ‘okay, we're still good.’ And the teams may say, ‘well we delivered more value.’ But what is value to the team? ‘Well, we've delivered more code.’ They're not looking at it from a customer-centric perspective.

The conversations that we're having with leaders and teams now has a lot to do with what is valuable and what metrics are valuable for us. It’s a shift from looking at velocity, which I think is ridiculous for any leader to worry about velocity for a team, to throughput and time to build or time to value. What did it take for us to actually deliver this to production at the end of the day? And when I say time to value, I mean from soup to nuts — from when we identified the problem to the moment that we said the problem is solved.

Now I'm helping leaders and teams identify where they have bottlenecks in their flow of work and why. And can we solve them? If we can't, then who can? The metrics that we're using have evolved from basic velocity to things like throughput cumulative flow. You can be really effective at solving problems versus just trying to hit a number.

Have you found organizations slowly changing their definition of value as they've gone more remote? Has there been more of a focus on hard metrics like defects found, or lines of code versus speed to market, etc.?

Jeremy Wood: I've seen a bit more focus on some of those traditional metrics that are not necessarily “the output.” I see teams playing the game of scrum right now, and they’re good at playing the game. And the game says that user stories should be completed in a short amount of time. That people should form in them. Get them open, closed, and then accepted.

But we often miss the whole point. It's called a user story for a reason, and the behavior we tend to see is wrong because we want to measure how good we're doing right
now, and that's subjective. User stories often are not complete user stories; they are pieces of a user story that probably won’t decompose. So, we will see a UI user story, a back-end user story, an integration user story, and a regression test user story.

But at the end of the day, all those user stories must be complete before there's any value to be given to the customer.

Too often we're so focused on playing scrum that we forget the goal of the framework, which is how long it takes us to get that work out the door. We are measuring some
metrics like cycle time. From the time we start working on it, how quickly can we solve that problem for our customer? We don't solve that problem with the user interface. We don't solve it with a backend suite. We don't solve it with regression testing. We do all those things to solve that problem.

If your cycle time or lead time is horrible, that’s unfortunate. But it's real. That's a great thing because the moment it gets less worse, then you've made progress. If
your average time to create a complete a user story goes from 21 days to 20, that’s progress. And every day we close in on that, we're doing it better. It provides a clear view of what's really going on within our team and our organization. We’re not just playing games to appease somebody who wants to look at a specific metric.

Phil Ricci: But we do have to bring our customers into that interaction. We must get feedback from them and find some way to measure that based on their perception. That's a difficult thing to do. It's not something as simple as the Net Promoter Score. It's got to be deeper than that. But we're trying to give customers a scorecard and have them rate us when they
come out of our system demos.

Bryan Agosto: As a remote team, that becomes sometimes even more challenging depending on how far removed you are from your customer. In one of the organizations I'm working with now, with COVID they have a big focus on switching from projects to product focus because they realize a lot of the people that are remote have no idea who their customer is and who they're actually working for. They're creating code because of the requirements they're given and they're not solving any problems.

With remote teams, we must focus on how do we get them closer to that customer? Getting them focused on becoming closer to the product, and aligning to the product, and creating value streams that identify their products.

You mentioned Net Promoter Score as a form of feedback. Have you seen any changes to the way you get feedback from your business partners? Have you seen an improvement or the inverse, now that we're remote, have people regressed back to that forming stage where they don't want to step on each other's toes?

Jeremy Wood: Within the organizations I've been involved with, there's still a bit of a gap. The middle person to come translate isn't always the product owner, so they get some influences from that. I would say that the teams also have tried to take on work that really starts to go to organizational structure conversation, which is by far the greatest challenge for any company.

Bryan Agosto: When they all had to go remote, it was a challenge to being at home. Some of the teams regressed and didn't have the opportunity to adjust. One of the exercises I did
with a few teams during their retrospectives was an exercise that's based on liberating structures.It’s an exercise called ‘being heard, seen and respected’. And we talked about that.Has there been a time where you haven't felt heard, seen and respected? Can we talk about that in the last two weeks? And then sharing that with leaders, how can they support their teams so they can be heard. seen and respected. It’s led to thinking about their customers. Are we hearing, seeing and respecting our customers in these situations? It's a really cool exercise. It takes about 25 minutes to do with the team. If you're ever interested, look it up under liberating structures.

Phil Ricci: One of the things I've noticed is that teams are less receptive to work in that fashion, because they are fundamentally focused on delivering. Getting them to take time to introspect and to think about what makes them a better team — it’s been a difficult sell.