Lesson: Agile Development with Ian McFarland

Step #8 Practice: Learn how Agile works in practice

One of the core Agile practices is also the retrospective. And this idea is really a Lean manufacturing idea, it's a Kaizen idea, continuous improvement idea that comes straight out of Toyota Production System, the idea that you want to be continuously improving. Six Sigma has this; basically any sort of management system knows that continuous improvement is part of how you produce. Continuous improvement and zero defects are kind of big things that come out of the Lean manufacturing movement.

A retrospective is basically we try to do one, again, we tend to be more structured at the beginning, less structured over time. But we try to do one at least ever couple weeks, sit down, get everybody in the room. On a project or in an office, depending on what team structure and what process you're evaluating, you get the people who are involved in that thing together as a team, and you sit down and talk through how it's going.

This is usually structured in the form of like, "Hey, what's going well? What's going poorly?" And then there's a third bucket that's like, "What are things that kind of need to get talked about more?" That can be things that are topics, but it's not clear whether they're good or bad; it can be things that are like the beginnings of an idea; things that you notice as issues. There's lots of different ways to structure them.

What I've done for the last eight years I've been doing them is basically, again, you have the "happy" column and you have the "sad" column and you have the "needs more discussion," the sort of "not sure" column. The fastest way to do this is you give everybody a Post-it pad and they write up, do five minutes at the beginning to go up and just post whatever things in the "happy" column, in the "sad" column, in the "eh" column. Typically you'll do a little bit of grouping exercise, different people go through it different ways. What's interesting is usually there's more "happy's" than not, but that doesn't deserve as much attention because honesty what we're trying to do is solve problems.

So it's kind of funny, you always end up spending all the time in "sad" column or in the "eh" column. But you kind of want to go through and read them all off as quickly as possible so that there's time to discuss how to solve what's going poorly. Usually you want to come out with some action items. How structured that is, again, depends on the team and how effective the team is in being able to translate discussion into action. How formal the action items are is the question of that. Often, just identifying the problem, you come to new working agreements inside of the discussion. It's really important that this is not a top-down thing and it's not a blame-oriented thing.

The quality of this exercise is only as good as the feeling of trust that people have that they're going to be respected, that they're going to participate. They're only going to participate meaningfully and tell you what's wrong if it's okay for them to tell you what's wrong. And I've seen this go incredibly badly with, especially, a certain kind of entrepreneur, that we've all dealt with, who's just like, "No, that's not right," who wants to deny that things are going poorly or wants to blame the people that bring up the problems. And the fact is in healthy organizations, in trust-based organizations, calling out a problem is a gift to the team. It's like, "This isn't working, I want to call this out." Because the idea is that, "Now we'll solve it," right? And as soon as you call it out you can actually start to work on a solution to the problem.

What happens, often there's beer involved, but the idea is that it's a couple hours that are set aside for the team to really talk about what's going on in the team. It's worth talking through what's going well, because you want to reinforce that these things, often it's like we took this action and it solved this problem, so it's good to get that feedback about things that are going well. But you do spend most of the time like, “Our build is too long, it's taking too long to run,” or “We had this weird interaction with a client and we want to solve that relationship issue,” or “There's a team dynamics problem,” or “We're not switching pairs often enough,” or whatever it is. The idea is by having a forum to continuously discuss what is working and what's not, you make more things work and solve problems that are causing drag in the organization.

Standups are great. They're a super lightweight, low ceremony way for communication to happen. All sorts of organizations have adopted them even if they don't do full agile. The idea of a standup meeting as a type of meeting comes from the idea that you don't want long meetings. You want really short focused meetings. And it's kind of quaint really, but the idea that if you do it by actually standing up people will become restless and will naturally not go on terribly long, because they become tired and they can't stand for an hour long meeting. Now, that said, I would be the guy that kept standing at desks and I tend to not actually sit most of the day. But the idea really is to have short meetings that are meaningful.

A typical standup, the classic standup was like, "What did you do yesterday? What are you doing today? Anything that you need help with?" That still works for small teams and I think that's a pretty effective place to start. What we've done with larger teams is to really focus it on not everybody needs to speak. If there's anything that anybody's stuck on and they need help with, the larger group can help with, that's something to share. If there's anything that they've discovered that's broadly useful or interesting, that's something to share like, "Oh, I found this quirk in IE that if you do this thing it's a workaround" Or, "There is this great article on gene structure. Go read it, if you're interested."

Those are the kinds of things to share, because again you want to have this be like a ten minute meeting or less, as short as possible. Because what you're trying to do is create general awareness of what's going on, organizationally, so everybody knows what everybody's doing. You get a lot of that if you're pairing people know what's on in the club base already, so there's not as much need for that deeper communication around the project. It's really just like, "Oh, here's generally where things are." So it's a way for people to re-contextualize themselves and go, "Oh, yeah. We're working on this, this, this. This is the arc. This is the progress that I'm making."

More and more of that stuff also gets externalized into other tools that create visibility. But the core idea is to have good concise rapid communication among everybody who's a stakeholder and so that where there are places where two people should be interacting where they may not be thinking they're interacting that there's a context to that for that relationship to be bridged. Like, "I'm having trouble with how iTunes RSS stuff works." "Oh, I did that last week. Here's what you do." Or, "Here's a resource." Or, "Oh, I have an answer to that. I'm going to come talk to you after standup." The idea is this just a brokering. No actual deep communication should happen during the standup. It should just be like, "Hey, does anybody need anything from anybody else?" "Yes, me. I need this thing." "You, I have the answer." You go talk after.

I talk about code debt a lot. So, first of all, let's define what code debt is. An example, your team is running around and you don't have time to do things right because you have so much pressure. This is one of the great fallacies of software development, and most things is the idea that doing things properly takes more time. It does in isolation. Doing something correctly once does sometimes take more time than doing it incorrectly at the time. But over the life of a project, you then have to pay for the failures of not doing it correctly over and over and over again. So that's the notion of code debt, of technical debt in general, or product debt, or whatever kind. Any time you think of systems and agile language around the space, when you're talking about debt, just like financial debt, you're talking about a thing that you didn't pay the full amount for, that you're then having to pay the ongoing cost over time.

It's really important to understand that dynamic. The simplest place to start, if you want to start one place, is to remember to pay down your technical debt. So, I talked about red-green refactor. That's a great example of that. You do a first pass implementation that gets the test green and it's not very good yet. So it's really useful at that point, at that very time to go through and clean up the code so that it's easy to maintain, because it's a lot cheaper to do then, you already know what code you're trying to work on, it's a much smaller incremental investment, you don't have to relearn the context. So it's a much smaller incremental investment at that point to finish making the code correct.

Doing refactorings from time to time in the code base, any time the code is becoming too hard to work in, and this happens all the time in code bases. Any project that isn't rigorously refactoring, you will find very rapidly that the things that used to take ten minutes to do at the beginning of the project are now taking an hour to do or two hours to do, or are really painful or nobody wants to do them. So that is a really strong signal that you have unsustainable technical debt.

So the way that you sustain unsustainable technical debt is you pay it down. You go and spend an hour a day, a week. And ideally you're doing this in smaller increments every day so it doesn't take a week or two weeks to get there. But you go through and you fix those core problems with the software. So if every time we do this it takes ten minutes to do this deployment, that means you have to look at that investment. If it takes you four hours to fix and you do this deployment five times a day, that's 50 minutes a day. By the end of the week, you will have paid for the four hours of investment.

There is a school of thought and the pivotal model was that you must always pay down your technical debt. There should be zero technical debt. That's not actually true. I think the main important thing to do is to be conscious of it. I'm not saying you should go out and measure and actually build a spreadsheet for every step that you take. Are you getting the return you want out of it? It's inefficient to do it that way. It may be correct, but then you have to count the cost of the accounting on top of the doing. So at some point you want to have lightweight heuristics. This is always the balance. You always want to sort of look at the lightweight heuristics that give you the result that you want and not really do deep analysis if things are working well enough and you're happy with the outcome. Every now and then you might want to do a deeper analysis.

But at core, looking at technical debt, there are a few things to consider when you're taking on technical debt. Sometimes you have a crunch time and you need to get that feature up by a certain date. So if that date is really important, let's say you have a week before that deadline and really doing the feature well would take two weeks, but you're going to do this sort of crappy job and get it out the door. Let's assume then that by doing it in a week, it's going to take another four weeks to fix it later. It still might be worth doing it because you have this deadline. So you have to decide is if it's worth having 5X the aggregate cost to get the result one week sooner.

Another special case is what I think of as code bankruptcy. That's where you're doing something that you're not sure is going to work. This happens all the time in early-stage startups. You're building something, but you never know if it's going to have value until you put it on the market. So don't over invest in the perfect code base for that first thing. So understand if it was going to cost you two units to do it correctly, and you can get it into market for one unit, and then you throw that away, and then pay the full two units for it if it actually has signal, then if there's a better than one in three chance that it's going to fail, it was actually a cheaper actuarial risk to do the short, cheap version, throw it out there and see if it works. So that's the big framework for me in terms of how I think about technical debt.

When people are new at this, being more structured about it, making an in to action items, usually there is not room for it, maybe there's room for two or three actions items that come out of a given retrospective. And it could be like, "Who is going to solve the building issue?" "Who is going to talk to the client and try to resolve that client engagement issue?" And different people have different skill sets around that.

Sometimes many engineering centric cultures forget that most of this is actually soft skills stuff, like that client engagement issues isn't "You told us this thing and you are wrong." Its "I see that you are trying to do the right thing here but I want you to understand the impact is has on the team when you give us last minute changes so that you could make informed decisions about whether you want to make last minute changes.” Figuring out who in the team is the right person to run to with the feedback is important. But honestly well functioning teams tend to do pretty well, translating that stuff into action or if not it shows up the next time as "We didn't resolve that problem with the..." whatever.

A lot of it is having an ownership culture and having people step up and say, "I really want us to solve this problem." The real pain points, it’s generally stuff that somebody wants to solve and if it’s not then somebody in the senior position in the organization needs to step up and say, "I'm going to figure out how to solve this problem." If you are the entrepreneur, if you are the team leader, its incumbent on you to make sure that you are taking that feedback and making something meaningful happen out of it. If it just turns into a discussion and the same problems shows up on the sad column all the time, people are going to lose interest in this exercise. You can't talk about continuous improvement without having any improvement for very long before people get frustrated.

Sustainable pace is really, really important. It's really easy for startup folks in particular to think, "Oh well we'll have sustainable pace when we get to scale." So, sustainable pace, life/work balance, these are all kinds of ideas in the same space. But the fact is you can only work on overdrive for so long and you're kind of using up your ability to do that. It's okay to have a crunch time. So, one, most deadlines that we think are deadlines are not deadlines. There are occasional deadlines. I used to work in architecture. The fact is in commercial real estate the only time that the store opens is the week before Thanksgiving because that month of revenue, that month of store open time is 40% of the revenue for the year. There are real deadlines in the world and it is important to recognize that they are real deadlines.

Sometimes there is a very specific thing that you need to be in line for but that is really kind of the exceptional case. And it is okay to have crunch time for that, but you have to understand that your aggregate throughput goes down when you have crunch time all the time. We know this. The reason we don't have finals week every week in college is because it wouldn't work. You can only cram for an exam for a certain amount of time before you really should just go to bed. So what happens a lot with startups and you see this in the gaming space all the time. There is even case law in California because EA was too hard on their developers. Now you have to earn more than $90,000 a year to be an exempt employee if you work in software because of EA. They were fine burning out their developers.

You can't do that in a competitive. One, you can't do it from an HR perspective. You're going to lose your people if you drive them too hard. But, the more important thing is it is just not even in your self interest. If somebody is working 80 hours a week, they are not doing good quality work after the first week. You're too tired. You can't make good decisions. What does a software developer or designer do? They make decisions all day. It's not the quantity of decisions that you're hiring them for, it's the quality of the decisions they make. So, having enough rest and having enough distance to be effective. That's kind of the thing you're managing for.

Honestly a 40 hour week is probably too long. But, it's something that's pretty sustainable and that's what we do here at Neo. People come in to work in the morning, leave at 5 or 6 and they have a home life. By the way if you're pair programming all the time, it's funny, most software developers when they go home will work on coding projects. Most people who hardcore Agile, they use it all up in the space of that 40 hour week and don't want to go home and program stuff. Some do, but definitely doing something outside of work that is not work is really enriching. That's the other thing. I think having people that are a little bit more diverse and have more interesting backgrounds makes for better products. You're going to have better empathy for people that you're building products for if you're out socializing and spending time in the world. I think just people are happier when they have a reasonable balance between their work and their home life.