How Amazon Raised the Hiring Bar and Leveraged Narratives for Decision-Making

Aydin Mirzaee (Fellow.app)  00:00

Colin, welcome to the show. 

Colin Bryar (Working Backwards)  03:09

Oh, thank you. Thanks so much for having me on the show.

Aydin Mirzaee (Fellow.app)  03:12

When I first reached out, I had mentioned that I had read the book twice. And when I say read, it usually means I doubled it. And you know, just in getting prepared for the discussion, I kind of power listens to a bunch of the sections all over again. So I’m very excited to get a chance to chat with you. The thing that I wanted to call out is you and Bill obviously, you know wrote this book, long time employees at Amazon, how many people worked at Amazon, when you first started working there?

Colin Bryar (Working Backwards)  03:45

When I started to work at Amazon, it was March of 1998. And we had about 100 people in the corporate area of Amazon and then 400 people additionally, and we had two fulfillment centers at this time. And then customer service. So about 500 total. And then 100 in the corporate area product development team was small enough to where we could have the whole team crammed into one medium-sized conference room, we’d be sitting around the table and then around the edges of the walls there too. But it was a little bit cozy, but we can all get together and talk about what we wanted to do and build next. Yeah, no,

Aydin Mirzaee (Fellow.app)  04:26

That’s super interesting. And then obviously, I think today Amazon has well over a million employees, something like that, right? 

Colin Bryar (Working Backwards)  04:32

The other thing is 1.3 and counting. So last time, I looked a little bit more than them when we started.

Aydin Mirzaee (Fellow.app)  04:38

Yeah, so this is super interesting. I mean, building a company like that. And keeping it a well-functioning machine is not easy. And so you talk a lot about various principles and also frameworks and so we’re gonna dive into a bunch of those. But I wanted to ask you so obviously you know when you first started working there and as you start to grow and even before that you held leadership roles at other companies as well. So one thing that we like to do on the Supermanagers podcast asks you, when you first started managing and leading teams, you may have made some mistakes or errors and wondering if there’s anything specifically that you’d like to call out something that you did in the early days. And later on through practice and learning. Maybe you did less.

Colin Bryar (Working Backwards)  05:23

Yeah, so I mean, it was probably the, I’m dating myself, but you know, the mid-90s when I started to manage people, and, you know, one mistake I made was not realizing when the best solution to a problem was to hire your way out out of that problem, where he just didn’t have enough people. And so you kept trying harder, or doing more yourself or asking your team to do more. And you know, there are only 168 hours in the week, and you need to do other things and work their work in those that time. And so those hours, so hiring often is the best solution. But it’s hard to set aside the time and effort to go find the right candidates to join the team. But ultimately, that is, is the only thing that will get you the bandwidth to solve those problems. I just say that’s one mistake I made early on. And now I feel like I’m cognizant of when that is the solution, and it put the effort in. And then a second one is, it’s not about you, as a manager or leader, it’s about the people that you’re leading. And so just keeping in mind that is as a leader or manager, your goal is to help make your team collectively do more than it could otherwise. And so you need to figure out what those things are, what the obstacles are, or what help you need to provide. And it took me a little while, to understand that, and to go out and seek what those obstacles are and then try to figure out how to remove them. So those would be two mistakes I’ve personally made and, and how I incorporated that into my career moving forward. Yeah, that’s

Aydin Mirzaee (Fellow.app)  07:01

Super interesting. I wanted to get on the higher your way out of the problem. How do you make that decision? And assuming that there’s no constraints, it’s, it is kind of hard to figure out, you know, when is this within our capability? Or, like, when do we need to find expertise to solve this problem? Are there any kind of rules of thumb or things that you look for to make that kind of a decision? Well,

Colin Bryar (Working Backwards)  07:25

I mean, one indicator is if you’re always prioritizing and reprioritizing because you know, there’s so much that you want to get done. But you can only handle a limited part of that. And then when you look at the list of the things that didn’t cut, that while we really should be doing these types of things, you know, that’s an obvious indicator that if you had more people or resources, you could, you know, accomplish more as a group or as an organization. You know, I think that is that’s one thing, and I’ve seen this more with, you know, first time or relatively new managers is their open headcount that the, you know, the tbh has to be hired, they sit on that spreadsheet for weeks and months and quarters. And that’s when you realize the team is just, it’s so busy working on the now and, you know, fighting fires, that you’re not devoting the time to build, to build, and grow the team. So those would be two indicators of, of when, when that happens when you have your annual planning process or even, you know, sometimes it’s quarterly, just be open and honest and looked at and ask yourself, you know if we could invest in these other activities, would we and what would those be? And you know, what, I would I spend my own money on this if I were looking at it as an investment? And if the answer is yes, then you move forward quickly, because you’ll get to that, that you’ll get to that outcome a lot quicker.

Aydin Mirzaee (Fellow.app)  08:52

Yeah. And so I guess one of the related questions is say that there is a, you know, a business that you’re trying to get into, or a problem that you’re trying to solve, you may have the bandwidth to work on it, but for whatever reason, you’re not making progress. And then you know, part of it is that there are many potential answers to this either it’s an unsolvable problem, either you don’t have the right team, you know, you just need to keep trying. So in those more nuanced situations, how do you figure out that like, maybe there’s the expertise that you can bring in someone, and they can unlock this problem in a way that maybe your current team can’t? 

Colin Bryar (Working Backwards)  09:28

Well, usually when I’ve seen the case, where there’s a team, maybe not a team, but a set of people working on an issue, and not making traction? A lot of times it’s said that the problem itself isn’t completely defined. And you know, Amazon has some practices, you know, which we may get into later about the working backwards process, which that’s an iterative process. And really, the first thing you do is you ask yourself, what is the customer problem or the set of customer problems I’m trying to solve and it sounds very But especially when you’re pioneering in a new area, it’s quite difficult. And, you know, use an example with web services. You know, the term cloud computing didn’t even exist at this point. But we were looking out the web services technology and figuring and knew that there was something there, but didn’t know what the right product to build was. And we didn’t focus on what should we what’s the product we should build, we focus on what are real problems are facing software developers today inside our organization and outside of Amazon, and how can we build a product that solves those problems, etc, to the It started as, as we wanted to help teams provision hardware so they could build and deploy services on the web. And as we peel off the layers of the onion to get to what the core of that problem was, it turned out wasn’t provisioning we were selling compute power and compute cycles. And the light bulb clicked on when you know someone wrote in I don’t know who it was, but they get a ton of credit for that we wanted to provide people in a college dorm room, you know, software, developer and, and college dorm room, the same access to World Class compute infrastructure, as someone working at an Amazon and Microsoft and Google and IBM, and that really, the light went on, then that told us, okay, here’s what we need to do. And there were some unsolved problems. And we had to go tackle on how to do that. And that, that you begin to realize then what are the what’s the expertise we need to have, that we don’t have on staff? And you know, do you have to partner with someone or do go the long path in and hire and acquire that resource, those types of skills on your own? But I think once you understand the problem, and what the challenges to solving that problem are, then it becomes much clearer on how to go do that?

Aydin Mirzaee (Fellow.app)  12:02

Yeah, I think that’s super smart. It’s a lot of times it’s the problem definition. I’ll ask you more about the working backwards in the PRFAQ, talking about hiring the expertise. So say that you decide that you are going to hire the expertise. You know, hiring is, is always challenging, but particularly as you start to scale. I know in the book you were talking about how you had new people who hired new people who hired new people, and you wanted to keep the standard of hiring and you want to keep the bar high. So you created this, this process called the bar raiser process would love for you to elaborate at a high level. What is that? 

Colin Bryar (Working Backwards)  12:44

Yeah, so it’s one of Amazon’s first scalable repeatable processes that I think allowed it to transition from this you know, relatively small 100 people corporate company into a fast-growing you know, large multi-billion dollar corporation but at its heart, it has a couple of components. One is it uses behavioral interviewing, in which you asked questions about past behavior on how candidates exhibited in embodies Amazon leadership principles to predict future performance that we found that was a better way to do it than brain teasers in it removed some bias to interview confirmation bias. The second part is the interviewing process. It’s a data-driven exercise where each of the interviewers is assigned a set of Amazon leadership principles to go vet a candidate, and they have a specific job going into the interview. By definition, they don’t have all of the pieces of the puzzle to understand, you know, to make a comprehensive hire no-hire decision, but they need to go in and get data for a subset of information about that candidate. And then it all comes together at the debrief for you to put those pieces of the puzzle together. Verde submitted your feedback. You can’t tell anyone else about your interview, hey, this person’s great, can you just do a cell interview that you don’t want that bias to creep in because everyone is trying to get their information, you write it down, you make your vote, and then in the debrief, you put this information together and you read it holistically where you get that’s the first time you get a whole holistic picture of the candidate. And at that point, it’s okay to change your vote, because now you’ve seen all the information and then the other part about the hiring processes there. It’s called the bar raiser process, but there’s also a specific role called the bar raiser. The bar raiser is not in the chain of command for the year it doesn’t report to the hiring manager. So it’s an independent role and their job is to ensure that the candidate if they hire them, it raises the bar at me you know, at Amazon, it makes you know, Amazon overall, the talent pool quality increased and so they don’t Have urgency bias or to hire the candidate because we need three people are not going to hit our goals. What they do is they’re trained interviewers, and they’re supposed to help the hiring team make the right decision. They do have veto power. Although in practice that’s rarely used, because a great bar raiser Will you know, in that debrief will collectively guide the debrief panel to make a hiring decision. So it’s you know, for new managers or it’s new to Amazon, it’s uncomfortable to say I don’t get a choice whether I want to hire this person or not someone else can Trump my decision. But you realize that having everyone come in with different sets of information and having an independent trained expert to help the whole group make the right hiring decision, just insert ensures consistency the, the company. And it’s, you know, it’s a lightweight process, and it’s scalable and repeatable and has a good feedback loop those debrief meetings are a great time to learn more about becoming a better interviewer. 

Aydin Mirzaee (Fellow.app)  16:02

Yeah, I have so many questions about this. First of all, I love the process, because I assumed that the original bar raiser was Jeff himself. And then you know, the question is, how do you continue to scale that as time goes on? My question is like talking about the leadership principles and getting every person to be responsible for teasing behavioral situations for each one of those however many they get assigned, are there then this is very tactical, but are there then a series of, you know, questions, or like a question bank that you start using? And everybody can kind of pick and choose from those? Or do you leave it up to each person to decide how does that part work,

Colin Bryar (Working Backwards)  16:44

There is a knowledge base in your question bank that you can, you can use or get inspiration from, you know, when you’re also when you’re first starting to interview, you undergo interview training, and you sit in on an interview, you’re paired with an experienced interviewer, the great thing about it is you can be trained to be a great interviewer. But it’s not always intuitive. So you need some guidance along the way. It’s a teachable skill, which is fantastic. But like, when I go into an interview, I already have my list of questions that I’m going to ask sometimes I go off script, depending on the answers that I hear, but I know that I’ve got to get this information. And I need to write my feedback and defended the feedback in front of my peers. So I know I have a very specific job, I need to get done. And I need to make sure that that comes out in the interview. And as you do it over and over again, you learn tips and tools, and you add to this knowledge base of questions or you ask well, how did you get that information out of this candidate? I couldn’t do that. And then you just, you know, that’s where that feedback loop comes in. So those are a couple of tips, you know, go into the interview, knowing what you need to come out of the interview with and have a plan of action on how you’re going to get that information. Rather than just you know, some people I’ve seen come into interviews at companies, they’re looking at the resume for the first time and oh, tell me about your resume. And they’re not even listening to the answer. They’re thinking about, well, what questions Can I ask this candidate? You know, that you can save some time, everyone some time by putting a little bit of effort in upfront about knowing what your goal and you know, is to achieve during the interview? 

Aydin Mirzaee (Fellow.app)  18:26

Yeah, I think that makes a lot of sense. So people then do the ranking, they write the written feedback, they don’t talk to each other beforehand so that you don’t have a group thing situation. And then during the debrief, this is not like the ultimate decision-maker is still the hiring manager, except if the bar raiser raises an exception. Right. But it’s not like a decision by committee. 

Colin Bryar (Working Backwards)  18:49

Yeah, the interviewing panel is there to help the hiring manager make a great hiring decision. So yeah, the hiring manager will ultimately decide, you know, am I going to hire this person, you know, assuming there’s no veto again, which rarely happens. But yeah, so it’s not, it is ultimately up to the hiring manager. And then it’s up to the hiring manager, by the way, to go close the candidate to it’s not let’s push this person over to HR, because people join companies really, because of the team they’re going to work with and the manager that they’re going to work with. That’s also the reason why a lot of people leave the company to is the relationship with their manager. So you want to from that first day, you want to establish a great relationship with the candidate and make sure that they’re on board.

Aydin Mirzaee (Fellow.app)  19:36

[AD BREAK BEINGS] Hey there before moving to the next part of the interview, quick interjection to tell you about one of the internet’s best-kept secrets, the manager TLDR newsletter. So every two weeks we read the best content out there, the greatest articles, the advice, the case studies, whatever the latest and greatest is, we summarize it and we send it to your inbox. We know you don’t have Time to read everything. But because we’re doing the work, we’ll summarize it and send it to your inbox once every two weeks. And the best news is completely free. So go on over to fellow dot app slash newsletter, and sign up today. And with that said, let’s go back to the interview. [AD BREAK ENDS]So one of the things that like, you know, the name bar raiser is so apt because it’s, you know, just the name of the process kind of reinforces this concept of trying to hire people that are at least in some way better than the average of what the team, you know, currently has. So what are some of the things that the bar raiser can do? Or some of the questions that maybe they can ask to make sure that that’s the case? You know, you know, running the process is very important. And if you do all these things, you’ll end up with better hiring, hiring decision, regardless, but are there specific things like for example, do you specifically ask how is this person better than in some way than whoever else we have on the team.

Colin Bryar (Working Backwards)  21:03

One of the things, when you’re going into an interview, is just finding out what that candidate specifically did in different instances versus, you know, it’s just by nature. Some people like to talk about I worked on a project and it saved the company $50 million over the year, but you’re understanding what was their role on the project? Would that project have happened without them? So you know, one, one question I’ll often ask is, well, if we remove you from the equation of what you worked on, what wouldn’t have gotten done? And, you know, sometimes, as well, I generated reports for the group or? Well, you know, and sometimes people say, humbly, to like, well, it wouldn’t have started because it was my idea. And I just kind of shepherded the whole thing through, you tease out what was their specific role. And then the second thing that we look for is, you know, one, Amazon’s 14 leadership principles, no one’s perfect at all, 14 of them. Yeah, no one and but just to find out, you want to know, find out a lot of, there’s some dimension, which their superstar, and you want to find out what that is, and if that dimension happens to help the group or the company or the team, you know, that is that is where you can, you can raise, that person can raise the bar, so understanding where they are off the charts, you know, on certain area or areas, knowing that trying to figure that out during the process interview process helps

Aydin Mirzaee (Fellow.app)  22:27

That makes a lot of sense in so and then if it is a job of a technical nature, that happens, there’s probably like a separate process for that or a test of some sort, you know, other ways of evaluating skill, this has passed the, you know, the person has the skills, now we’re trying to figure out if they’re a good fit for Amazon.

Colin Bryar (Working Backwards)  22:46

Yeah, the functional skills, especially for things like software engineers, data scientists, lawyers, you know, those are certainly important. And it’s just, you know, each company has kind of a different way to vet those were. So that is important, we just cover the mapping it to the leadership principles. Because what you want, especially for a young organization, you want to bring on people who are going to reinforce your culture or leadership principles, and you want to be deliberate about that because if you’re not deliberate about that, they’re going to have their principles are cultures that they’re going to abide by. And if you’re an organization that’s going from 10 to 30, let’s say over the year, you know, at the end of that year, you’re going to have a different company. Because you know, the 20 new people who came in are just going to be doing typically what they did at their old company. And so when we find organizations who say, Well, you know, our company’s just not like it used to be it’s either because they didn’t define who they were, with a, you know, a very codified set of leadership principles that this is what’s unique about us as an organization, or they didn’t have a deliberate hiring process to make sure that the candidates mapped on to that those are two good ways to lose control over who you are as a company.

Aydin Mirzaee (Fellow.app)  24:01

So your leadership principles are also not I mean, these are your other people might call these their company values. It’s not, it applies to everyone in the company.

Colin Bryar (Working Backwards)  24:11

Yeah, everyone’s a leader and your company’s common core values, leadership principles. You know, those are interchangeable, but it really defines who you are, and then how you want employees at your company or in your group to make tough decisions very quickly, when they don’t have all the information they need, and you’re not in the room. So you want to make sure they have something to fall back on. And it’s the same set of first principles to make that decision. And that’s I think, another reason why Amazon has had remarkably consistent types of decisions and behavior across a wide variety of businesses is that they all are falling on the back on those 14 leadership principles to make the tough calls. You know, when Jeff Bezos is not in the room, or when the senior leaders aren’t in the room, got it.

Aydin Mirzaee (Fellow.app)  24:59

So the bar raiser helps you bring the people into the organization. Is there any sort of involvement, you know, afterward, you know, months afterward or years afterward or the official role of the bar raiser stops at the hiring?

Colin Bryar (Working Backwards)  25:13

The bar raiser it’s just for the hire you know, just for hiring but what doesn’t stop is those 14 leadership principles woven into every important process that the company does. And there you know, they’re those processes were built from the ground up, keep with those principles in mind. So that part never stops. But the bar raiser process, it’s a hiring process.

Aydin Mirzaee (Fellow.app)  25:37

I mean, we talked about this a little bit in you know, debrief. It sounds like there’s a written report everybody’s writing, you know, feedback, and I assume if the feedback is mostly in narrative form format, I know that you know, you at Amazon, have, you know, been famous for getting rid of PowerPoints and shifting over to narratives and specifically like you culminating in this six Page Six pager format, I’d love for you to describe, you know, some of the key highlights of like, why to ditch a PowerPoint and like why a narrative is better. And then I have some specific questions too.

Colin Bryar (Working Backwards)  26:04

Narratives are a better tool for decision making for still use PowerPoint if it’s a one to many presentations, you know, an all-hands meeting, but just for a decision-making forum, we found narratives or just allows you to make better decisions. So that is why we switched to narratives. It was back in 2004. And it started onis sayingb basis as management, team meetings, teams would come in and we prayed they would present a PowerPoint to, you know, to us about either a decision that they needed to make or an update on a business, or, hey, should we go and acquire this company are moving to new market issues are getting more and more complicated, the stakes are getting higher, and higher, and we realize we’re not making we’re getting through these presentations, people, you know, we’re jumping around asking questions, that presentation would never get finished. And we’ve been studying different ways on how to improve decision-making. And, you know, kudos to Edward Tufte. He was one of the things we he would read some of his books and essays and he came in and spoke, you know, inspired us and Jeff one, one day after particular painful esteem, that’s his management team meeting, there were four hours every Tuesday and Wednesday, he said, Okay, we’re gonna switch to narratives. A couple of things that was notable about this, she said, there’s no more PowerPoint, you need to come in and read with the written narrative. When it was an experiment, we didn’t know whether it would work or not, if it didn’t work like it was reversible. So we’ve kind of gone back to using slides like every other company was using for those types of forums. The second thing that we wanted to try to do in the first narrative is not very good, to be honest. So it takes a couple of iterations to get your narrative stride. But the reason we switched and Jeff later called this move, one of the best decisions Amazon had ever made, was several reasons. One is that it forces the writer or writing team because they’re often not written just by a single individual. To have clarity of thought, it’s, it’s easy to throw a bunch of slides together and get up and talk about a topic. It doesn’t take that much time to do quite honestly. But to write a well structured six-page narrative, where that clarity of thought shines through from the first page to you know, to the last is hard, it takes more than one night, you can’t write a narrative the night before, it’s incumbent upon the writing team to get their story straight before they’re coming to another group and asking for feedback or help. The second thing is just pure information density, you got about seven to nine times pixel density on a written document versus big bulleted Point slides. And people also read faster than then they talk so you can get through a lot more information. And you can cover multi-causal arguments a lot faster and a lot better in narratives versus slides, the hierarchical slide and the next slide, the next slide. And everyone reads the whole document before they comment on it too. So all in you get about 20 times, you know, but anywhere between 10 to 20 times as much information flowing from the presenting team, to the audience, and you’re gonna make better decisions. If you have more information in the same unit of time, you know, an hour of listening to and asking during a slide presentation is the same as an hour of reading where the way that Amazon meetings work as the first 20 minutes you read the narrative and there’s a massive amount of information being transferred from the writing team to the audience. Then you have 40 minutes of high-quality discussion, debate, and decision making based on that information. Let’s Still the same hour, you just get 20 times as much information. So you’ll leave that meeting with a better decision more often than not, is if you use a tool where you get 20 times less information. So that’s why Amazon did it and why we found that they work better.  Yeah, that’s super fascinating. And I 100% agree with the notion of forcing you to write something like this is going to force clarity of thought. And it will be obvious if you wrote it the night before. So 100% agree with those concepts. It sounds like this is a tool that you can use for decision-making. And it’s particularly great for decision-making. So one of the things that were ambiguous for me is when do you use the six-page narrative process in general? And like, what kind of meetings are they used for? Are they used for, you know, recurring sort of staff or weekly Business Review meetings? Or are they use for only meetings where the purpose is to get to a decision? Yeah, so things like your annual plans use narratives for that quarterly business reviews where you’re recapping progress, what are you know, what went well? What didn’t we? Or what are we going to do next, where the obstacles we need help on removing monthly business reviews, if it’s a weekly business reviewer, a data deck, that’s not a narrative, that’s typically a well structured set of data that you look through, there is a specific type of narrative, called a PR FAQ that Amazon uses presser stands for a press release, and frequently asked questions, Amazon uses narratives, to take an idea to vet it to decide where they want to greenlight it to turn it into a product or service. So that is what it is. And also, if you have any new idea, or if you’re trying to present something, rather than just talk about it, at a meeting, say, let’s get everyone together that typically what happens is you’ll be asked to write a document, it could either just be a six, you know, three, or you know, six pages or less than an hour, so you don’t have to be six pages. Or if it’s a few, it’s an idea, you want to turn into a product. It’s a very specialized form of narrative called a PR FAQ document, you’ll get asked to write that first before you get a group of people together to you know, to decide what to do next. And then the cases where it’s not used, you know, weekly Business Review, where you’re looking at lots of data, well-structured data, you know, all-hands meetings, or things like that you don’t pass out narratives to everyone and, and have them comment online.

Aydin Mirzaee (Fellow.app)  32:35

Yeah, so it’s not like I mean, just a day in the life of calling the leader, how many? How often would you end up writing one of these narratives,

Colin Bryar (Working Backwards)  32:44

I would review more narratives. When I was working with Jeff, and then when, you know, when, when I went into other parts of the organization afterward, and then other companies, I would read and comment on two narratives two to three narratives a day, you know, sometimes with working with Jeff Bezos, when I was chief of staff was more than that, you know, go from one meeting to the next, you know, 60 ish minutes, we’d read a narrative, you know, comments, I’d have a discussion go on a completely different topic that was the primary way of, you know, getting information over to, to Jeff in a very efficient managed manner. So he could give high-quality feedback and detailed feedback on the issue at hand. So, you know, senior leaders at Amazon several a day, they would read in terms of writing them, you know, again, you can’t write them overnight. So you can’t, if you did it every day, you’d have to have a long pipeline of working on it. You know, I would write narrative is still do write narratives. Occasionally, we wrote a narrative when we’re, before we were starting the book, rebrand or a PR FAQ document, about working backward to see if there’s anything there 

Aydin Mirzaee (Fellow.app)  33:58

So that’s interesting. So if you have to say, a, you know, recurring monthly business update, something like that, that’s where you would use this sort of narrative format, like that would be a prime case of that.

Colin Bryar (Working Backwards)  34:09

Yeah, yeah. So that would be a prime cause. Or you go in and the team has an update on or has a new idea that they want our feature that they want to propose it could be the search team has got a new idea on on on something, and they would read in a narrative about that, even redesigned, by the way, that are largely UI driven. The first part would be a narrative. And then you could look at mock-ups, but the narrative helps set the stage on what is it we are trying to accomplish? What are you know, what are the goals? Why are we doing this? What does success look like? Then you can look at those mock-ups and give high-quality feedback. Whereas if you just say, Hey, we’re redesigning this, let’s go look up mock-ups. Everyone has a slightly different, you know, set of first principles on what it looks like and everyone’s an expert. On UI, so they’ll give different feedback that’s not entirely helpful for, you know, for the team moving forward. So narratives, even in that case, can help set the stage.

Aydin Mirzaee (Fellow.app)  35:10

So you mentioned a few cases where narrative can be useful. Do you then have templates are there must be like, you know, example ones that circulate in the company where it’s like, oh, proposing a new idea or redesign. And you can use the same format.

Colin Bryar (Working Backwards)  35:25

Yeah. So that once you start in this way, it took Amazon a little, and I’m talking a matter of weeks or months. So it’s not an organization that can start writing narratives. Tomorrow, it’s an easy change to make. And it takes a little bit, but not that long to get good at it. And really, what happens is narratives that are, you know, examples of shining clarity of thought get passed around the company, even though it’s a different, completely different area of business, you’ll see, oh, this is what good looks like this are the the, you know, the information density that I need. This is, you know, the story, our plot would like, for instance, flowery language is discouraged in, in narratives, because it’s more about uncovering the truth. And then it is about trying to sell your idea. And so you can see when, and then some documents that didn’t go too well, those get circulated too, by the way. But so you don’t have to have templates because you have the actual narratives. To know that, hey, this is a great one, I heard this went well, can I read this? Or you know, you go, you go find it. So that is one of the nice things about narratives is, you know, where the bar is relatively quickly? And is, you know, in terms of what makes a great narrative.

Aydin Mirzaee (Fellow.app)  36:40

I wanted to spend some time talking about the weekly business review. I’m just curious, like, on an ongoing basis, it sounds like, you know, at a very senior level, there were many business updates, or maybe that was a recurring meeting that you might have had with different groups. There’s a weekly business review at the executive or the esteem level, I assume you had someone on one meeting with, you know, individuals that reported directly to you. What other types of meetings would you have normally had that were say? I guess just recurring in nature. 

Colin Bryar (Working Backwards)  37:18

In terms of a leader, how you spend your time is probably one of the most important decisions you can make. And so you can step back and think about, you know, what percentage of the time do I want to devote to these different issues? So first of all, I think one on ones with your directs are incredibly important. So I, you know, tried to schedule those and then schedule things around that, you know, I found it particularly frustrating I did have some managers were, the one thing I would know is my one on one would get moved three or four times if I had it, you know, both before it came up in the week. So I try not to do that I tried to get predictability for my direct reports. Because again, my time as a manager is to help them be successful. It’s hard to do that when they don’t know when they can ask questions, or they don’t know when they’re going to meet with me next. So I think having one on ones is incredibly important to stay connected to the details of your organization, or what’s going on with your business is incredibly important to I’ve seen some other companies, managers, or leaders think it was my role increases and in scope at the company, I can get more and more removed from the details of the business. So at Amazon, we rejected that approach. And we said, you know, for that same investment of time, how can we stay connected, more connected or connected at a deep level to a wider variety of activities that are happening? And so the with something like a weekly Business Review, is incredibly important to you to look at all of the details of the business to know What did my customers experience last week? Are we on track meeting, you know, too, to our plans, and are we trying to get better, you as a manager need to know that know those details, and you need to figure out a way to get that information, you have it for instance when you’re a sole proprietor, you’re running spring or a store or a coffee shop, you can see how long the lines are, you can see if people like the coffee, you know, when people look in and the lines too long, they walk away, you know, if your vendors are late and you don’t have all the product you need to sell the impact of that. You get more and more proxy layers between you and your customer metrics are the way that you remove those proxies to end you have to have the right types of metrics which are called customer-facing controllable input metrics to know what in my customer’s experience last week, so to try to figure out how to get that information is incredibly important. So there’s the one on ones staying connected with your business. And then another principle to keep in mind is I tend to focus on what the put my time on what are the biggest opportunities for my Group, not necessarily what’s the biggest business? Or what is the most pressing problem? And, you know, to give an example, when when I was working with Jeff Bezos, in the mid-2000s is his chief of staff. You know, if you’re looking at Jeff’s calendar, then, you know, multi-billion dollar business, but probably about half of his time was focused on figuring out what if this webs, there’s anything to this web services thing, which would eventually become AWS, figuring out how to make the digital transition, those two businesses combined had revenue of roughly zero, but he was spending half of his time on that to try because he knew those were two big opportunities for the company, he had capable leaders and you know, Jeff Wilkie and the fulfillment inside of the business, and then you know, Diego PSN teeny on the retail side, so he could have the bandwidth to focus on those details. So that’s the other thing, I would just advise us to know what your big opportunities are, and where you want to make your mark and figure out how you can organize some of your time to focus on that.

Aydin Mirzaee (Fellow.app)  41:07

Yeah, that’s incredible advice to dig in on one of the things that you said about the weekly Business Review, I had a tactical question about this and you kind of referenced it in the book as well. But if you’re looking at a bunch of metrics, and you know, say that people say that, well, we have a live dashboard and you know, anybody can we have a screen in the office and you can just look at it, I have a tab open in my browser, and I can just see all the dashboards live, what is then the point of the weekly business review? Or what happens that still makes that useful? Get together? 

Colin Bryar (Working Backwards)  41:42

Yeah, so dashboards can be useful, but I think what the power that weekly business review is, it’s an end to end consistent look at the activities of your company or your organization, it’s a nice fractal quality. So it doesn’t have to be for the whole company, but it can be for my group, who are my customers, they, By the way, it could be internal customers or external customers, what did they experience because, you know, a single metric doesn’t tell the whole story. And often you need a group of metrics to, paint a picture, of what happened. And when you look at a, an end view, you see how different activities are, you know, impact one another, you know, if something So for instance, if you see customer service, contacts go up, it could be because you have some late vendor deliveries, or you know, there’s a website outage, you get to see the linkage between those different activities on and how they impact one another. And when you view it as yourself or as a management team, everyone gets that global view. And, and so you can optimize globally versus focusing on you know, what, what is my area, the other thing is, you don’t just want to look at vanity metrics. So you know, that consistent thing means sometimes you don’t spend any time everything’s normal, so you just skip right over that. But you know, every once in a while there will be a blip where you need to pay attention to so instrumented, right? It’s a very quick and efficient way to get a holistic view, again, of what my customers experience last week, um, did we meet expectations? And are you know, we’re hitting planning, are we getting better, and we’re on the track of where we need to, you know, attract where we need to be. And that’s hard with the dashboard. dashboards are point in time or, you know, they may have some graph, but it’s, it changes and it’s just one metric.

Aydin Mirzaee (Fellow.app)  43:35

Yeah, and that’s interesting. And the other thing that stood out to me was you were saying that it’s, you don’t come into the meeting and view the data for the first time if you’re an owner of a series of metrics, like if there is a blip, you will have spent some time and have some things to talk about. So are you then ever problem-solving during this meeting or do you identify issues and if they need to be solved? You know, people look into it and then bring it back to the next iteration of that meeting? 

Colin Bryar (Working Backwards)  44:05

Ideally, the teams or the people who own the metrics have spotted that may come to the meeting hopefully they have detected it and either has a plan of action or solved it. And if not, then you know that this is not a-okay let’s figure out how to solve it that you do assign action items or issues after that so and you don’t get caught up in the strategy about are we doing the right thing you know, the weekly business review is really a tactical view on the business but there are action items that do come out of that and then you can report on those later either at the beginning of the end or offline through, you know, some other Slack channel or email, what have you. One of

Aydin Mirzaee (Fellow.app)  44:46

The other things that I was going to ask you about are just communication at such a large organization, where there are a series of practices, you know, to just basically keep everybody informed You know what, what the mission is, what the vision is, what things are being worked on. I know you had this annual planning process with Obi one and Obi two. And that that brings a lot of alignment, I guess in the beginning, but business is very dynamic and things start to change. I’m curious, like, how did you continue to keep everybody steering in the same direction and also be aware of the things that are going on?

Colin Bryar (Working Backwards)  45:29

In terms of everyone in the direction? So I’ll just talk about the retail business at Amazon through a bunch of other ones. But there’s a concept of the flywheel you know, the tip of the hat to Jim Collins in his book, Good to Great where he introduces that, but Amazon has a flywheel that’s all about what, what are the customer experience vectors that I need to work on. And if I do these things over the long run, it’s going to yield the right output metrics for the business, you know, so in terms of price, selection, convenience, there’s a whole lot of sub metrics and activities there. Everyone in the retail business knows that flywheel it’s simple to understand, you also know what I’m working on. Is it spinning this flywheel faster? Or is it doing something else? And if it’s doing something else, you question, whether that activity in itself is worth doing. So that alignment is powerful. And you don’t then have to litigate? Well, if you know what the flywheel is, which are your, your durable customer needs, that you’re trying to improve over time, you know, if I improve it today, it’s going to pay dividends. tomorrow, next year, 10 years down the road, you don’t have to litigate, whether it’s worth lowering your cost structure or not, you know, that’s good and virtuous and of itself. So then you’re just sequencing, you’re not trying to pick from a series of activities and trying to evaluate whether they’re good or not. So I would encourage trying to figure out what your flywheel is for the organization. And then making sure everyone understands that that is a powerful alignment tool in and of itself. You’re right, that the annual plans, start to go stale The day after they’re written and improved because things move fast things are changing, you know, to stay updated, on what’s going on, just being immersed in the metrics. And this isn’t only the weekly Business Review, but just knowing what’s going on knowing when things aren’t normal, the only way to know that is to know what normal looks like. And so you have to look at data, you know, dashboards weekly, daily email flashes, or whatever. So you know what, what normal looks like so you can detect when something needs to be acted on. And then the last thing is, with the leadership principles, you don’t want people to memorize them. I suspect that they’re not that many people at Amazon who could verbatim recite all 14 leadership principles, and all the other three or four sentences have with them a behind each of them without making a slip-up. But they deeply understand them. Because they’re embedded in every process the company does, it’s hard to go through a day or a week at Amazon without encountering several of these leadership principles directly. In terms of when you’re presenting a plan. You’re looking at metrics, you’re figuring out what to do next, you know, so those leadership principles have to shine through they have to have teeth we call them they have to be actionable.

Aydin Mirzaee (Fellow.app)  48:17

Yeah, no, that’s awesome. I mean, everybody should go out and get the book there’s a lot of details there’s a lot of stories and it’s this inside look and you know, I wanted to start by saying thank you for to both you and bill for producing this this work, sharing the advancements in Management Science has been great, and so many people can learn from it. But one of the questions that we like to ask everybody, you know, for all the managers and leaders constantly looking to get better at their craft of managing and leading teams, are there any, you know, tips or tricks or final words of wisdom that you’d like to leave them with?

Colin Bryar (Working Backwards)  48:55

I’ve had some great mentors and my career, you know, and just even when I started in school and started up my job, different people, so you know, I think having mentors, whether they work at your company or not, that you know, that doesn’t matter helps because having a sounding board sometimes being a leader can be a little lonely at times, where you know, yeah, and so having someone who you can trust and rely on. The other thing and this is one thing Jeff Bezos told me is that seek out leaders who are like you effective leaders who are like you and personality-wise, and then you try to emulate those people. Don’t try to emulate a great leader who’s not like you because it won’t it will come across as is fake or not genuine and it won’t be as effective and you can’t become someone who you aren’t. So you know, try to seek out leaders who have some similar traits as you and find out what makes them great leaders and then try to incorporate some of that behavior into your leadership style because there are plenty of leader different leadership styles that work, and you need to figure out what is the right leadership style for you.

Aydin Mirzaee (Fellow.app)  50:04

That’s incredible advice and a great place to end it calmly. Thanks so much for doing this.

Colin Bryar (Working Backwards)  50:10

Oh, thank you, and think this is a great question. Thanks for having me on the podcast.