The Blog

Schedule a Consultation

Have you read our blog and still have questions? We offer no-cost consultations to portfolio firms and organizations seeking further advice or assistance in strengthening and growing their Product and Tech teams. 

Sign up now to schedule your session with one of our expert principals. 

Recent Posts
What's next?

Interna principals present at events worldwide. We send out a monthly newsletter with information on where to find us next and how to stream our talks to wherever you are. Our newsletter is filled with additional tips and tricks for executive leadership and the latest stories in global tech news.

 

Stay up-to-date by subscribing to our newsletter using the button below.  

Modes of CTO Success and Failure



David's notes:

How do you define CTO ‘failure modes’ and ‘success modes’ and how can you effectively communicate this with your team? In this panel, we discuss CTO success modes and failure modes, metrics, delivering value, and how to effectively communicate this with your team and your CEO.

Transcript:

Zoe Cunningham:

Welcome to the latest CTO Craft Bytes. Today, we're going to be looking at CTO failure and success modes. If this is your first time on CTO Craft Bytes, let me tell you a bit more about what CTO Craft is. CTO Craft is a mentoring and coaching community for technology leaders around the world, focusing on supporting technologists in their leadership growth. Community members are over 5,000. In fact, I don't know if everyone can see, there are a large number of people who signed up to watch this event, which is super exciting. You're all super welcome.


CTO Craft provides these 5,000 members with one-to-one coaching and mentoring groups, a curated Slack community, and events like this one. I am Zoe Cunningham and I am your host or MC which sounds a bit flashy to me, but I am-- or like, I'm MC Hammer. A reference probably younger members of the audience won’t get. I'm the director of Softwire-- I had to check my notes for that, how embarrassing. I've been in Softwire for over 20 years, having started as a coder in 2000, and going on to do almost every single different job in the company.


A fun fact about me is that I have a side hustle as an actor, which is very unusual, I think. Although, I always say, if there are any coders and actors watching, drop me a line because there are more of us than you would think. I would love-- Yes, and someone got my MC Hammer reference.


I'm delighted to introduce David Subar, who is the CTO and CPO at Interna. David is the Founder and Managing Partner of Interna, a consultancy that enables technology companies to ship better products faster, to achieve product-market fit more quickly, and to deploy capital more efficiently.


Interna has worked with companies ranging in size from small six-member startups to The Walt Disney Company, and has helped Pluto on the way to its $320 million sale to Paramount, and Lynda.com on the path to its $1.5 billion sale to LinkedIn. Engage them, they can make you a lot of money.


Prior to founding Interna, David was CTO and CPO of a number of companies including ZestFinance, Break Media, and Interthinx. He currently serves on the board of directors of the LA CTO Forum.


On my right-- or possibly on your left or your right, I'm not entirely sure how this works-- I would like to also introduce Emanuele Blanco, who is the CTO at Moneyfarm. Emanuele is Chief Technology Officer at Moneyfarm, a digital wealth manager. He's got more than 13 years of experience in the software industry. He's passionate about how concepts like Agile, DevOps, and continuous delivery help businesses to create value and scale up organizations. Welcome to both of my guests. Thank you both so much for coming on to talk to me.


I just want to do a quick shout-out to the audience: please do drop your questions at anytime. You can put them in “Ask a Question,” and you can also put them in the chat-- I think, it's not the official rule, but I think that will be alright. I have the chat on my screen, so I will see it quicker than the “Ask a Question.” Please do ask questions from the beginning. I have my own questions. I will always take questions from the chat in preference to my own questions, because David and Emanuele are here to tell you what you want to hear, not me what I want to hear, although they're super interesting-- I'm very much looking forward to asking my own questions.


Let's get started, and let's have a chat about how to use metrics effectively, deliver value, and communicate that with your team-- which is your whole job, right? Or shall we start at the beginning? What is your ultimate aim as a CTO? What are you trying to achieve?


David Subar:

You want to start or should I?


Emanuele Blanco:

After you start.


David:

Okay. I have a very specific opinion about this. As CTO or as Chief Product Officer, your job is to deliver a product that makes an effect in the market. It's not just to write code, it's not just to write user stories. Those are necessary, but it's all about: did you ship something that mattered, that created value for somebody else? That's your job.


Zoe:

Super simple. I'm guessing almost-- if anyone watching disagrees, put it in the chat because that will be super interesting. I'm guessing, Emanuele, that is your experience too. Has that been a change from moving to being an engineer to being a CTO?


Emanuele:

Well, definitely. Even as a CTO, sometimes you come from a technology background, like myself. I used to be a software developer. I agree with David. It's really about making an impact on your customers; then everything you do, with how you use technology, how you organize yourself and your work, it is about delivering value to your customers. When we talk about success and failure, one way to reach success is keep that in mind, that with every step you take, your ultimate goal is to ship features to your customers; then, everything you do is how you can do that better.


Better features, doing it more efficiently... that's very, very important. As an engineer, I would say sometimes you don't think about that directly. Some great engineers do think about that, but not everybody. I think it's also us as leaders, we need to teach this to our people. It's about delivering value to customer.


Zoe:

Yes. Fantastic.


David:

Yes. I'll vamp on that a bit, if I might, is that there's a reason why it's important engineers now. There's four ways you can code anything. If an engineer doesn't have the context of the why, they're not likely to consistently pick the right way of doing things. Scrum does a really good job on this. Thinking about user stories in the format of user stories, even at that lowest atomic level talks about the why.


I'm not saying Scrum is the only software development methodology to use, there's Kanban, there's SAFe if you're doing it, there's XP, there's a variety of them. They don't all pull that through to the smallest level, but as a leader, you should make sure your team knows the why.


Zoe:

Yes. Fantastic. I want to pick up on something that Emanuele said, which is you need to communicate this to your stakeholders. You need to communicate the value that you've created and be like, "Hey, look at all my value." Who are your stakeholders? Who are you accountable to? Who are you serving as a CTO?


Emanuele:

I think for me, there was quite a revelation in my career when I became CTO. It's really, it's the executive team. Well, it's the company, first of all, it's not the executive team. It's the company, you're there to serve the company, you're there to create value to the company via technology. It's very important that you align with your peers. I know that we have a job, we have a goal. My job is to make sure we reach that goal via technology, and that's key. I have two teams, in a way. I am part of the executive team. I am part of the technology team.


Ultimately, I'm there. I think as a CTO, you're there to serve the-- it's more senior management than technology. You really need to be focused on that. That's how you create value. Then, of course, your technical expertise helps you in having those conversations, as David was saying, to give context to people to understand why we're doing certain things instead of others. It's very important. The key relationship with me is with my CFO, with my CEO-- that's very, very important because then you create those synergies that then allow you to deliver better as an organization.


David:

I'm going to disagree.


Zoe:

Yes. First fight, first fight of the day.


David:

[laughs] He's in London, I'm in Los Angeles, so it's not likely to get physical. It's also not going to get physical because that's just not one of my things. It's true that you create value. I would say you create value with your colleagues, not for your colleagues. You're creating value for your customers. You and your colleagues need to be looking in the same direction. I'm not trying to make the salespeople's lives better. That should happen as a-- I'm trying to do that too. Primarily, I know that if we create value for customers-- and then from a transactional standpoint, there's margin there-- then we're going to create profit for the company.


It's about looking in the same direction and being focused on who we serve and why we serve and how we serve and how do we know whether we created value, with the other executives. Then we can talk about, “I can help sales better by doing this, or they can help me better or I can get data from them or the finance market.” We can talk about harmonizing collaboration, but if you're not looking in the same direction, you're going to have a problem.


Zoe:

Fantastic. This is my favorite type of disagreement because you're kind of agreeing. It's like a different light on the same answer, right?


David:

Yes.


Zoe:

You're delivering value to your customers and you've got to communicate that with the rest of the senior team so that they understand what creates value. I think that's something that we've all bumped up against at some point, right? Where you know your team is creating value and you just cannot get other people on board with that, or perhaps it's just me. [laughs]


Great comment from Andrew about "with" rather than "for", and so important that in an organization, you're all on the same team. Actually, this is something that's always been super important to me, you're on the same team as your customer, right? You don't want to be at odds with your customer, you're making money by making them money, right?


David:

For sure, that's exactly right.


Zoe:

You've got value for your organization by creating value for them. Okay, so let's get on to it. Everyone wants to talk about it, everyone wants to know, what are the metrics, how do we do this? I'm going to ask you a really broad question. What does success look like on a project? As a heads up, I'm also next going to ask, what does failure look like on a project? I'm sure those two will actually interweave?


David:

I'll let you start.


Emanuele:

Go for it.


David:

Oh, I was going to let you start. Okay. Well, now we're being overly polite.


Zoe:

Welcome to Britain, welcome to Britain, that's how we do things here.

[laughter]


David:

I live in the States, we're not used to that kind of thing.

[laughter]


David:

You want to start?


Emanuele:

I can start, of course.


David:

Yes, go ahead.


Emanuele:

I agree, by the way, with what you said: it's about the customer. Success is really-- if you are part of a customer-centric company, a modern product company, in my opinion, should be-- is really the success of your customer. Sometimes we have limited resources depending on where you work, sometimes you may need to make choices. I think it's important to understand that you're getting value for your customer, and how do you measure that? Well, I think you need to understand if people really stick with your products. How are they using the product, how can they become better? Because I think at the end of the day, success for a product organization is success of the product. I can even build the most scalable infrastructure, but if nobody uses the product... well, it's not really relevant, I think.


Zoe:

Right. [chuckles]


Emanuele:

I did a fair share of different work in my career. I did work for a non-product organization, I worked for consulting firms; and sometimes, there was a lot of attention to the technology, but technology per se doesn't solve a problem. We create organizations to create value to solve problems. I think that's like, you can be successful if your product is successful. I think the two things are very intertwined. Then, of course, we can talk about, do I deliver software fast enough? Do I deliver value fast enough? How often do I deploy to production? They are metrics for acceleration. What's the lead time for my things? Those things that really serve in the product. You can be even great at delivering software and being very efficient, but if nobody uses your product, well, is it success? I don't think so.


David:

I'm going to get a little more prescriptive. I have a belief that everything that goes on the roadmap should have a specific statement about what you expect for value delivery and how you're going to know. There's this thing, an epic statement and it's a very specific structure. You don't have to use this, I suggest you use some structure like this. An epic statement says, "We believe by doing this epic, this major feature," specifically called out, "for this customer," specifically called out. "We will create this value for the customer and we'll know it when we see this metric move." It's all very objective and discrete.


My suggestion is, the roadmap is a series of bets about what steps we think we're going to take to create value for customers. My expectation-- I think a good expectation-- some of them are going to be wrong. If none of them are wrong, you're not trying hard enough. You should have an expectation that you're going to place a bunch of bets. In the aggregate, you're going to create value for customers.


I actually almost don't care if four out of five fail if, in the aggregate, you're creating value for customers that's measurable. Because it's on the roadmap, it's public. You should have a conversation with the other executives, particularly the Chief Product Officer, but then the other executives. "Here's what our bets are for this quarter, or for this six months, or for this whatever period they are." They should know.


The conversation about velocity is not interesting for the CEO, or probably shouldn't be interesting for the CEO. It is interesting conversation within engineering, or metrics like lead time. Those are efficiency metrics that are important to manage the organization; but to Emanuele's point, that's about how you do it better once you're on the right road. First, we get on the right road; then, we run fast.


Zoe:

Right. There are lots of roads, you don't know which one you're getting down, and the road changes. You're going in one direction, and then actually, suddenly, that's not the right direction anymore. It's complex.


David:

That's right. Every time you release something into the market or your competitors do, you've changed the road. There's an expression I remember, maybe it was Satya who said it: "You can never walk in the same river twice." By being in there, you've changed it. That's why every time you do a release-- Sorry. Now, I feel like I'm monopolizing the conversation, I didn't mean to.

I think one of the things that people miss is the value of retros after release. People do retros after a sprint and say, "How did that sprint go?" What people don't typically do is, "We released this thing three weeks ago." [The retro] can't be right when you release it. "We had this bet, why we put this on the roadmap. We're right or we're wrong. In either case, what did we learn so we can get smarter about the why?"


Zoe:

Yes, super, super interesting. I'm all about practical, like practical tips you can take away and try in your own organization. I think that is a great one to think about. Emanuele, yes.


Emanuele:

I'd like to add something because I saw Andrew's comments regarding the IT cost center and all of that. I think what David is saying is, it’s key to not treat IT as a cost center. If you're just a feature factory, you're just delivering stuff that other people tell you-- you're not a cold shipping machine, you're not delivering value. Instead, technology products should work together and make bets, like, understanding, "Oh, we're doing this. We want to spend six months of our time doing this bet, what am I going to get in return? Do I know if I pull the plug before if I don't get my results?" It's very important. Only like this, we can stop IT being treated as a cost center and really part of the decision-making of the business.


Zoe:

Absolutely, and stop yourself being demoted by having to have another layer in between you and the decision-making of the company, who then does that translation. I think that is absolutely spot on. This has kicked off some discussion from the audience, so let's go. I'm going to try and take the questions in the order they come in even though they're coming in from two different places. That's how good we are at CTO Craft. Do you treat roadmap items in the same way you treat OKRs? I don't know if we need more context on that question.


David:

I think I understand the question. I'm glad to explain what I think my understanding is, and we can see if I'm mapping well. OKRs are a management technique to set the goals of the company. Here are the objectives and here are some key results that we're going to look to to fulfill the objective. The objectives tend to be subjectively defined and the key results are metrics-driven, they're objectively defined. They're generally tools that-- Andy Grove wrote about this. It was done at Google. It was brought in by the guy from Kleiner Perkins whose name is escaping me. There's a lot of companies who have done this. It's a very lightweight way of managing companies.


OKRs and roadmaps are related, but I would argue that they're not done the same way. The things you put on the roadmaps fulfill the key results because they're objectively measured, and fulfill the key results. OKRs, you're having several sets of initiatives to generate key results that get you to that higher-level objective. “We want to be the number one in the market” or, “We want to get a data monopoly here.” You have a series of initiatives-- things on the roadmap are some of those initiatives that a company might have, certainly not the whole set but the ones that product and engineering can contribute to-- achieving key results and objectives.


Zoe:

All right, let me jump over. Again, I'm going to be less British and I'm not going to force you both to answer all questions, so just jump in if you have something to add. Otherwise, we've got so many questions I want to make sure we talk about all of them because they're amazing. How about consulting companies? Emanuele, I know, you said you previously worked at a consulting company, right? How is it? How can they measure success? Is it the same or it's not going to be exactly the same, right? So how to do it?


Emanuele:

Well, of course, when you don't have skin in the game and if you are consulting, you may say, "Well, I delivered the project, I got my invoice paid that said, ‘job done.’" I think for me it is really key important-- we talk a lot about net promoter score with customers-- it's important that you do a great job for your customer. If I hire a consultant, I want them to help me, and maybe I need them because I don't know how to do X, Y, and Z; but for them, their success should be my success. There's a lot of conversation we can do about how to contract with consultancy. Is it worth it just to pay for the time they spend with me, and not having any part of the compensation related to the success of the project? I mean, it's a very, very interesting conversation.


I don't have an answer to that. I just think that success is important with skin in the game. For me, it's really key. Otherwise, if I’m from a third party and I say, "Well, actually, I'm successful if I delivered this project," and then I go somewhere else and then I haven't served my purpose, it's a bit tricky, right? I don't have an answer on that, but I think it's important to understand like, when you are a product organization, you know that your success, we were saying before, is a lot of customer success. As a consultancy, well, it’s somebody else's idea; so that they can be successful because it can be just financial, but then what about your clients, right?


Zoe:

Right. Actually, I just agree with you so much. I would just like to ask a little question that I understand about having skin in the game, and that that is a great motivator for everyone. Does skin in the game have to be financial, though? With your employees, would you tie your employees' remuneration to success? No, you just only work with employees who are committed to delivering success, maybe? Contracting is not covered. Maybe we'll do another talk just on like, how do you contract with the consultants? I'm sure there are actually people who are super interested in that.


All right, so let's move on. I really, really like these two questions. Oh no, they're not two questions. One is a clarification of the other question. What are the risks with using metrics? Because in life as human beings, nothing is actually a metric. It's always back to this---all of our goals. No, okay, not all of our goals are subjective, but I believe that the most important goals in life are always subjective, not metric-driven. We're always chasing happiness and success, and none of these are actually directly mapped onto metrics. Is there a risk if you focus too much on metrics that you will miss the bigger picture or fail to be ambitious enough?


David:

I think that's an interesting point. Two things, one is OKR is the objectives talk about the things that are a little bit harder to measure, but I would start at things that are metrics-driven. There are things like employee happiness that are important. You can ask questions, you can measure that, that will probably be directionally correct. Maybe not specifically correct. Some of this is you just have to feel, but customers are buying-- specifically in B2B corp companies, customers are buying for people that promote a metric because they're running a business.


B2C companies are different. Sometimes they're promoting a feeling like, why do people buy Coca-Cola? Because of Pepsi? It's actually not because it tastes better. Because people do blind taste tests, and people prefer Pepsi. I mean, some of this is objective, and some of this just has human value, but I would suggest starting at a thoughtful way of looking at what you're going to do for customers and then modifying.


Zoe:

Fabulous. I hope everyone's paying attention at home because I'm going to change the rules. It would actually be much easier for me if all the questions go into “Ask a Question,” I have learned in the last 30 minutes. If you don't mind, I will just take the, or I know there's been a bit of discussion about this on chat, but maybe I will take the last question I can see in the chat and then I will take questions only from “Ask a Question” so that everyone can chat in the chat without me losing your questions. I hope all these words made sense as I said them in that order.

How do you move from a feature-delivery shipping machine where everyone is focused on, “ship, ship, ship as quickly as possible,” to being focused on delivering value? There's some context to this. The culture is very focused already on how vast engineering can go.


I can totally see this being something that people bump up against. You're trying to partner with the rest of the senior team and say, "Let's make this decision together," and they're like, "No, no, you are engineers." Like, "It's all about how quick. That's all we care about." Any advice on that?


Emanuele:

I learned, to be honest, as an engineer, that progressing to CTO quite recently, I learned that companies are about numbers at the end of the day. We are here to maximize value for our shareholders, so we are here for the numbers. I would argue that you need to demonstrate the fact that even if you ship faster, it doesn't necessarily mean better numbers for the company.


I think I saw another question about, well, how do we deal with the CEO if they keep asking about velocity? David was saying the CEO shouldn't care about velocity. I do agree; yet at the same time, most people that maybe work in early-stage companies as well, have had the CEOs breathing on their neck and saying, "When are you going to get this done, when are you going to get this done?" The key thing is to demonstrate the value you're creating.


You don't want to pay for a bunch of people that chunk code on a keyboard. You want to pay-- you want to invest money to add value. You add value by measuring those metrics. If I do a feature that improves my conversion rate by 50%, that's much better than doing 10 features that increase the conversion rate by 1%. I think every CEO or CFO would agree with that statement. It's important to show the relationship between what you're doing and how the numbers are moving.

David's talked about how we measure-- say, we want to do this better than the six months, but then we want to give this value and we'll demonstrate by that. Once you have a track record established, you're doing this and the numbers are following, you get a bit more credible as well and you keep saying, "Well, actually, we should focus on what are the outcomes we achieved."


Marty Cagan from the Silicon Valley Product Group wrote a lot of bodies about productive feature teams. I highly recommend following his work because that's just amazing. Sometimes it's just a matter of talking to the people in the organization and letting them understand that, because once you stabilize that the value is in the numbers, nobody ever in the business will disagree with that statement.


David:

I agree with that and I think there's an incremental step. When CEOs ask about velocity, there could be several reasons. One is, maybe they were an engineer themselves and that's what they understand. Maybe they're sales-driven CEO, and they're completely ignorant and it's the only thing they can grab on to. The conversation that we had about delivering value, what Emanuele says is correct; but I don't want the point to be missed is that, that is credibility you gain over time by being successful. The trick that I've found that's useful is comparing this with a sales funnel. The VP of Sales or the Chief Revenue Officer and the CEO don't just talk about what's the revenue you delivered last quarter, or what's the revenue you can do next quarter. They have a sales funnel and they talk about where sales are in place and what the percentage of completion is, and that type of thing. There's a little bit more transparency. We have the opportunity to do that in engineering as well; but it's different.


I have no problem about if they really want to talk about velocity and having that be one of the conversations, but the way that you get that credibility is by demoing frequently. That's where they get the contextual understanding of how far you're going in saying-- I keep going back to Scrum but you can do this in Kanban, you can do this in XP, you can do this in other methodologies-- is, "Let me show you what we just produ