Interna Talks 4 - OpenAI, Vendor Dependence, and What has Changed About Product Management

Welcome, everyone, to another set of Interna talks, where I get to talk with my colleagues here at Interna. We talk about some of the trends that we see with our clients and going on the industry in general. Today we have a few different interesting conversations. Of course, we talk about the thing that went on in OpenAI where Sam Altman got fired and then rehired, and what we think that impact is not just on open in the industry. We talk about CTO´s and their vendors and vendor dependence, vendor reliance, and what that relationship looked like. That's particularly interesting because so many people are reliant on OpenAI and companies like that for now, strategic things. And then we talk about, generally the relationship with product management, what's changed in the world, product management, how OpenAI, how Gen AI has changed that and just has changed what's in the industry. And then I have a lightning round, so stay tuned I hope you enjoy it.

Transcript:

[00:00:01] David Subar: Welcome, everyone, to another set of Interna Talks, where I get to talk with my colleagues here at Interna. We talk about some of the trends that we see with our clients and going on the industry in general. Today we have a few different interesting conversations. Of course, we talk about the thing that went on in OpenAI where Sam Altman got fired and then rehired, and what we think that impact is not just on open in the industry. We talk about CTO´s and their vendors and vendor dependence, vendor reliance, and what that relationship looked like. That's particularly interesting because so many people are reliant on OpenAI and companies like that for now, strategic things. And then we talk about, generally the relationship with product management, what's changed in the world, product management, how OpenAI, how GenAI has changed that and just has changed what's in the industry. And then I have a lightning round, so stay tuned to hope you enjoy it.

[00:01:01] David Subar: Welcome, everybody, and welcome to another session of Interna Talks, where we at Interna talk about things that we've seen with our clients and happening generally in the industry. So say hi to all my colleagues. I'm David Subar. Jeff, would you like to introduce yourself?

[00:01:21] Jeff Yoak: Sure. My name is Jeff Yoak. I've been with Interna for a few years and most of my career as a CTO in the Internet technology space.

[00:01:31] David Subar: Ross?

[00:01:32] Ross Webb: Hey, I'm Ross Webb. I'm based in north London. I'm originally South African, like another one of my colleagues here at Interna that you're going to meet. And I've been a principal at Interna for a number of years. And maybe I'll hand over next to Eddie.

[00:01:46] David Subar: Hello.

 [00:01:47] Eddie Shek: It's Eddie. Eddie Shek, the principal at Interna, longtime CTO, senior entrepreneur with a particular interest and background in enterprise software and fintech. Mark.

[00:02:02] Mark Goldin: And I'm Mark Goldin. I'm the other South African that Ross referred to. I was a CTO in Southern California for about 30 years, and now I'm a consultant working with David and the rest of the team.

[00:02:15] David Subar: Well, welcome, everyone. So we have a bunch of topics to talk about. Let's talk about OpenAI in the once in future king at OpenAI. Sam Altman, what happened? What do you guys think happened? AI safety. What does this mean for OpenAI? What does this mean for the world? I'm going to start with Mark. What do you think?

[00:02:40] Mark Goldin: Well, it was a roller coaster. It was embarrassing from a board standpoint. I've never seen such poor board governance and board dysfunction ever. That's got to be one for the record books to fire a guy on one day and invite him back the next, I mean, that's just crazy. Think through the consequences of your actions before you do anything. We don't even know the full story yet, but it seems it was. Most people seem to think it was a debate between speed and safety, and the board must have felt that Sam was emphasizing time to market over safety, and that's what they reacted against. Having said that, it still doesn't excuse the way things operated.

[00:03:16] Mark Goldin: I think it's good that it ended up the way that it did. It's good that Sam Alman went back. He's been an effective CTO. The Lord of the Rings movie that you could most approximate to this would be return of the king, I think. And so the king is back in his seat. It also showed you just how important this was to Microsoft, given that they were willing to hire him almost overnight and then got themselves a board seat after the fact. And of course their investment is huge and it makes perfect sense.

[00:03:42] David Subar: So I have an opinion about this open AI stuff. I completely agree with Mark about the lack of good management skill in getting rid of Sam and then having to bring them back. Clearly they didn't have a sense of where their employees were. I think the other thing that this points out is the reliance on a vendor who becomes critical. So many companies are using OpenAI, creating foundation models and using them, or anthropic or one of the other competitors, and it's fundamentally strategic, this things that people are doing with GenAI. And so I think it should ask, companies should say to themselves, what do I do about vendor reliance in this or anything else that's strategic. Generally, I recommend people do not buy something that's strategic. They should probably build it.

[00:04:45] David Subar: And they buy licensing for things that aren't. But where things are so costly, like building a foundation models, you don't have a choice. And so I think it raises a big question about vendor reliance on something like this. And what do you do? OpenAI could have just disappeared. It didn't, thankfully, but it could have. And I think that raises a significant potential issue. Jeff, you have anything here?

[00:05:10] Jeff Yoak: Yeah. The way I had kind of framed it is it was a conflict between the idea of commercialization on Altman's part and on AI alignment from the board's part. And to paraphrase something I think it was, Andrew Ring said they're unable to get alignment in a six person board of directors. Why are they worried about AI alignment? But as much as that's just a cute comment. I think it goes deeper. Even if there are dangers, which I think are generally more of a concern to people than they should be, I think it's still the right thing to do fast, because if the current state of the technology allows something which can be potentially dangerous, it's certainly going to happen. And I want it to happen in organizations like open AI, rather than, say, happening in China and not in the United States. The fast moving is going to happen.

[00:06:12] Jeff Yoak: I think where we ought to dedicate our efforts is in trying to mitigate real problems as they emerge, rather than very theoretical problems that could emerge later and that move fast as the right answer. And that's definitely the side Sam Altman was on, and that's definitely the side that's won here.

 [00:06:31] David Subar: Excellent. So let's move on to a different topic. This topic is a natural follow on for what I was suggesting about vendors and dealing with vendors. In this case, OpenAI was a vendor. But let's open the aperture and talk about it more generally. CTO`s and their vendors, you can have a variety of different kinds of relationships there. What should that look like as a CTO? Is it same for all the vendors? Do you treat them differently? Mark, I'm going to start with you. What do you think here?

[00:07:04] Mark Goldin: I think it's an interesting opportunity to develop a relationship and to really make vendors strategic. Of course, not all vendor has to be strategic, and some are more strategic than others, let's say. But companies have different approaches to managing this. I'm just thinking of a particular example. In my case, one job where I worked with a company for a number of years, and the company grew over time. At the beginning, there was no purchasing department, right? So it was really just me and my team negotiating and managing the relationship. And that made us naturally close with the vendor because we were very directly involved. As the company grew, we implemented a purchasing function.

[00:07:40] Mark Goldin: And at first I kind of resisted that a little bit because I was worried about losing that direct relationship. But then I came to embrace it, realizing that you can really separate pricing from other parts of the relationship. And it was actually useful to have an objective, less emotionally involved party involved who could negotiate price and make sure that we were getting the best bang for the buck. At the same time, I could also make it clear to my team that we should stay involved and maintain the relationship and be able to see vendors as strategic partners, whether it's a massive cloud provider, like an AWS, for example, where you're really in bed with them deeply and you need them and you want them to be responsive to your needs. And it's more than just about pricing. Or if you're dealing with something else, like a large software vendor or even, I don't know, say, a storage vendor, for example, as another example that comes to mind, you want to be able to have a close relationship. You want to have input into their roadmap. You want to see them as partners and not just as companies that will do your bidding because you're spending a lot of money.

[00:08:42] Mark Goldin: You get much more out of people that way than you do by just driving them hard on price. And sometimes it's not even worth getting the lowest possible price. Sometimes it's worth having that relationship and being able to have influence on one another's planning than to just focus on the bottom line. So I would say ctos should focus more on building relationships where it matters, either themselves or senior people on their team, so you can have that input into the vendor roadmap.

[00:09:12] David Subar: Got it. So, Eddie, you've been on the other side of this equation. What's it look like from there?

[00:09:18] Eddie Shek: I can't agree with Mark anymore about how CTO´s should absolutely have a strategic relationship with strategic partners and vendors. And having spent a lot of time in the enterprise software world, where the products are very much a dependent part of a lot of our clients, I think a lot of ctos underestimate how much vendors actually appreciate the input and the connections with significant customers to learn from and to really to build the roadmap out of. We are all human. So from having a relationship with a vendor, when something goes wrong, when you need the support, having that relationship matters, trying to get something pushed through a little faster.

[00:10:19] Mark Goldin: I'd like to add a little bit to what Eddie was saying. I also agree with what you're saying, and I think maybe I learned to work with vendors because of the way certain of my customers treated me. I was also in the enterprise space, and customers specifically said, we don't want to treat you like a vendor. We want you to be a partner. I want to be able to call in you when I have a problem. I need to be able to get to you quickly if I have a need. And I felt that I was being treated properly, it was good for my self esteem and my self respect and improved the relationship with the customer. So I think maybe I learned from my strategic customers how also to handle my strategic vendors.

[00:10:51] Mark Goldin: And I appreciate that.

[00:10:54] David Subar: I'm going to turn this over to Ross with a follow on question as well, because the information you learn from your customers needs to go back into product.

[00:11:03] Ross Webb: Sure.

 [00:11:04] David Subar: Right. And so I want to open the aperture once again, talk about product. And let's talk a minute how the information from your customers should go through the CTO into product, or should it go directly to product? And then we'll talk about the future of product management. But Ross, why don't you tackle that first question first?

[00:11:27] Ross Webb: Yeah, I think it should go directly to product, and product need to be the shield before it gets to technology or the engineers. I think product need to really validate requirements before CTO´s and the engineering teams get it. They should really be acting as that shield. Totally.

[00:11:48] David Subar: So do you want product to be having that direct contact with the customers and not engineering, or do you want them to be both?

[00:11:59] Ross Webb: 100%. I've worked at quite a few organizations where I think having maybe not the CTO, because I can see a CTO having kind of contractual relationships with vendors that in some organizations that make sense from a procurement perspective. But I think the whole point of product management is the product needs to own that relationship with the customer. They need to be able to explain the needs and the requirements almost better than the customer can themselves for the product. So absolutely the primary relationship with the customer should live with the product manager. I think you can make an argument that if the product managers don't have that primary relationship, why do you need them? Right. And they need to be able to validate the relationships. They need to be able to do cost benefit analyses, etcetera.

[00:12:52] Ross Webb: But they primarily, I think that is the primary difference between, say, a product manager and a project manager. And a project manager, you can argue, might not need to have such a close relationship with customers, for example. But a product manager, I think that's the reason they exist.

[00:13:11] David Subar: Okay, so Ross just fired all the product staff from every company, or not? Probably not.

[00:13:21] Ross Webb: ...The product.

[00:13:19] David Subar: Well, you said, why do you need them?

[00:13:22] Ross Webb: 100%. I truly believe that.

[00:13:28] David Subar: So let's put the future of product management. I keep open the aperture. I'm going to do it again. So what has changed about product management from a year ago and rolling back to before? Does GenAI change this stuff or not? Or is it just general changes?

[00:13:48] Ross Webb: Okay, so I've got strong views on both of those. What I've seen happen maybe in the last year and probably in the last three years is product management has gone from being incredibly close to engineering and putting engineering first and moving over to really putting the business first and having to have good understandings of that. I remember once being a director of product and having almost 200 product managers, and almost all of them had very, very poor understanding of being able to have a proper financial understanding of how the business works, of basics, of being able to read a cash flow statement or a balance sheet. And I'm not talking about being at an MBA level, which I'm fortunate enough to be, but just having those real, real basics that when they want a new feature, for example, are they able to put a proper cost benefit analysis together for that? And in previous organizations I've actually put educational programs for product managers to be able to do that. And that is where I see the future of product going. That yes, we need to partner with customers and give customers features what they want, but we need to be able to really analyze how we're going to do that in a way that is cost effective for the client and gives an ROI for the business. The second part of your question, really. As far as...

[00:15:15] David Subar: Actually, let me just pause you on that. What you're talking about, and I just gave a presentation on this at a company in the bay, is really aligning with the company and the company strategy. It's not just about features, it's about how you're driving the company forward. You need to have some kind of metrics about it. Is that a good summarization?

[00:15:33] Ross Webb: 100%. You just said it in 30 seconds. That's fantastic. And product managers need to be educated for that. I think that's one thing that people don't realize. They may have 15 years experience, ten whatever years experience in working with engineering, but a lot of them don't have that experience on the business side.

[00:15:49] David Subar: Great. Okay, so you said there was a second part.

[00:15:52] Ross Webb: Oh, you asked me about the AI.

 [00:15:54] David Subar: Yes.

[00:15:55] Ross Webb: So currently I see AI as a tool. I see AI very much of where the web was in 95, 94, however far back we want to go, but very much I remember being back then, it's, oh, the Internet is going to come and it's going to take all our jobs. It's going to change everything. And yeah, it did change the world and change huge amounts of things. Or you can go back ten years before that on computers. I see AI as just the latest evolution of that, that it is a tool. I think it's a fantastic tool. I've got an eleven year old daughter, I've created a little GPT that does revision for her.

[00:16:31] Ross Webb: I don't see it as her like cheating. I remember talking to her teachers and they're all excited and I know I'm talking moving a little bit away from product management. But I see that with a lot of product people that I speak to also. I'm in Europe, so I work with a lot of people whose first language might not be English, and I see a lot of them going and saying, hey, rewriting what they may have done and hey, let's put it in, let's say agile format or in the format that it needs to be. So I see it as an incredibly useful tool. I don't see it getting rid of product managers as long as product managers up their game and are now able to make real strategic business impact to the enterprise.

 [00:17:11] David Subar: Going. And I'm going to toss this over to Mark, is the thing with GenAI, one of the things that's very different about it. In previous days, we had something we want to have happen. We would write code to do it, and then we'd have a small metadata in the beginning, a small data day in the end, and we would do QA and test it. And so there was a lot of concentration on writing code. In the GenAI world. It's much more of a data problem and an architecture problem. Here's all we have, data.

[00:18:02] David Subar: Yeah, that's fine. So we have a bunch of data, and we have to clean the data, and we've got to mark the data, and then we have the outputs, and we use that for training. But the cost of doing that, at least in the initial development of some model, might be quite expensive. And so as opposed to the build an MVP, iterate really quickly and learn that way, which in the traditional engineering product management model, it's what we've been doing the last, call it ten years. This is very different. The kind of investment that requires even before you start getting value out of it. And actually, Jeff, if you want to talk about this, or Eddie, that's fine, too. Does this change the relationship between product management engineering? Because the capital investment is so much higher to get started.

[00:19:01] Eddie Shek: So I think in terms of the relationship between product management and engineering and technology in this GenAI world, it's like the evolution of level of abstraction of the technology, just that with more and more tools. And again, I agree with Ross. All of these are just more and more sophisticated tools, with more and more, harder and hard work to explain what it's actually doing. Anything that is sophisticated looks like magic generated, just an acceleration of that. But my point is we just create another level of abstraction in terms of what product management can do as it pushes into the traditional realm of engineering work, from automatically generating all the test cases to regression testing to actually writing code. I have product managers generating pseudocode or actual code because they can now with Chat GPT or other tools they have. Right. It just pushes the product management more and more into engineering.

[00:20:13] Eddie Shek: There'll always be software developers, but that I see that line between product management, product ownership and engineering continue to shift to the right. Engineering being on the right.

[00:20:26] David Subar: Okay, we're going to do a lightning round. My niece is thinking about going to computer science. Should I encourage her to do it or encourage her not to do it? We'll do a lightning round. Give me a yes or no. Jeff.

[00:20:40] Jeff Yoak: Encourage her.

[00:20:42] David Subar: Encourage her. So it's still a good profession, Ross?

[00:20:45] Ross Webb: Yes, absolutely.

[00:20:47] David Subar: Ross says yes. Eddie says, I think Eddie just said no.

[00:20:51] Eddie Shek: I have a long discussion with a bunch of students at UCLA just last week on that, and the short answer is no.

[00:21:00] David Subar: Mark.

[00:21:01] Mark Goldin: I'm on the no side. I'm going to tell her to become a plumber.

[00:21:07] David Subar: We'll always need plumbing. Okay, thanks, everybody. And we'll do this again next month.

[00:21:15] Mark Goldin: Thanks, David.

[00:21:17] Jeff Yoak: Look forward to seeing you.