135: Jim Highsmith

Listen to full episode:

Joe Krebs speaks with Jim Highsmith about his book "Wild West to Agile - Adventures in Software Development Evolution to Revolution" (Amazon)

Transcript:

Joe Krebs 0:10

Agile FM radio for the Agile community. www agile.fm. Thank you for tuning in to another episode of agile FM today. I have Jim Highsmith here. Jim Highsmith just released a book called Wild West to Agile the adventures in software development, evolution and revolution. And it came out by Pearson as a publisher. Well, before we get started taking the book apart, welcome to the podcast. Jim, I'm so happy you're here. Thanks. So

Jim Highsmith 0:46

I'm glad to be

Joe Krebs 0:47

A lot of people know, Jim for one of the 17 signatories of the Agile Manifesto. And so this is, this is a very interesting book you wrote, because it's about a time period of 60 years of software development, software engineering, but also management leadership topics, and you group them into four eras. And we'll talk a little bit about those, obviously, and discuss if that's if that's possible. But you did write a book in 1999, or something. And the book was called adaptive software development. And without that, without that book, our entire industry would have been possibly called adaptive instead of Agile.

Jim Highsmith 1:36

Yeah, it's interesting, you know, before the Agile Manifesto meeting, Kent Beck and I swapped books, or manuscripts before they were published, I read XP, for it was published, and he read my adaptive book for it was published. And so we, we had those went into the Agile Manifesto meeting. And it was it was, is I remember, we had like, 20 words up on the board. And we whittled them down to agile, but adaptive was one of them until the board and I made the point that I didn't think that the name ought to be something that one of us already had.

Joe Krebs 2:15

Yeah. And and then we you guys chose the word agile became the Agile Manifesto. And, you know, and that was just the starting point of the fourth out of your four areas you are highlighting in your book. There's three before that, right. And this is this is this is the, this is the interesting piece here is did you take journal, did you write journal for those last 60 years? Or how do you remember, going all the way back when I looked at your book is fascinating to see all of those topics? But by no way? Could I remember all of those things, how you wrote them down? How did you do that? Well,

Jim Highsmith 2:54

it's interesting, because the things that I had to look at changed abruptly in the mid 90s, when I started having emails and computerized documents. And the other parts of it, particularly the early years, was basically from memory. It's interesting, as I, as I looked at things as I began to remember, other things came to me. So it was it was interesting how one memory led to another memory.

Joe Krebs 3:26

Wow, that's amazing. Yeah. So even Nike made it into the book, right? Yeah. Nice. So what's interesting about this book is I looked at the title. And obviously, it's about a reflection on 60 years of software engineering, from Apollo to SpaceX, if you want to see that. Right. I think that was one of those subtitles. What's interesting is when I first picked it up, I thought it was a book about that wasn't sure let's put it this way, if it's about you, or is it about a historical book about all of what's going on? And then when I started reading it, I was like, Oh, my God, this is fascinating, I's both. It's, it's a reflection on the the eras ors of what was happening in the last six years of software engineering, plus a personal touch from you, and how everything came together. Why did you decide off of putting this together, like your personal experience? And, you know, what do you what do you think is benefiting from the historical aspect of the book?

Jim Highsmith 4:32

Well, one of the things about the history that I think is important is that it helped by understanding some of the history, it helps us prepare for the future. I don't try to predict the future in the book. And I say this is, you know, part of being ready for the future is preparation. And it's interesting how this book got started, and why the personal is in there, because it actually started out as a family oriented memoir to my grandkids. And as I, as I developed that and tried to figure out how to make something that would be interesting to teenagers, because they're in their mid teens now, I decided on this kind of scope of 60 years and breaking it into arrows. And once I did that, I realized that a lot of it was my personal stories. And I kept, I kept asking people, which do I emphasize? Do I emphasize historic history? Or do I emphasize the personal and people like Martin Fowler, who was a reader of the manuscript and had a lot of great information and feedback for me? Yeah, pushing me to do more personal or like a memoir. So it is kind of a historical memoir. And I think that it also helped me reduce the scope of the book. As I tell people, it's not the history of software development, it's a history of software development, it's really important, because there are a whole lot of areas that I never really got into. And so they're not in the book. So for example, I worked with people who did object oriented programming, but that was sort of different from what I did. So there's not a lot of history in there about object oriented programming. There's nothing about aerospace, there's nothing about Unix, there's nothing about a whole range of topics that I didn't have any interaction with. And by doing it like that, I was able to scope the book to something reasonable. Yeah.

Joe Krebs 6:35

Well, I it's, it's fascinating, right, so you just mentioned those four areas, just to give readers or listeners a little bit context, here is the Wild West. In the beginning, this is how it all started. We got structured, and we got the roots. And obviously, then the Agile space. Now, you just mentioned that a little bit in how it could be helpful for for anybody who to look back into history to make, you know, not predictions, but to learn from history for future events, possibly reflect on it. Now, if somebody and because the Agile era itself is already quite long, at this point, we're recording this in 2023. So some of the listeners right now might only have experience in that era. Right? So what do you think if somebody who is relatively new into software engineering, possibly coming out of college right now, and this is like, this is all I know, this is the way of how I have learned and worked in this is the only thing I know, what are the aspects you feel like you would like to point people back to until I get this, this is interesting stuff, and you should be aware of it.

Jim Highsmith 7:45

Um, I had a colleague at ThoughtWorks, who is in her late 20s, she read some of the manuscript help some with it. And it was really interesting talking to her, because in college and and her work, work environment, she had never done anything except that. And so looking back at the history of things, she, she really enjoyed it. And she thought it was very helpful to her to kind of understand, for example, what was the conditions in the world that made agile, kind of take hold in the early 2000s? Was it just because it was a better way to do software, because people really liked it. There were business conditions, technological conditions, that kind of came together at that point in time to make a pivot point. And I think people need to understand these things didn't just grow. Boom, but, they had some background and the other background background, I thought was important was to bring out some of the individuals, some of the people who were pioneers of those different eras, who really contributed to the evolution of software development. I asked people if they did they know who Tom DeMarco can or or Larry Constantine do they know that these people were and most didn't? So I wanted to bring those people forward in people's mind. It's

Joe Krebs 9:32

interesting yeah, no, I and it's nicely written beautiful graphics. And in there too you see like the the era and you saw like with, you know, where technology was produced with the mainframe computers, and you see people like interacting with the machine and you see today are people enjoying technology in their living rooms. So a lot of these kinds of visuals that go in, there's also a visual and that was striking to me that was interesting because you always have like these comparisons in your book where you would say the "then", right? And the "now" piece where you you highlight the different windows here in terms of time. And what's interesting about several times the org charts of organizations comes up. And and then poor was like a hierarchy of organization and the now part is very different. I don't and this is this is something I noticed in the book is that I definitely see that there is a trend towards that. But when I read that, I was like, there are a lot of organizations still out there that are having an old org chart kind of thing they are, they're still today operating in an agile era, with the org chart of, you know, the structured, maybe right kind of approach. What's your advice to them? I mean, there's there seems to be like a less of learning in terms of adaptation?

Jim Highsmith 10:56

Well, I think that this is, you know, a big topic now is digital transformation, becoming a digital organism. And I think there are multiple different parts of it. And I think until, well, for example, if you really want to be a digital organization, you're going to have to think about how you measure success, with different measures of success. And then you have now, just like in project management, we had to move from the Agile triangle to something I call the Agile triangle, from the iron triangle to the Agile. And in business, I think you've got to do some work. And so I think organization structure is another one of those things that become digital, and become fast acting and innovating. You've got to look at the organization structure, and have it malleable. meet the needs of a growing company, or of a company that's moving into making some major changes. I think there's there's some people doing that. But it's one of those areas. That's it's just emerging, and I don't think the right model are there yet other than other than Germany and Apple whose unfix model, which I talked about in the book, and it's just getting started, but it's seems to be really taking hold in europe.

Joe Krebs 12:23

Yeah, it's interesting. Like, we'll get definitely get there. You just mentioned business one more time, right? So the agile movement is a reaction to the business needs, right? It's not just like you guys thought about, hey, let's work differently. Right? It was business needs that require that. And I think that need is still obviously here. So how did the like, because 95 somewhere in that neighborhood? That would be in your roots era? That was the significant event of the.com bubble burst? How did that influence like business and that era? Do you see like, historically, while you were working on the book, and you're just on the material? Did you see any correlation? Like what happened was that like, also like a massive impact on the way of how people worked?

Jim Highsmith 13:16

Well, I think the thing that was the massive impact on how people work was really not connected to the.com bubble. But it was connected to something else. And this is the transition from automating interim systems. Automating customer facing system. I think that was a that was one of the impacts of the internet. And that was a major transition. So for example, there was a late 1980s, my wife and I went together, my chair, and we went to this place. And I finally picked out the right chair and hook it up to the counter, or took the slip up to the counter, wrote the guy check. Now those checks are those little small pieces of paper that we used to use. And said, we helped me put the chair in the car. And he said, Well, you have to come back tomorrow. And I said, well, the chair is right there. My car is over here. Why can't we put it in the car today? He said, Well, our computer system prints out picking tickets overnight. And I can't give you the chair without picking. That's the sort of computer interfaces that we were dealing with in the late 80s, early 90s. And so that move from internal facing systems to external facing system was a big movement and to me that was a bigger thing than the.com.com bust was a temporary reaction, the moving too fast. You can anticipate the same thing for AI now.

Joe Krebs 14:59

Maybe Yeah, yeah. That's a wonderful example of how you connect the paths to possibly future events. So I was like, Well, are we possibly going into first year too? Well, that would be for a totally different recording here. Right? That would be awesome to catch up on that as well. Now, I do when I was going through this material in your book, that was also obviously, you know, I have lived through professionally, almost three, I touched on the second one, but then the the roots in an agile myself. What's interesting is there's several topics where you look back, and you're like, oh, wow, I totally forgot about this, right? We did exactly what you did too right. It's like, there are certain steps where you find yourself in your personal story, I found myself, for example, in domain modeling, for example, right? technique I find very useful. Still, today, sometimes I scribble a little bit on a napkin and do these kinds of things. Obviously, Martin Fowler follow, which you mentioned before, right analysis pattern, huge book and everything, but you don't see these things necessarily anymore. I just want to use that as an example. Right? Not necessarily make this a conversation about analysis, patterns ones. But is there anything where you would look back and say like, Okay, we are in the Agile era, but there is something in those previous three eras, we would say that's a shame that they went away, there wasn't useful techniques. They are always like, Oh, why we're not doing this anymore. It might be still a good idea. Is

Jim Highsmith 16:29

it true? Interesting, as I began looking at some of the stuff that was used, for example, in the structured era, I found out that people are still using data flow diagrams, maybe not to the extent they were before, but there's still a useful tool. So there's some of the diagramming methods that people are still using. And I'm sure some of the diagramming methods in UML are still being used. One of the interesting things that's still being used today, I think a lot of people don't know the origin of it. Was the idea of coupling and cohesion. Yeah, that actually, Larry Constantine, developed those in the 1960s. And so, one of the interviews that I have in the book is with Larry Constantine. Another one is with Tom DeMarco, who, those two people and a few other really kind of started the structured methods movement in the 1980s.

Joe Krebs 17:33

Yeah, if I remember correctly, even Larry Constantine even went to the started initiating use case driven approach why and so there was certain I think there was part of that, and that popularized this technique, among others.

Jim Highsmith 17:47

I'm not sure he was involved in use cases, but he may have been,

Joe Krebs 17:52

yeah, there was there was definitely a ton of movement here. That very interesting, you just mentioned the the unfixed model. And maybe that is something I actually do want to ask you about that. So we have these four eras, which is great material. But there's also topics like unfix, for example, right? You have mentioned in your book, and that's a little bit forward thinking. Now, I myself, I'm a little biased here, because I'm writing about agile kata. But there's also lean change management, flight levels, there's evidence based management is beyond budgeting. There's agenda shift as fast goals, I mean, there's topic after topic after topic. And if I, when I came to reading about the Agile era, I was also like, fascinated about all of those things. Again, I'm a little focused on one of them myself with the Agile Kata. But what I noticed is, are we right now with business agility, the digital transformation you mentioned, are we entering? are we approaching a fifth era right now? Because there is a diversity of techniques right now. It feels like very energetic right now. There's a lot of things that are happening right now. And like in islands, and we're trying to put things together into this business agility right now. Do you feel like we're in the beginning of a new era? Something business?

Jim Highsmith 19:17

I think it could be a new era, people have asked me about that quite a bit. I don't know if agile methodologies per se, will continue there as they are today. I think there's a lot of stuff happening and people going in different directions. And somebody asked me the other day, if I thought the 17 would get back together and rewrite the manifesto. And I said no, we're in a completely different era. You know, and and agile is now been spread kind of worldwide. And back then, in 2001, there was a very small contingent that was working in what was then called lightweight methodologies. Right? And so the times are very Very different. So I think that for the future, I think the important things are how do we build agility and adaptive leadership into our organizations? That's the real challenge. And I think agile can be a part of that. I think what we have to do is we have to look at, what do we keep from agile? And what do we change? Yeah. What is it that persists? And one of the things that I think the manifesto did, it was both inspirational and aspirational. I think in some of the newer things that we're seeing, they've lost that inspiration part of it, got some new new project, new principles or new processes, or new names, but it doesn't have the inspiration. The original manifesto. I think that's one of the things may be modified a little bit. But keep Yeah. And then we need to figure out what what goes on beyond that. And whether it's a new methodology called Excalibur doesn't matter to me, as long as it keeps on focus on Agility and adaptively leadership.

Joe Krebs 21:15

Yeah, well, I do think like, from from whatever I noticed is I think we're moving forward with the, with the ideas in mind, right, I think, I don't think there's any kind of dead end or anything in terms of the journey. I think this is going to continue. I think it's an expansion right on. Where do we go with this topic in general. And I see like, somewhere in your, in your work, I see parts within the evolution where there's a high increase of new ideas, and then there's a new arrow coming out of it. And I was just wondering if you with all the oversight and things you see and read and hear about, if you feel like and this might my stuff I just mentioned is probably not even a complete list? Definitely not. If there is anything where you would say there is a big, big pool in arsenal of ideas right now, for how do we approach the problems of the future?

Jim Highsmith 22:12

Well, I think that there's a lot of new stuff coming down. And both in management, organizational design, software development, and I think you it's going to require integration, we've got to, you've got to be able to use all those different topical areas, and somehow integrate them into something that an organization can use. And I think it's going to be different for every organization. You know, I think that this idea of one methodology fits a lot of different companies, I think one methodology to one company that everybody has to have sort of their role their own, appropriate for them. And I think that's actually the more difficult part. And the difficult part that I've seen all through the eras, which is, there's, there's a number of people who take whatever methodology and say, This is it, we're gonna follow these steps, and we're gonna do these processes, we're gonna fill out these documents. And that's the way we're going to do things. Yeah. As opposed to this is a framework, a guide a guidelines need to be adapted to every different project or every different organization. I think that's the, that continues to be one of the more difficult things to do for organizations is to allow them enough flexibility in how they approach. Yeah.

Joe Krebs 23:44

I couldn't agree more with you. And this is you just make me think about all of those things that are ahead of us. As a as a community as a as an industry. When you just mentioned earlier in your book that you had the intent of writing this book for your grandchildren in the beginning, and then add a little bit more other things to it. And the book grew in both sides. It still both it's still personal as well, a historic document you put together. Is it any point that you like, because it we've been up? It's going public, right? In Pearson in here as a book? It's not just for your grandchildren? Did you soften your tone a little bit your when you did this were like, because some of the experience you had you were like, you could read between the lines that it was not necessarily easy. There was some frustration, right? Did you so it's a little

Jim Highsmith 24:41

bit so maybe a little bit and you'll notice that with organizations where things went pretty well attended to use the name of the organization, but it didn't go so well. tended to use pseudo name Yeah, yeah. And one of the things that that happened during a book is, you know, I had been used to in my previous books, writing stuff, writing about engineering methods, writing about management methods. And here I was faced with writing about myself. And that's a very different perspective to write from. And luckily, I had a number of people that pushed me to do more and more of that, I think it was the right direction. But it was difficult, but I really challenge other people in our industry do more of that write about themselves and what they're doing, not just write about stone.

Joe Krebs 25:43

Yeah. That's, that's interesting. Why because it's the personal touch and the struggles. It's also like, you know, it's not like polished in a way where you would say, that doesn't sound like reality, you can really feel with you in some of the situations, you know, you know, some of them were further back where I can picture like a cubicle or something like that, like, you know, like, all these kinds of things. And it's like, oh, he's going through this, but you see the path of where this is going and how you found your path. So I read by or any kind of personal story that goes along with it. It's, it's makes it more real. Jim, this is a great conversation. Thank you. And I do want to say everyone who is listening to this and has an appetite for hearing more about this and obviously going into those four eras of Wild West structure routes and agile as you grouped them and labelled them. I can only recommend to pick up the book wild west to Agile by Jim Highsmith. Thank you so much, Jim, for your time.

Jim Highsmith 26:45

Thank you Joe, I enjoyed it.,

Joe Krebs 26:48

Same here., thank you. Thank you for listening to Agile FM, the radio for the Agile community. I'm your host Joe Krebs. If you're interested in more programming and additional podcasts, please go to www agile.fm. Talk to you soon.