Transcript
Sven Johann: Welcome to a new conversation about software engineering. Today we are talking with Andrew Hamelor from ThoughtWorks, author of Facilitating Software Architecture and also Architecture Metrics. I didn't know about it until I checked your website. Exactly. Yeah, we are happily talking about the book Facilitating Software Architecture, How We Can Make Better Architecture Decisions.
Andrew Harmel-Law: Just one chapter. Yeah, just one chapter.
Sven Johann: Yeah, so welcome everyone. Welcome Andrew. Welcome Heinrich. Welcome Alex.
Andrew Harmel-Law: Thank you for having me.
Heinrich Hartmann: Hello everyone.
Alexander Heusingfeld: Thanks for being on the show.
Andrew Harmel-Law: Yeah, you're welcome. Thank you for inviting me.
Sven Johann: Yeah, I think the book came out already two years ago. It's quite what? really? One year. OK. I gave a training course in software architecture last year. I think it was last year, but maybe it was beginning of this year.
Andrew Harmel-Law: One year, one year, one year. Yeah, one year, one year this month. So it's literally a year old.
Alexander Heusingfeld: On you.
Alexander Heusingfeld: So happy birthday!
Sven Johann: And then one of the attendees showed me the book and I was totally shocked kind of, know, there is an O'Reilly book on facilitating software architecture and I don't know about it. I mean, it was terrible, terrible feeling. Then I bought the book and I must say, we come back to it later. After the first pages, I was like, that's not a book for me. And then I, I skipped some pages and I started like chapter two, let's say chapter two. And then I was like, I had this, I must say I had this kind of team topologies moment, which means, yeah, we know this for quite some time. And how can it be that it took so long that someone thankfully wrote it down? ever since I'm really happy. I give everyone in my course not the book, but I always show the book and say, please buy this book. All right, so now what is the book about?
Andrew Harmel-Law: Nice, thank you.
Sven Johann: Brief outline what we want to talk about today. We want to talk about architecture significant decisions, architecture decision records, software architecture advice forums, and software architecture advice process. And a little bit of context setting. We are just assuming now maybe we can move course later that the teams we are talking about, they already own their decision. or they had at least some say in the decision so that we exclude the whole ivory tower architect thing we have to skip.
Andrew Harmel-Law: Yep.
Sven Johann: Alright, so...
Heinrich Hartmann: Can I ask a beginner question? What is software architecture?
Andrew Harmel-Law: That's a good... I dodge this, if you read this... I dodge this question almost explicitly in the chapter because... I do have opinions about it?
Heinrich Hartmann: But just to kind of set some boundaries, because I think it goes into modeling, right? How do you send your classes? And it goes into distributed systems, components, and it goes into teams and organizational design. So at somewhere there are boundaries. And how would you kind of delineate it?
Andrew Harmel-Law: Hmm... Yep.
Andrew Harmel-Law: Yep. Yep.
Andrew Harmel-Law: So it's a good question. And I think chapter two, one of the chapters is me kind of trying to tackle this because it kind of depends. But I am now going to try and answer the question. what I, like Sven mentioned as well, what I tried to say is the thing that seemed important to me and the thing that I think when you look at it from this perspective, it is a way to unblock the system of practicing architecture. To me, software architecture is kind of like the sum total of all of the significant decisions. specifically the ones that got into the code because I was haunted by this quote from Alberto Brandolini which is something like your software architecture is the developer's assumption as to what you said or what you know what like it's what everyone heard and then they turned into code it's not necessarily the pictures on a whiteboard or any of those other things so it's kind of the sum total of all of the decisions implicit and explicit intentional and unintentional and architecturally, this is the boundary where I try and draw the line between like people talk about design and architecture, the architectural ones are probably the bigger, sharper, scarier ones you hope you got right more than the ones which are like smaller or hopefully have less impact. I do spend quite a lot of time, maybe too much time some people would say, in the book trying to define what is significant because like You can update your maven dependency in Spring, which will then bring in a transitive dependency, which will later on completely depth charge your entire architecture. That feels like a very small thing, which isn't even an architectural, specifically an architectural decision. Other times you can spend months and months thinking about something which doesn't really matter because you think it's big. So, but yeah, it's the sum total of the decisions which have a longer term, bigger picture impacts.
Sven Johann: Thank
Sven Johann: you
Andrew Harmel-Law: maybe across more teams or across a broader scope or whatever. And that kind of goes to like think Martin Fowler's definition of things which are hard or expensive to change. It goes to some of the other definitions, but my framing was specifically around about decisions because what Sven said, I tried to bring that together and say this is the core of the practice, then everything else sits around it. And if we get good at deciding and communicating those decisions, then
Sven Johann: Thank
Andrew Harmel-Law: architecture kind of unlocks and becomes useful and valuable and in the code and all of those kind of things. So that's kind of... I mean, whoever reads the book can be the judge of whether I was successful, but that was kind of my goal.
Sven Johann: Martin Fowler also had another interesting definition because there is this official ISO or IEEE definition. It's about components and relationships between components or major components and cross-cutting concerns and principles guiding the development. And Martin Fowler once said, he has a very famous article. Who needs an architect? IEEE is like 20 years old. And I hope I say it wrong. Right. Sorry. What is architecture? It's about the important stuff. What is important? It's what the senior developers say is important. So it's a bit wishy-washy, but I think it's true. If you have very senior people, they know exactly.
Andrew Harmel-Law: Yep, that's quite, I quote that. Yep.
Heinrich Hartmann: Thanks
Sven Johann: This is something we shouldn't do lightheartedly. We cannot decide this now. It's like a big thing. Or we can decide it now. cares?
Andrew Harmel-Law: Yep, yep.
Andrew Harmel-Law: Yep, yep, precisely.
Alexander Heusingfeld: Talking about this significance, I guess that not everyone starts as a senior, And I'm quite often asked, like, okay, how do I identify whether I can take a decision or not? Andrew, what would you say?
Sven Johann: We
Andrew Harmel-Law: Well, so the quick answer is, as I outline in the book, if you follow the advice process, and the book talks about the architecture advice process, but I just stuck the word architecture on the front of the advice process. The advice process comes from trust organizations, and the answer is anyone can take any decision as long as, and there's two important caveats. Number one, they seek advice, which doesn't mean permission, it doesn't mean approval, it doesn't mean consensus, but they seek advice and not opinions, advice. which is like reasoned thinking. They seek advice from people who are affected and a nice way of thinking about that is either will you put something on someone else's backlog or are you spending money that doesn't belong to you or are you deploying resource which is maybe the backlog thing, right? You're making someone do some work which you don't control, right? And people with expertise. So people... If you just speak to people who are affected, maybe there's some smart person like Sven said, right, there's an expert who's done this before and will tell you that graph databases do not solve all of your problems, but they're not in an affected team, you definitely want to find that person and say, please tell me what went wrong in 1987 when you built your own graph database and you thought you were the smartest people in the room and stuff blew up. And then within that, yeah, exactly, graph databases, I may have had bad experience with
Sven Johann: It's not.
Alexander Heusingfeld: Interesting that you picked this example.
Sven Johann: you No.
Andrew Harmel-Law: And so this is, because this is the, I do agree that the more, I think architecture is theory and practice, so I'm spending a lot of time, I don't know if I want to write another book, but the thing I'm interested in now is like how people learn to become architects and this theory and this practice. And I think one of the problems with architecture is people think they just need to know a lot, but I think they devalue the things that they've done. And I also think you can, You knowing everything is an anti-pattern. Everybody, the collective, knowing everything, that's important, right? If you know a smart person and that person wants to share their advice and things, then you can power up everybody. But it's like that person needs to be willing to share that advice. And so this is where the book, this is what I think is powerful about the book. It kind of unlocks, like it turns, this is a Diana Montalian phrase, knowledge stocks, which is what we value. I'm smart, I know these things, I'm a senior developer because I've seen it and done it. and you turn it into knowledge flows and then you can maybe junior developers or people with less experience are still junior but they can benefit from the experience more frequently, more real time of like pair programming is another way of doing this right? You sit with someone and you see and learn because if you get that kind of stuff you can suddenly it removes a lot of the boundaries and a lot of the smart stuff kind of comes in
Sven Johann: Six months.
Andrew Harmel-Law: and then can be and then people can it's it's the acceleration learning for me is is terrifying people pay attention to stuff and likes and listen and learn and things it's very interesting
Alexander Heusingfeld: What do you think about unlearning? Because I had this wonderful example where someone said, but Alex, I have 10 years of experience. Isn't it sufficient if I say we always did it like this? So let's do it like this.
Sven Johann:
Andrew Harmel-Law: Yeah, this is the hardest thing is I think, because when I was at my previous company before Thorworks, tried to, we were like, TDD and pair programming is really good, we should all do it. The hardest people to get to adopt those practices and even to acknowledge that they were okay, maybe not even great, right? We weren't saying you have to love them. We're the people who had done it differently or we've always done it differently the other way. They were very closed off apart from a few who were like, okay, let's see. And suddenly they would learn amazing things but they'd have first to unlearn and that was very interesting and it's not for everybody right but yeah it's very it's like the curse of too much knowledge right it can sometimes blind you to stuff and that's the anyone can decide thing is quite nice because then it it makes you you can't just say no because it's the advice process right I'm not asking for your approval I'm just asking for your input if it works when it does work you're like right don't do this because
Sven Johann: Hmm.
Alexander Heusingfeld: Yeah, that's a good quad.
Andrew Harmel-Law: Here is the reason why. These are the things I know and then the knowledge is shared and all that kind of stuff and it becomes very powerful.
Sven Johann: Now you mentioned unlearn and also the advice process. I want to jump to the advice process, but one word about unlearn. In our, let's say, pre-show talk, we mentioned Barry O'Reilly, which one, but the other, the thought works, the ex-thought works, Barry O'Reilly, by the way, wrote a book about unlearning.
Andrew Harmel-Law: Yep. Yeah, which better?
Sven Johann: which is probably an interesting read.
Andrew Harmel-Law: Yeah, totally. It's very... And this is the thing that's interesting about unlearning. As things move fast, unlearning becomes more important, right? Because the things that were correct today are definitely not correct tomorrow. And that happens a lot faster these days. It's very interesting.
Alexander Heusingfeld: Exactly.
Sven Johann: No.
Alexander Heusingfeld: Exactly. I think that's the biggest challenge that we have nowadays to understand. If we look back two years from now, most of us are actually doing a completely different job. And would we only have the skills we had two years ago, we wouldn't be able to do our jobs today. So this unlearning is, in my point of view, prerequisite for rapid skill acquisition.
Andrew Harmel-Law: Yeah, this is that Kent Beck quote, like after Kent spent three days working with LLMs, like coding LLMs and was like, I've just spent three days working, I'm guessing how much longer it I've just spent three days, 90 % of my skills were irrelevant, 10 % of my skills are now completely worth more than anything. And Kent's right, and Kent's thinks about this a lot, right? It's very, it's very interesting. Yeah, Kent Beck, yep, yep, yep, yep.
Sven Johann: Can't back you, Yeah.
Alexander Heusingfeld: Mm-hmm.
Sven Johann: Architecture for advice process. You mentioned it so many times now. So we have architecture significant requirements or decisions to make. It's whatever affects other people, what senior people say is important. Maybe important stakeholders are. have an interest in this quality attributes like security, performance, availability, stuff like that. if we do in this area, usually we have architecture significant requirements. So now the question is, how can we as a team make a good decision? And you already, you mentioned it already a few times. If you follow the advice process and if you take the advice. So what is the advice process?
Andrew Harmel-Law: So yeah, the advice process is, like I say, it's a very simple process and so the idea is, like I said actually, anyone can make a decision as opposed to... So it moves responsibility and accountability to anyone as long as they want to. it doesn't mean everyone has to make a decision, but it means if I want to make a decision I can, I can say there is a decision which needs to be taken and I will take responsibility for it and I'm accountable for it, which is very important, right? It doesn't mean I'm going to decide something... it's still Sven's fault because Sven is the architect. And to do that, to take a decision you need to like go and figure out what the decision is in front of you, kind of scope it. If you're doing an ADR you'd write the context of the thing and maybe identify some of the options for different ways of doing it. And then you would seek advice and like I said advice is and you seek it from people who are affected and experts. The advice is specifically reasoned feedback and input into the thing. And it might be advice on the options, right? So like I want to do, we think we're gonna pick a new programming language and therefore the options are this, this, and this. then the advice might be, use Java because it's better and there's lots of people who know how to program in it. But the advice might also be in the framing of the context. It might be in the... the identification of the options, maybe there's a fourth option that someone's missed or whatever. It can be in lots of different things or the advice can be, and this is where I think is very interesting for the practice of architecture. One thing that architects have is like this large network of human beings that they know, right? Their job is not so focused in on one team. Their job is typically to know about people. They know that they need to speak to these people. They know that this is the technical strategy. They know that this old system has got...
Sven Johann: Hmm.
Alexander Heusingfeld: .
Andrew Harmel-Law: things that you can't touch on it. So they can also provide advice and say, right, you should speak to these people, you should listen to these people, I would worry about this kind of thing, have you remembered about this kind of thing? And they can, through that, not just say, this is good, this is bad, but they can say, here is how to think, here is how I have thought in the past, here is how I like to consider these kind of things. And so then you get this, the theory is you get this kind of spreading slowly of architectural thinking around about things. So you kind of build this knowledge system of people doing the practice of architecture. And in my experience, as long as there is trust, this doesn't work everywhere to be very clear, it has worked everywhere I've tried it apart from one place and that one place had... shockingly low levels of trust and they had far bigger problems than the fact that they couldn't make good architectural decisions. But everywhere else it was really, it was very interesting to see because most people want to do a good job and they want to build systems which they can live with and evolve and be proud of and that was, is a nice kind of way of doing it. And the advice process is it removes the unnecessary hierarchy and just kind of centres the skills and the knowledge and expertise of people.
Sven Johann: The final thing is just to expect it to stand.
Andrew Harmel-Law: which is, given that software is a collective act, it's not just like someone thinks really hard and then everyone else types. Everyone is doing some level of architecture design thinking at all stages in the process, right? It's just that we value some of them more than we value others.
Sven Johann: Thank
Alexander Heusingfeld: From personal experience, what if the person that wants to take a decision and says, I'm accountable, doesn't see or understand the relevant criteria or that they actually are not accountable. So they cannot take the accountability.
Sven Johann: Yep.
Andrew Harmel-Law: Yep, it's a good question.
Sven Johann: Maybe just before you answer that, we are talking about a team. And the team has some sort of lead developer. And the team is working on their service, on their system. That's what you mean, Alex, right? Yeah.
Andrew Harmel-Law: Mm-hmm. Yeah, typically it's a team.
Andrew Harmel-Law: Yep. Yep. Yeah, I think so, right? Yep.
Alexander Heusingfeld: does not necessarily have to be the service or the system. It could be any decision that the team takes, but that is actually beyond the accountability of the team.
Sven Johann: out
Andrew Harmel-Law: Yep. And you're right, because there will be. So this is the thing, again, so with more traditional ways of doing architecture, which we're not talking about, right, there is, you have architects who have general accountability, so this doesn't, the problem doesn't arise because, well, usually doesn't arise because I am accountable for everything, Whereas now, I could like do things like you said that could have impact where I'm not accountable. And so that's where, The key part is, and this is not in the advice process, like the vanilla advice process that's used in like, I don't know, like nursing companies and like nuclear power generation companies and things, the architecture one is like you need to write decisions down as an ADR. So when you make your decision, you write an ADR. Then when you say we're thinking of doing this, then you seek advice. The advice gets written down on the ADR as well, either as comments or whatever, and then, you know, and then... ideally is incorporated into the ADR. The key thing is though is that the ADR continues to exist with that advice, right? So you could have, in a bad scenario, sociopathic person who could like make a decision, go and seek all the advice, everyone says don't do this, this is completely a disaster, you're spending money that doesn't belong to you, you're exposing us to security risks that you you have no power over to control or whatever.
Sven Johann: Thank you.
Andrew Harmel-Law: But if those things are written down in the ADR, I've never seen anyone who then says, right, I'm going to ignore all of this valid stuff and just completely take this decision anyway. So there's a kind of, as Rita, my O'Reilly editor said, there's basically a social contract. I trust you to decide well and listen to my advice, and you trust me to decide well and listen to your advice. And it's very self-correcting. And the transparency of the information is what keeps it working, unless someone's a sociopath. I have not yet come across associate path and then if you are hiring associate path it is probably a bigger problem. that keeps it... What's good is though, and this is the other thing I learned this from Kat Morris who's an ex-colleague, people are really worried about it and what's really nice is you can say instead of doing a decision let's do a spike because people just can't decide, right? They're like, I know I'm right, you're wrong. And the other people are like, I know you're wrong, I'm right.
Sven Johann: Yeah.
Andrew Harmel-Law: You can spike something, which is very safe, but it's actual real code, and you can see if it works. So you still take the decision, but in a small, definitely reversible, code wins arguments kind of thing. That's a thought worker colleague used to say. We can all have our opinions, but then you write some code, and then you're like, OK, cool, I admit it. Or typically, when you write code, we're both wrong, neither of us are completely correct.
Sven Johann: Thanks.
Andrew Harmel-Law: We've built a proper spike which properly addresses the question that we've got in front of us and proved whether it was right or wrong.
Sven Johann: Now you
Alexander Heusingfeld: In the just let me stick for a second when in the concrete example, it was actually not malice, but it was people not knowing that they should consider a certain fact so that they should involve someone and specifically the team decided to introduce a new database without thinking of disaster recovery. So it was completely like an unknown unknown and that can happen to any one of us, right?
Andrew Harmel-Law: Yeah, yeah.
Andrew Harmel-Law: yeah, nice.
Andrew Harmel-Law: Yeah, this is the thing, yeah. Yeah, 100%. And so this is where people who traditionally would play these roles, that's where they step up and say, you definitely need to think about this. What about this? That's where those people become very important. And then if no one knows it, then it was an unknown unknown completely outside the organization, which also occurs, right? But the role of that becomes very important. What I've experienced is people like that because typically those people find out after it's too late. If you're doing the advice process, maybe they're not included in the advice giving, but if there's an ADR, these people are responsible, like InfoSec, for example, or QA can subscribe to the Confluence space where all of this is happening, or they know they can listen to all the pull requests and listen to a Slack channel. They can get awareness of all of the decisions and then they can read the ones that seem important.
Sven Johann: See you
Andrew Harmel-Law: and inject themselves into them. Like, for example, I did stuff with a bank and regulatory, they were like, we always get told when it's too late, but it's me that goes to court and maybe goes to prison. When they could see these things, most of them, were like, we don't care. I don't even know what React Native is, right? But some stuff they were like, whoa, you just did something which sounds like it's going to break GDPR hard and we're all going to prison. Or not everyone, me. And so the fact it's written down,
Alexander Heusingfeld: Mm-hmm.
Sven Johann: Thanks.
Andrew Harmel-Law: that can make and in the open that's a big thing right that can make it make it work yeah yeah yeah like pay like this is the thing like putting in the tools and like subscribing to it it's very useful
Alexander Heusingfeld: Yeah, yeah. That's valuable advice, thanks. Like the subscription is a good thing.
Sven Johann: Hmm.
Sven Johann: And now you said subscribe to it, subscribe to the space. You often mentioned the ADR. One thing in the book, which I was particularly happy, I must say, was a full chapter on ADRs, Architecture Decision Records. Why? There was this... 15 years ago, Michael Nygaard writes his article about architecture decision records. Some stuff appeared around it, but it was all short. I'm not aware of, maybe I'm wrong, but I was not aware of, let's say a decent good. description with everything you need to know. And I think in your book, if you are interested in ADRs, go to the book. There is a lot to learn. Maybe briefly for everyone, what is an ADR? What are the elements of an ADR?
Andrew Harmel-Law: Yeah, me neither.
Andrew Harmel-Law: Good question. So this is, and I should actually say ADRs are architectural decision records because I did a presentation in Germany and at the end someone said you didn't explain what an ADR is. So the whole half the audience, this might not be a German problem, I think it might, but I was like, oh my god, I should have actually said so ADRs are architectural decision records. it was popularized, this I learned from Ruth Mallon, it was popularized by Michael Nygaard, but Michael didn't invent the idea. There's a paper from I think 1980 something or 90 something. Someone suggested them and then but but it was a paper so we don't read papers right so I famously someone will write a computing science paper from 1950 and then this year will read it and we're like wow effect-oriented program is amazing apart from Kevin Henney who reads all of these papers right
Sven Johann: The cohesion paper from Constantine, it's like 50 years ago, I read it like three weeks ago or something. Anyway, just yeah.
Andrew Harmel-Law: Exactly right and you're like my god right this is the thing so Ruth is amazing so Ruth was like right Michael didn't invent them okay and so an ADR is it's a short technical document that you write which documents a single decision and I then spend later chapters trying to say how to make it single how to make it small etc etc and it has a standard in my experience although people add stuff which is fine but I've tried in the book to be like these are the bare minimum things so it has a standard set of things so it has a title, has an identifier, whatever you want to call it, ADR001 and then sections and I put them in this order because you write them once but you read them a lot, hopefully so the order I have it in is the decision is the next section which you write last like an abstract in a paper right but if I want to click on the ADR I want to Read the title which tells me what the decision is. I want more information I click and I read the decision paragraph. Below that is the context. Below that are the options considered. And you don't just say we considered three of them and we picked option C because each of those options should have pros and cons for the options. And then if you're following... So that's all standard Michael Nygaard stuff although I changed the order. Because Michael Nygaard I think has context first. If you want to just know the decision, have to read through a bunch of stuff and you're like, just tell me what you decided. Yeah, exactly. I spent too long speaking to Bruce Eccles. like pyramid, like tell me at the start. And then, and it works, right? You can then read and then if you're like, why is this an important decision? Then you read the context and then you're like, right, what were the three options you consider or seven options you considered? Keep scrolling. If you want to know what were the pros and cons of the third option, keep scrolling. And the last thing is the advice. So that goes in the comments, if it's a confluence page or whatever.
Heinrich Hartmann: Pyramid principle.
Andrew Harmel-Law: because ideally if people follow the advice process they roll the advice into the ADR they're not obliged to but it still keeps that history and so things like little tricks that I've learned which are really good are if someone says this will be a bad idea you can say despite this advice we are still picking option A so you're acknowledging that someone disagrees with you and you're saying yeah we know we hear you look at the questions we had before or despite this downside we are adopting this or despite this positive benefit of Java 1.3 we are choosing not to adopt Java 1.3 because Java has moved on. So you can kind of acknowledge those things. The other final thing is, so ADRs are read-only when you finish writing them. So writing them, and Michael Nygaard points this out, writing them is a creative act and architecture is creative. So you learn by thinking. So you learn by writing the ADR. And one of the key things as an architect you can teach people to think by helping them write. When the decision has been made, it's immutable. So it's like an immutable change log. People, this winds me up more than anything and I think I made this very clear in the book. People are like, we should re-decide. Don't reopen the immutable ADR. Have a subsequent ADR and say, right, that was right at the time, but now we think this, so therefore we're going to do this instead. Because the amount of ADRs I see that just get opened again and again.
Sven Johann: Hmm.
Andrew Harmel-Law: and you can't see the original history because it's now been overwritten 17 times you should have like an immutable changelog like Prometheus or something of stuff that goes all the way through
Sven Johann: Yeah, is also, I mean, everything you said, there are also nice templates. We put them in the show notes, like markup, architecture, decision records. You can just adopt them as a listener and have those ideas.
Andrew Harmel-Law: I was glad you liked the chapter actually because the formatting in the chapter was the hardest thing in the world to get right. It was an absolute nightmare. So it took the... of all the chapters in the book, I'm glad someone likes it because... because as you can tell, I got really... I got really annoyed about like... People are smart people and they make good decisions but they can't express them in an ADR. So if you... if people get this right, suddenly...
Sven Johann: Mm.
Andrew Harmel-Law: lots of the transparency and stuff. It doesn't mean that you don't need other architecture documentation etc but like a nice clean set of ADRs can really have a significantly transformative effect on the practice of architecture in my experience.
Sven Johann: Yeah, yeah, I mean, it's also.
Alexander Heusingfeld: talking about... go ahead man
Sven Johann: No, I just wanted to say it's scaling knowledge at its best, right? So if someone asks why the hell are we doing it like this, nobody explains anything. You just explain it by, here is a link, read it. If you don't agree, make comments. Or if we missed something, I think it's one of the best.
Andrew Harmel-Law: 100%.
Andrew Harmel-Law: Yep. Yep.
Sven Johann: The best tools we have to make great decisions.
Andrew Harmel-Law: so powerful.
Alexander Heusingfeld: Well, it's
Andrew Harmel-Law: When one thing, and there's a case study, can send you it for the show notes actually. I was at Zappo after, so there was the company where I kind of initially experimented with this, which is called OpenGI. Then afterwards I went to this Bitcoin bank called Zappo. So there's a Martin Fowler case study. One of the chief architects there, or architects at Zappo, had a history in technical writing. So they not only... helped everyone write really good ADRs. They curated like a kind of narrative page on top. So they were like, if you've just joined, you don't need to read. And they loved ADRs because they went, there was a lot of ADRs. But they created a page which was like if you're a new joiner because they were a scale up. Here are the key things you need to know because this will tell you the story of why our architecture is like it is. And they even retrospectively went back and documented some of the key decisions. because they knew they weren't perfect but they were like at the time this was the best we could do based on what we knew and as an onboarding and kind of historical record so powerful and all this person had to do was like just the ADRs were there they were just you know they just decorated it with a kind of master page and then you could see some of the older architects expressing why this is important why this is historical why we you know I know this feels weird but we tried something different three years ago and it turned out that it
Alexander Heusingfeld: Hmm.
Andrew Harmel-Law: for some weird edge case it doesn't scale. All of that was there. was beautiful. was beautiful, beautiful.
Alexander Heusingfeld: Also from personal experience, it's sometimes hard to understand whether a decision that has been made is still relevant today. So we ended up adding two more attributes to our ADRs. One is the minimum time to live.
Sven Johann: So,
Andrew Harmel-Law: by a prophet.
Alexander Heusingfeld: to say how long is an ADR actually supposed to be valid and after which amount of time do we want to revisit. The second is the minimum validity period saying we take this decision now and for at least six months everyone can rely that we're not taking a different decision.
Andrew Harmel-Law: Yeah, yeah, perfect. We had a similar thing at Zappo where we took a decision because they were a very product focused company and they were trying to move really fast. There was one thing which broke some of the principles and this is a lovely thing to like actually nice things to call out in ADRs are, there's boring ADRs which are everyone agrees and blah blah blah, right? We tried three options. We know that we're a Python company that uses Django. So we're going to decide to use Python and Django. But there was a few decisions which went against some of the corporate standards, but for product management reasons to address a new market quickly. They were like, we are going to break the rules explicitly because it will give us this benefit and then in six months time we will revisit this so we're incurring architectural debt and we're writing about it, we're being clear and then in six months time, like you said Alex, right, then we're going to come back and say, okay, do we still want to keep this because, you know, it means that we've like, we've done something that we're not proud of, but it got us out. down the path but now we have a record of this thing. Do we want to then invest the recovery time to bring it back or are we comfortable living with it etc etc. And it's super powerful. It builds people's trust because you'll see people who are scared of deciding because they're like this will be used against me later on. You're like write it down. Say you're not happy about it. This is why we do this despite the fact it's a bad idea. You can say that out loud right?
Alexander Heusingfeld: Absolutely.
Sven Johann: Mm. Mm.
Andrew Harmel-Law: I mean, Rebecca verse Brock has a thing where she said another field which you can add is like how you feel about the decision. So there's another paper somewhere which Rebecca, I'll dig it out and send it to you for show notes. But she's like, not everyone's happy with every decision they make, right? They're not super comfortable, but there's like a reason, right? Someone told us it was super important and we had to just do it. Okay. So was very, very interesting to see that.
Sven Johann: Now
Heinrich Hartmann: Could you talk a little bit about the scaling of the system and how to organize the knowledge around it? Because I can see this as something that you do maybe in a team of two, maybe in a single code base. I could see something that you do that at the company scale of 100 people and you probably need different tooling for this. the second bit is related, which is you talked about architecture depth, very familiar. You sometimes take the shortcuts.
Andrew Harmel-Law: Mm-hmm.
Sven Johann: Okay.
Heinrich Hartmann: And just documenting that there was a shortcut is already good step, right? But how do you deal with that from a kind of knowledge organization slash process side that you are bringing these decisions back to the front?
Andrew Harmel-Law: Yeah, that's great question. the last chapter of the book, so this was we were talking again before we started recording about... I tried... So as someone pointed out when I wrote the book, the book is fundamentally about the advice process. So someone suggested I should use the advice process to write the book. So I wrote it for a publisher, so I can't really do that. I wasn't writing completely out in the open, but every chapter I wrote I could publish on the O'Reilly Early Access Program. And then I got... And I put my email address at the start. It was like, please send me feedback. I would love your advice. Many of the later chapters come from people going, yes, but what about? And one of them was, how does this scale? And so I kind of dodged this question. So there were three inspirations for the book. One was Ruth Mallon, who said the architecture is conversations. And I was like, right, the center is conversations completely.
Sven Johann: you
Andrew Harmel-Law: Number two is Alberto Brandolini that I mentioned, you know, what gets shipped to prod, the real architecture is what gets the developers' assumptions about what you meant, right? That's what's the architecture running in prod. And the third thing was the power of self-organization, which I'd experienced in Bruce Eccles' Java Posse Roundup and the Wintertech Forum, Open Spaces, and I've now been to loads of more Open Spaces since. And so I once had a question to Bruce, which was like, how big can an Open Space get? Because it's self-organizing.
Sven Johann: Thank you. Thank you.
Andrew Harmel-Law: how big can it get? And Bruce said it can go as big as it needs to get and when it gets too big it'll split into two. Like if you have... His experience was like 80, 90, 100 then you can have all of those people participating in the same event. If it gets much bigger than that your event will just split into two events whether you like it or not which are happening at the same time in the same venue because human beings... brackets Dunbar. Dunbar's number is all a bit pseudo-scientific. I've done a psychology degree, right? So things like Dunbar's number are a bit pseudo-scientific but...
Sven Johann: Thank you for joining us.
Andrew Harmel-Law: At some point humans and the knowledge system decides it can't be that big. So I use this to kind of dodge the question in chapter 17 of the book which is like you start small because we all know starting big always goes wrong, right? You start small and then incrementally move this up and you add, and this is why the book is structured with like the core pieces which is the advice process and ADRs, then you add extra things if you need to to kind of make it scale. But at some point you will notice that there are conversations happening which many people don't care about or there will be things which are so completely diverse or whatever, people working on completely separate pieces of the system. And in that case, you can just like start splitting stuff off and assuming that you can have like two sets of advice forums, which is like a weekly meeting, which a like an architectural review board, but you're offering advice, not asking for approval. those things split up, can start subsetting your ADRs into like these are the ones that everybody cares about, these are the ones only my team cares about, so you can kind of manage the cognitive load. Fundamentally you kind of just end up using tricks or experiments, right, not tricks, you try things and see if they work, you try and manage the cognitive load to build your knowledge architecture, which hopefully follows the knowledge architecture of your systems, right, you're hopefully doing bounded context, so you've... You of manage the cognitive load from a systems perspective, you're kind of managing the cognitive load from a teams perspective and then you're kind of doing team topologies and stuff. You're hopefully pulling out things like platform type products, internal products and things to basically take your core principles and turn them, make them into codes to make those principles easy to conform to. I know I'm dodging the question. You kind of, this is where I said O'Reilly wanted to know what to do. The answer is when you feel the stress of it not working, that's when you try and split it and you try and find a natural seam. And in my experience, although I love domain driven design, so I would say this, the domain is where things will split. And so then you split it into kind of anarchistic, the word anarchy doesn't appear in the Martin Fowler blog post, but it does appear in the book.
Sven Johann: Hmm
Andrew Harmel-Law: It self-organizes because it's like, the human beings are like, this is not working, there's too much happening in one place, so let's split it into two separate pieces and then use platform teams to join everything together or use principles to join everything together or use a tech strategy to join everything together and it kind of self-organizes. But then you need to make sure that you are...
Sven Johann: Hmm.
Andrew Harmel-Law: as someone who is traditionally higher up in a hierarchy that you're making space to say, it feels like people are not happy about this, it feels like people are not, you know, they're not following around, it feels like conversations are too big or too small or too many people don't care about them or our architecture is too coupled, you know, this is the benefit why I think it works. And I know it's quite a long rambling answer. They're like...
Sven Johann: But I think...
Alexander Heusingfeld: No, but it really resonates.
Andrew Harmel-Law: We're good at like, know, our spidey sense from software is we're like, this is too coupled or this is not, you know, it's too, it's not cohesive enough. We apply that thinking to human being organizations. That's when we know it works. That's my kind of, that's what I should have said at the start. If you kind of apply that thinking, that's, know, this is, the conversations aren't cohesive. There is no, there's too much coupling, et cetera, et cetera. That's the way to do it.
Alexander Heusingfeld: Reminds me.
Sven Johann: Hmm.
Alexander Heusingfeld: Reminds me bit of this programmer anarchy talk from Fred George. have been like 2012, 2013, somewhere.
Andrew Harmel-Law: Yes.
Andrew Harmel-Law: Yeah, yeah. I need to properly re-watch that again actually, it's... yeah.
Sven Johann: Yeah. But I thought it's the best way to organize it is to do this hierarchically, to have a hierarchy. I'm working on, no, how do we organize seeking advice and organize, let's say, the amount of
Alexander Heusingfeld: You mean decision records or what?
Andrew Harmel-Law: Mm-hmm.
Sven Johann: The other thing is you have a certain amount of architecture decision records. Let's say you have a confluence space with all the architecture decisions of the whole company. And if you Google, you get every second an email that something has changed. actually, the company where this person recommended the book to me, they don't do this. But when I talk to them, I
Andrew Harmel-Law: Yep. Yep.
Sven Johann: I was like, their setup, I would immediately think that the organizational setup works for advice forums. We haven't talked about the advice forums yet. We do it in a second. So you are a team and you own your decisions. everyone can, if you're interested in the team, subscribe. The team is part of a larger domain. So now,
Andrew Harmel-Law: Yep. Yep.
Andrew Harmel-Law: Yep. Yep. Yes.
Sven Johann: Let's say you're a domain architect. You subscribe to all the decisions of all the teams. And if you're a team, you subscribe to all the decisions of the teams you're interested in. What you said, do you have a cohesion and coupling thing? Which teams I have a strong coupling to or I'm dependent on? So there I subscribe. And then above the domain, I think they call it
Andrew Harmel-Law: Yep. Yep.
Sven Johann: work streams or some sort of streams. And then you can again subscribe if you're interested in this very high. It's basically on that level, they only have principles. They make only decisions about guiding principles. By the way, we have an episode about guiding principles with your colleague Brigitte Böckeler. And I think if you have this hierarchy,
Andrew Harmel-Law: yeah, nice.
Sven Johann: it's probably a good start to organize it.
Alexander Heusingfeld: So if you're, just for clarification, you're saying hierarchy, you are referring to like the impact or reach or blast radius of a decision?
Sven Johann: Yep. Yep. Yep.
Andrew Harmel-Law: So I think, yes and... because I did a talk in Romania yesterday about this. So one of the reviewers of the book was Trond Shortland. So Trond is a big socio-technical thinker and proper socio-technical thinking, right? Like the Emerys and Trond's got various talks about the history of socio-technical thought when it came out of the mines in Wales in the 1950s and it were 1940s. And so I was very anti-hierarchic because I'd read too many books by Anna. And Tron was like, hierarchy isn't bad as long as hierarchy is just like you said Alex, Like broader scope, broader time scale, bigger blast radius, more. But it comes with lots of dangers because number one, the only way we see to organize is a fixed standard hierarchy, which is just copied org design, which like we just, all companies do it, right? And then we're like, right, then there's this level of management, then this level of management, then like.
Sven Johann: Hmm.
Sven Johann: Hmm. Exactly.
Andrew Harmel-Law: So like it needs to be functional and it needs to come from the work. The good thing is, like, and this was my argument in my talk yesterday in Romania, we know what our work is, right? We're very clear. We know what we need to build. We're very good at refactoring our code. We're getting better with things like team topologies, like refactoring our org designs, which I think is very good, right? We should be able to, and I've seen...
Alexander Heusingfeld: Yeah.
Andrew Harmel-Law: This thing is mentioned in the book, you could do a social decision record which is like we decide to merge these two teams. The two teams have got together and decided they should merge, they shouldn't seek someone else's approval. Although they should seek their advice. So if you have a hierarchical level, and just like you described Sven, we're very light touch, we're not, I don't like the term guard rails because that tells you where you can't go, but it's more like paved roads, which I like more. We want you to go in this direction so we've made it easier. and this is the word that Netflix uses as opposed to guardrails. So therefore, but we know you might go somewhere else, right? Maybe there's a reason to go somewhere else, but our principles and our maybe they're baked into our non-functional requirements or our platforms and stuff. We've made this easy for you because we want you to hopefully go in this direction. And then lower down this some more constraints, but just like this is the other thing I was talking about yesterday, Christopher Alexander's pattern language. The patterns start at a regional level, They're like a whole valley or a whole flat plain of land with loads of different things and it goes all the way down to where you'll put your windows in a building. So this is the book which inspired the pattern language that we use in software.
Andrew Harmel-Law: The thing is though, and this is the mistake we make, we have higher level, bigger, broader scope things, but we over-specify them. So Christopher Alexander's pattern, is something like villages joined by country roads, It tells you very little. It's just like there'll be a village and there'll be some country roads. Whereas we're like, sometimes at the high level, we're like, please use SAP version this, this, this, and this, and use this programming language, and we over-specify.
Sven Johann: Thank you.
Andrew Harmel-Law: as opposed to setting more broader kind of things. And I think if you, the layers but where you kind of, have just enough touch to kind of direct things but without constraining the subsequent decisions which sit inside of those things, that's when I think it works. And that's kind of to Heinrich's point. Finding that thing and like checking when it makes sense and like you said as well Sven, right? Like revisiting a decision because you thought it worked but now it doesn't.
Sven Johann: Thank you.
Andrew Harmel-Law: keeping all of that stuff flexible because the system will change and evolve and find its natural place and as like Simon Wardley would say, right, like we build something but then it's continually evolving because technology is changing and what was true yesterday is not true in three months time. Keeping all of this stuff open, so this I think is like chapter 14 where I'm like, we pretend like we write code and that's it, but this stuff is changing all of the time and there's like a kind of, there's a top down
Sven Johann: Hmm. Hmm. Hmm.
Andrew Harmel-Law: thing but there's also a bottom up and listening to both of those feedback loops is super important right like one team trying to build one thing will be experiencing the knock-on effects of someone making a decision about a corporate LDAP server 17 years ago you want to hear that problem as much as you want someone to say strategically we're going to go in this direction right we're going to adopt open source or something and you want those two things to come together and that's where the advice process I think is good because it
Sven Johann: Hmm.
Andrew Harmel-Law: It gives anyone the power to challenge these things, but then they want to like, the conversation needs to happen and stuff.
Alexander Heusingfeld: I what I'd like to reiterate here is that hierarchy is absolutely not the company hierarchy. Someone on the same hierarchy level, but in another subsidiary taking a decision can have an impact on you. Right. So it's more. Yeah.
Andrew Harmel-Law: 100%.
Andrew Harmel-Law: Yeah, totally, totally, totally. And when the two, when we confuse the two is when we get into trouble, right?
Sven Johann: Hmm.
Alexander Heusingfeld: Exactly. So it's maybe it's even better to look at the communication structures like Conway's law suggests, but it's definitely not the company hierarchy or how the company is structured.
Sven Johann: Mm. Mm.
Sven Johann: Hmm. Hmm.
Andrew Harmel-Law: Yeah, there's a super interesting talk actually where, so, Dr. J. Bloom, who does the most crazy, like super deep, let's read the papers type talks, he did a talk at DDD Europe where basically he went back to the Conway paper and then said, what was everyone else writing about at the same time? But he properly read the Conway paper, right? Not just the one sentence, but all of it. And he talks about like, sympoesis and autopoesis, which is like the natural evolution of different teams finding their boundaries and finding
Sven Johann: think we should not.
Andrew Harmel-Law: who has a broader scope, who has a... You know, like if we're peers, then we need to find out where we interface. If we're like broader hierarchy, so I think that's simple poetic, and I'm maybe straying into territory I don't know so much about, but if there's like hierarchies, like I have a higher level, broader, but maybe less detailed view, that's auto poetic. They're self-organizing concepts, and like as Conway pointed out, they happen whether we like it or not, But like...
Sven Johann: Hmm.
Sven Johann: Hmm.
Andrew Harmel-Law: focusing on those things and again, and this is where the socio-technical thing comes in, I think the practice of architecture is not less important, it's more important, but being aware to these broader things like the org design, how specific a broader, bigger design is, you want it to be directional but not restrictive because you want people to still be able to practice architecture within the scope of that decision. This is why it's interesting. think this... I would say this because I wrote the book, but I'd like it to unlock a kind of new way of practicing all of this kind of stuff because I think it's... I think it's interesting.
Sven Johann: Hmm. Hmm.
Sven Johann: Damn it! I thought I just take the organizational structure. So now there is work. Yeah, but it should work.
Andrew Harmel-Law: Only you could! But yeah, just don't want to assume... I mean, this is the classic domain-driven design trick, right? You're like, what are the bounded contexts? You just look at the org chart, and that's what the org thinks it is. Maybe they're wrong, because like... But then you've got like version 0.1 of your domains bounded context. And you've also got a list of all of the pain points, because everyone will tell you exactly what they hate. So it's not bad, but making it like...
Sven Johann: Yeah
Andrew Harmel-Law: Just another thing that can be designed and changed and updated, think is that's the superpower. I think as software professionals, we should understand this way more. We don't, right? We're like, we're big now, so we need to have a hierarchy. I'm like, but you can make it, right? doesn't, you can, can, it's up to you, right? But it isn't because of power dynamics and stuff.
Sven Johann: Yeah, yeah, yeah.
Sven Johann: Hmm.
Alexander Heusingfeld: And now that you, like now that you mentioned the bounded context, that's actually something that, that I try to emphasize a lot in terms of, we always assume a certain bounded context in terms of like, that, that is not relevant for me. Right. But understanding that actually there is no green field without any integration.
Andrew Harmel-Law: Yep. Yep.
Andrew Harmel-Law: Yep. Yep.
Alexander Heusingfeld: That is pretty important, I would say. You always have a relationship to someone and you better understand what type of relationship that is in the domain-driven design, strategic design context.
Andrew Harmel-Law: Yeah. yeah, totally. And there's another thing, because I've re-read, because I do this training course on O'Reilly, I've re-read the book, the blue book, many times. One thing I realized, but I'd not realized for the first three times reading it, I had read it for a lot of times, assuming you just kind of like, you find the domain and the domain tells you what the software looks like. But Eric's very clear. He's like, you you co-create, like the domain experts tell you what they think. And then you're like, right, we've got software. But you don't just copy what they tell you. You're like, right.
Sven Johann: Hmm, hmm, hmm, yeah, yeah.
Andrew Harmel-Law: We have the power to do this differently. Let's co-create a thing so you know what you need to do because you understand the business problem or the business domain. We understand software, so we can make this easy, right? That's simple, we can fix this. And co-creating something, which was a big... I was like, oh my God, I'd never even noticed that before. it's very... And I think the org design stuff, this is why I spend ages talking to Trond about this. We just assume that the organization is the organization. Who said? Just because someone... And this is chapter 17 of the book, it's like... Half of chapter 17 is how to do this by stealth so that everyone in the rest of the organization, like HR don't think you've changed anything. Right, so HR are like, there's still seven teams and they seem to be doing software development, but meanwhile you've changed all of the stuff, right, so...
Heinrich Hartmann: That is exactly doing it in stealth. I think that is the really interesting bit. And that was like also the scaling aspect. My intention was what more like a whole small kind of start, right? I think the entry barrier zero. You can start this by yourself if you really want to. Yeah.
Andrew Harmel-Law: Yeah, 100 % you can do it. If you want to like, if you have the power to decide, just start following the advice process because people will be like, wow, you're doing this in the open. If you don't have the power to decide, you can still do it. And then you just take your ADR to the people to seek advice and then propose it to the person who does have the power to decide and say, we would like to take this decision. We don't have the power to, but we would love you to see what we think and get your feedback on it. And in my experience, like, Because sometimes I've run this and the architects are like, no, no, no, no, I don't approve this. And then it gets to the point where they're like, I trust you. Just make sure you do it properly. I'll just rubber stamp all of the things. So we don't even change the approval process. There is still technically a sign-off process. But they're like, we get it. We've read all the ADRs. I can see you've spoken to this person. I understand that this, I can see you've done the work. And that's another, that's exactly it. Heinrich, they're right. You start the small and then you go.
Heinrich Hartmann: Yeah.
Andrew Harmel-Law: And what's also cool is, so I mentioned OpenGI where we first tried this, we were trying this on one program of work. So I'm lucky because I'm a consultant, we get brought in, we don't get asked to do everything to fix everything because that would be expensive. We're like, here's one project, fix this or work on that project. So we came, me and this Pete Hunter basically did a bunch of mucking about because we'd read the same books. We were like, we think this might work. But we weren't. Another team which were maintaining a Java legacy app were not within scope but they were advisors so they kept coming to our architecture advice forum which was this weekly meeting where people would talk about ADRs. I discovered after I left that they set up their own version 2 completely separately but they were like, this is working really well and people are asking our opinions and we can ask their opinions. And that was super gratifying that people were just like, this seems to be people are talking about architecture. This is my big thing.
Heinrich Hartmann: Yeah.
Andrew Harmel-Law: People think that architects talk about architecture. Everyone talks about architecture. It's just that if you don't have any power, you complain about it. And if you do have the power, you worry that someone's going to catch you out because you haven't thought about everything.
Heinrich Hartmann: Yeah, this is very important kind of principle with general organizations that you kind of either have to percolate information up to the decision maker and write all these reports. You can kind of empower a higher manager or somebody up there to kind of make a informed decision, right? Or you are handing down the authority and the agency to do decisions further down. And that is, it's scary for many organizations to do because then what if something goes wrong?
Andrew Harmel-Law: Yep. Yep.
Andrew Harmel-Law: Yep.
Andrew Harmel-Law: Yep. Yep.
Heinrich Hartmann: So I think this ADR is right at the heart of this and the transparency I think is the big differentiator here.
Andrew Harmel-Law: Yeah. Yeah, 100%.
Sven Johann: You just mentioned the advice forum. I wanted to come to the advice forum like 20 minutes ago, but now we made it. What is this architecture advice forum?
Andrew Harmel-Law: I did it on purpose, Sven, because I knew it was on your list. Yeah. Yeah, sorry. Sorry, sorry, sorry.
Andrew Harmel-Law: So it's good question. So it's not, I think this is maybe the first sentence in the chapter, it is not an architectural review board, although it looks very similar. So it's basically a regular meeting because, like I said, this is based on trust and I did a bunch of research on trust and one way to build trust is you see people, other people doing things, in this case making decisions, and the more you see them do it in the open, the more you will trust them. Even if I trust them to like annoy me, right, I'm like, they're gonna make, Sven's gonna make that decision again, which I hate.
Sven Johann: Mm.
Andrew Harmel-Law: At least I can trust that Sven is consistent. So it's a regular meeting. I thought it was an hour every week until I worked for Zappa, who are a fully remote company, and they were like, we do not want a weekly meeting with 40 plus attendees, because it's an open meeting, anyone can come. And the idea is there's two standing agenda items. Anyone got any new spikes, tell us what spikes you're doing. And that's just for information, because spikes, like I said, Kat Morris figured out that spikes pretty much turn into ADRs. Does anyone have any ADRs? In the larger meeting, people offer advice in the meeting and then are responsible for writing that down. So then you get the practice happening out in the open, which is good, but also people attending can see the practice of architecture happening. So for example, at Zappo, Noosh Streets, the CTO would come along and basically say nothing, but they could just see that their teams were thinking about architecture and Noosh didn't need to do anything because it was happening. That was super good. And then, like, so then you could see it happening. In Zappo, because they were like, we don't need no flippin' meetings, and they were very good at written communication. They would share the things, they were like, for your information, here's a big significant one coming up. Asynchronously, everyone would write stuff, because they were very, very good at written, their Slack discipline was the best I've ever seen in my entire life, it was terrifying. And they would capture all the stuff and they would write communication. Other things I've seen be added, which are really cool, OpenGI would talk about their, and this goes to the other chapters I wrote, because basically that's a case study of what we did at OpenGI, they would report the 4K metrics, because the 4K metrics turns out to be, if you need a business case to do some architecture work which doesn't directly have product value, I guarantee it will improve your 4K metrics unless you are doing a vanity project.
Sven Johann: Hmm.
Sven Johann: Forky metrics are the Dora metrics, we would say.
Andrew Harmel-Law: Dora 4K metrics, so lead time for changes, frequency of deployments, change failure rate and mean time to recovery. If you are doing an architectural improvement or something to decouple your architecture or make it work better, unless it's a bad improvement, it will have a positive impact on your 4K metrics, which is why it's the first chapter in the software architecture metrics book.
Sven Johann: Hmm.
Andrew Harmel-Law: because it's kind of not a software architecture metric but I'm like I don't think I could believe in software architecture metrics but like it's a second order metric it's like if this is good it will be more deployable more stable more easy to change etc etc
Sven Johann: Hmm. Yeah. One comment definitely on doing it in the open. At my current client, we also introduced those advice forums, of course. I must say the first time I've seen them was probably eight years ago here. Alex Heusinkfeld introducing them to the place I worked. So he is for a long time in it.
Andrew Harmel-Law: Yeah, none of these ideas are new.
Sven Johann: So I also introduced them at my current client. And one thing is it used to be a regular weekly meeting. We are not so many teams. But the thing is there is just a channel, Microsoft Teams channel. Whenever you have to make a decision, post. We have the rule you have to post it like three days before so that everyone can read the ADR.
Andrew Harmel-Law: Mm-hmm.
Andrew Harmel-Law: Yep. Yep.
Andrew Harmel-Law: nice
Sven Johann: My, at least my feeling is, and that's also the feedback, you feel very well. You know, what you said like half an hour ago, like what are you feeling about this decision? We kind of all feel very safe because we heard a lot of people. We just heard a lot of people. And what you also said, the opposite is so true. Decisions made beneath the curtain. you know there is a central service going to build. you just don't know, you are affected like everyone else is affected and they just make this decision in secret. It scares everyone like from morning to evening and we are always like do this in the open. It's a much better outcome. Just it...
Andrew Harmel-Law: Yep.
Andrew Harmel-Law: Yep.
Andrew Harmel-Law: Yep, yep, totally.
Sven Johann: And it's so important. And yeah, I just want to underpin everything. Yeah.
Andrew Harmel-Law: Yeah, and what's interesting is the last thing, and I think I major on this in the book, the problem with architecture review boards is it's like, I'm right, you're wrong. Someone has to win and someone has to lose. Because it's advice, it removes the kind of ego, I need to be right, you need to be wrong. We can just share, I can tell you what I think. You can ignore me, but then I still get to say what I think and then I should write it down.
Sven Johann: Mm.
Andrew Harmel-Law: And so what I've seen is the quality of discussions, because someone doesn't have to win and someone doesn't have to lose, the focus is on the quality of the advice, the reasoning and not just an opinion, so Diana Montalian pointed that out to me. In review boards it's opinions, Java's slow, right? Okay. But advice is like, think this will have, Java will be problematic because Java has a known problem with this and then people can ask you questions, etc. etc. right? And then you can get to the focus of the real point as opposed to, here is my opinion, I will
Sven Johann: Mm-hmm.
Andrew Harmel-Law: be unshaken in this and like my dominance of Kenny Barschwegler and Heen, I shouldn't have said Heen's name because I can't remember Heen's surname, but the authors of, and Evelyn, the authors of collaborative software design, they talk all about like power, because there is all these power hierarchies, right? They're all over the place. And it kind of, in many ways, not always, but it removes the power from lots of stuff. not everything. There's a whole other chapter about approaching the people for advice who you know will disagree with you, seeking, you know, like, what's it called? Not consent farming, but farming for dissent. When it's the advice process, the person who's going to disagree with me has no power to make me say no. I'm more inclined to hear what their concerns are. Because I know that I'm not going to lose an argument.
Sven Johann: Hmm.
Andrew Harmel-Law: But I do want to know why they think that Oracle is a terrible database. OK, go and tell me why Oracle is a terrible database.
Sven Johann: Yeah, yeah. Yeah, also my engineering manager there, he also joins those meetings and he makes it very clear from his behavior that, you know, I'm the, everyone knows he's the hippo, the highest paid person in the room, but he's giving advice. And I think that's also important that the expensive people say,
Andrew Harmel-Law: Yeah, yeah, yeah, yeah, yeah, yeah, yeah.
Andrew Harmel-Law: super powerful.
Sven Johann: I'm here to give advice. You can say no. You can disagree. It's just my opinion.
Andrew Harmel-Law: Yep. Yeah, totally. Totally. Because it is, it means like... And it's super interesting how many people who are higher up don't realize this stuff. And then as soon as it becomes advice they're like, wow, people are disagreeing with me. They've never disagreed with me before and you're like, they told me, they complained to me about it a lot. But like, they won't. And so, yeah, this is why it's like...
Sven Johann: yeah.
Andrew Harmel-Law: So, the word magical is not in Martin Fowler, the blog post which turned into the book, because Martin said... It's maybe not the right word to put in a software development paper, but frankly, it's kind of close to magic. Like, when you see this stuff, you're like, okay, suddenly, because we're a bunch of smart people who don't know everything, and we all know we don't know everything, but we're trying to do the right thing, build software that makes us happy, that makes us, you know, that we can live in. I think this is my book... Something about living in software is in my next book.
Alexander Heusingfeld: Nice.
Andrew Harmel-Law: That's where we spend all our time talking about it and staring at it. And yet we're like, we have all these power dynamics. So if you can remove lots of the power dynamics, because we have the skills to think about how we structure ourselves, because we think about how we structure our code all the time, I think that's a really powerful way of doing things. So this is the pitch for the new book. I'm not really sure what it is. I don't think O'Reilly will publish it.
Sven Johann: Hahaha
Alexander Heusingfeld: Mm-hmm.
Andrew Harmel-Law: Something about taking these skills all the way from the level of software and writing code and type systems all the way up to organizing teams I think is really... I think we're smarter than we know but we don't realize that we're smarter than we know.
Sven Johann: All right, yeah, maybe final questions for everyone.
Andrew Harmel-Law: I'll try and be brief.
Heinrich Hartmann: Yeah, mean, as we are, it's as it's 2025, right? I think we have to ask the AI question.
Alexander Heusingfeld: Mm-hmm.
Andrew Harmel-Law: Yes.
Alexander Heusingfeld: You
Heinrich Hartmann: There are like a lot of angles that just listening to this and I admit a lot of this is first time for me I'm not more on the SRE side than on the architecture side so a lot of interesting learnings But there immediately some sometimes the bell was ringing Hey, isn't that a good use case for AI and I'm sure you must have had that thought so What I from your perspective kind of the most promising angles on kind of weaving in AI into those kinds of processes
Andrew Harmel-Law: Hmm
Andrew Harmel-Law: So it is really good question. think... So this was Kent Beck said this was... I'm going to name drop again. So I was judging an O'Reilly Cata yesterday, Architectural Cata with Kent Beck. And Kent Beck's big thing was... And this is a Toyota thing, right? This is to bring it to SRE. This is explicitly in the Toyota way. Like, AI and robots in the Toyota case, like as partners of human beings, right? They're helping you out and doing stuff. I think... if you know how to do prompting and all this kind of stuff. Having a second brain or a second way of thinking about things or something which is very good at going out and collecting lots of information, something which is very good about like pointing out flaws in your thinking, there's a whole, Sven might remember this, buried somewhere deep in the book is like blind spots, like we talked about the unknown unknowns and all this kind of stuff right? You could use something like that to like AIs and LLMs that could If you've got your ADR history and your code base and all of the other stuff written down, could bring all of this stuff together, First with RAG and things which are well established these days, although well established a few years old. But it could be incredibly powerful. I still think one thing would be terrifying, I would definitely always want to a human being who's accountable and responsible for the decision. It shouldn't be an AI deciding to do it. And I would still want to explicitly prompt it and feed it to make it do things because I should... My big thing is you shouldn't outsource your thinking to an AI because the thinking is how you understand it and it's not just like... It's how you understand what you have as a result of the thinking, but as a thinking aid and a thinking tool, just like a robot in a Toyota production system, I think is incredibly powerful. And I think that would be super interesting to see all of this kind of stuff, right?
Heinrich Hartmann: Yeah. Yeah. Yeah. I want to just point out that this is the fuel angle. This is like, okay, you have all this knowledge base and then kind of fueling an AI that can help you as an advisor or a code reviewer in your engineering. I think that's really powerful also, like consistent with things I've seen elsewhere.
Andrew Harmel-Law: Yep. And also things like other stuff like I spend another chapter talking about like how to make decisions smaller because one of the curses I read the Principles of Product Development Flow by Don Reinertsen and his big thing was like small, small, small, small, like single piece flow, right? But single piece flow in a non-deterministic system, right? Which is far, you know, not like Toyota. You want Toyota production to be deterministic. We're not building recipes, right? We're building individual things every single time.
Sven Johann: Thank
Andrew Harmel-Law: small pieces like helping you like split decisions down doing things like what's the time bound on this decision etc etc kind of stuff like Alex mentioned it could be a really good coach for all of the kind of ways of thinking about this kind of stuff be really interesting to see it would remind me because i talk about this stuff but i forget about after
Sven Johann: mean, exploring options, for example, is for me one of my major, let's say, the killer use case for AI when I write ADRs. That's,
Andrew Harmel-Law: Yep, 100%.
Alexander Heusingfeld: It actually helps me to reflect on all the note pieces that I take in my note taking system. So every week I do a review session and that helps a lot to like distill what were the thoughts that I want to revisit. How do they connect to previous thoughts from months ago? That helps a lot. Like a saddle custom.
Andrew Harmel-Law: And nice.
Alexander Heusingfeld: thingy. I have to say I'm still like my thoughts are still wrapped around what you said when we were talking about the advice forums, Andrew. It basically struck me when you said, well, don't call it architecture review. And the reason why this struck me is I'm doing tech reviews with
Andrew Harmel-Law: Yep. No, definitely.
Andrew Harmel-Law: Hmm
Sven Johann: Thank
Alexander Heusingfeld: with our teams because I want to learn what they learned. I want to look at what they looked at, what they thought about the decisions that they made. And more often than not, I get this feeling that they think they need to justify in front of me. And like I was just thinking, maybe it's because I called the meeting a tech review and not
Andrew Harmel-Law: Mm-hmm.
Andrew Harmel-Law: Yep. Yep. Yep.
Alexander Heusingfeld: a tech, I don't know, presentation or something. So could it really be in the wording?
Andrew Harmel-Law: think could be like, yeah, the names of stuff is super powerful, really interesting. And that's why, like I said, I spent the first bit is like, this is not a review board. No one is here. You're not asking for permission. It's very interesting. Because when people are safer, again, there's a whole chapter on this, very DEI, it's like it's all before Trump, right? But it's like software is built by groups of people and software is built best by groups of people who have different perspectives, different skills, different, you you want, there's a...
Alexander Heusingfeld: No, it's a-
Andrew Harmel-Law: There's a junior developer in one of the examples in my book called Rita. Rita is the name of my editor, who is not a software professional, but Rita kept asking really good questions. I was like, everybody has a role to play in this. if you get this, I mean, like, the best thing I've learned from people, junior developers, is like, they'll ask about things which aren't clear. And you're like, if this was better, this would be clear.
Alexander Heusingfeld: Mm-hmm.
Sven Johann: Okay.
Andrew Harmel-Law: We would have properly managed the naming, would have properly managed the cognitive load, we would have properly managed the bounded context. And it took me 15, 20 years of my career to realize that me feeling stupid was not always me being stupid. Sometimes it was a signal that things were more complicated than they needed to be. But like we build a big ball of mud, right? Because we're like, some people are very good at it. I think my superpower is I am not as good as most of architects.
Alexander Heusingfeld: That's...
Alexander Heusingfeld: Mm-hmm.
Andrew Harmel-Law: So my tolerance for big balls of mud is way lower than everyone else's because I cannot cope with a big ball of mud and everyone else is like I'm pretty smart I can cope with a medium sized ball of I can't cope with
Alexander Heusingfeld: That's actually the perfect bridge to what I wanted to ask as a closing question, because I saw on the companion side for your book that you thanked Rita for saying, you wrote, I've tried to put the most relevant information in the book and other important but not essential information here. And I was like, yeah, there's always more to share. how do you actually decide what not to share because it might easily overwhelm people. So how do you select, for example, in such an advice forum, what not to share because it's important, but it's not essential. And maybe you can, you can resonate with that and give some advice, like maybe how you did it with the book.
Andrew Harmel-Law: Mm-hmm.
Andrew Harmel-Law: That's a good question.
Andrew Harmel-Law: So I'm the worst, the irony is, I'm the worst person to ask, I can't remember if I mentioned this during the podcast or before, Bruce Echol, who's read the book, said, there's lots of good bits in there and then there's a lot of other, listen, he didn't say this, but like he basically went, it's too long. And it is, I will admit, it is very long. Hopefully each chapter is kind of standalone, Sven can tell you what he thinks, but there's a lot and...
Alexander Heusingfeld: Hahaha
Alexander Heusingfeld: Mm-hmm.
Andrew Harmel-Law: The act of making things, it is a lot shorter than it was. But it could be a lot shorter. The reason I was speaking to Bruce is because I spoke to him in another podcast about the book he'd written. The air book is 80 pages long. And it's like super focused. so to take it to the advice thing, I'm probably the worst person to ask because I...
Alexander Heusingfeld: Mm-hmm.
Sven Johann: Thank
Sven Johann: Thank you.
Alexander Heusingfeld: But you had the experience with Rita. So what did she do to like...
Andrew Harmel-Law: Yeah, exactly. So what Rita does is actually... So yeah, this is good. Rita would encourage me, and Diana Montalian would encourage me to do this too, like you write... What you need to do is get comfortable with editing and losing things. And it's very hard. I realized, like a lot about my life changed when I wrote this book, because I didn't... Like, you're like, I'm smart, O'Reilly are asking me to write a book, so I must be pretty good at this stuff. I was not good at writing.
Sven Johann: I'm I'm
Alexander Heusingfeld: I second that,
Andrew Harmel-Law: It turned out. so Rita would, who had no technical background, obviously she's an editor of technical books, but she's not a software engineer or architect, but her questions were so... They went to my ego. I'm a lot less egotistical about my code, but it turns out I'm very egotistical about my writing. And each month, like we'd have a chat every month or every two weeks I think, it was hard because Rita would give me some very direct, very honest, very valid feedback. She would dissect. I think we got to chapter 14 before Rita said this is a good chapter. So there's a lot, you should have seen some of the earlier drafts. But Rita made this book, this is why Rita is very early on in the thing. It's very hard, but what I've learned is, you can see this in how I speak. I'll talk a lot and then I'll eventually come to a conclusion.
Alexander Heusingfeld:
Andrew Harmel-Law: It's good but then editing it, bringing it down, like I'll go broad and then you'll bring it down. And I think... You need to have the things, you put them out in the open and like with code, right, you'll do it then like, you know, kembeck, like red, green, refactor, what does it need to do? Make it do it, then you refactor it. I could have refactored this book a lot more, let's be honest, please buy my book. I would love your feedback. But, But, There are bits which aren't, won't be for everybody. It's like, like, I spoke to Eric Evans and he was like... Eric... I think regrets putting the second half of his book at the end and the first half at the start. I think you would put them the other way around. I disagree, I think his book is in the right order. But I guarantee half the people who read my book won't read the end. I think all of that stuff is more important. But people are like, right, Andrew's told us how to do it, now I'm gonna get going. The rest is advice. It's like, right, when you try this, like you said Sven, right, you'll try this, this won't work. You'll try this, this will fail. You'll try this. It's just things that I've learned. not telling you to do it, but I'm like, here is stuff I've learned. So I couldn't edit it out. That's my problem.
Alexander Heusingfeld: So would you have to write a second book you would start the same way? Like write it out and then condense?
Andrew Harmel-Law: What I would do is, so I've learned this from Bruce, because they talked about this. They were very, very, very, very strict about who the audience for the book was and what that book reader was trying to achieve. And they would then experiment with removing even more concepts. So they, because it was a book about code, it was about effect-oriented programming, they were like, wonder if... It took them longer to write, but they were like, I wonder if we just completely remove this concept. Does the whole book fall apart? and they would remove the concept and the book would fall apart so they put it back in but then they'd remove another concept and they're like this is a lot better and I'm in awe like I'm like wow that's it's amazing but I only I had two years to write the book and it was taking a long time so like I need to stop now
Heinrich Hartmann: Yeah, I think the dynamics of this are actually changing. think audio content is becoming a lot more relevant than a lot of the written content. And also producing stuff just by mumbling and having AI kind of have massage it is really changing the game and it comes to content creation. And yeah, I mean, the old saying of like start at the top, not at the front, right? You want to kind of put the framing in place before you kind of start piling on.
Andrew Harmel-Law: Yep. Yep.
Andrew Harmel-Law: Hmm.
Andrew Harmel-Law: Yeah. Yeah, totally. And I think, and hopefully you could read the first part of each of my chapters and then it would still make sense, because later on this is like Rita did, like you mentioned Heinrich, right? Like the pyramid. It definitely is. The mistake I kept making was I would bury, because I do too many talks and conferences, where you bury the reveal at the end and Rita was like, why do I have to get to the end of the chapter before you tell me why it's important? Tell me at the start.
Heinrich Hartmann: That is still good advice.
Sven Johann: Thank
Heinrich Hartmann: Yeah.
Heinrich Hartmann: Yeah.
Andrew Harmel-Law: Hopefully you could read the first bit of each chapter and then you're like, this is good. And then keep going and at some point you're like, I'm bored and I'll skip to the next piece. That might work.
Sven Johann: Okay.
Heinrich Hartmann: And then there is this books where it's like they have three good chapters and then it's just anecdotes about why this is again true. it's.
Andrew Harmel-Law: Yeah, yeah, yeah. So hopefully mine is like, this is the key thing, here's a bunch and it'll get less... more personal. Boof, another thing. More personal. Boof. So it's hopefully... but yeah. We'll see.
Alexander Heusingfeld: Mm-hmm.
Heinrich Hartmann: All right.
Sven Johann: Good. Thank you very much, everyone. Yeah, it was a good chat. Awesome.
Alexander Heusingfeld: Thanks a lot, Andrew.
Andrew Harmel-Law: Thank you for having me. Lovely to meet you.
Sven Johann: Again, we mentioned a lot of things, a lot of books, or not we, but mostly Andrew.
Alexander Heusingfeld: Yeah. It's gonna be long show notes.
Andrew Harmel-Law: Yep, sorry. Yeah, sorry about that. Yes! Longest show notes ever.
Sven Johann: It will be probably the longest show notes we've ever had. Yeah. Yeah, thank you very much for coming and looking forward to your new book. Also, I personally say I'm looking forward. I have the Matrix book, but I haven't read it. yeah. So.
Andrew Harmel-Law: and I figure out what it's gonna be.
Andrew Harmel-Law: The first chapter is great, I'm not sure about the rest.
Alexander Heusingfeld: Ha ha ha.
Sven Johann: Thanks everyone for listening. you made it so far, please give us feedback. Buy the book and...
Alexander Heusingfeld: Yeah. Leave some LinkedIn comments on what you want to hear next.
Sven Johann: See you next time.
Andrew Harmel-Law: Yeah. Yeah.
Heinrich Hartmann: Bye bye.
Andrew Harmel-Law: Cheers.
Alexander Heusingfeld: Thanks a lot guys. Bye.
Sven Johann: Cheers.