Software Circus: Thinking Fast and Slow with Software Development

We’re only human (and so is our decision making)…

My talk is based on the superb book of the same title, “Thinking Fast and Slow“, by Daniel Kahneman. The central premise of the book is that human decision making is prone to unconscious bias, and that we use sometimes flawed heuristics that may keep us safe when we encounter a tiger (or road rage), but don’t work very well in an office environment. The book is well-worth reading, and it taught me to fundamentally question some of the approaches to thinking I use throughout my daily life. However, I have arguably gained the most from this book when it comes to my professional decision making, and the decision making of those around me when developing software.

As a consultant at OpenCredo, I get to work with lots of fantastic organisations and people (many more than a typical developer or architect would) and as we often get brought in to help rectify troubled or failing projects, I have discovered a series of patterns of why things typically go wrong in projects, and have developed techniques to allow my team and I to add the most value when helping to put these things right. A large part of these patterns revolve around thinking, planning and decision making, and I’ve tried to codify some of my experience into this presentation.

Biases and heuristics affecting software development

I covered five primary biases and heuristics within the Software Circus talk, and related them to everyday challenges with software development:

• “Availability heuristic”: This heuristic states that if something can be recalled, it must be important. We see this all the time within software development, with viscous (or virtuous) cycles of popular technology and methodology. We are often a fad-driven industry, and can be prone to ‘cargo culting‘ e.g. with microservices. In this part of the talk I try to get people to look outside of their typical choices, undertake experiments, and strive to constantly learn and grow as an individual.
• “Optimistic bias”: This bias relates to the tendency of people to be overconfident, and here I suggest that goals should be defined, shared and evaluated constantly. I also suggest that it is often beneficial to attempt to remove complexity and reduce risk early within the project lifecycle (when we know the least), and I also strongly recommend increasing collaboration, as software development is inherently a team sport. This goes for interaction with the customer/user (three amigos, impact mapping, lean enterprise), and also the entire technical team dev -> QA -> ops.
• “Planning fallacy”: I believe that anyone who has worked within the software development industry for more than a few months has witnessed our tendency to underestimate software projects, both in terms of complexity and cost. In this part of the presentation I talk about ‘feedback and federation’, and the need to divide requirements/problems and teams to provide the best chance of delivering value.
• “Sunk cost fallacy’: This fallacy states that we often factor costs that have already been paid and cannot be recovered into decisions where we should not. We typically do this because we don’t like being wrong, and we hate to ‘waste’ all of the effort we have put into a project, even if a large portion of this adds no value (or worse, this is causing problems now and in the future). Here I talk a little about the use of techniques such as the OODA loop to try and avoid sinking costs in the first place, and I also discuss the need for regular honest and open retrospectives to learn whether we did make the right choices, and how we can improve in the future.
• “Anchoring bias”: This bias is our tendency to rely too heavily on the first piece of information offered when making decisions. For example, does your manager ‘anchor’ you when asking for estimates (“that story will take two weeks, right?”)? Here we discuss techniques for ‘upwardly managing“, learning to collaborate better on story definition, and also techniques for better estimating (or using #noestimates!)

“The people you meet and the books you read”

One of the core themes in many of my talks is the need to collaborate better and ensure that we have strong foundations of knowledge, hence my love of a quote by Charlie ‘Tremendous’ Jones:

“You will be the same person in five years as you are today, except for the people you meet and the books you read”

I believe people who come to conferences like the Software Circus have already committed to meeting interesting fellow engineers and practitioners, but I would like to see more people reading the IT classics and also books about decision making, psychology and the other ‘soft skills’ that are often under-valued within our industry.

In summary, the key takeaways of this specific talk are to apply lightweight process and models where appropriate, use hypothesis-driven development and the scientific method more (and throughout the software development process), and collaborate more and better, both with external stakeholders and within your team. The final piece of advice is to ‘think more deliberately’…

“Think more deliberately”

As I chatted to several of my fellow attendees at the Software Circus conference, many people questioned whether the primary message of “think more deliberately” was too vague or generic, and whether it was akin to telling children to ‘be careful’ (you know they won’t!). My reply was that hopefully within the context of my entire presentation, the advice is cogent, but even in general case I meant it as a call to action to recognise when you are working on something that could benefit from a more process-driven approach, or when we should be aware that our decision-making may be subject to bias.

Often when I start acting clumsy (typically when juggling knives in the kitchen) I still hear my parent’s voices “be careful with knives”, and this does cause me to at least question what I’m doing (am I trying to do too much, did I plan appropriate for the recipe, do I need to clean my work surface etc?). The message of being more deliberate is the primary goal of the talk, and hopefully you’ll think back to this presentation when you’re next about to make a big decision when developing software… 🙂

A video from an earlier version of this talk (with slightly different slides) can be found on the Parleys jfokus conference video channel.

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.