Sherman On Software

Sherman On Software Latest

  • The Anna Karenina Principle of Scaling Software Systems

    The Anna Karenina Principle of Scaling Software Systems

    The Anna Karenina Principle says “All happy families are alike; each unhappy family is unhappy in its own way.”  The same is true for software scalability.  All scalable systems are alike; each unscalable system is unscalable in its own way.

    Scalable Systems Scale Linearly

    Scaling software systems means doing more work with more resources.  As software scales various state management issues will require ever more resources.  In the beginning processing 2x more requests will require less than 2x more resources.  Over time, the ratio will become 1-to-1 and continue to slide.  2x more requests will require 4x more resources.

    So long as the ratio is linear, the system is scalable.  When the ratio becomes exponential, for example 2x more requests require x^2 more resources, the system is no longer scalable.

    Scalable Systems Are Fault Tolerant

    A system with 5 nines will process 99.999% of all events correctly.  Failures are 1 in a million.  A system doing 1 million events per second will have a failure every second.

    Scalable systems have defined ways of handling failure – they will return an error and make the caller handle it, they will do retries, they may even fail silently.  What they won’t do is crash or error out.

    Scalable Systems Are Idempotent

    Exactly once delivery is nearly impossible.  As a system scales, it becomes inevitable that some events will be processed multiple times.

    Scalable systems are idempotent and indifferent to how many times an event is processed.  It doesn’t matter how many times an event enters the system, once, twice, 100 times, it is all the same.

    Non-idempotent systems are much easier to build, but they will fail in all kinds of different ways as customers and networks send in duplicate events.

    Scalable Systems Have Minimal Human Interaction

    Humans are expensive and do not scale.  The more human intervention a system requires, the less scalable it is.

    Scalable systems do not require humans to deploy code, scale up or down, or watch the logs.

    Conclusion

    Remember, all scalable systems are alike:

    1. Linear Scaling
    2. Fault Tolerant
    3. Idempotent
    4. Require Minimal Human Interaction

    Systems that are not scalable are not scalable in their own unique way.

    If you don’t know why your system isn’t scalable, these 4 points are a great place to start looking.

  • The Never Rewrite Podcast, Episode Eighty-Six, Annual Reminder – Prune Your Meetings

    The Never Rewrite Podcast, Episode Eighty-Six, Annual Reminder – Prune Your Meetings

    This week, Isaac and I remind you to Prune Your Meetings! It is important to reassess whether meetings are necessary on a regular basis; but if never else, the beginning of the year. We emphasize why you should cancel unnecessary meetings and optimize the structure of meetings you keep. As a leader, it is also important that you empower team members to decline meetings and regain control of their calendars as well!

    Watch on YouTube or listen to it at Spotify, Apple Podcasts, or Amazon Music, and let us know if you have ever been involved in a rewrite. We would love to have you on the show to discuss your experience!

  • Confusing The Feature With The Value

    Confusing The Feature With The Value

    I was called in to consult on a project recently where the developers, and the product managers, had confused the features they were building with the value they were delivering.

    The developers had written an initial integration to a new marketing channel.  The goal was to deliver something minimal, but high quality, that they could get in front of customers in under 6 weeks.  The project hadn’t specified any UI for the customers, and the initial version offered users no feedback.  Customers could set up a campaign, but they couldn’t find out if it had completed, or how many people had clicked on a link.

    With two weeks left of the original six week project, product managers stepped in with a 12 week project to add reporting.  The reporting may have been beautiful, intuitive, and powerful, but none of that mattered.  The project’s value was in the new marketing channel, not the report.  Having a polished report that would make the value easy to understand was a feature.

    The project owners decided not to go with my advice and release the feature with a minimal data table report.  Instead a 6 week project became an 18 week project, and delivered no customer value for an additional 3 months.  When customers finally got their hands on the polished report, they didn’t like it and needed revisions.

    Months were lost, and customers ultimately got less value, because the team had confused the features they were building with the value they were delivering.  An MVP today is worth infinitely more than a highly polished guess in 3 months.

  • ShermanOnSoftware 2024 Year In Review

    ShermanOnSoftware 2024 Year In Review

    By The Numbers

    In 2023 I set a personal goal of publishing 50 podcast episodes, 100 articles on this site, 6 articles on other sites, and 1 conference talk.

    That turned out to be overly ambitious.  In 2024 I created:

    • 50 podcast episodes (100%)
    • 44 articles on this site (44%)
    • 0 articles on other sites (0%)
    • 1 conference talk (100%)

    I didn’t have a word count goal but I wrote over 23,000 words.  More than double last years’.

    For 2025 I’m going to adjust my ambitions a little bit:

    • 1 Conference Talk
    • 52 PodCast Episodes
    • 100 Articles on this site

    Last year I blindly increased my goal and hoped to hit 2/3 of it.  This year I intend to work on my systems to achieve my goals.

    Top 3 Most Popular Articles in 2023

    1. Fixing All The Bugs Won’t Solve All The Problems – Deming’s Path Of Frustration
    2. Patterns of Data Loading – Topics and Broadcast
    3. ASCII Boobs in Your Codebase Won’t Fix Themselves

  • The Never Rewrite Podcast, Episode Eighty-Five, 2024 Year In Review

    The Never Rewrite Podcast, Episode Eighty-Five, 2024 Year In Review

    Isaac and I reflect back on NeverRewrite in 2024. We look back through our episodes, thank our listeners, and outline our plans for 2025.

    A true thank you to everyone who listens, we are honored by your gift of time.

    Watch on YouTube or listen to it at Spotify, Apple Podcasts, or Amazon Music, and let us know if you have ever been involved in a rewrite. We would love to have you on the show to discuss your experience!

  • The Never Rewrite Podcast, Episode Eighty-Four, The Impact of AI on Recruiting

    The Never Rewrite Podcast, Episode Eighty-Four, The Impact of AI on Recruiting

    Special guest Paul Giron returns to discuss how AI is transforming the interview experience. Learn how AI is changing how candidates are assessed, the shift towards remote interviews, and the evolving expectations of candidates. The discussion delves into the implications of AI in recruitment, including potential biases and the need for organizations to adapt their interview processes to attract top talent.

    Paul’s contact information: https://www.linkedin.com/in/paulpgiron/

    paul@g-techrecruitment.com

    Watch on YouTube or listen to it at NeverRewrite, Spotify, Apple Podcasts, or Amazon Music, and let us know if you have ever been involved in a rewrite. We would love to have you on the show to discuss your experience!

  • The Never Rewrite Podcast, Episode Eighty-Three, The Impact of AI on Recruiting

    The Never Rewrite Podcast, Episode Eighty-Three, The Impact of AI on Recruiting

    Special guest Paul Giron discusses the transformative impact of AI on recruiting and interviewing processes. He highlights the challenges faced by recruiters in adapting to AI tools, emphasizing that while AI can streamline resume sorting, it does not necessarily improve the quality of hires. The discussion also touches on the importance of understanding high performers within organizations and how structured interviewing can enhance recruitment outcomes. Ultimately, the conversation reflects on the need for organizations to invest in training recruiters to leverage AI effectively and improve hiring practices.

    Watch on YouTube or listen to it at NeverRewrite, Spotify, Apple Podcasts, or Amazon Music, and let us know if you have ever been involved in a rewrite. We would love to have you on the show to discuss your experience!

  • Cost To Serve For Staff+ SaaS Developers

    Cost To Serve For Staff+ SaaS Developers

    Cost To Serve is a critical metric for Staff+ SaaS Developers to understand because it shapes your understanding of the business.  Extremely oversimplified, Cost To Serve is the cost of  everything that goes into producing, running, and supporting software, divided by the number of customers.

    Cost To Serve = [The cost of everything] / [Number of Customers]

    This is extremely inaccurate, but it helps to point out two major concepts.  First, a large part of Cost To Serve is fixed.  Developer salaries, aka producing software, do not change based on the number of customers.  This leads to the second concept, SaaS pricing models work on leverage.  As the number of customers goes up, the fixed costs to serve each one goes down.

    So, the path to success is to get as many customers as possible right?  No, because not all costs are fixed, and not all customers are profitable.

    Not All Customers Are Profitable

    The first problem with the oversimplified view is that not all customers cost the SaaS the same amount of money.  This is why SaaS pricing includes feature and usage tiers.  The problem is that the features and tiers don’t exactly align with usage and cost.

    Pricing based on Contacts is standard for CRMs, charging for imports is rare.  A customer who imports 1 million contacts once has a lower cost than one who reimports a million contact list every week.

    Webhooks never make it into pricing models, but their costs add up.

    There are dozens of legitimate usage patterns that can make the difference between profitable and unprofitable customers.  You won’t find them on the pricing page, but chances are your finance department has at least a vague model.  It is expected that some customers will be more profitable than others; and that some will even be unprofitable.

    You Need A Model For Cost To Serve

    It doesn’t matter what your company’s Cost To Serve is and you don’t need to know the fixed costs.  What you need is a model to evaluate customer usage.  You need to be able to explain which usage patterns are expensive and estimate the potential costs of new features.

    If your SaaS is complex, try breaking it down like Big-O notation.  What costs scale linearly, which ones are exponential?

    Your model can improve over time, the important thing is to start today.  Cost To Serve is a critical metric for SaaS companies; as a Staff+ Developer you need to understand it, and how it differs from your company’s pricing model.

  • In A Remote First World, Being Local Is The Only Leverage A Developer Has

    In A Remote First World, Being Local Is The Only Leverage A Developer Has

    I was speaking with a VP of Software Development last week and he gave me a brain bursting bon mot: In a remote first world, being local is the only leverage a developer has.  We were discussing the situation at a true remote-first SaaS; they have developers in the USA, Ireland, Poland, Costa Rica, Brazil, and Australia.  Becoming globally remote first is not cheap or easy, but once you’ve done it, the marginal cost of a developer in Poland is 60% of one in the USA.

    He continued, “from a cost perspective, you can either hire more developers, or more senior (expensive) developers.  It’s no longer about where anyone inside the company is, or how hard it is for them to work together.  It’s about how well they understand the customer’s culture.”

    Thirty years ago outsourcing turned out to be a mirage because the difficulty of working together outweighed the cost savings.  Today, being able to assemble people from everywhere is part of being a global SaaS.

    Which begs the question, how well do you understand your customers?

  • The Never Rewrite Podcast, Episode Eighty-Two, Can You Like Tests And Also Believe You’re Faster Without Them?

    The Never Rewrite Podcast, Episode Eighty-Two, Can You Like Tests And Also Believe You’re Faster Without Them?

    Isaac and i discuss the balance between unit testing and the need for speed in software development. The discussion delves into the contexts in which testing is most valuable, the challenges of rapid development, and real-world examples of testing practices. We wrestle with the idea of adapting testing strategies based on the specific project and team dynamics.

    Watch on YouTube or listen to it at NeverRewrite, Spotify, Apple Podcasts, or Amazon Music, and let us know if you have ever been involved in a rewrite. We would love to have you on the show to discuss your experience!