ShermanOnSoftware 2025 Year In Review

I set ambitious goals for 2025. 

I was doing well, then my plans changed.

Original Goals

My goal was to write a blog post every Monday and Wednesday; and then do a Podcast episode every Friday.  I gave myself a little slack, I only wanted to write 100 blog posts.

In the end, Isaac and I published 52 podcast episodes.  I also wrote 34 blog posts and 19,500 words.

This wasn’t a case of running out of steam, I was on pace until Isaac and I decided to write a book.

The Rewrite Trap

In March, Isaac and I decided to write a book about rewrites.  You can pre-order it here, I guarantee it will be the greatest book on the fallacy of software rewrites ever written.  It will also be the first book ever written on the subject.

Right now the book is about 100 pages and 18,000 words.  The first draft is complete - we have said everything we intend to say.  Now we are working to make our arguments persuasive and well written.

Total Words

I didn’t have a total word goal for 2025.  In 2024 I wrote 23,000 words and 44 posts.  In 2025 I wrote 19,500 words on the blog, and at least 9,000 more on the book, for a total of 28,500.

I wrote 23% fewer posts last year, but I also wrote 24% more words.

Conferences

I hoped to speak at 1 conference in 2025, and it didn’t happen.  I did write 6 speaking proposals; none were accepted.

Unlike blogging, podcasting, and writing, this goal isn’t entirely in my control.  Going forward I will adjust my speaking goals accordingly.

Goals For 2026

With 2025 in the rear view mirror, what do I want to achieve in 2026?

  1. Another 52 episodes of Never Rewrite.  No change in the headline.  I do want to spend more effort getting guests on to talk about their experiences.
  2. 100 blog posts on ShermanOnSoftware, or 30,000 words.  I want to publish twice a week.  I was on target until deciding to write a book.  This year I am adjusting the goal to take the book into account.  I can write a lot of pieces, or I can write a lot of words.  I am also changing how I block out time for writing, and I am hopeful that stronger systems will enable me to be more consistent.
  3. Publishing The Rewrite Trap.  The book will probably be self published, but I am also committing to submitting book proposals to the 4-5 publishers who might publish it.  If it does get picked up by a publisher, it won’t be out in 2026, and that’s ok too.
  4. Speaking at 1 conference, or submitting at least 10 conference talks.  Unlike the first 3 goals, I can’t control whether I get booked to give a talk.  I can control how many submissions I make, so I will keep at it and keep revising talk ideas based on feedback.

I am intentionally not putting down goals for things I hope will happen next year like:

  • Book sales
  • Consulting jobs
  • Teaching classes about rewrite alternatives
  • Increasing podcast listeners
  • Increasing mailing list subscribers

I hope all of those things will happen, but I am not making them a priority.

Check back next year to see how things played out!

Incremental ETL Patterns For SaaS Developers

An ETL is a programmer’s way of saying “I’m moving data from one system to the other, and I have to do a bunch of things to make the data fit in the other system.”  ETLs tend to get written by database administrators, business intelligence developers, and backoffice programmers.  Developers that work with long lived processes that take 8-12 hours, often run overnight, and usually need to be finished first thing in the morning have different habits than SaaS developers.

SaaS developers live in the web development world - massive numbers of short lived processes.  User latency is measured in milliseconds and some amount of failure is inevitable and acceptable.

Because SaaS developers are unlikely to be woken up at 3am to restart a ten hour process that needs to be finished by 9am, they aren’t aware of how incremental ETLs are structured.

  1. The code is procedural; a series of specific steps, executed in a specific order.
  2. Every step begins by loading state and ends by writing state to a file.
  3. Every step is idempotent and deterministic, state is immutable.
  4. Every step can be run manually so long as the previous step has completed successfully.

The Code Is Procedural

This isn’t about the code, it is about the system running the ETL.  The code might be written in an object oriented language like python and java or a procedural language like sql.  The system running the ETL, on the other hand, is designed to execute a series of steps, with retry logic and built in alerting.

Or, it can be as simple as a shell script.  No retries, no alerting.  Just a series of commands and echo statements that you will look at when you remember, or when someone tells you things are broken.

What an ETL is not, is a highly stateful piece of software that runs a complicated process.  Procedural steps allow the process to resume close to where it failed.  ETL runtimes are measured in hours, best case starting over will only cost you a day.

Every Step Begins By Loading State From Persistence And Ends By Writing State To Persistence

Persistence is a file, database, or any other durable system that will let you load or write the data over and over again.

Each task stands alone because each task can fail.  The output from the last task is the input for the next.  When something goes wrong, you can examine the inputs and outputs.  When a simple transformation process that reads from FileA and writes to FileB stops on line 35, you can be pretty sure that there is something unexpected in line 36 of FileA.

Fix the input, fix the code, and resume.  When the input is easily accessible you can often fix it manually, get the process moving, and then come back and fix the code.  Remember, you are often doing it at 3am and racing the clock.

Every step is idempotent and deterministic, state is immutable

Every step needs to be tried and retried as many times as it takes to succeed.  This means it can’t change or overwrite the original input, and it needs to produce the same output each time.  The step ends by writing the results to persistence.  So long as the step is run in isolation, with no process manager turning the outputs into inputs, it is idempotent.

You don’t need it to be perfect to rerun the job.  You can fix one issue and run it again to see if it blows up again.  Yes, in production because it is 3am and you can’t drag the data to a staging environment.  But with immutable state, deterministic runs, and idempotentancy, even testing in production at 3am is safe.

Every Step Can Be Run Manually

The ETL is a series of steps, and those steps can be run over and over again until they are correct.  Each step runs in isolation.  Job running software is helpful, but when there is a problem, you can run it yourself.

Everything is laid out for you.  You can examine how far the process went and where it died.  There is no global state to worry about, no disabling parts of the process to get to where it failed.  

How ETL Thinking Helps SaaS Developers

Outside of the database and business intelligence teams, SaaS developers rarely build ETLs.  Which is a shame because ETL thinking makes it much easier to assemble complex workflows.  Work incrementally, work idempotently, and your work will be powerful and easy to debug.

Context Required For Static Vs System Optimizations

A static analyzer can go through code and find big-O type problems.  A developer can go through and refactor the code to make it run more efficiently.

Neither of these tasks requires much context about the larger system.

For example, a function can take a list of contacts and load each one from the database.  After loading the contact, it can then load each activity that the contact has done, one at a time.  This is a standard nested for loop with O(n^2).

With no further context, you could modify the code to load all of the contacts in a single query.  You could then load all of the activity with a second query.

From the perspective of querying the database, the function would go from O(n^2) to O(1).  The only context you would need to know is if N could be large enough to exhaust your software’s memory.  (There would still be O(n^2) work in memory to assemble the data objects, but that is negligible against the cost of DB calls)

But if you had deeper knowledge you might realize that you don’t need the data in the first place.  No database calls are not only faster, but deleting the code is a whole lot easier than refactoring.

AI and other tools make static type efficiency optimizations much easier; but they can’t ask “Should the code even be doing this?”  System context is where developers still shine.

Announcing Never Rewrite: The Low Risk Guide To Modernizing Your Legacy Software

Isaac and I are excited to announce that we are taking our podcast to a book!

Never Rewrite: The Low Risk Guide To Modernizing Your Legacy Software is for everyone who maintains, or relies on, systems that you hate.

Systems that are poorly designed, unreliable, untestable, won’t scale, or make you tear your hair out.  Where every change creates 2 new bugs.  Systems that are so bad that they can’t be fixed.  

Where it seems like the only option is to start over from scratch.

When it seems that you need a rewrite, you need Never Rewrite: The Low Risk Guide To Modernizing Your Legacy Software.

Because no one actually wants a rewrite -

Developers don’t want a rewrite, they want to work with well designed and tested code that doesn’t fight them every step of the way.  They want to take pride and get joy from their work.

Managers don’t want a rewrite, they want the people they manage to be happy, for bug reports to be few, and for work to be delivered at a consistent pace.

Leadership doesn’t want a rewrite, they want to empower their people, have reliable systems, and consistent delivery.

If no one wants a rewrite, why does it seem like a reasonable solution?  Because it seems like there are no other options.

We’re here to tell you that there is another way -

Never Rewrite will teach you how to modernize legacy software without a rewrite.  The book will show you how rewrites destroy teams, increase turnover, and prevent growth.  We will show you how to escape a rewrite in progress.

Along the way we share real stories of rewrite; from Sonos’ recent $500 million debacle, companies that paused new features for years, and projects that cost millions and never made it into production.  Our case studies are full of lost opportunity, wasted money, and derailed careers. 

If you’d like to learn more, please sign up to our mailing list and be the first to hear about our progress!

ShermanOnSoftware 2024 Year In Review

By The Numbers

In 2023 I set a personal goal of publishing 50 podcast episodes, 100 articles on this site, 6 articles on other sites, and 1 conference talk.

That turned out to be overly ambitious.  In 2024 I created:

  • 50 podcast episodes (100%)
  • 44 articles on this site (44%)
  • 0 articles on other sites (0%)
  • 1 conference talk (100%)

I didn’t have a word count goal but I wrote over 23,000 words.  More than double last years’.

For 2025 I’m going to adjust my ambitions a little bit:

  • 1 Conference Talk
  • 52 PodCast Episodes
  • 100 Articles on this site

Last year I blindly increased my goal and hoped to hit 2/3 of it.  This year I intend to work on my systems to achieve my goals.

Top 3 Most Popular Articles in 2023

  1. Fixing All The Bugs Won’t Solve All The Problems – Deming’s Path Of Frustration
  2. Patterns of Data Loading – Topics and Broadcast
  3. ASCII Boobs in Your Codebase Won’t Fix Themselves

In A Remote First World, Being Local Is The Only Leverage A Developer Has

I was speaking with a VP of Software Development last week and he gave me a brain bursting bon mot: In a remote first world, being local is the only leverage a developer has.  We were discussing the situation at a true remote-first SaaS; they have developers in the USA, Ireland, Poland, Costa Rica, Brazil, and Australia.  Becoming globally remote first is not cheap or easy, but once you’ve done it, the marginal cost of a developer in Poland is 60% of one in the USA.

He continued, “from a cost perspective, you can either hire more developers, or more senior (expensive) developers.  It’s no longer about where anyone inside the company is, or how hard it is for them to work together.  It’s about how well they understand the customer’s culture.”

Thirty years ago outsourcing turned out to be a mirage because the difficulty of working together outweighed the cost savings.  Today, being able to assemble people from everywhere is part of being a global SaaS.

Which begs the question, how well do you understand your customers?

SaaS Project Deadlines Are Artificial

In SaaS development, almost all delivery deadlines are artificial.  The code won’t be burned onto a chip, CDs don’t go gold, and no physical media will be created or sent anywhere.  Outside of regulatory changes, SaaS companies simply don’t have external deadlines.

And yet, almost every project I’ve ever worked on had a deadline.  Usually an aggressive deadline.

Artificial deadlines are social contracts; you have to commit to needing the work and you have to commit enough people to get the work done on time.

In the SaaS world, missed deadlines are management failures.  The work didn’t need to be done, as evidenced by the fact that it wasn’t done and the sun still shines.  Management didn’t allocate enough people or time, otherwise the work would have been completed.  Every missed deadline breaks trust and erodes your teams.

The alternative is simple, don’t set artificial deadlines.

Talk From StaffPlus NYC 2024

I spoke about business strategy at this year's StaffPlus NYC.

If you've ever been paralyzed trying to decide which projects fit with your company's strategy, or how to even identify your company's strategy, this talk is for you!

I cover the three main types of business strategy, how to learn which strategy your company is pursuing, and how to discuss projects through the lens of strategy. Learn how to identify projects with clear value that do not align with strategy; and how to discuss the problem with stakeholders.

The talk is now online, free registration required:

Focus on project value using businesses strategy

And yes, the videographer goofed the spelling of my name. :)

5 Tactics To Start Scaling Software

I can’t blindly tell you why your software is slow and won’t scale.  Everyone’s software and situation are different.  If you’re stuck with no idea where to begin, these are my top 5 tactics for getting started when you don’t know what to do.

These tactics will improve the performance and stability of your system.  They will help you understand the system’s constraints and bottlenecks.  They probably won’t be the optimal solution to your problem, but they are a great place to start

  1. Fix the errors in the logs and stop logging things that don’t matter.  Not all errors are bugs, and not all bugs produce errors.  Noisy logs that are full of meaningless errors make it hard to see “real” problems.  Fixing or suppressing the noise will help you, learn the code, and fix a bunch of small problems.  Exceptions are also slow, so this will make the system faster and more stable
  2. Reduce the amount of db usage.  Add caching for data that gets requested and doesn’t change.  At scale even trivial queries add up. Ex: loading countries or states from a db. 
  3. Make work async - move it to queues.  This allows you to keep api/ui moving and sets you up for eventual parallelism. 
  4. Make work restartable/idempotent/recoverable.  Things break, make it easy to just try again 
  5. Shadow users.  Learn what they think is important and broken. It probably isn’t what you thought. 

ShermanOnSoftware 2023 Year In Review

It’s the time of year to look back, reflect, and highlight some interesting pieces you might have missed.

Also known as the self indulgent year in review post!

By The Numbers

In 2022 I set a personal goal of publishing twice a week, or at least 100 articles in 2023.  I published 65 different pieces, almost ⅔ of my goal!

I published 34 Podcasts, 31 Articles, and wrote over 11,000 words.

For 2024 I’m going to stretch further and shoot for 50 podcasts and 100 articles; for a combined schedule of 3 items/week.

I’m also aiming for 6 published articles on other sites and 1 conference talk.

Top 3 Articles Written in 2023

Top 3 Most Popular Articles in 2023

Thanks for reading!

Site Footer