Scaling software has tension between the needs of today and tomorrow. How do you resolve the tension? Where does the work of today end? What makes the next step part of the work of tomorrow?
Consider this simple rule:
For any piece of software in your system, you should scale it when it is the primary constraint, and stop when a different part becomes the new primary constraint.
Easy to say, and easy to do…if you can measure the performance of your system in part and as a whole.
If you don’t know which piece of your software is failing to scale, if you’re guessing about the work of today, don’t be surprised when your scaling efforts don’t impact the system as a whole. Sometimes the scaling work of today isn’t scaling, it’s observability.