Do You Punish Customers For Loyalty?

Does your Customer’s experience with your service get better over time?

Does it get worse?

SaaS software often punishes long term clients in subtle and frustrating ways.

Do your CRM customer screens show a decade of buying history?

How many emails can a contact open before you can’t open the contact?

Do marketing campaigns, contact lists, and tags accumulate over the years?

Do database inserts slow down as you write the 10 millionth row into a log table?

There are countless ways to punish customers for staying with you for years.  It’s not a startup problem, it sneaks in as you become a scaleup.  The flood of new customers blinds you to the slow leak as your most loyal customers churn.

When your longest customers complain about performance more than your largest, chances are your software is punishing them for being loyal.

The Implications Of Your Characteristics

Part Three of my series on iterative delivery 

You have a goal, you have characteristics, now it’s time to ponder the implications.

The implications are things that would have to be true in order for your characteristic to be achieved.  They are levers you can pull in order to make progress.

Let’s work through an example

In part 2 I suggested that a Characteristic of a system that can support clients of any size is that API endpoints respond to requests the same amount of time for clients of any size.

What would need to happen to an existing SaaS in order to make that true?

  • The API couldn’t do direct processing in a synchronous manner.  It would have to queue the client request for async processing.  Adding a message to a queue should be consistent regardless of how large the client, or the queue, becomes.
  • For data requests endpoints would need to be well indexed and have constrained results.
  • Offer async requests for large, open ended, data.  An async request works by quickly returning a token, which can then be used to poll for the results.

Implications are measurable

How much processing can be done asynchronously today?  How much could be done last month?

How many data endpoints allow requests that will perform slowly?  Is that number going up or down?

How robust is the async data request system?  Does it exist at all?

Implications are Levers

Progress on implications pushes your service towards its goal.  Sometimes a little progress will result in lots of movement, sometimes a lot of progress will barely be noticeable.

Speaking to Implications

It is important that you can speak to how the implications drive progress towards your goal.

Asynchronous processing lets your service remain responsive.  It doesn’t mean you can process the data in a timely manner yet.  It sets the stage for parallel processing and other methods of handling large loads.

Next Steps

Before continuing on, try to come up with 3 implications for your most important characteristics.

You’ll want a good selection of implications for the next part – blockers.

We will explore what’s preventing you from moving your system in the direction it needs to go.

Picking an Iterative Goal at a Scaleup

Note: This is part of my series on Iterative Delivery

When you are in Scaleup mode, picking a goal to iterate on should be straightforward.

In Scaleup mode, picking an iterative goal should be straightforward.

What can’t you deliver?

Are you attracting larger clients and discovering your software can’t handle their size?

Do you have a swarm of small clients overwhelming the backend?

Does throwing money at your problems keep the software running smoothly, but unprofitably?

Your goal should be a single, short, aspirational sentence.  

If you get stuck, try the “We should be able to ___”  template:

We should be able to support clients of any size!

We should be able to support any number of clients!

We should be able to support clients profitably!

You don’t need to have any idea how to achieve your goal, your goal might not even be achievable.

The important thing is that you can clearly state your goal and explain it to others.

The Only Client Experience

The only client experience you can offer clients is the one you have today.  Yesterday’s experience is a memory, and tomorrow is a promise.

Delivering iteratively, baby step improvements to your client’s experience, makes the experience better today, tomorrow’s promise more believable, and memories of things improving.

Promising a moonshot keeps the only experience your client’s have exactly the same; and if it was a good one, you wouldn’t need a moonshot.

When Your Customers Struggle

When your customers struggle with a non-intuitive interface, would they rather:

Wait 6 months while you work on an amazing redesign and brand refresh, or get weekly tweaks that make using your product easier and more intuitive?

Branding and color schemes are about you, improving the UI is about your customers.

Don’t put off customer steps while working on a moonshot like a branding refresh.

Moonshots vs Baby Steps at A Scaleup

You’re A Scaleup!  What’s a Scaleup?

You become a Scaleup when your SaaS’s service offering becomes compelling and you start attracting exponentially more clients.

All at once you have a lot more clients, clients with a lot more data.

Solutions that support 1,000 clients buckle as you pass 5,000.  Suddenly, 25,000 clients is only months away.

Services that support hundreds of thousands of transactions a day fall hopelessly behind as you onboard clients with millions of transactions.

You finally know what customers want.  You quickly find the edges of your system.  Money is rolling in from customers and VCs.  You can throw money at the problems to literally buy time to find a solution.

But you’re faced with a looming question – moonshots or baby steps.

Moonshots Are About You, Baby Steps Are About Your Clients

It’s not about you or your SaaS, it’s about your client’s outcomes.

Moonshots are appealing because they take you directly to where you need to be.  Your system needs to scale 10x today and 100x next year; why not go straight for 100x?

Baby steps feel like aiming low because the impact on you is small.  But it’s not about you!  Think about the impact on your clients.

From a technology perspective, sending emails 1% faster is ::yawn::

But for your clients, faster emails means more engagement, which means more sales.

Would your clients rather have more sales this week, compounding every week for the next year, or flat sales for a year while you build a moonshot?

Clients who churn, or go out of business, won’t get value from the moonshot.  Even if you deliver greater value eventually, your clients are better off getting some value now.

Are you delivering value to your SaaS or your clients?

The Opposite of Iterative Delivery

Iterative Delivery is a uniquely powerful method for adding value to a SaaS.  Other than iterative, there are no respectably named ways to deliver features, reskins, updates and bug fixes.  Big bangs, waterfalls, and Quarterly releases fill your customer’s hearts with dread.

Look at 5 antonyms for Iterative Delivery:

  • Erratic Delivery
  • Infrequent Delivery
  • Irregular Delivery
  • Overwhelming Delivery
  • Sporadic Delivery

If your customers used these terms to describe updates to your SaaS, would they be wrong?

Iterative Delivery is about delivering small pieces of value to your customers so often that they know you’re improving the Service, but so small that they barely notice the changes.

Don’t be overwhelming, erratic or infrequent – be iterative and delight your customers.

3 Signs Your Resource Allocation Model Is Working Against You

3 Signs Your Resource Allocation Model Is Working Against You

After 6 posts on SaaS Tenancy Models, I want to bring it back to some concrete examples.  When your SaaS has a Single Tenant model, clients expect to allocate all the resources they need, whenever they want.  When every client is entitled to the entire resource pool, no client gets a great customer experience.

Here are 3 signs your Resource Allocation Model is working against you:

  1. Large clients cause small client’s work to stall
  2. You have to rebalance the mix of clients in a cell for stability
  3. Run your job at night for best performance

Large clients cause small client’s work to stall

This is a classic “noisy neighbor” problem.  Each client tries to claim all the shared resources needed to do their work.  This isn’t much of a problem when none of the clients need a significant percentage of the pool.  When a large client comes along, it drains the pool, and leaves your small clients flopping like fish out of water.

You have to rebalance the mix of clients in a cell for stability

When having multiple large clients in a cell affects stability, the short term solution is to migrate some clients to another cell.  Large clients can impact performance, but they should not be able to impact stability.  Moving clients around buys you time, but it also forces you to focus on smaller, less profitable clients.

Run your job at night for best performance

This is advice that often pops up on SaaS message boards.  Don’t try to run your job during the day, schedule it to run in the evening so it is ready for the morning.  When clients start posting workarounds to your problems, it’s a clear sign of frustration.  Your clients are noticing that performance varies by the time of day.  They are building mental models of your platform and deciding you have load and scale issues.  By being helpful to each other, your clients are advertising your problems.

Conclusion

These 3 issues have the same root cause; your SaaS’s operational data is mixed in with client data.  If you have any of these three problems, the time has come to separate your data from the clients’.

Fixing these problems won’t be easy or quick!  

The good news is that you can separate the data and change your resource allocation model in an iterative fashion.  Start by pushing your job service across the tenancy line.

Get value and regain control one incremental step at a time, and never do a rewrite!

Cell Based Single Tenancy

This is part 4 in a series on SaaS Tenancy Models.  Parts 1 , 2 , and 3.

SaaS companies are often approached by potential clients who want their instance to be completely separate from any other client.  Sometimes the request is driven by legal requirements (primarily healthcare and defense), sometimes it is a desire for enhanced security.

Often, running a Multi-Tenant service with a single client will satisfy the client’s needs.  Clients are often willing to pay for the privilege of their account run Single Tenant, making it a potentially lucrative option for a SaaS.

What is a Cell?

A Cell is an independent instance of a SaaS’ software setup.  This is different from having software running in multiple datacenters or even multiple continents.  If the services talk to each other, they are in the same cell regardless of physical location.

Cells can differ with the number and power of servers and databases.  Cells can even have entirely different caching options depending on need.

The 3 most common Cell setups are Production, Staging (or Test), and Local.

Cell Properties

Cell architecture comes with a few distinct properties:

  • Cell structures allow SaaS to grow internationally and offer clients low latency and localized data policies (think GDPR).  Latency from the US to Europe, Asia and South America is noticeable and degrades the client experience.
  • Clients exist in 1 cell at a time.  They can migrate, but they can’t exist in multiple cells.
  • Generally speaking, Cells can not be part of a disaster recovery plan.  Switching clients between Cells usually involves copying the database, and can’t be done if the client’s original Cell is down.

Cell Isolation as a Single Tenant Option

In part 3 I covered the difficulties in operating in a true Single Tenant model at scale.  A Cell with a single client effectively recreates the Single Tenancy experience.

Few clients want this level of isolation, but those that need it are prepared to pay for the extra infrastructure costs of an additional Cell.

Conclusion

For SaaS without global services, a Cell model enables a mix of clients on logically separated Multi-Tenant infrastructure and clients with effectively Single Tenant infrastructure.  This allows the company to pursue clients with Single Tenant needs, and the higher price point they offer.

The catch is that Single Tenant Cells can’t exist in an architecture with global services.  If there is a single service that must have access to all client data, Single Tenant Cells are out.


If you are enjoying this series, consider subscribing to my mailing list (https://shermanonsoftware.com/subscribe/) so that you don’t miss an installment!

Infrastructure Consolidation Drives Early Tenancy Migrations

For SaaS with a pure Single Tenant model, infrastructure consolidation usually drives the first two, nearly simultaneous, steps towards a Multi-Tenant model.  The two steps convert the front end servers to be Multi-Tenant and switch the client databases from physical to logical isolation.  These two steps are usually done nearly simultaneously as a SaaS grows beyond a handful of clients, infrastructure costs skyrocket and things become unmanageable.

Diagram of a single tenant architecture become multi-tenant

Considering the 5 factors laid out in the introduction and addendumcomplexity, security, scalability, consistent performance, and synergy this move greatly increases scalability, at the cost of increased complexity, decreased security, and opening the door to consistent performance problems.  Synergy is not immediately impacted, but these changes make adding Synergy at a later date much easier.

Why is this such an early move when it has 3 negative factors and only 1 positive?  Because pure Single Tenant designs have nearly insurmountable scalability problems, and these two changes are the fastest, most obvious and most cost effective solution.

Complexity 

Shifting from Single Tenant servers and databases to Multi-Tenant slightly increases software complexity in exchange for massively decreasing platform complexity.

The web servers need to be able to understand which client a request is for, usually through sub domains like client.mySaaS.com, and use that knowledge to validate the user and connect to the correct database to retrieve data.

Increased complexity from consolidation

The difficult and risky part here is making sure that valid sessions stay associated with the correct account.  

Database server consolidation tends to be less tricky.  Most database servers support multiple schemas with their own credentials and logical isolation.  Logical separation provides unique connection settings for the web servers.  Individual client logins are restricted to the client’s schema and the SaaS developers do not need to treat logical and physical separation any differently.

Migrations and Versioning Become Expensive

The biggest database problems with a many-to-many design crop up during migrations.  Inevitably, web and database changes will be incomparable between versions.  Some SaaS models require all clients on the same version, which limits comparability issues to the release window (which itself can take days), while other models allow clients to be on different versions for years.

Versioning and Migration Diagram

The general solution to the problem of long lived versions is to stand up a pool of web and database servers on the new version, migrate clients to the new pool, and update request routing.

Security

The biggest risk around these changes is database secret handling; every server can now connect to every database.  Compromising a single server becomes a vector for exposing data from multiple clients.  This risk can be limited by proxy layers that keep database connections away from public facing web servers.  Still a compromised server is now a risk to multiple clients.

Changing from physical to logical database separation is less risky.  Each client will still be logically separated with their own schema, and permissioning should make it impossible to do queries across multiple clients.

Scalability

Scalability is the goal of Multi-Tenant Infrastructure Consolidation.

In addition to helping the SaaS, the consolidation will also help clients.  Shared server pools will increase stability and uptime by providing access to a much larger group of active servers.  The client also benefits from having more servers and more slack, making it much easier for the SaaS to absorb bursts in client activity.

Likewise, running multiple clients on larger database clusters generally increases uptime and provides slack for bursts and spikes.

These changes only impact response times when the single tenant setup would have been overwhelmed.  The minimum response times don’t change, but the maximum response times get lower and occur less frequently.

Consistent Performance

The flip side to the tenancy change is the introduction of the Noisy Neighbor problem.  This mostly impacts the database layer and occurs when large clients overwhelm the database servers and drown out resources for smaller clients.

This can be especially frustrating to clients because it can happen at any time, last for an unknown period, and there’s no warning or notification.  Things “get slow” and there are no guarantees about how often clients are impacted, notice, or complain.

Synergy

There is no direct Synergy impact from changing the web and database servers.

A SaaS starting from a pure Single Tenant model is not pursuing Synergy, otherwise the initial model would have been Multi-Tenant.

Placing distinct client schemas onto a single server does open the door to future Synergy work.  Working with data in SQL across different schemas on the same server is much easier than working across physical servers.  The work would still require changing the security model and writing quite a bit of code.  There is now a doorway if the SaaS has a reason to walk through.

Conclusion

As discussed in the introduction, a SaaS may begin with a purely Single Tenant model for several reasons.  High infrastructure bills and poor resource utilization will quickly drive an Infrastructure Consolidation to Multi-Tenant servers and logically separated databases.

The exceptions to this rule are SaaS that have few very large clients or clients with high security requirements.  These SaaS will have to price and market themselves accordingly.

Infrastructure Consolidation is an early driver away from a pure Single Tenant model to Multi-Tenancy. The change is mostly positive for clients, but does add additional security and client satisfaction risks.

If you are enjoying this series, please subscribe to my mailing list so that you don’t miss an installment!