There’s no upper limit on the number of Contacts that customers will want to add to your CRM. Until you can support billions of contacts, there’s always an argument that you should support more.
On the other hand, Relationship and Management options are constrained by the choices you present to your users. You can’t support everything and need to consider which options make sense for your CRM. Every option adds complexity and cost to you; and cognitive load on your users. Adding features often decreases value.
When it comes to contacts, think about what scales. For everything else, remember that people have limits.