Conway's law tends to make this decision for you and engineering concerns are usually secondary. Who has power in your organization? Is authority centralised or distributed? It is pretty similar to the process that results in workload getting split across cloud accounts or collocated in the same. Often not really "designed" so much as the result of path-dependent org politics.
Some common top-level patterns I've seen:
1. A politically powerful central platform team builds & operates the whole infra foundation for developers. This tends to lead to fewer cloud accounts, and large shared clusters (it can be done in a multi-account, multi-cluster kind of way though).
2. Less powerful cloud engineering team(s) that do guard-rails, interconnect, patterns etc, but leave product engineering teams to manage the actual workloads. Typically results in "cluster per team or project", ideally stamped out from some shared IaC.
3. Various forms of anarchy (I don't mean this pejoratively) that result from widely distributed authority. Product engineering teams are powerful enough that no central team can really "tell them what to do". This tends to result in multiple clusters that are built and run in different ways.
Very large organisations can have pockets of all 3 models.
From my perspective the strongest forces that "push" towards big, centralised clusters are:
1. Powerful central teams that also operate. They're OK with trading off fewer things to look after for potentially larger blast radius. Also the capacity to set up the more complex security controls required for multiple teams and projects to co-exist in the same cluster.
2. Large homogenous workloads are a natural fit for big clusters. Homogenous workloads tend to have 1 "entity" looking after them, so no matter what organisation model, you can end up with big clusters.
3. Tight coupling between workload components. If you have a lot of different things that all need to talk to each other quickly and cheaply you will want to put them close together if you can.
Conway's law tends to make this decision for you and engineering concerns are usually secondary. Who has power in your organization? Is authority centralised or distributed? It is pretty similar to the process that results in workload getting split across cloud accounts or collocated in the same. Often not really "designed" so much as the result of path-dependent org politics.
Some common top-level patterns I've seen:
1. A politically powerful central platform team builds & operates the whole infra foundation for developers. This tends to lead to fewer cloud accounts, and large shared clusters (it can be done in a multi-account, multi-cluster kind of way though).
2. Less powerful cloud engineering team(s) that do guard-rails, interconnect, patterns etc, but leave product engineering teams to manage the actual workloads. Typically results in "cluster per team or project", ideally stamped out from some shared IaC.
3. Various forms of anarchy (I don't mean this pejoratively) that result from widely distributed authority. Product engineering teams are powerful enough that no central team can really "tell them what to do". This tends to result in multiple clusters that are built and run in different ways.
Very large organisations can have pockets of all 3 models.
From my perspective the strongest forces that "push" towards big, centralised clusters are:
1. Powerful central teams that also operate. They're OK with trading off fewer things to look after for potentially larger blast radius. Also the capacity to set up the more complex security controls required for multiple teams and projects to co-exist in the same cluster.
2. Large homogenous workloads are a natural fit for big clusters. Homogenous workloads tend to have 1 "entity" looking after them, so no matter what organisation model, you can end up with big clusters.
3. Tight coupling between workload components. If you have a lot of different things that all need to talk to each other quickly and cheaply you will want to put them close together if you can.