Source hydrator fails with controller is configured to ignore cluster

[crenshaw-dev]

Describe the bug

When running the app controller with multiple shards, source hydration will fail if not all apps for a given repo/sourceRevision/targetBranch group are destined for clusters on the same shard.

To Reproduce

Set up Argo CD with multiple shards and multiple clusters. Make sure two of the clusters are on different shards.

Set up two apps destined for the two different clusters. Set up sourceHydrator on each to point to the same repo, sourceRevision, and targetBranch.

Watch the hydrator fail with failed to get cluster version for cluster... controller is configured to ignore cluster.

Expected behavior

The hydrator shouldn't try to get the cluster server version and API versions for each app at all. The hydrator doesn't even use those values, because they wouldn't be deterministic.

Screenshots

Version

2.14.0