Hi, okay... so I had a similar thought to you.
We are using a distributed switch with probably around 20 VLANs on different port groups on it. The vDS switch that we were just failing over to for this test had 499 ports available, although each of the port groups are configured with Ephemeral port binding, and I wonder whether this could be causing the issue.