VRR+120Hz mode makes controller unresponsive

In some games (COD Vanguard, Gears 5, Plague Tale), when I have 120 fps and VRR mode enabled, the controller always becomes unresponsive at some point. In Vanguard and Gears 5 it happens after each match and in Plague Tale, it happens at a random point. The controller doesn't turn off, but it doesn't register any input.

The only band-aid fix is to turn off 120 fps mode or disable VRR.

The controller has the latest firmware.

Does anyone else have this issue?

:)