How do you fix unresponsive controller?

For the past few days, my xbox one has been having a strange problem. Randomly during play, the controllers will become unresponcive. It is as if the xbox believes the controllers are still connected, even when you turn the controllers off and on again.
The xbox menus are not accesable. No control does anything. Restarting the xbox does not fix the issue, upon restart the controllers contuinue to yield no results on screen. I know the xbox is not freezing because whatever you were doing when this happens
continues.

For example I was playing Ark Survival Evolved split screen and then suddenly the controllers no longer give controls to the characters. You can see the animals moving and time passing. The third time this happened, i left the xbox to sit and watched what
happens. Eventually the xbox put the screen into sleep mode, displaying the time and stuff, but still the controlelrs did nothing. I have only seen this happen with Ark Survival Evolved as its the only game I have played lately, I dont really play games much
at the moment. I will update here if this happens in other games or not as i have the time to find out. Seems like an xbox issue not a game issue the way it effects xbox controller button pulling up the xbox menu.

the only fix I have found, more of a work around, is to do a hard reset by holding my finger on the power button/light until it shuts down. Upon restart the xbox works fine, but its no more than a few hours before it happens again.

Has anybody had this issue? Anybody know what might cause this or a possible fix?

Thank you in advance for any responces, and reading through this short novel

Edit: typos

:)