My headset keeps turning off after I turn it on.

Hi. My name is Brilliant and I am happy to assist you today.

Here is a thread with a possible fix to a similar issue to the one you're currently face with: https://answers.microsoft.com/thread/221c9d7f-dc91-4067-ba81-33ab671179b5

You can also try troubleshoot your headset and see if this will resolve your issue:
1. Check your headset settings.
2. Restart your headset.
3. Update your headset.
4. Reset your headset.
For more details on the above kindly visit: https://support.xbox.com/help/hardware-network/accessories/troubleshoot-xbox-wireless-headset

Depending on how long you have used your headsets, you might also consider checking if the battery is still functional.

You can also get your device repaired.
With Microsoft, you can register your device so you can request and track repairs and check your warranty status: [https://support.xbox.com/help/hardware-network/warranty-service/register-your-product].

If you've registered your device, sign in at Device services to see the device’s warranty status. If it’s within warranty, repairs are free. If your device is out of warranty, you’ll be charged a repair fee.

To get your headsets serviced, kindly follow: https://support.xbox.com/help/hardware-network/warranty-service/getting-your-product-serviced

You can also get your headsets repaired at the retail you purchased them from if they do repairs.

I hope the above helps resolve your query.

UPDATE: I popped off part of the back and noticed it is turning on now but when I try and connect my headset it completely shuts off and won't turn back on until I remove the headset.

The same thing is happening to me. The power is off then the console just turns itself on. I turn it off at the button and a little while later it turns on again.

If you have a plug and play controller plugged in it keeps it on. Otherwise IDK.

The issue could also be caused by other devices that somehow interfere with the console. It's a long shot,but who knows.