Specifics ports for chat on XBox One that can be QoS/DSCP prioritized?

You would need to do this at the network level on your Local network. I would assume one of the UDP ports is used for voice. If you really want to know what port is being used you need to run wire shark on your network and do some packet inspection. I did
google but nobody seems to have done this in the past.

If you have a firewall or network hardware, such as a router, you might need to make a configuration change in order for your Xbox One console to communicate with Xbox Live. This configuration change is sometimes called "opening ports" or "port forwarding."
Xbox Live requires the following ports to be open on your router:

• 
  Port 88 (UDP)
  
• 
  Port 3074 (UDP and TCP)
  
• 
  Port 53 (UDP and TCP)
  
• 
  Port 80 (TCP)
  
• 
  Port 500 (UDP)
  
• 
  Port 3544 (UDP)
  
• 
  Port 4500 (UDP)
  

Sorry, no solution. My guess is that Microsoft is applying some kind of QOS to the connection that prioritizes chat so that it isn't interrupted by other traffic. I've noticed that even downloading demos has no effect on the party chat. While in theory this
is a good idea...in the real world you want the game traffic prioritized. Especially in FPS games. QOS will also add a small amount of lag just due to the packet processing.

Game chat definitely doesn't prioritize the chat channel. The quality difference between the two is very noticeable.

check your router settings. your router may be set to prioritize your internet traffic. specifically check your Qos settings

Ok great. Were you able to set the QOS setting to prioritize the xbox?

It sounds like you have the Xbox One IP traffic prioritized via QoS.

Try restoring the router back to default settings.