I can't talk in a party while playing Xbox 360 games on my Xbox One.

Here is the link that should help you, https://beta.support.xbox.com/help/hardware-network/display-sound/troubleshoot-xbox-one-party-audio
Side note, My husband said he has the same trouble if it is game chat he can join in, but he can't join the party if they are playing on 360s, his friends change to game chat for him.

I think people use party chat when they play Skyrim. They can't play together but they just talk to one another while playing it.

Hey, Paiste! You will not be able to play in parties in game with users on Xbox One consoles while you are playing 360 games. You will need to create an Xbox One party on the console to be in a party with those users. You should be able to create parties
in game for 360 games with other 360 users by pressing the Menu and View buttons on your controller while in the 360 game to bring up the 360 Guide. Party options should be on the first slide there.

Let us know if you are still having issues after that explanation!

Whenever I launch an xbox 360 game on the xbox one x my party chat is not allowing me to talk with my friends. My friend has the same issue.

Is this without the other players being on 360? Ive found that same issue when playing on the 360 and having friends still playing One games. But ive never had this issue with others on the 360 as well. Try it with everyone once your in the 360 mode
and then party up. Thats what worked for me.

If you're playing an Xbox 360 game on the Xbox One, you can already play with people who are on a physical Xbox 360. I believe you can also start and join Xbox 360 parties from the Dashboard. (The option for parties was there but I didn't try it.)

With the exception of World of Tanks, what you can't do is play multiplayer between the Xbox One and Xbox 360 version of the same game.