How could I restrict access from others accessing my Xbox account?

You will need to ask the other console owner to remove your account. I agree that changing your password to restrict others from accessing your account could also help.

you could log into their profiles and manually hide the games but other than that there are parental controls available on your microsoft account to restrict access.

I think it's possible it sets a global restriction, but when you try to access that content it asks for the parental passcode. As such, the two parental accounts would be able to access the restricted content by knowing the passcode whereas the minors' accounts
could not.

I'm having the same problem but I set up my account. How do I access the "content restrictions"

Hi there,

If you purchased the content on the parent account, then the DLC can only be accessed by the adult account. The only way the child account could access the DLC content is if both accounts are on the same Xbox and that the parent account has the Xbox set
as the Home console. This will allow digital content owned or purchased from the parent account to be shared with the child account or other accounts on the same Xbox. The parent account will also need to allow access to this content for the child account
due to the age rating of GTA 5.

Full information on the Xbox Home Console feature can be found
here.

Allowing content restricted for child accounts can be done from
https://account.microsoft.com/family/ or on the Xbox One when the child tries to access this content.