My brother and I share a PC, he can't play the game pass games I have downloaded and vice versa

My brother and I both got game pass on our accounts recently, he downloaded Gears of wars 5 in his account first. When I try to play gears 5 it prompts me the option to install in the xbox app, the download auto finishes since I believe it detects the game
files are already in the disk D: and a message from microsoft store pops up saying the game is ready to launch but on the xbox app it goes back to Install option. If I click the windows store message then the xbox app crashes and I have to quit it and open
again. My brother and I tried to prove how it worked, we logged into his account and he could play Gears of wars 5 with no problem, but when he tried to play any of the games I've downloaded using my account, then the same issue that I had with gears 5 would
happen. I had one game installed in disk D: using my account, then he installed the same game but in disk C:, and this allowed him to finally play the game, but does this mean I have to have each game installed 2 times on my pc in different drives just so
the both of us can play it? Kinda sad since we are both paying for the service.

I tried logging into my brother's xbox account being ligged into my user on the PC and it would propmpt a message saying that my xbox account and microsoft store account accounts were different, there stating that if I logged into the microsoft store account
of my brother too whilst being in my pc user still probably the app would let me play the game, but its kinda dumb since in the end it means I would have to be logging into his xbox and microsoft store accounts each time I want to play the game?

I believe the issue lies on the app detecting that the permissions are granted to the first microsoft user that downloaded the game on the device and not being able to switch permissions based on who is logged into the PC although it can recognize that the
game files are on the PC already creating an issue regarding permissions between the users of the device itself.

I just want something like my and the microsoft app having good communication so they can say: Oh, he is a different user on the PC but he has game pass too, lets let him play the game that its already installed on his PC although it was downloaded by the
other user because in the end they both have acess to it because they are subscribed to the same service. I myself can locate the game files on disk D:, but I cannot gain permissions for my account to use it manually for obvious microsoft security reasons,
but its still frustrating that something I could do manually cannot be done by the PC and the app itself.

I hope the devs can fix this issue soon, cause paying 2 times for the same service and not being able to use cause of dumb reasons like this is just plain stupid. I tried to contact support but the guy was telling me the reason might be that gears was not
included in Xpass, and that if I wanted a refund to subscribe to ultimate to I could play gears cause it was included in ultimate pass. I mean the guy didn't even know the product he himself was suppossed to work with and the list of game it actually includes.
Even if i subscribed to ultimate I would just gain access to the ultimate edition of the game, which in the end if it is treated as the same game files that are already installed on my PC then the problem would obviously persist. I don't blame the guy on support
since this is a fault on the developer's end, but its still really frustrating thet even support can't fix something like this. Hope this post reaches to someone working on the app and they can give a solution. Thanks for your time reading this and i'll be
waiting for answers.

:)