Is everyone's XBox One X so slow and clunky?

Sorry, I know that title screams clickbait and suggests the op is a Sony loving console warmonger, but I promise that's not the score. I owned an XBox One, then S, and I've had plenty enjoyment gaming on my One X Scorpio edition since launch, but there's
always been one thing that's threatened to ruin the experience, and that's the OS. It's just so freaking slow, laggy and clunky, and it suffers errors if I leave the console in quick start mode but unused for 3-4 days.

So often I'll come back after a week of no play and start the X up only to find the UI won't load properly, or my external hard drive isn't recognised, or it won't connect to the Microsoft live network, or it's just monumentally slow at doing anything.
Then every time I have to turn it off, pull the mains lead out, disconnect the external hard drive, wait 10 seconds and plug it all back in (obvs) before watching the U N B E L I E V A B L Y sssslllloooooooooooowwwww One X boot up sequence and then the UI
loading sequence.

It can literally take 5+ minutes to get into a game and it's so frustrating. Great when I'm playing though! But it's so slow and painful when it's not working, and that's such a frequent occurrence I'm genuinely put off playing much anymore. I mean, this
thing cost £450 and is meant to be the fastest console available, right?! And it's the second Scorpio I've owned as the first one was just as slow (or slower) to boot and load the UI (I returned it after a few days at launch and was given my current one).

I spoke to someone else who owns one and was surprised that they experience the same issues and thought it was just something XBox owners put up with!? So I was just wondering, is it normal and do other owners experience the same issue when leaving their
XBox One X unused for a few days (especially with an external hard drive plugged in and fast start turned on)?

:)