Combat Encounters and World Scaling

In an effort to make the world feel "real" or lived-in, and while the players definitely are the stars of the show, having decisions that influence the world tenfold when compared to the average passerby - I don't scale the world to the PC's. I think Oblivion did this, but I am not sure. One of the Elder Scrolls games did this, where at the beginning the bandits are wearing leather armor and wield iron blades, later on when you have become stronger, so have they. Now wearing dragonscale armor wielding magic weapons and trinkets talking about how they want to rob me for my COPPER. The "benefit" or I guess the point of this type of world scaling is to make every encounter challenging, every encounter "difficult"...but does it? I argue it does not. For anything to be, I can't think of the word right now, BUT BASICALLY - without a reference point you can't gauge whether something is this or that. For example, if ALL my encounters are difficult then none are. In the example of the bandits, the bandits that you encounter with dragon scale armor and shit are NOT harder than the bandits with leather armor. They are just as hard as before and that's the point of scaling the world to the player. Which is fucking stupid. It feels like a thought idea halfway. "Players like when things are challenging, so let's make everything challenging" - which may be to someone hearing that sounds accurate, but it's not at all. No one likes anything challenging. Hear me out, thinking of something challenging for you that you cannot do. EVER. Do you like that thing still? I am guessing, like me, you don't. Now imagine something that WAS challenging that you accomplished - I bet you like that thing. THAT"S WHY THE BANDITS NEED TO BE CHALLENGING ONCE. And when I come back with fuckin dragon armor I laugh at their feeble attempts and relish in remembering when this was a challenge to me. Doing this type of world scaling constantly and subtly reminds the players how far they've come and gives the dopamine injections of progressing towards something even if they don't know what that is. EXCEPT, with this type of world scaling there are places and things above their capabilities. There will be bandits in dragon armor they encounter when the PC's are in leather armor (hopefully there were warnings) this in turn gives the PC's subtle and constant goals to aspire to. Which will make them more engaged in your world and game, when the quests or plot hooks run dry or maybe just don't appeal to that specific player, they will think in the back of their head "well, if I keep going along this quest/storyline/whatever I'll surely get items and experience, enough maybe to go "kill/steal/enter that bandit camp full of dragon armor!" - hopefully you get what I mean. This is why in Amith the world is how the world is. If you go to a kobold camp at level 15, don't expect a dracolich to be secretly controlling the Kobolds or something, if the only reason there would be is to make that encounter more difficult. Due expect to go make the hills run red with kobold blood, as you eradicate this kobold settlement, and DUE expect the in-world consequences. In that specific situation, Kobolds may start settling further away, they may start attacking more indiscriminately due to peaceful living still leading to their devastation, etc etc. TL;DR:
The encounters are not built with the PC's in mind, not solely anyways. The encounters are structured in order of (most important to least important)
1. What makes sense narratively/geographically 2. What has been foreshadowed or otherwise told to the players 3. Time(in-game) 4. Player's reasoning(yeah just being honest) 5. When the last encounter was(not in-game time) 6. How did the player's feel about the last encounter 7. Lastly, how strong the PCs are.