Mind Grubs

Little buggers are so small nobody has even seen them!
They've never been definitively proven to exist but any wizard will tell you they can be found in the far reaches of space. When all is quiet and safety is finally at hand they find their way into a wizard's brain and eat away. Spell power is what moves most ships across space, so any loss in spells directly equates to a loss in ability to traverse space and perform emergency maneuvers.
Captains across the multiverse are less convinced of their existence. Insisting the ship's wizard is just lazy and malingering. Unfortunately for them, they lack the negotiating power required to force them back to work. Even dissecting wizards who allegedly died from mind grubs have no turned up any sign of them. The only clues to their existence are the feelings that wizards describe as wormy or grubby. Not exactly the most convincing of arguments.
The exhausting nature of being a ship's wizard and high-level talents needed to run an elite ship means getting bodies to fit the role is difficult. The occasional bar found rogue can make for a suitable wizard for tight pursed captain but mainly for short excursions. For deeper probes you need a full fledged wizard so their concerns over Mind Grubs, real or imagined, must be treated with serious consideration.
When wizard's contract the grubs they slowly lose more of their spells and with them the ability pilot the ship each day. The grubs are contracted in deep outer reaches. More time there and the more they multiply. As fast as space rabbits! Time spent in at home port is the only way to remove the grubs. Perhaps the home environment gives the brain the home port advantage in fending off the grubs. Perhaps the copious amounts of alcohol kill them off.