I really hope they ditch it and go back to the skill-stat-leveling system that worked better in Morrowind and Oblivion if they do a VI. That was a better system allowing for superior character customization and could radically change gameplay from character to character. I feel dumbed-down for the idiot masses by some of these changes they made.
I think the balance issues thing isn't quite right, for the most part yes...but a well-developed high-level magic user is much more powerful, it's just way too easy to f**k it up and not be well-developed.
Other than that, the thing that peeved me was the massive time jump. For someone whose love of this series was the lore and interconnectedness/cohesiveness of the story (one game's plotline flowed into the next), jumping ahead a few hundred years after the first four games occur over the span of about 40 years (the last 3 over the span of about 15 years) was a serious piss-off for the people who were into the story. Now I concede that they painted themselves into a corner with the end of the Septims, but the aftermath of the Oblivion crisis would have made a more compelling storyline IMO.