I honestly think it would be a dandy idea to have height in the game but I think it'd be best left for sims 5. Height would be a crazy addition and think of all the stuff the animators have to do to make it work well in the game seeing as I've used the height mod in my game before(specifically sims 3) and the amount of distortion between two different heights in sims is a little hilarious.
The height slider mod I use is only a 3 kb package file & using it vs. not using it makes no real difference in CPU or memory usage that I can tell. No bloat in save file sizes either. My teen - elder Sims vary in height from +20% to -15% of normal Sims. At those sizes the Sims do not look distorted at all. Virtually all animations look fine for tall or short Sims, even for things like holding hands or hugging. There must already be some form of compensating for height differences built into the mod, otherwise their hands would be off, like they are playing patty-cake and not holding hands! xD. The only thing that doesn't line up well for Sims at the extreme ends of my settings are their lips when they are kissing, but even then you have to look really close to notice it. I imagine anything the game devs would come up with, if they ever decided to add something like this to the game, would be even more polished than that.
I'm not trying to encourage the use of mods here, that's up to each player as long as they understand and accept the risks that go along with using them. I do like the variety that having different sized Sims brings to my game so I am still Me Too with the original post.