I know how you feel. I've had to deal with sims canceling tasks you've given them, and lag especially going into and leaving build mode since the day I got this game. Recently I started having the speed up and the standing around before doing a task problems which makes the game almost unplayable because your sim will not move for a a long period of time before doing literally any task you give them. I'm 99% sure the speed up problem is the cause of the sims standing around before doing a task you gave them because I've had the standing around problem ever since I started having the speed up problems. I'm going to try and play the game without using the speed 3 to see if that's the problem.
I get the little stutters too but the biggest problem for me is when my child is about ready to go to school, the game stops for at least a minute. I only have 3 sims running in the house. I never have problems with the adults when they go to work.
Im playing on ps4 and have city living installed as well.
I have a time glitch or bug
When shes sleeping or at work it will do the triple fast forward but time moves like its set to normal. If i switch back and forth it will go fast for a moment then usually time will jump back 10 to 15 seconds and go slow again and keep jumping back and forth idk what to do but its aggravating