[COLLECTION THREAD] Autonomy and Simulation Lag (Seasons Patch Edition)

[crinrict]

There's some tips on the first post, other than that, it would be great if you could provide the info asked for on the first post.

[msgeekness]

How long do you think this will take to fix? I just bought Seasons and I was super excited about it just to see that it is completely unplayable right now. I tried everything that you told me to do like turn the graphics down delete things in household inventory all of it and its still completely unplayable for me.

[crinrict]

Hi @msgeekness

would be helpful if you could provide the info asked on the first page.

Thank you.

[ToxicKitKat]

Look, I respect what you're doing, but it is not a hardware problem. It is a developer problem.

[Zhentarrrr]

Product: The Sims 4
Platform:PC
Which language are you playing the game in? English
How often does the bug occur? Every time (100%)
What is your current game version number? 1.46.18.1020
What expansions, game packs, and stuff packs do you have installed? All of them
Steps: How can we find the bug ourselves? 1. Load the game A large volume of custom content makes the issue significantly more visible, but it is present even without any.
What happens when the bug occurs? The game takes [significantly] longer to load.
What do you expect to see? With a large amount of custom content (~14GB), the game loads in <= 5 minutes, rather than 9 minutes and 21 seconds.
Have you installed any customization with the game, e.g. Custom Content or Mods? Not now. I've removed them.
Did this issue appear after a specific patch or change you made to your system? No

‌

Basic machine specs: i5-8600, 16GB RAM, 480GB Intel 730 SSD

My game takes a long time to load. I've got a ton of custom content, I don't expect it to be fast. But when I took a closer look at what exactly was taking so long, I found that much of the time was for a silly reason.

I recorded a trace of the game starting up using xperf (part of the Windows Performance Toolkit). (I can share this trace privately if desired; I do not want to post a link for it publicly because traces can contain personally identifiable information). When I took a look at the CPU sampling, I noticed something rather interesting. 

The attached cpu_samples.png shows it: 4 1/2 minutes of CPU samples came in on just six assembly code instructions. And zooming in (in cpu_sampls_closeup.png) we can see it's actually alternating between two sets of just three CPU instructions. Earlier in the trace, each one takes a pretty small amount of time, just 2-3 milliseconds each. Near the end of the loading process, each one is taking 200 milliseconds! 

My curiousity piqued, I took a look at what those instructions were:

loc_B45400:                             
 inc     rdx             
 and     rdx, rcx
 lea     rax, [rdx+rdx*2] 
 cmp     qword ptr [r9+rax*8+10h], 0FFFFFFFFFFFFFFFEh
 jnz     short loc_B45400

(I guess technically that's 5 instructions, not 3, but it seems the and & cmp instructions pipeline/superscalar well enough to be almost free in practice)

or, decompiled:

    do
      idx = this->buffer_size & (idx + 1);
    while ( *(&this->buffer + 3 * idx) != -2 );

 Ah, so this is a hashmap! -2 means an open slot, -1 means an entry in the appropriate slot/bucket, otherwise it's the slot/bucket it wants to be in. The hot instructions are looking for the first open slot when there is a collision! But this seems like a pretty good hashmap implementation - it looks like it's going for a 66% capacity factor, and there's some sort of search distance reduction scheme (robin hood hashing?), so why would it be seeing so many collisions?

So I took a look at the hash function - and here we have our problem! To me, it looks like this (my deepest apologies for butchering your code; C++ does not decompile very cleanly at all but I've tried to neaten it up as best I can):

unsigned __int64 sub_B45BA0(void* thing_to_hash)
{
  hash_flags_union flags; // r9
  unsigned __int8 ones_bit; // bl
  unsigned __int64 hash; // r8
  list *list_field; // r11
  signed __int64 i; // rdx
  __int64 list_field_value; // rcx

  flags = thing_to_hash->field_x22;
  ones_bit = 0;
  hash = 0;
  if ( (flags >> 6) & 1 )
  {
    list_field = thing_to_hash->list_field;
    if ( list_field )
    {
      hash = 0x1505;
      for ( i = 4 * (list_field->count + 1); i; hash = list_field_value + 65599 * hash )
        list_field_value = *(list_field + --i);
    }
  }
  if ( (flags >> 1) & 1 )
    hash ^= thing_to_hash->field_x18;
  if ( (flags >> 2) & 1 )
    hash ^= thing_to_hash->field_x1A << 8;
  if ( (flags >> 3) & 1 )
    hash ^= thing_to_hash->field_x1C << 12;
  if ( (flags >> 4) & 1 )
    hash ^= thing_to_hash->field_x1E << 16;
  if ( (flags >> 5) & 1 )
    hash ^= thing_to_hash->field_x20 << 18;
  if ( (flags & 0x80u) != 0 && (flags & 0x8000u) != 0 )
    ones_bit = 1;
  return hash ^ (flags >> 1) & 0xC00 ^ (ones_bit | (flags << 9) | (flags >> 8) & 0x6);
}

 Unfortunately, this is not a very good has function. The bitfield is the only unconditional source of entropy, and it gets shifted out of the lower 9 bits, almost guaranteeing that there will be a lot of contention for just one out of every 512 hash buckets. A brief sampling in WinDbg suggests that it is actually much worse than that (not unexpected, since the bitfield values are unlikely to be randomly distributed), and that 0 is a particularly common hash value. 

Suggested fix: I don't understand the context well enough to suggest what a better hash function might be. However, at a minimum, multiplying by 11400714819323198485llu (the golden ratio/fibonnaci constant for a 64-bit integer) provides a computationally cheap method to better mix what entropy is already present across the bits of the hash code.

[crinrict]

Hi @Zhentarrrr

thank you for your detailed report.

I merged this with the autonym and Simulation lag thread.

[thisisarealpainx]

Product: The Sims 4
Platform:PC
Which language are you playing the game in? English
How often does the bug occur? Often (50% - 99%)
What is your current game version number? Latest patch
What expansions, game packs, and stuff packs do you have installed? Get to work, get together, Seasons, Cats and Dogs, vampires, parenthood, jungle adventure, dine out, bowling stuff, romantic garden stuff, my first pet stuff, city living
Steps: How can we find the bug ourselves? Make a household with some adult sims and children. Send them to work or school a few times.
What happens when the bug occurs? What happens when the bug occurs? For example, children are meant to come home at 3pm. But, they will not come home until 1am the next morning.
What do you expect to see? The sims to come home on time.
Have you installed any customization with the game, e.g. Custom Content or Mods? Not now. I've removed them.
Did this issue appear after a specific patch or change you made to your system? No

‌

NOthing more to add.

[crinrict]

Hi @thisisarealpainx

what kind of mods were you using before removing them ? Have you tried this on a save that never saw any mods ?

Do they physically not come home from work or just show as at work when they're not ?

[thisisarealpainx]

I’ve only ever had MCCC. I moved it to my desktop and it doesn’t seem to have made a difference. The sims are staying at work hours after they are meant to have left (not just showing as at work when they are not). The line finishes going around action “at work” indicating they should have finished, but they don’t come home.

[thisisarealpainx]

I haven’t tried it on a save which never saw mods, but it has happened across multiple saves.