a month ago
Had a stable game when I set all clock speeds to default, turned Virtualization Technology back on and set XMP 2.0 profile for my RAM @ 3200 MHz, which oddly puts the RAM voltage at 1.45v versus 1.35 as per the tech specs.
a month ago
I have not heard or seen any devs reply about this...
Dev 1: "Hey guys lets add a crash log in our game"!
Dev 2: "Yeh, then let's not support it!"
Dev 3: "Yeh great idea, and let's keep the Mozambique how it is cause for the bique memes, even though it's a meme because people want it fixed".
EA: "Great jobs guys. You're all promoted and still get the same everything... cause we take all lootcrate money of course".
a month ago
Guys please like my tweet on twitter so respawn will see this and fix it since ea doesn't care about us and doesn't reply on the forums.
a month ago
Hi, I'm an engineer at Respawn. Thank you for posting this log!
This particular crash is in MSVCRT, which is MicroSoft's Visual C++ RunTime dll. Microsoft's DLL crashed when Apex was trying to exit normally. This crash happened inside sapi_onecore, which is part of Microsoft's Speech API.
Unfortunately, there's not much we can do about this crash in 3rd party software.
It looks like it only happened when you were already exiting, so I hope that it doesn't negatively impact your experience enjoying Apex.
a month ago
Thanks for these logs. I'm the engineer who implemented the crash log system. I'm also the one who used the logs when we first implemented the voluntary crash reporting system to find and fix the bug causing the vast majority of the crashes we were seeing. After that fix, we saw the rates of players vanishing from servers plummet.
I just found this thread and am now looking into each of these logs. I see four different crash reports in this thread.
One of them is the one I just commented on, in msvcrt.dll when quitting. As I said, I don't think we can do anything about that one, since it's Microsoft's code that is crashing, but at least it only seems to happen when you already wanted to exit anyway.
The other three are in r5apex at three different offsets: 00000000002F2DCA, 00000000002F2E2A, 00000000002F2E9E. I'll comment on these in reverse order.
The last one (00000000002F2E9E) happened once for TlElTlRlIlS and is the most confusing. That's executing code in the middle of an instruction. That shouldn't be possible.
The middle one (00000000002F2E2A) happened for two people, kaolinAngel_TTV (original poster) and umarusann. This looks like it might be a legitimate stack corruption bug, but it might also be an overclocking / overheating bug. I'm in the process of analyzing the code to figure out how this might be possible. This is hard, and may take some time. I can't just analyze the human-readable code we actually wrote; I also have to analyze the machine-readable code the compiler generated from what we wrote.
The first one (00000000002F2DCA) happened for 12 people in this thread, making it the most common. This one is also bizarre. It's executing an instruction that just asks if a CPU register is zero. (In very basic terms, a register is a place inside the CPU that holds a number that's being used in its computations. In one sense, it's like a super tiny cache of super fast memory. If that doesn't make sense, don't worry about it!) That instruction is causing either a CPU breakpoint to be hit (10x), or causing a write access violation (2x). This instruction should not be able to do either of these things. It's not a breakpoint, it's a bit test. It doesn't write memory, it reads a register.
Interestingly, all three of these crashes are in the same function. This is also the function that sometimes reports an execution access violation on a memory location that isn't the current or next instruction, which should also be impossible for a properly functioning CPU. When that crash happens, I made the latest patch of Apex tell you that the crash is probably due to overclocking or overheating.
So, the most likely explanation I can come up with for these other crashes is that your CPU is overheating, possibly due to overclocking. xjungleWarrior even helpfully mentioned that he had a 3.7 GHz CPU overclocked to 4.9 GHz. Thanks for that extra tidbit!
I have worked as a software engineer in the games industry for nearly 20 years, but before that I earned a Master's in electrical engineering from Stanford, with a specialization in microprocessor design. So, while not an expert on overclocking/overheating behavior of modern Intel CPUs, I do have a lot of relevant experience and background. I'm a very logical and scientific guy. So, I don't suggest overclocking/overheating lightly. I suggest it because it's the most logical explanation.
Overclocking / overheating can cause very rare CPU faults. These can even show up only when you execute a particular sequence of instructions, so you can think that your computer is perfectly stable when it's overclocked, and then one day you install a new program and it crashes a lot. It's only natural to think that it's the program's fault, since everything else worked fine for so long. But CPU clock speeds are set to make sure that the CPU *always* finishes every possible instruction sequence in time for the next instruction to start. If you increase the clock speed, you give it less time to finish instructions. That will usually work, since there will usually be some "buffer" time at the end of each instruction. But in very rare cases, it needed the whole clock cycle to finish this particular instruction with that particular data, and something else need the results right away, and then your CPU will crash.
Overclocking can also lead to overheating, but overheating can happen even without overclocking, such as in a hot room. These can have similar effects as overclocking, but overheating can be even worse. It can lead to more thermal noise inside the CPU, especially when the CPU is working hard, which can cause bits to flip inside the CPU when they shouldn't. If that happens, your CPU is likely to crash.
All that to say, the most likely explanation for the vast majority of these crashes is overheating and/or overclocking.
If you're experiencing these issues, please try these things and reply to this thread. I really want Apex to be stable for all our players, which is why I implemented the crash log in the first place!
Thanks again for all these logs, and hopefully together we'll be able to figure out how to keep you all in the arena!
a month ago
Thanks for feedback. I've reverted my 8700k to 4.7Ghz with temps not exceeding 85C also run default RAM settings. Still same result and crash every other game.
It's not temperature or OC related as I've been running OC settings for the past months without any single crash.
There is something wrong with the code after last update, please sort it out.
a month ago
Please check the code for some sort of "CPU Parity error" corruption, if that is possible.
I do know that one reason for these crashes, which I have managed to log myself, is a "correctable" (meaning: not blue screen) WHEA error, called "CPU Internal Parity Error". This seems to happen randomly and happens usually on overclocked systems but if it is happening at STOCK (that same R5Apex: 00000000002F2DCA) error, then it means there is something seriously wrong. Intel designs their CPU's to run at stock up to 100C (at the highest official turbo frequency). This is absolutely *NOT* an overheating bug at all. These errors can happen even as low as 45-50C. In most people's cases, reverting to pure stock settings will prevent this.
I do know that this "Parity error" was first reported after some old microcode updates (originally for Haswell processors) for Spectre/Meltdown mitigation, although the OS had to be aware for these protections to actually work. What I believe is that these crashes are caused purely by AVX code doing some..."strange" things that I've never seen before.
In my case, I know that programs that use AVX and hit the L2 cache heavily, like Realbench 2.56 and Cinebench R20 (new version) start going pear-shaped in an overclocked system at about 94 degrees celsius. However the errors generated here are (CPU Cache L0 errors) (correctable WHEA Errors), and if it is not correctable, a System service exception, Kmode exception not handled, or IRQL Not Less or Equal BSOD will be generated. (funny enough it's never Clock Watchdog Timeout (stop 0x101--frozen cores) or WHEA uncorrectable error (stop 0x124--usually a cache error). A CPU Cache L0 error isn't even a cache error relating to normal CPU caches--it's some sort of instruction register (CPU's don't have a L0 cache). There have been some comments that this is an "Operand Cache" or Register file, or access to the stack.
These L0 errors are the "overheating" errors you are referring to. You only get these errors when you are overclocked and your CPU exceeds a certain temperature in a stress test or full usage application.
Apex isn't generating these errors. It's generating "Internal Parity Errors". These are most likely causing the game to crash. But these errors are happening at VERY Low temps.
This could also be some sort of Intel Microcode bug as well. But I have no way at all to check on such a thing. That being said, the newest 9900K microcode you can download and patch (in windows, without bios flashing) yourself is ucode AE, which doesn't do anything helpful for this.
Please check on this however. This "CPU Internal Parity Error".
But for those who are getting these crashes at *STOCK* speeds, this is not an overheating issue and should not be happening. There may be a bug with the AVX code or something Apex legends is doing.
a month ago - last edited a month ago
I am also seeing this crash at both stock and overclocked settings.: R5Apex: 00000000002F2E2A
I have been a PC building hobbyist for the last 20 years and I also work as a software engineer. I have also been overclocking pcs for the last ~15 years. I've run extensive tests under both stock and overclocked settings to determine component stability. HWINFO, Event Viewer, and all benching software /stability software are showing no errors, crashes or any indication of hardware failure.
I am 100% confident temperatures are not an issue under either clock speed as the cpu and gpu are both being cooled under water (I have HWINFO logs to show temps are fine even for extended heavy stress testing (24 hr stress tests)). Additionally, I have a wireless probe within the case to provide temperature readings.
I am currently allowing GSync to cap my frames with VSync disabled both in game and via nvidia control panel. I could switch over to using command line args if that helps you rule out some interaction with GSync.
I have recently wiped my computer, updated bios, upgraded firmware on everything that was out of date, and installed up to date drivers.
I honestly think you have something wrong in your stack, compiler, or some interaction with the micro processor code. If you want me to provide any further information let me know, happy to help.