I dont think I am near the limit because I have 16GB ram. You never know though. I think it has to do with the linkage system. (a scriptable object in the resources folder that points to a texture). I way that because when I tried to do that very thing I had the issue again. BUT, when I built not using the any Unity editor build buttons, just the build command, I had no problems. Ill update if this changes. I think it has to do with the editor thread. So if any one else is suffering from the same issue, make a custom build panel and build from there.
UPDATE
Scratch that, still crashing