Atlas creation and memory problems

[VGT]

I currently am running 64-bit windows Unity 3.5.1 with 2D Toolkit version 1.7 + patch 2.

We recently started adding animation sequences of different sizes and lengths.  (most were large animations being 1024x1024 and 70 + frames).

As expected Unity ran out of memory quite frequently resulting in crashes as we tried estimating the right collection sizes.  I have also noticed 2D toolkit gets buggy after crashing.  Frequently I have to delete the entire sprite collection prefab because it won't even show up in the Inspector.  Other times I will click on 'Open Editor...' and it will open a white window where the only way to fix this is to re-open unity and hope it doesn't occur again. 

Sometimes when creating an atlas, Unity will freeze but not appear as 'Not Responding' in task manager.  The Sprite collection window won't be frozen and I can click different frames. 

Other times I will get the error that 2D toolkit cannot cast to source destination.  This means that I have too many frames and there isn't enough memory for what I'm trying to create.  I will see that there is a corrupted image file in the data folder.  I'll delete the data folder and if I try to create the collection again Unity will immediately crash due to running out of memory.

Sometimes it will appear that the collection successfully created all the files but when I assemble the animation the collection will now show up. (deleting the .asset file and 'rebuild index' doesn't fix this)

Sometimes unity will crash saying it ran out of memory while selecting multiple or single images (around 1024x1024) while having the sprite collection editor open.

When successfully creating an atlas, data prefab, and material, I will go onto the next collection and unity will immediately crash almost as if the memory used to create the previous collection wasn't freed.

I have also noticed that the version of 2D toolkit's sprite collections aren't as efficient as previous versions.  I can try to re-create a collection using the same exact image sequence, same amount of frames, and same settings and unity will just endlessly crash due to being out of memory.

Changing the Max size to 2048 instead of 4096 does help a lot when fitting more frames into a single collection but there are certain days that even this won't help.  I'll have a 1024x1024 image (which needs to be trimmed) crash unity if I try to make a collection over 10 frames.  When I have an animation that is over 100 frames at this size it can get quite tedious.

The animations look great once I can put them together and see them playing in game but the collection editor has been frustrating to work with lately.  Is there any possible fixes to these or improvements coming in new versions?

[unikronsoftware]

Can I have specific numbers from you, please, so I cant attempt to reproduce this? Eg. how many textures, and the sizes of each per sprite collection. Also, do you use the atlas spanning feature? This feature ends up using a lot more memory due to requiring all the textures to be resident, which isn't ideal, but there isn't really a workaround in Unity, apart from loading and unloading the textures multiple times which will make everything a whole lot slower.

This version of 2D Toolkit actually frees a lot more data which was left for the garbage collector than the previous version. The only explanation I've got for the increased crashing is increased fragmentation due to the fact that more data is freed at odd times.


You or someone from your organization once requested for the ability to create sprite collections in a separate project. This is now doable (the last bug should be resolved by 1.75 final due out any day now). This might be the best approach with one of the larger ones. The workflow then becomes -

1. Create new "Sprite Collection creator" project.
2. Create your sprite collection in that project.
3. Rename the data folder to something sensible (you must do this within the Unity interface, not from explorer)
4. Export a unitypackage of the data folder alone and import into your other project.
5. 2D Toolkit > Rebuild Index to rebuild the index and recognize your sprite collection.

The only remaining issue with this is that sprite thumbnails won't work - in the current beta 1.75b4, they will appear as pink boxes - I will just be disabling them in the final release.

Let me know if you'd like to try that, and I could expedite that release slightly.

[VGT]

Inside the unity editor each 1024 x 1024 image takes up about 4 MB. (around 198 kb in windows).  Each frame has alpha that needs to be cut off but that's needed since the animations goes from side to side throughout the sequence.

The specific size of the animation I'm working with is 295 frames long but I can't even create a collection of 10 frames without unity crashing on me (I tried with atlas sizes of 4096 and 2048).  I am also selecting the 'Compressed' option.

I'm not sure what you mean by atlas spanning but I am aware that checking multiple atlases can end up taking a lot more memory but unity still runs out of memory when I try to make a collection of 10 frames (with the information I provided above) without checking the multiple atlases box.

Your suggestion of creating our collection data folders on other machines hasn't worked so far for us.  We have tried having artists create these collections on their mac machines then transferring them over to windows.  This usually results in us having to re-create the collections because the files are corrupted. 

[unikronsoftware]

In the Unity editor, each 1024x1024 image takes 4MB of VRAM, but takes quite a bit more RAM when loaded in, unfortunately due to how Unity works. In addition to that there's the 400MB chunks Unity allocates memory in. Would it be possible to send some assets of the same specs over so I can try to reproduce this on my end? By that I mean any blanked out textures of roughly the same size as your source sprites (including the trimming). I might be able to modify the build process to run the slow way as I mentioned earlier to save a bit more memory.

Transferring the sprite collection data should work OK - Just checking that you use unitypackages to transfer them, and that you've got the same versions of 2D Toolkit on both? In terms of creating the sprite collections, I've tested it a fair bit in 1.75, but not as much in 1.7. I'll give it a quick go when I get a chance in case there's something wrong there. I can prioritize this over the next couple days to see if we can work something out here. Its probably worth doing this over the other random memory saving opportunities, which Unity may (and is likely to) change in the next version.

[VGT]

Here are 10 frames (all the same) that I created from one frame of our original animations. The character is colored over but the portion of the image it takes up is the same.  I also want to add that our projects are setup to have .meta files generated for all files (helps with version control).  The meta files are generated when the asset is imported and I wonder if this is overwhelming Unity in conjunction with 2D Toolkit.  I might test this out at some point today.

[unikronsoftware]

Thanks for those files.

I'm trying to see how many sprite collections I can create before it actually crashes. Starting with a blank project, and your directory duplicated 7 times - I created sprite collections here about 20x in sequence and I've not managed to get it to crash yet. 10 sprites per sprite collection, ends up with a 2048x4096 atlas.

Are you doing your sprite collection construction in a blank project or do you have anything else in them? Perhaps something else is interfering with this?

Edit: Unity 3.5.2f2, 2d toolkit 1.7final + patch2

[unikronsoftware]

Did a few more tests - I think we need to narrow this down further somehow. Maybe with comparing memory sizes. Could you get in touch by email so we can try to sync something by IM or something.

[lelag]

I have exactly the same problem, it seems that I reached the max number of sprite collections i can create.
When I want to add a new one (I already have 9 collections), unity crashes, fatal error saying i allocated too much memory.

Even when i launch unity, the editor takes approximately 2 go of memory, and i can't do much inside the editor. I'm quite stuck with my project.

And also, i have pretty long animations (120 pictures), and i wan unable to add all the images inside a single sprite collection, i was forced to divide the sprite into 4 collections. Above 60 frames, the editor crashes. Computer i use is good (4 go ram, quad core, etc...).

I use multiple atlas generation (because final app needs to be high definition), compressed and size = 2048.

Help.


Thx.

Ps : great product anyway.

[unikronsoftware]

About the editor taking up 2gb of memory on load - this is unexpected. Is this before or after you load a scene? If this happens when you initialize into an empty scene, we will need to find out what is loading up these objects - they aren't loaded unless something requests it to be loaded.

What version of 2D Toolkit & Unity are you using?

Drop me an email at support@unikronsoftware.com and we can discuss this further. There are ways around this, but we need to work out the issue above before anything else.

[sebastian]

Just wanted to say I had a similar problem when I had a bad crash and a lot of my tk2d sprites got disconnected.  When restarting my project it would reimport all my assets-- and would cause unity to crash every time (Fatal Error Could Not allocate memory).

Turns out a solution that worked for me was to close unity, then remove my TEXTURES folder from my unity project.  Reopen Unity, then drag my textures back into the texture folder one by one.

I guess Unity was choking trying to reimport all of my textures at once.  Doing it a few at a time worked.

Hope someone else finds this helpful!

[unikronsoftware]

There was an issue with large animations being loaded in all the time - this has been fixed in 1.80 which I hope to submit tonight. Memory usage should drop quite a bit if you have very large projects.