Huch, seems you stumbled over an old one here. The internally used GraphicsData node was not working correctly with mutable collection types. It was using a cache region which does not trigger for a spread builder when its count changes. Fixed in upcoming preview build, thanks for pointing it out and posting a patch showing it.
Regarding the lag you get when you call ToSpread - if a newly allocated object is large enough (> 85,000 bytes), the dotnet runtime allocates it on the so called large object heap (LOH), which in turn will only get cleaned up by a Gen2 garbage collection which is the most expensive one. In other words, if your patch allocates large objects every other frame, it will force a Gen2 collection at some point.
There was some hope on the horizon just recently regarding that topic with a different garbage collector for dotnet, however after some initial tests we had to abandon it for now. Hopefully Microsoft recognizes the demand for it and adds it as an option to choose from in the future. Until then we’ll need to handle these cases ourselves, by (like you did in your patch) allocating such large objects / collections only once and re-using them as best as we can.