In which I describe some of the struggles that I had to deal with during development.
Asynchronous Scene Loading with Multiple Simultaneously Active Scenes
As the development of the game proceeded and new optimization and streamlining processes were introduced, there existed a possibility to spread certain game aspects across multiple scenes. For instance, to reduce amount of loading and memory garbage collection performance impact I have decided to offload all of the particle effects and other non scene dependent elements into their own scene ("[Pool Manager]" one on the screenshot below). This way those could be pre-allocated at the start of the game and persist throughout scene loading process omitting the need to destroy/allocate them every time the new scene is loaded as well as streamline the process of creating a particle by having the ParticleManager script instance persist in a similar manner.
However, there came an unexpected issue due Unity's way of handing scene loading and the API they provide for such operations, especially involving asynchronous loading. UnityEngine.SceneManagment namespace provides two options for loading:
1. Scene can be loaded in LoadSceneMode.Single which would unload all of the existing scenes (except "DontDestroyOnLoad" scene) and set the loaded one as new active scene.
2. Scene can be loaded in LoadSceneMode.Additive which would load and append the new scene to the existing list of scenes.
The issue with the existing Unity's scene management structure is that there are only minimal way to control this process. Developer can only set one scene to be active at the time but it must be loaded for it to work however there is no API to check if a specific scene is loaded. The scenes can't be loaded as inactive either which means that upon finishing the load they would immediately process its awake and start calls. All those issues are only perpetuated by the fact that there was a need to have multiple scenes loaded at the same time as well as having active scene load new active scene while still having behavior happening within it due to the nature of async loading.
To make the matters worse, scene's can only be refereed to by their buildID or name string. I have tried to set up a system that would allow to use scene ".unity" object to read its ".name" property in order to allow for easier organization and handling but using this property while working perfectly fine in editor caused nullref errors in the built version of the game forcing me to scrap it and hard-code scene names directly within the script.
My solution to this mess was to set up a system that would utilize a script instance connected to the object in "DontDestroyOnLoad" scene (so its always active) to set up a series of carefully ordered coroutines that would first load a new scene, upon the load finish wait for the end of the frame of the current active scene (to avoid any scene change reference issues), set the newly loaded scene to be active and only then unload the old scene. This solved the issue but had a side effect of forcing me avoid using Awake() call in any of the objects. However, I was able to substitute it with "sceneLoaded" event delegate which required some more boilerplate code but achieved pretty much the same effect.
UnityEngine.Object faulty handle reference
This one is partially connected to the scene loading issue as well. Because Unity's input management had not changed since Unity 4 version and was inherently faulty when it came to multiple controller input, we have been forced on relying on ReWired editor extension we purchased from asset store which in turn introduced few more interface layers into the existing code base (but solved so many other issues so it was worth it).
We have ended up designing a system that relied on using event delegates connected to a non-monobehavior class that sent input information to each connected player based on the scene that was active at the time.
While this method worked just fine for the most part, at the point where I started working on multiple scene streaming this system produced quite a weird bug where if the gameloop would work just fine for its initial iteration but would throw a c# object null reference error on its second iteration.
After much debugging, rewrites and googling I was able to pin down the issue: due to how unloading was handled, some of the scripts never unsubscribed from aforementioned input event delegates and thus after being unloaded had reference to them within Unity's core code related to the internal event system specifically while everything else was destroyed by a garbage collector. Fortunately, modifying the scene loading process to wait until the end of the frame solved the issue altogether.