Light maps not loading when a model is disabled

Bit of an odd problem, but my directional light appears to stop when I disable a certain model within my editor. Launching the scene doesn’t change the light.
This is the link to the project, if the dice model in the scene is disabled the table top turns black and cannot be seen unless you zoom the view back a certain distance.

Kind of found a fix, changed Distance on the light to a lower value and it seems to be working.

It was at 15 when I was having this problem, anything higher and it would get worse.