Do you see a red light in the optical output?
Seems Roon is still running. Can you SSH into your computer, and then run:
sudo systemctl stop roonbridge
sudo systemctl disable roonbridge
Second step, we need to check first if it's a NAS issue (I don't think so, but let's try that anyway). To do this unmount the NAS share, and then copy a single file over to the the music shared folder. Now try and play that high res music, does it work well?
If it's still a problem. Try and use Squeezelite + LMS. Does it still have a problem?