Well I tried to force VoLTE using the code *#*#86583#*#* and then making sure is active in the SIM options, so far have not had to to restart the phone yet.
Update: Can confirm this did the trick for me 24h without restarting and still have sound on calls