What strange was, I did a dalvik and cache wipe before installing the update 4.20 and the play store was still there. I never had that with an update. But now with every restart the old market keeps returning. I will try it later with a wipe of the market cache. See what happens then.
A solution (got it from xda I think) that worked for me was to first delete phonesky.apk from /system/app. Then reinstall playstore and move the apk from /data/app to /system/app. I have rebooted several times after and have not seen it revert to an old version