You don't need to buy and manage property if you just want a place to drink margaritas in February. You can accomplish the same thing in a rental, B&B or hotel suit yourself.
Besides, if you want to go there for a few months a year what are you going to do, kick out the tenants every 6 months? Good luck with that.
I'm not a huge fan of swiss cheese.