Longtime residents felt the proposed nudity ban ended up being as a result of demographic shifts