Yes, removing a user from a site doesn't delete their account - they stil remain a user on Voice, though they are no longer a member of your site.
You can either invite them to join your site using the link on the members page in the admin pages of your site, or if your membership policy allows it, there will be a "join this site" link at the top of your site's homepage which they can click on in order to rejoin the site.
Thanks
Joe - Voice Admin