You can remove Users from your Portfolio via the Manage Users app if you have Admin access.
Entering the App shows all the users with access to your Portfolio, along with their level of access.
If a User is no longer required to have access to the Portfolio, click on their name and choose 'Remove User'.