You can do this from the subscriber list. From the left menu, click Manage > Membership/Product/Podcast, etc. In the overview menu for your Podcast, in the Members/Subscribers area, click "View Active" or "View All".
There you'll see your subscriber list. You can filter this list to show between 20 to 1000 records per page. Set that to include the scale of subscriber records you want to see. Note that there is a checkbox by each name, and that there is a Check/Uncheck All link at the bottom of the list.
Check all that you want to remove, then at the bottom, you'll see a "Do the following to checked:" option.
Select either "Unsubscribed" or "Deleted" and then click the "Update Status" button. This will apply the selected status to the checked subscribers. Note that selecting "Deleted" will permanently delete a user, which cannot be undone. Unsubscribed removes access to the content but does not delete the subscriber record.