The reason you see the upsell in the old offer that you thought you had deleted is because it is still attached to it somewhere in your complex funnel.  To resolve this without recreating the membership, go into each product, click on the Edit Upsells link, and then click the save button.  This will ensure that if there were "orphaned" IDs still saved in the database from deleted products, it will remove them from the upsell sequence.