@Trev settings -> authorized followers might be a little easier for at least removing follows to you