The Rise and Fall of “Vanilla” Labeling Challenges