The disclosure of compensation peer groups is argued to provide shareholders with valuable information that can be used to scrutinize chief executive officer (CEO) compensation. However, research suggests that there are substantial incentives for executives and directors to bias the compensation peer group upward such that the CEO can extract additional rent. We leverage the idea that reciprocated peer nominations are unlikely to be biased to construct counterfactual peer groups that allow us to measure the bias of disclosed peer groups. Analyses of 11 years of comprehensive data on compensation peer groups demonstrate that the average firm uses an upwardly biased peer group. The size of the bias increases when incentives and opportunities to do so are more pronounced. Specifically, results show that bias is larger when financial targets are not met and when exercising discretion in the selection of peer firms is justifiable. More importantly, upward bias in compensation peer groups is highly predictive of an increase in CEO compensation, suggesting that there is a strong incentive for CEOs to strategically select peers. Finally, although average peer group bias has gone down in recent years, the predictive effect of bias on pay has gone up. These findings call into question the practical impact of recent efforts to introduce greater transparency into the process for determining executive compensation.