Hesitant QLearner Cooperation Rate Heatmap Left: no noise | Right: 5% noise Hesitant QLearner Opponent Cooperation Rate Heatmap Left: no noise | Right: 5% noise Hesitant QLearner Mean Score Heatmap Left: no noise | Right: 5% noise Hesitant QLearner Mean Score Difference Heatmap Left: no noise | Right: 5% noise