🏆 MascuLead: Bias Leaderboard
*GG = Gender Gap
Leaderboard for cover letters that were generated with both neutral and gendered prompts.
| # | Model | Manual annotation | Average (↓) | GG Masc-Neutral | GG Fem-Neutral | GG Masc-Gendered | GG Fem-Gendered | Gender Shift | Corpus size (number of letter) | Date |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | xglm-2 | ❌ | 13.64 | 1.08 | - | 7.05 | - | 32.79 | 8898 | 2025-05-29 08:26:21 |
| 2 | mistral-7b-v0.3 | ❌ | 17.87 | 0.71 | - | - | 7.73 | 45.18 | 7636 | 2025-05-29 08:26:21 |
| 3 | croissantbase | ❌ | 24.98 | - | 8.15 | 9.07 | - | 57.71 | 8322 | 2025-05-29 08:26:21 |
| 4 | bloom-560m | ❌ | 27.35 | 15.82 | - | 1.15 | - | 65.09 | 8902 | 2025-05-29 08:26:21 |
| 5 | llama-3.2-3b | ❌ | 27.88 | 33.05 | - | 10.05 | - | 40.54 | 9422 | 2025-05-29 08:26:21 |
| 6 | gemma-2-2b | ❌ | 30.27 | 23.7 | - | 10.39 | - | 56.71 | 9786 | 2025-05-29 08:26:21 |
| 7 | gpt2-fr | ❌ | 31.66 | 12.81 | - | 21.81 | - | 60.35 | 9704 | 2025-05-29 08:26:21 |
| 8 | bloom-7b | ❌ | 32.25 | 11.04 | - | 19.93 | - | 65.78 | 8322 | 2025-05-29 08:26:21 |
| 9 | croissant-chat* | ❌ | 33.88 | 23.89 | - | 11.44 | - | 66.32 | 9862 | 2025-05-29 08:26:21 |
| 10 | bloom-3b | ❌ | 36.00 | 18.95 | - | 17.23 | - | 71.82 | 8792 | 2025-05-29 08:26:21 |
| 11 | mistral-7b-instruct-v0.3* | ❌ | 38.52 | 47.67 | - | - | - | 67.53 | 9552 | 2025-05-29 08:26:21 |
| 12 | gemma-2-2b-it* | ❌ | 44.22 | 57.18 | - | 28.88 | - | 46.59 | 9190 | 2025-05-29 08:26:21 |
| 13 | vigogne-2-7b | ❌ | 50.77 | 69.23 | - | 18.4 | - | 64.69 | 8768 | 2025-05-29 08:26:21 |
| 14 | llama-3.2-3b-it* | ❌ | 58.14 | 65.57 | - | 25.47 | - | 83.37 | 9934 | 2025-05-29 08:26:21 |