TLDR: A new study has found that popular Large Language Models (LLMs) consistently suggest lower salary expectations for women compared to men, even when presented with identical qualifications. This bias, observed across various AI models like ChatGPT, Claude, and Llama, highlights concerns about AI perpetuating existing gender inequalities in the workplace.
A recent study conducted by researchers at the Technical University of Würzburg-Schweinfurt in Germany has brought to light a significant gender bias embedded within widely used Large Language Models (LLMs), including prominent AI chatbots such as ChatGPT, Claude, Qwen, Llama, and Mixtral. The findings indicate that these AI tools consistently advise women to seek lower salaries than men, even when both genders are presented with identical professional profiles, including job title, location, education, and experience. This alarming discovery underscores a critical issue of bias in AI models that could inadvertently perpetuate and exacerbate gender inequality in the professional sphere.
The research involved testing five popular LLMs by providing them with user profiles that differed only in gender. When asked to recommend a starting salary for a job interview, the models consistently suggested lower figures for female applicants. For instance, in one striking example, ChatGPT’s o3 model recommended a female job applicant request a salary of $280,000, while the same prompt for a male applicant resulted in a suggestion of $400,000 – a substantial difference of $120,000 annually. Similar disparities were observed across various industries, with the most pronounced pay gaps appearing in fields like law and medicine, followed by business administration and engineering. Only in the social sciences did the models offer comparable advice for men and women.
Out of 400 gender-based comparisons conducted by the researchers, over 27% showed statistically significant differences in pay advice. This suggests that the bias is not merely an isolated glitch but is deeply ingrained within the models, likely stemming from the vast datasets they were trained on, which may reflect historical wage gaps and societal gender stereotypes. Ivan Yamshchikov, a professor of AI and robotics at the Technical University of Würzburg-Schweinfurt and co-author of the study, noted that a mere two-letter change in prompts—from “he” to “she”—could lead to such a significant divergence in salary advice.
Beyond salary recommendations, the study also revealed that the AI models offered different career advice, goal-setting tips, and behavioral suggestions based on the user’s gender. Women were often encouraged to be more cautious and agreeable, while male users received advice that leaned towards assertiveness and confidence. This further highlights how AI systems, if not carefully designed and trained, can reinforce existing societal biases rather than promoting equitable outcomes.
Also Read:
- AI Chatbots Found Spreading Dangerous Health Misinformation, Study Reveals
- AI Expertise Commands Significant Salary Premium, Lightcast Report Reveals
The implications of these findings are profound, especially as individuals increasingly rely on AI tools for career guidance, resume drafting, and negotiation coaching. If AI systems continue to perpetuate these biases, they risk entrenching systemic inequality and further disadvantaging women in the workplace. Experts emphasize that addressing this issue requires not only policy changes but also a fundamental re-evaluation of how AI tools are developed and trained, ensuring that the datasets feeding these models are free from outdated or biased norms to prevent a feedback loop of discrimination.
