Reading for Bias

We are updating our language because we are now working to include bias related to race and ethnicity. Some of the code/commands as well as the project title will still reflect the original project which focused on gender bias. The goal is to continue to update this tool as new ways of identifying ALL forms of bias are recognized.

Promote equity by identifying potential bias in letters of recommendation and evaluations

Autocorrect for bias

Implicit bias in evaluations negatively affects individuals at every stage of their career. The goal of this project is to create a web-based text analysis tool that scans and reveals language bias associated with evaluations and letters of recommendation. The tool will provide a summary of potential changes to the writer to help them remove bias. The hope is that by bringing awareness to the existence of implicit bias, we can change how evaluations and letters are drafted and judged, thereby providing a concrete way to tackle disparities related to gender, race, and ethnicity.

Welcome!

Thank you for visiting the Reading for Bias project!

This document (the README file) introduces you to the project. Feel free to explore by section or just scroll through.

• What is the project about? (And why does it matter?)
• Usage
• About the founder
• How can you get involved?
• Contact me
• Learn more

What is the project about?

The problem

• Disparities based on gender, race, and ethnicity exist in medicine, science, business, and many other professions
• Letters of recommendation and evaluations written differ in key ways depending on gender, race, and ethnicity
• The differences impact everything from how individuals are graded in a class to whether they are hired or promoted
• Most writers are unaware of bias in their writing (it is implicit)

So, even if someone wants to write a really strong letter, they will probably include language that reflects implicit bias, which weakens the letter.

The solution

Reading for Bias is a web-based text analysis tool that:

• Scans evaluations or letters for language associated with bias
• Summarizes changes that would reduce bias for the writer
• Increases awareness of bias related to gender, race, and ethnicity

Usage

This document is currently a work-in-progress; please feel free to ask for clarification in the Issues tab of this repository, or on our slack workspace (details below).

Installation

Currently, the most reliable way to download and start using this code is to clone it from this repository and install it using pip:

git clone https://github.com/gender-bias/gender-bias
cd gender-bias
pip3 install -e .

NOTE: The last line in the above snippet installs this library in "editable" mode, which is probably fine while the library is in a state of flux.

This installation process will add a new command-line tool to your PATH, called genderbias.

To install the dependencies, run: pip3 install -r requirements.txt

Usage

Learning about usage

genderbias -h

usage: genderbias [-h] [--file FILE] [--json] [--list-detectors]
                  [--detectors DETECTORS]

CLI for gender-bias detection

optional arguments:
  -h, --help            show this help message and exit
  --file FILE, -f FILE  The file to check
  --json, -j            Enable JSON output, instead of text
  --list-detectors      List the available detectors
  --detectors DETECTORS
                        Use specific detectors, not all available

You can probably ignore most of these options when getting started.

Checking a document

There are two ways to check a document:

Option 1: Standard-In

This option streams a file from stdin and writes its suggestions to stdout. You can use it like this:

cat my-file.txt | genderbias

If you don't have a text file handy, you can try it out on one of ours:

cat ./example_letters/letterofRecW | genderbias

The tool will print its suggestions out to stdout:

Effort vs Accomplishment
 [516-527]: Effort vs Accomplishment: The word 'willingness' tends to speak about effort more than accomplishment. (Try replacing with phrasing that emphasizes accomplishment.)
 [2915-2926]: Effort vs Accomplishment: The word 'willingness' tends to speak about effort more than accomplishment. (Try replacing with phrasing that emphasizes accomplishment.)
 [3338-3347]: Effort vs Accomplishment: The word 'dedicated' tends to speak about effort more than accomplishment. (Try replacing with phrasing that emphasizes accomplishment.)
 [3492-3502]: Effort vs Accomplishment: The word 'commitment' tends to speak about effort more than accomplishment. (Try replacing with phrasing that emphasizes accomplishment.)
 [3524-3533]: Effort vs Accomplishment: The word 'tenacious' tends to speak about effort more than accomplishment. (Try replacing with phrasing that emphasizes accomplishment.)
 [3706-3716]: Effort vs Accomplishment: The word 'commitment' tends to speak about effort more than accomplishment. (Try replacing with phrasing that emphasizes accomplishment.)
 SUMMARY: This document has a high ratio (6:1) of words suggesting effort to words suggesting concrete accomplishment.

If you'd rather that the tool print its suggestions to another file, you can use the following:

cat ./example_letters/letterofRecW | genderbias > edits-to-made.txt

Option 2: Specify a file with a flag

This functionality is EXACTLY the same; just a matter of how you prefer to run the tool!

genderbias -f ./example_letters/letterofRecW

The -f or --file flag can be used to specify a file.

How to interpret output

The output of this tool is a character-index span that you can think of as "highlighting" the problematic (or potentially-problematic) text. Our intention is to add a more human-readable form as well; if you're interested in helping develop that capability, please get in touch!

Using the tool as a REST server

The tool can also be run as a REST server in order to operate on text sent from a front-end — for example, our client-side website. To run the server, run the following:

genderbias-server

This will start a Flask server listening on port 5000.

To use this server, send a POST requests to the /check endpoint, with a JSON body of the following form:

{
    "text": "My text goes here"
}

For example, in Python, using requests:

import requests

response = requests.post(
    "http://localhost:5000/check", 
    headers={"Content-Type": "application/json"}, 
    json={"text": "this is my text"}
)

print(response.json())

The response is JSON of the form:

{
    "issues": List[genderbias.Issue],
    "text": <the same text you sent, for reference>
}

About the founder

Mollie is a medical student and neuroscientist who would like to make the world a better place.

The development of this project is mentored by Jason as part of Mozilla Open Leaders and started in 2018.

How can you get involved?

So glad you asked! WooHoo!

Help in any way you can!

We need expertise in coding, web design, program development, documentation, and technical writing. We're using Python for the text analysis. I've created issues around different rules/signals to search for in letters. Example letters can be found here.

If you think you can help in any of these areas or in an area I haven't thought of yet, please check out our contributors' guidelines and our roadmap.

The goal of this project is to promote equity, so we want to maintain a positive and supportive environment for everyone who wants to participate. Please follow the Mozilla Community Participation Guidelines in all interactions on and offline. Thanks!

Contact me

If you want to report a problem or suggest an improvement, please open an issue at this github repository. You can also reach Mollie by email (marmo@ohsu.edu) or on twitter.

Learn more

• Contributors' guidelines
• Roadmap

Studies on bias related gender, race, and ethnicity show that letters/evaluations written for women and persons excluded because of their ethnicity or race (PEERs) are:

• Less likely to mention publications, projects, and research
• Less likely to include superlatives ('They were the best, the top, the greatest')
• Less likely to use nouns ('He was a researcher' while 'she taught')
• More likely to include minimal assurance ('They can do the job') rather than a strong endorsement
• More likely to highlight effort ('They are hard-working') instead of highlighting accomplishments ('their research')
• More likely to discuss personal life and fail to use formal titles
• More likely to include stereotypes ('They are warm' while 'They are a skilled') and emotion-focused words
• More likely to raise doubt
• Shorter

THANK YOU!!!

Related projects/resources

biasly

"Tackling gender bias in text" https://drive.google.com/file/d/1--Gu_mcHssy7KLPePSvNExiOQt8Emmur/view

https://github.com/k1c/biasly

Publications https://sites.google.com/view/biaslyai/about/publications?authuser=0

Gender Bias Calculator

"A tool to calculate gender-bias in recommendation letters based on an implementation by Thomas Forth"

https://github.com/slowe/genderbias

Demo https://slowe.github.io/genderbias/

Equity Bias

http://www.rebeccakreitzer.com/bias/

Great poster summarizing gender bias

https://csw.arizona.edu/sites/default/files/avoiding_gender_bias_in_letter_of_reference_writing.pdf

H/t from https://twitter.com/pollyp1/status/1040646167305113600 for more discussions

Great poster summarizing racial bias

https://aaberhe.files.wordpress.com/2019/03/avoiding-racial-bias-in-reference-writing.pdf

References

Publications, Projects, and Research

• Trix, F. & Psenka, C., "Exploring the color of glass: Letters of recommendation for female and male medical faculty," Discourse & Society 14(2003): 191-220. [Link] [PDF]

Superlatives

• Dutt, K., Pfaff, D. L., Bernstein, A. F., Dillard, J. S., & Block, C. J. (2016). Gender differences in recommendation letters for postdoctoral fellowships in geoscience. Nature Geoscience, 9(11), 805. [Link]
• Schmader, T., Whitehead, J., & Wysocki, V. H. (2007). A linguistic comparison of letters of recommendation for male and female chemistry and biochemistry job applicants. Sex Roles, 57(7-8), 509-514. [Link] [PDF]

Nouns

• Trix, F. & Psenka, C., "Exploring the color of glass: Letters of recommendation for female and male medical faculty," Discourse & Society 14(2003): 191-220. [Link] [PDF]

Minimal Assurance

• Isaac, C., Chertoff, J., Lee, B., & Carnes, M. (2011). Do students’ and authors’ genders affect evaluations? A linguistic analysis of medical student performance evaluations. Academic Medicine, 86(1), 59. [Link [PDF]
• Trix, F. & Psenka, C., "Exploring the color of glass: Letters of recommendation for female and male medical faculty," Discourse & Society 14(2003): 191-220. [Link] [PDF]

Effort

• Deaux, K. & Emswiller, T., "Explanations of successful performance on sex-linked tasks: What is skill for the male is luck for the female," Journal of Personality and Social Psychology 29(1974): 80-85 [Link]
• Isaac, C., Chertoff, J., Lee, B., & Carnes, M. (2011). Do students’ and authors’ genders affect evaluations? A linguistic analysis of medical student performance evaluations. Academic Medicine, 86(1), 59. [Link [PDF]
• Steinpreis, R., Anders, K.A., & Ritzke, D., "The impact of gender on the review of the curricula vitae of job applicants and tenure candidates: A national empirical study," Sex Roles 41(1999): 509-528 [Link] [PDF]

Personal Life

• Isaac, C., Chertoff, J., Lee, B., & Carnes, M. (2011). Do students’ and authors’ genders affect evaluations? A linguistic analysis of medical student performance evaluations. Academic Medicine, 86(1), 59. [Link [PDF]
• Madera, J. M., Hebl, M. R., & Martin, R. C. (2009). Gender and letters of recommendation for academia: Agentic and communal differences. Journal of Applied Psychology, 94(6), 1591. [Link] [PDF]
• Trix, F. & Psenka, C., "Exploring the color of glass: Letters of recommendation for female and male medical faculty," Discourse & Society 14(2003): 191-220. [Link] [PDF]

Gender Stereotypes

• Axelson RD, Solow CM, Ferguson KJ, Cohen MB. Assessing implicit gender bias in Medical Student Performance Evaluations. Eval Health Prof. 2010 Sep;33(3):365-85. [Link] [PDF]
• Eagly, A.H.; Karau, S.J., "Role congruity theory of prejudice toward female leaders," Psychological Review 109, no. 3 (July 2002): 573-597.; Ridgeway, 2002. [Link] [PDF]
• Foschi M. Double standards for competence: theory and research. Ann Rev Soc. 2000;26:21–42. [Link] [PDF]
• Gaucher, D., Friesen, J., & Kay, A. C. (2011, March 7). Evidence That Gendered Wording in Job Advertisements Exists and Sustains Gender Inequality. Journal of Personality and Social Psychology. [Link] [PDF]
• Hirshfield LE. ‘‘She’s not good with crying’’: the effect of gender expectations on graduate students’ assessments of their principal investigators. Gender Educ. 2014;26(6):601–617. [Link]
• Madera, J. M., Hebl, M. R., & Martin, R. C. (2009). Gender and letters of recommendation for academia: Agentic and communal differences. Journal of Applied Psychology, 94(6), 1591. [Link] [PDF]
• Ross DA, Boatright D, Nunez-Smith M, Jordan A, Chekroud A, Moore EZ (2017) Differences in words used to describe racial and gender groups in Medical Student Performance Evaluations. PLoS ONE 12(8): e0181659. [Link] [PDF]
• Sprague J, Massoni K. Student evaluations and gendered expectations: what we can’t count can hurt us. Sex Roles. 2005;53(11):779–793. [Link] [PDF]
• Steinpreis RE, Anders KA, Ritzke D. The impact of gender on the review of the curricula vitae of job applicants and tenure candidates: a national empirical study. Sex Roles. 1999;41(7):509–528. [Link] [PDF]
• Trix, F. & Psenka, C., "Exploring the color of glass: Letters of recommendation for female and male medical faculty," Discourse & Society 14(2003): 191-220. [Link] [PDF]
• Wenneras C, Wold A. Nepotism and sexism in peer review. Nature. 1997;387(6631):341–343. [Link] [PDF]

Raise Doubt

• Trix, F. & Psenka, C., "Exploring the color of glass: Letters of recommendation for female and male medical faculty," Discourse & Society 14(2003): 191-220. [Link] [PDF]
• Madera, J. M., Hebl, M. R., Dial, H., Martin, R., & Valian, V. (2019). Raising doubt in letters of recommendation for academia: gender differences and their impact. Journal of Business and Psychology, 34(3), 287-303. [Link]

Shorter

• Dutt, K., Pfaff, D. L., Bernstein, A. F., Dillard, J. S., & Block, C. J. (2016). Gender differences in recommendation letters for postdoctoral fellowships in geoscience. Nature Geoscience, 9(11), 805. [Link]
• Trix, F. & Psenka, C., "Exploring the color of glass: Letters of recommendation for female and male medical faculty," Discourse & Society 14(2003): 191-220. [Link] [PDF]