CS 6983 - Bias / Ethics Implications in AI | Enhancing Healthcare Decision-Making through Comprehensive Multimodal Data Fusion

This study investigates the potential benefits of integrating chest X-ray images into predic- tive models trained solely on tabular electronic health record (EHR) data to improve predic- tive accuracy for clinical decision-making tasks. Leveraging the Medical Information Mart for Intensive Care - EYE (MIMIC-EYE) dataset, we analyze the performance of multimodal data fusion models in binary classification tasks re- lated to healthcare outcomes. Additionally, we explore the variations in model performance across diverse demographic groups, including age, gender, and race/ethnicity. Our findings shed light on the efficacy of integrating mul- timodal data and highlight the importance of considering demographic factors in healthcare predictive modeling.

Project Directory Structure

  ├── Project Root Directory                   # Root Folder
    ├── models/                                # Folder contains the models used for the project.
        ├── config                             # Configurations directory
        ├── evaluations                        # Folder consists of the predictions generated by the model and the code for computing the performance metrics values.
        ├── logs                               # Folder consists of the model training logs for all the different demographics and experiments.
        ├── MedBERT                            # Folder consists of the MedBERT encoder weights and model code.
        ├── weights                            # Folder consists of the trained model weights
        ├── new_training.py                    # Python file used for model training.
        ├── new_inference.py                   # Python file used for model inference.
    ├── preprocessing/                         # Folder contains the source code for generating Pseudonotes from the EHR tabular data
    ├── rgrg/                                  # Folder contains the source code of the chest x-ray radiology report generation code
    ├── .gitignore                             # Python gitgnore file
    ├── requirements.txt                       # Python requirements file which specifies the library dependencies.
    ├── README.md                              # Project README file.

Installation

1. Clone the repository into a folder and open a terminal of your choice.

2. Create a virtual environment

python -m venv .venv (or) python3 -m venv .venv

3. Activate the virtual environment

   On Windows run

   .venv\Scripts\activate.bat
   

   On Linux and MacOs run

   source .venv/bin/activate
   

4. Install the dependencies for the project in the virtual environment

pip install -r requirements.txt

5. Then run the following command

python models/new_inference.py -c config/<required_config>.json

Team Members

The project is undertaken by Kishore Sampath, Ayaazuddin Mohammad as their class project for the course CS 6983 - Bias / Ethics Implications in AI. Contact the authors for more information about the project.

Acknowledgments:

We extend our heartfelt appreciation to Professor Malihe Alikhani and Dr. Resmi Ramachandrapillai for their invaluable support and guidance throughout the duration of this project. Under their mentorship, we have acquired substantial knowledge and skills in conducting research, which have greatly contributed to the success of this endeavor. Additionally, we express our gratitude to Northeastern University for graciously providing access to their High Performance Computing resources, which were instrumental in training and finetuning the machine learning models utilized in this study. The codebase has been inspired from the work done by MEME and RGRG.