Skip to content

Commit

Permalink
Merge branch 'release/3.2.0'
Browse files Browse the repository at this point in the history
  • Loading branch information
dermatologist committed Jan 20, 2023
2 parents b43b394 + 07d5c3c commit 483f2b1
Show file tree
Hide file tree
Showing 11 changed files with 55 additions and 40 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/bump.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:

steps:
- name: Checkout Latest Commit
uses: actions/checkout@v2
uses: actions/checkout@v3

- name: Generate changelog
uses: charmixer/auto-changelog-action@v1
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v2.3.4
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v2.2.2
uses: actions/setup-python@v2.3.1
with:
python-version: '3.8'
- name: Install dependencies
Expand All @@ -22,6 +22,7 @@ jobs:
- name: Create docs
run: |
make -C docs/ html
cp docs/_config.yml docs/_build/html/_config.yml
- name: Deploy Docs 🚀
uses: JamesIves/github-pages-deploy-action@4.1.5
with:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v2.3.4
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v2.2.2
uses: actions/setup-python@v2.3.1
with:
python-version: '3.8'
- name: Install dependencies
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ jobs:
python-version: [3.7]

steps:
- uses: actions/checkout@v2.3.4
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2.2.2
uses: actions/setup-python@v2.3.1
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/tox.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ jobs:
python-version: [3.8]

steps:
- uses: actions/checkout@v2.3.4
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2.2.2
uses: actions/setup-python@v2.3.1
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
Expand Down
4 changes: 2 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Changelog

## [Unreleased](https://github.com/dermatologist/pyomop/tree/HEAD)
## [3.1.0](https://github.com/dermatologist/pyomop/tree/3.1.0) (2021-09-17)

[Full Changelog](https://github.com/dermatologist/pyomop/compare/3.0.0...HEAD)
[Full Changelog](https://github.com/dermatologist/pyomop/compare/3.0.0...3.1.0)

**Closed issues:**

Expand Down
16 changes: 14 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,32 @@
# :notebook: pyomop
# pyomop

![Libraries.io SourceRank](https://img.shields.io/librariesio/sourcerank/pypi/pyomop)
[![forthebadge made-with-python](http://ForTheBadge.com/images/badges/made-with-python.svg)](https://www.python.org/)
[![PyPI download total](https://img.shields.io/pypi/dm/pyomop.svg)](https://pypi.python.org/pypi/pyomop/)
[![Build](https://github.com/dermatologist/pyomop/workflows/Python%20Test/badge.svg)](https://nuchange.ca)

* *Inspired by [@jbadger3's](https://github.com/jbadger3) [inspectomop](https://github.com/jbadger3/inspectomop)*

### [Documentation](https://dermatologist.github.io/pyomop/)

## Description

The [OHSDI](https://www.ohdsi.org/) OMOP Common Data Model allows for the systematic analysis of healthcare observational databases. This is a python library to use the CDM v6 compliant databases using SQLAlchemy as the ORM. **pyomop** also supports converting query results to a pandas dataframe (see below) for use in machine learning pipelines. See some useful [SQL Queries here.](https://github.com/OHDSI/QueryLibrary)

## Installation
## Installation (stable)

```
pip install pyomop
```

## Installation (current)

* git clone this repository and:
```
pip install -e .
```

## Usage

```
Expand Down Expand Up @@ -91,6 +101,8 @@ pip install sqlalchemy==1.3.24
* SqLite
* More to follow..

## Give us a star ⭐️
If you find this project useful, give us a star. It helps others discover the project.

## Contributors

Expand Down
10 changes: 5 additions & 5 deletions dev-requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ charset-normalizer==2.0.6
# via requests
commonmark==0.9.1
# via recommonmark
coverage==5.5
coverage[toml]==5.5
# via pytest-cov
distlib==0.3.2
# via virtualenv
Expand Down Expand Up @@ -64,7 +64,7 @@ pytest==6.2.5
# via
# -r dev-requirements.in
# pytest-cov
pytest-cov==2.12.1
pytest-cov==3.0.0
# via -r dev-requirements.in
pytz==2021.1
# via
Expand Down Expand Up @@ -101,18 +101,18 @@ sphinxcontrib-serializinghtml==1.1.5
# via sphinx
toml==0.10.2
# via
# coverage
# pytest
# pytest-cov
# tox
tomli==1.2.1
# via setuptools-scm
tox==3.24.4
tox==3.24.5
# via -r dev-requirements.in
urllib3==1.26.6
# via requests
virtualenv==20.8.0
# via tox
wheel==0.37.0
wheel==0.37.1
# via -r dev-requirements.in

# The following packages are considered to be unsafe in a requirements file:
Expand Down
2 changes: 2 additions & 0 deletions docs/_config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
theme: jekyll-theme-leap-day
include: [_sources, _modules, _static]
8 changes: 4 additions & 4 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@
#
# pip-compile
#
click==8.0.1
click==8.0.3
# via pyomop (setup.py)
inflect==5.3.0
# via sqlacodegen
numpy==1.21.2
numpy==1.21.4
# via pandas
pandas==1.3.3
pandas==1.3.5
# via pyomop (setup.py)
psycopg2-binary==2.9.1
psycopg2-binary==2.9.3
# via pyomop (setup.py)
python-dateutil==2.8.2
# via pandas
Expand Down
36 changes: 18 additions & 18 deletions src/pyomop/vocabulary.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,45 +67,45 @@ def set_concept(self, concept_code, vocabulary_id=None):
def create_vocab(self, folder, sample=10):
if sample < 1000: # nrows=sample
try:
df = pd.read_csv(folder + '/DRUG_STRENGTH.csv', sep='\t', nrows=sample, error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/DRUG_STRENGTH.csv', sep='\t', nrows=sample, on_bad_lines='skip')
df.to_sql('drug_strength', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/CONCEPT.csv', sep='\t', nrows=sample, error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/CONCEPT.csv', sep='\t', nrows=sample, on_bad_lines='skip')
df.to_sql('concept', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/CONCEPT_RELATIONSHIP.csv', sep='\t', nrows=sample, error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/CONCEPT_RELATIONSHIP.csv', sep='\t', nrows=sample, on_bad_lines='skip')
df.to_sql('concept_relationship', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/CONCEPT_ANCESTOR.csv', sep='\t', nrows=sample, error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/CONCEPT_ANCESTOR.csv', sep='\t', nrows=sample, on_bad_lines='skip')
df.to_sql('concept_ancester', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/CONCEPT_SYNONYM.csv', sep='\t', nrows=sample, error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/CONCEPT_SYNONYM.csv', sep='\t', nrows=sample, on_bad_lines='skip')
df.to_sql('concept_synonym', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/VOCABULARY.csv', sep='\t', nrows=sample, error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/VOCABULARY.csv', sep='\t', nrows=sample, on_bad_lines='skip')
df.to_sql('vocabulary', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/RELATIONSHIP.csv', sep='\t', nrows=sample, error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/RELATIONSHIP.csv', sep='\t', nrows=sample, on_bad_lines='skip')
df.to_sql('relationship', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/CONCEPT_CLASS.csv', sep='\t', nrows=sample, error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/CONCEPT_CLASS.csv', sep='\t', nrows=sample, on_bad_lines='skip')
df.to_sql('concept_class', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/DOMAIN.csv', sep='\t', nrows=sample, error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/DOMAIN.csv', sep='\t', nrows=sample, on_bad_lines='skip')
df.to_sql('domain', con=self._engine, if_exists = 'replace')
except ValueError:
print("Oops! Could not write vocabulary")
else:
try:
df = pd.read_csv(folder + '/DRUG_STRENGTH.csv', sep='\t', error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/DRUG_STRENGTH.csv', sep='\t', on_bad_lines='skip')
df.to_sql('drug_strength', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/CONCEPT.csv', sep='\t', error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/CONCEPT.csv', sep='\t', on_bad_lines='skip')
df.to_sql('concept', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/CONCEPT_RELATIONSHIP.csv', sep='\t', error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/CONCEPT_RELATIONSHIP.csv', sep='\t', on_bad_lines='skip')
df.to_sql('concept_relationship', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/CONCEPT_ANCESTOR.csv', sep='\t', error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/CONCEPT_ANCESTOR.csv', sep='\t', on_bad_lines='skip')
df.to_sql('concept_ancester', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/CONCEPT_SYNONYM.csv', sep='\t', error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/CONCEPT_SYNONYM.csv', sep='\t', on_bad_lines='skip')
df.to_sql('concept_synonym', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/VOCABULARY.csv', sep='\t', error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/VOCABULARY.csv', sep='\t', on_bad_lines='skip')
df.to_sql('vocabulary', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/RELATIONSHIP.csv', sep='\t', error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/RELATIONSHIP.csv', sep='\t', on_bad_lines='skip')
df.to_sql('relationship', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/CONCEPT_CLASS.csv', sep='\t', error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/CONCEPT_CLASS.csv', sep='\t', on_bad_lines='skip')
df.to_sql('concept_class', con=self._engine, if_exists = 'replace')
df = pd.read_csv(folder + '/DOMAIN.csv', sep='\t', error_bad_lines=False, warn_bad_lines=True)
df = pd.read_csv(folder + '/DOMAIN.csv', sep='\t', on_bad_lines='skip')
df.to_sql('domain', con=self._engine, if_exists = 'replace')
except ValueError:
print("Oops! Could not write vocabulary")
Expand Down

0 comments on commit 483f2b1

Please sign in to comment.