Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Type hints and changes to class inheritance #76

Open
wants to merge 33 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
7f0fd3f
verion number updates
bmos Feb 4, 2024
3ff7ba7
cleanup linter errors in pytests
bmos Feb 4, 2024
c14b88a
consistent formatting at 160 line length
bmos Feb 4, 2024
71f1816
add type hints
bmos Feb 4, 2024
f75431d
don't use setattr with static attribute
bmos Feb 4, 2024
eb1d394
reduce complexity of _start_chrome_browser
bmos Feb 4, 2024
3b2214b
define stacklevel
bmos Feb 4, 2024
fce33c1
fix typo in error message
bmos Feb 4, 2024
710c6c4
ensure error message always has formatting string
bmos Feb 4, 2024
c60bf47
RequestiumResponse inherit requests.Response
bmos Feb 4, 2024
7d796bb
define RequestiumChrome as class inheriting from webdriver.Chrome
bmos Feb 4, 2024
1b9531d
don't use mutable default args
bmos Feb 4, 2024
58c6e12
don't use except Exception
bmos Feb 4, 2024
798298a
move module docstring to init
bmos Feb 4, 2024
ecdd420
include By in init
bmos Feb 4, 2024
80cd6ce
split requestium into multiple files
bmos Feb 5, 2024
3253286
add dependabot pip and github-actions
bmos Feb 8, 2024
8bdfb9d
add python linting workflow
bmos Feb 8, 2024
e158e53
add pytest workflow
bmos Feb 8, 2024
c336b5f
add license in tool.poetry
bmos Feb 8, 2024
d8c4a76
only lint on one OS
bmos Feb 8, 2024
ee52347
install dependencies in lint action
bmos Feb 8, 2024
43ca36c
add dependency to pytest action
bmos Feb 8, 2024
68ded78
Update README.md
bmos Feb 8, 2024
725a568
disable pytest emoji
bmos Feb 8, 2024
b56be4b
disable pytest workflow (headless issues)
bmos Feb 8, 2024
6deb922
re-enable pytest workflow
bmos Feb 8, 2024
319a197
don't pytest action on ubuntu
bmos Feb 8, 2024
c3a0cca
use union for 3.9
bmos Feb 8, 2024
c6e1ec7
update version number
bmos Feb 14, 2024
df3db34
uv package installer/resolver for speed
bmos Feb 29, 2024
9561710
fix Missing return statement
bmos Mar 1, 2024
58106a2
improved typing
bmos Mar 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
19 changes: 19 additions & 0 deletions .editorconfig
@@ -0,0 +1,19 @@
# EditorConfig is awesome: https://EditorConfig.org

root = true

[*]
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true

[*.py]
charset = utf-8
indent_style = space
indent_size = 4
max_line_length = 160

[*.yml]
indent_style = space
indent_size = 2
trim_trailing_whitespace = True
16 changes: 16 additions & 0 deletions .github/dependabot.yml
@@ -0,0 +1,16 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates

version: 2
updates:
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"

- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
59 changes: 59 additions & 0 deletions .github/workflows/lint-python.yml
@@ -0,0 +1,59 @@
name: Ruff

on:
push:
paths:
- "**.py"
- .github/workflows/lint-python.yml
- pyproject.toml
pull_request:
paths:
- "**.py"
- pyproject.toml
workflow_dispatch:

jobs:
ruff:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version:
- "3.12"
- "3.11"
- "3.10"
- "3.9"

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: pip

- name: Install uv and create venv
run: |
pip install -U pip uv
uv venv

- name: Enable macOS/Linux venv
if: runner.os != 'Windows'
run: |
source .venv/bin/activate
echo $PATH >> $GITHUB_PATH

- name: Enable Windows venv
if: runner.os == 'Windows'
run: |
.venv\Scripts\activate
echo $env:path >> $ENV:GITHUB_PATH

- name: Install dependencies
run: |
uv pip install -e . ruff ydiff

- name: Lint the code with ruff
run: |
ruff check $(git ls-files '*.py') --diff | ydiff -s
64 changes: 64 additions & 0 deletions .github/workflows/pytest.yml
@@ -0,0 +1,64 @@
name: Pytest

on:
push:
paths:
- "**.py"
- .github/workflows/pytest.yml
- pyproject.toml
pull_request:
paths:
- "**.py"
- pyproject.toml
workflow_dispatch:

jobs:
pytest:
strategy:
fail-fast: false
matrix:
os:
# - "ubuntu-latest" # doesn't work without headless
- "windows-latest"
- "macos-latest"
python-version:
- "3.12"
- "3.11"
- "3.10"
- "3.9"
runs-on: ${{ matrix.os }}

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: pip

- name: Install uv and create venv
run: |
pip install -U pip uv
uv venv

- name: Enable macOS/Linux venv
if: runner.os != 'Windows'
run: |
source .venv/bin/activate
echo $PATH >> $GITHUB_PATH

- name: Enable Windows venv
if: runner.os == 'Windows'
run: |
.venv\Scripts\activate
echo $env:path >> $ENV:GITHUB_PATH

- name: Install dependencies
run: |
uv pip install -r requirements.txt
uv pip install -e . pytest pytest-cov

- name: Test with pytest
run: |
pytest --junitxml=junit/test-results.xml --cov=com --cov-report=xml --cov-report=html
3 changes: 3 additions & 0 deletions .gitignore
Expand Up @@ -100,6 +100,9 @@ ENV/
# mkdocs documentation
/site

# Poetry
poetry.lock

# mypy
.mypy_cache/

Expand Down
14 changes: 7 additions & 7 deletions .travis.yml
@@ -1,13 +1,13 @@
sudo: false
language: python
python:
- '2.7'
- '3.4'
- '3.5'
- '3.6'
- '3.9'
- '3.10'
- '3.11'
- '3.12'
install:
- pip install tox-travis
- wget -N https://chromedriver.storage.googleapis.com/99.0.4844.51/chromedriver_linux64.zip -P ~/
- wget -N https://chromedriver.storage.googleapis.com/100.0.4896.20/chromedriver_linux64.zip -P ~/
- unzip ~/chromedriver_linux64.zip -d ~/
- rm ~/chromedriver_linux64.zip
- sudo mv -f ~/chromedriver /usr/local/share/
Expand All @@ -24,7 +24,7 @@ deploy:
on:
all_branches: true
tags: false
python: 3.6
python: 3.12
# Real PyPI in tags (ie. GitHub releases)
- provider: pypi
distributions: sdist bdist_wheel --universal
Expand All @@ -33,6 +33,6 @@ deploy:
on:
branch: master
tags: true
python: 3.6
python: 3.12
addons:
chrome: stable
1 change: 0 additions & 1 deletion README.md
@@ -1,7 +1,6 @@
![Requestium](https://user-images.githubusercontent.com/14966348/32966130-8bb15b00-cbb7-11e7-9faf-85963ec5bd82.png)
========

[![Build Status](https://travis-ci.org/tryolabs/requestium.svg?branch=master)](https://travis-ci.org/tryolabs/requestium)
[![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)

Requestium is a Python library that merges the power of [Requests](https://github.com/requests/requests), [Selenium](https://github.com/SeleniumHQ/selenium), and [Parsel](https://github.com/scrapy/parsel) into a single integrated tool for automatizing web actions.
Expand Down
11 changes: 10 additions & 1 deletion pyproject.toml
Expand Up @@ -4,6 +4,7 @@ version = "0.3.0"
description = ""
authors = ["Joaquin Alori <joaquin@tryolabs.com>"]
readme = "README.md"
license = "BSD-3-Clause"

[tool.poetry.dependencies]
python = ">=3.9,<=3.13"
Expand All @@ -13,7 +14,15 @@ selenium = "^4.15.2"
tldextract = "^5.1.1"

[tool.poetry.group.dev.dependencies]
pytest = "^7.4.3"
pytest = "^7.4.4"

[tool.ruff]
line-length = 160

[tool.ruff.lint]
select = ["E4", "E7", "E9", "F"]
extend-select = ["B", "W", "C"]
ignore = ["C400", "C401"]

[build-system]
requires = ["poetry-core"]
Expand Down
5 changes: 4 additions & 1 deletion requestium/__init__.py
@@ -1,5 +1,8 @@
"""Requestium is a Python library that merges the power of Requests, Selenium, and Parsel into a single integrated tool for automatizing web actions."""

from selenium.common import exceptions # noqa: F401
from selenium.webdriver.common.by import By # noqa: F401
from selenium.webdriver.common.keys import Keys # noqa: F401
from selenium.webdriver.support.ui import Select # noqa: F401

from .requestium import Session # noqa: F401
from .requestium_session import Session # noqa: F401