Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add bulk writer #396

Merged
merged 5 commits into from Aug 11, 2021
Merged

feat: add bulk writer #396

merged 5 commits into from Aug 11, 2021

Commits on Aug 10, 2021

  1. feat: bulk writer 555 rate_limiter (#368)

    * added 555 throttle utility
    
    * Update google/cloud/firestore_v1/throttle.py
    
    Co-authored-by: Tres Seaver <tseaver@palladion.com>
    
    * added ability to request a number of tokens
    
    * replaced Callable now parameter with module function
    
    * updated tests
    
    * renamed throttle -> ramp up
    
    * improved docstrings
    
    * linting
    
    * fixed test coverage
    
    * rename to RateLimiter and defer clock to first op
    
    * linting
    
    Co-authored-by: Tres Seaver <tseaver@palladion.com>
    craiglabenz and tseaver committed Aug 10, 2021
    Copy the full SHA
    5a0ad83 View commit details
    Browse the repository at this point in the history
  2. feat: added new batch class for BulkWriter (#397)

    * feat: added new batch class for BulkWriter
    
    * updated docstring to use less colloquial language
    craiglabenz committed Aug 10, 2021
    Copy the full SHA
    783d9e3 View commit details
    Browse the repository at this point in the history
  3. feat: BulkWriter implementation (#384)

    * feat: added `write` method to batch classes
    
    * added docstrings to all 3 batch classes
    
    instead of just the base
    
    * updated batch classes to remove control flag
    
    now branches logic via subclasses
    
    * fixed broken tests off abstract class
    
    * fixed docstring
    
    * refactored BulkWriteBatch
    
    this commit increases the distance between WriteBatch and BulkWriteBatch
    
    * began adding [Async]BulkWriter
    
    * continued implementation
    
    * working impl or BW
    
    * tidied up BW impl
    
    * beginning of unit tests for BW
    
    * fixed merge problem
    
    * initial set of BW unit tests
    
    * refactored bulkwriter sending mechanism
    
    now consumes off the queue and schedules on the main thread, only going async to actually send
    
    * final CI touch ups
    
    * 🦉 Updates from OwlBot
    
    See https://github.com/googleapis/repo-automation-bots/blob/master/packages/owl-bot/README.md
    
    * 🦉 Updates from OwlBot
    
    See https://github.com/googleapis/repo-automation-bots/blob/master/packages/owl-bot/README.md
    
    * moved BulkWriter parameters to options format
    
    * rebased off master
    
    * test fixes
    
    Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
    craiglabenz and gcf-owl-bot[bot] committed Aug 10, 2021
    Copy the full SHA
    70e255d View commit details
    Browse the repository at this point in the history
  4. feat: add retry support for BulkWriter errors (#413)

    * parent 0176cc7
    author Craig Labenz <craig.labenz@gmail.com> 1623693904 -0700
    committer Craig Labenz <craig.labenz@gmail.com> 1628617523 -0400
    
    feat: add retries to bulk-writer
    
    * fixed rebase error
    craiglabenz committed Aug 10, 2021
    Copy the full SHA
    510f15b View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2021

  1. Copy the full SHA
    8956f42 View commit details
    Browse the repository at this point in the history