Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WhyLabsWriter refactor #1484

Merged
merged 39 commits into from May 13, 2024
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
c90eaee
WhyLogsWriter refactor
richard-rogers Mar 15, 2024
071aa9c
pre-commit
richard-rogers Mar 15, 2024
6f8c677
Integration test progress
richard-rogers Mar 16, 2024
3bef949
bug fixes
richard-rogers Mar 21, 2024
5bb1f7c
pre-commit
richard-rogers Mar 21, 2024
a0bd7c9
more fixes
richard-rogers Mar 28, 2024
4382e9f
oops print
richard-rogers Mar 28, 2024
544d35d
notebook fixes
richard-rogers Mar 29, 2024
02ad2a0
notebook fixes
richard-rogers Mar 29, 2024
039b658
pre-commit
richard-rogers Mar 30, 2024
6b63663
more transaction work
richard-rogers Apr 2, 2024
3a8f1fe
bug fix; transactional segmented batch upload
richard-rogers Apr 3, 2024
2c565b7
Update python/whylogs/api/writer/mlflow.py
richard-rogers Apr 5, 2024
51094d6
Update python/whylogs/api/writer/mlflow.py
richard-rogers Apr 5, 2024
eada9e9
update lockfile
richard-rogers Apr 5, 2024
5e133d7
notebook fix
richard-rogers Apr 5, 2024
a87ca2b
EstimationResult fixes
richard-rogers Apr 12, 2024
9d3f111
add WhyLabsClient::write_feature_weights() test
richard-rogers Apr 12, 2024
693a6b2
skip zip tests
Apr 22, 2024
f53e84f
pre-commit
richard-rogers Apr 22, 2024
2726a40
notebook update
richard-rogers Apr 22, 2024
da697aa
Merge branch 'mainline' into dev/richard/wlw
richard-rogers Apr 22, 2024
70d46c5
Update python/whylogs/core/feature_weights.py
richard-rogers Apr 25, 2024
6daf4d4
Merge branch 'mainline' into dev/richard/wlw
richard-rogers Apr 25, 2024
9088e02
transaction API updates
richard-rogers Apr 26, 2024
f86f8dd
Merge branch 'dev/richard/wlw' of github.com:whylabs/whylogs into dev…
richard-rogers Apr 26, 2024
55907da
segmented batch profiles vs transactions
Apr 26, 2024
f253434
Update Transaction_Examples.ipynb
richard-rogers Apr 26, 2024
d13bc5b
transaction updates
richard-rogers Apr 26, 2024
2b5b0e9
pre-commit
richard-rogers Apr 26, 2024
153e4f7
version bump
richard-rogers Apr 27, 2024
f90a6f1
Update smoketest.py
richard-rogers Apr 30, 2024
d4788e2
Merge branch 'mainline' into dev/richard/wlw
richard-rogers May 7, 2024
b750e05
fix segmented custom perf/output metric tagging
richard-rogers May 11, 2024
169185f
Merge branch 'mainline' into dev/richard/wlw
richard-rogers May 11, 2024
cc1d95e
precommit
richard-rogers May 11, 2024
1e71383
Merge branch 'dev/richard/wlw' of github.com:whylabs/whylogs into dev…
richard-rogers May 11, 2024
510504f
green
richard-rogers May 13, 2024
ed072ec
pre-commit
richard-rogers May 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
10 changes: 5 additions & 5 deletions python/examples/advanced/Transaction_Examples.ipynb
Expand Up @@ -41,7 +41,7 @@
"from whylabs_client.api.transactions_api import TransactionsApi\n",
"from whylogs.core.schema import DatasetSchema\n",
"from whylogs.core.segmentation_partition import segment_on_column\n",
"from whylogs.api.writer.whylabs import WhyLabsWriter, WhyLabsTransaction\n",
"from whylogs.api.writer.whylabs import WhyLabsWriter\n",
"import os\n",
"from uuid import uuid4\n",
"from whylogs.datasets import Ecommerce\n",
Expand Down Expand Up @@ -593,7 +593,7 @@
"cell_type": "markdown",
"source": [
"## Uploading multiple profiles with the same batch timestamp\n",
"The `WhyLabsTransaction` context manager can simplify error handling."
"The `WhyLabsWriter` can be used as a context manager to simplify transaction error handling and ensure `commit_transaction()` is called."
],
"metadata": {
"id": "K7hEBIyuzXh0"
Expand Down Expand Up @@ -630,7 +630,7 @@
"cell_type": "code",
"source": [
"try:\n",
" with WhyLabsTransaction(writer):\n",
" with WhyLabsWriter() as writer:\n",
" print(\"Started transaction\")\n",
" for i in range(5):\n",
" batch_df = list_daily_batches[i].data[columns]\n",
Expand Down Expand Up @@ -670,7 +670,7 @@
{
"cell_type": "markdown",
"source": [
"If a `write()` call returns a `False` status, the profile will not be included in the transaction. You might want to retry writing it. If not, that profile will be left out of the transaction, but those successfully written will still be included."
"If a `write()` call during the transaction fails (returns a `False` status), the transaction's commit will fail raising an exception."
],
"metadata": {
"id": "PaAQy-RDftXU"
Expand All @@ -692,7 +692,7 @@
"source": [
"schema = DatasetSchema(segments=segment_on_column(\"output_discount\"))\n",
"profile = why.log(df, schema=schema)\n",
"with WhyLabsTransaction(writer):\n",
"with WhyLabsWriter() as writer:\n",
" status, id = writer.write(profile)\n",
"\n",
"print(f\"{status} {id}\")"
Expand Down
4 changes: 2 additions & 2 deletions python/examples/basic/Getting_Started.ipynb
Expand Up @@ -75,7 +75,7 @@
"outputs": [],
"source": [
"# Note: you may need to restart the kernel to use updated packages.\n",
"%pip install whylogs"
"%pip install whylogs==1.3.29.dev0"
jamie256 marked this conversation as resolved.
Show resolved Hide resolved
]
},
{
Expand Down Expand Up @@ -394,7 +394,7 @@
"metadata": {},
"outputs": [],
"source": [
"why.write(profile,\"profile.bin\")"
"why.write(profile, \"profile.bin\")"
]
},
{
Expand Down
4 changes: 2 additions & 2 deletions python/examples/integrations/Feature_Stores_and_whylogs.ipynb
Expand Up @@ -3183,7 +3183,7 @@
" # If new request is from the next day, close logger, save profile in-memory and start logger for the next day\n",
" if request_timestamp.day > day_to_log.day:\n",
" # let's write our profiles to whylogs_output folder\n",
" why.write(profile,os.path.join(\"whylogs_output\",\"profile_{}_{}_{}.bin\".format(day_to_log.day,day_to_log.month,day_to_log.year)))\n",
" why.write(profile,\"whylogs_output\",\"profile_{}_{}_{}.bin\".format(day_to_log.day,day_to_log.month,day_to_log.year))\n",
" day_to_log = request_timestamp.replace(hour=0, minute=0, second=0, microsecond=0)\n",
" print(\"Starting logger for day {}....\".format(day_to_log))\n",
" profile = None\n",
Expand Down Expand Up @@ -3221,7 +3221,7 @@
" else:\n",
" profile.track(assembled_feature_vector)\n",
" \n",
"why.write(profile,os.path.join(\"whylogs_output\",\"profile_{}_{}_{}.bin\".format(day_to_log.day,day_to_log.month,day_to_log.year)))"
"why.write(profile,\"whylogs_output\",\"profile_{}_{}_{}.bin\".format(day_to_log.day,day_to_log.month,day_to_log.year))"
]
},
{
Expand Down