Add synthetic profile dataset generation

[jkpage00]

Added code to SyntheticData.ipynb to generate a profile dataset in addition to the Events dataset. User IDs are consistent across the two datasets so that profile attributes and events will be linked to the same profile

Description

• Added Profile schema creation
• Added Profile dataset creation
• Added function for generating synthetic data for the Profile dataset
• Refactored data generation and ingestion code:
  • New ingestBatch() function takes a dataset_id and data object and ingests the data into the specified dataset, returning the batch id
  • New ingestSyntheticBatches() function generates and ingests synthetic data for experience events, profiles, or both, for a given number of batchs and number of users per batch

Related Issue

Motivation and Context

How Has This Been Tested?

Notebook executes without errors in Cloud ML stage environment (cmle-governance sandbox)

Screenshots (if appropriate):

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• I have signed the Adobe Open Source CLA.
• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.

[liqunc]

Do you plan to change anything in Week1Notebook? If not, you can revert changes in this file.

[jkpage00]

I didn't actually change anything (intentionally) in that file. How do I revert?

[cmenguy]

This is very hard to review because I have no idea what was changed in that new notebook from week 1 notebook since they are currently tracked as 2 separate files.

Can you just rename the week 1 notebook to that and do your changes on that renamed one, like that Github will just show us the proper diff? You can do a git move operation for that.

[cmenguy]

Not sure about these configs, we should not put developer specific configs otherwise this list is going to be gigantic :)

Why do we need more than just config.ini?

[jkpage00]

sorry, I have created multiple config files for my own testing in different orgs, and added them to .gitignore so that they wouldn't be merged into the repo. but your right, no reason to update those changes to my .gitignore. I'll clean it up and re-commit. Sorry, I am a noob at git workflows 🙃

[cmenguy]

If you are changing the format of the config file you need to also update the README in that same PR so it reflects the new configuration options.

[jkpage00]

Okay, that's what I was going to work on next. I'll keep it in this branch.

[jkpage00]

@cmenguy I submitted a PR for updated README in the update-README branch. It reflects the changes to Oauth2

[jkpage00]

This is very hard to review because I have no idea what was changed in that new notebook from week 1 notebook since they are currently tracked as 2 separate files.

Can you just rename the week 1 notebook to that and do your changes on that renamed one, like that Github will just show us the proper diff? You can do a git move operation for that.

Okay, I have another branch (the base to this one) that added the original version of the synthetic data notebook. I'll update that so that SyntheticData.ipynb is just a copy of the original week 1 notebook, then merge this with that branch so that you can see the changes? Will that get what we want? Sorry again for my inexperience with git workflows