Building an automated pipeline in Google Cloud Platform to decompress, prepare and perform visual analytics on responses collected with Google Form surveys. For a comprehensive explanation of the application of Cloud Functions to automate a Cloud Dataprep pipeline, please refer to the use case discussed in this article.
Assets for demonstration of the blog post Automated End-to-End Solution to Process and Visualy Analyze Google Form Survey Responses (https://docs.google.com/document/d/1hEo_qrrykVZq19rN2kZ5ZgjNFID9odsEehN0Tlj9v-w/edit#heading=h.ep3hsi5hfvb9)
-
flow-Google-Form-Analytics-Design-Pattern.zip: Dataprep Flow package example to import from Dataprep UI.
-
mockarooAKS.csv: Example of survey responses CSV file used as one of inputs of the flow example and to be stored in the Google Cloud Storage you want to monitor.
-
mockarooAKS.zip: Example of survey responses ZIP file used as one of inputs of the flow example.
-
cf-decompress-trigger-dataprep.py: Background Python function to decompress the file and trigger a Dataprep job when a file ZIP is created in a Google Cloud Storage bucket folder. Dataprep job started with REST API call and new decompressed file as parameter.
-
requirements.txt: Code complement to be include as requirements in the Python Cloud Function definition.
Google Cloud Functions https://cloud.google.com/functions
Google Cloud Dataprep by Trifacta https://cloud.google.com/dataprep
Google Cloud Dataprep API https://clouddataprep.com/documentation/api
All Trifacta API https://api.trifacta.com/