The Smart Receipts Scanning for Reward Programs Solution Accelerator is an easy-to-deploy solution, that creates an automated process of scanning receipts, extracts insights and information, approves or rejects receipts based on business rules, and consumes the outcome in one wholistic report.
The solution aims to accelerate information extraction using the pre-built receipt model in Form Recognizer. It applies intelligent business rules to check the eligibility of a receipt based on pre-defined criteria. It can be considered as a complete and automated end-to-end solution that leverages the power of Logic Apps, and creates a summarized view with a Power BI Report to consume the extracted insights.
Reward Programs are a proven marketing strategy, designed to encourage customers of a business, to continue using their services and buying their goods. One way of achieving such results is by analyzing purchase receipts and rewarding the customer with redeemable points during a campaign period.
This solution accelerator aims to create an end-to-end solution, which automates the process of scanning receipts, extracting purchase information like brand details and amount spent, and apply business rules to approve receipts and reward qualified customers with equivalent points, by leveraging the capabilities of Form Recognizer and Azure Cognitive Services.
The following are the Azure resources that are required to deploy this accelerator, along with the architecture of the solution:
- Storage Account -- to store the input receipts and the output CSVs.
- Cognitive Services (Form Recognizer) -- to extract insights from receipts with the pre-built models.
- Logic App -- to work as an orchestrator and automate the whole process.
The sample receipts dataset used in this accelerator is from an open-sources dataset, containing 200 sample receipts, and acquired from ExpressExpense.com.
Using the pre-built receipts model in Form Recognizer, we can extract the following infrormation from the receipts:
- Brand name
- Transaction date and time
- Total (sub total, tax, total)
- Line-items (name, quantity, price)
There are four, customizable business rules currently supported in this solution accelerator, that approves or rejects a receipt, which are:
- Brand Eligibility -- checks the Brand Name and qualifies the receipt if accepted or not.
- Campaign Period -- checks the Transaction Date and qualifies the receipt if accepted or not.
- Points Earned -- checks the Total Spent and rewards the user with points if the receipt is accepted.
- Confidence Score -- checks the Confidence Score and qualifies the receipt if accepted or not.
- Azure Subscription
- Sample Receipt
- Creat a new Resource Group in your Azure Subscription and provision the followng resources:
- Storage Account
- Logic App
- Form Recognizer
You can also deploy the required resources using this ARM template:
- Create a blank Logic App.
- Search for "Azure Blob Storage" and select "When a blob is added or modified" trigger.
- You will be required to create a connection to the storage account. Fill in the information and click on Create to proceed.
- You will be required to select a container in the storage account to monitor. Fill in the information and click on New step to proceed.
5.From the "Azure Blob Storage" list of actions, select "Get blob content".
- Fill in the information and click on New step to proceed.
- Search for "Form Recognizer" and select "Analyze Receipt" from the list of actions.
- You will be required to create a connection to Form Recognizer. Fill in the information and click on Create to proceed.
- Fill in the information and click on New step to proceed.
- From the "Variables" list of actions, select "Initialize variable".
- Create an empty array variable that will hold the line items. Fill in the information and click on New step to proceed.
- From the "Data Operations" list of actions, select "Create CSV table".
- Search for "documentResults" and select it as the input for the CSV table. Select "Custom" for the "Columns" option.
- Fill in the table with the following information: Receipt information
"brand_name": "<Merchant name field Merchant name>"
"brand_confidence": "<Merchant name field Confidence>"
"transaction_date": "<Transaction date field Transaction date>"
"transaction_confidence": "<Transaction date field Confidence>"
"total_spent": "<Total field Total>"
"total_confidence": "<Total field Confidence>"
Business rules
"eligible_brand": "<IF merchant_name CONTAINS eligible_brands_list THEN true ELSE false>"
"campaign_period": "<IF transaction_date CONTAINS campaign_period THEN true ELSE false>"
"reward_points": "<MUL total BY conversion_rate>"
"accuracy_flag": "<IF confidence_score LESS THAN threshold THEN true ELSE false>"
- From the "Azure Blob Storage" list of actions, select "Create blob".
- Fill in the information. For "Blob name" use a concat function to append ".csv" to the file name for the generated CSV file. For "Blob content", use "Outputs" of the "Create CSV table" action. Save your Logic App to proceed.
- Select "Add a parallel branch" after the "Initialize variable" action.
- From the "Control" list of actions, select "For each".
- Search for "documentResults" and select it as the loop parameter. Click on Add an action inside the loop to proceed.
- Add another "For each" loop. Search for "items" and select "Items field Items" as the loop parameter. Click on Add an action inside the second loop to proceed.
- From the "Data Operations" list of actions, select "Compose".
- Use the following structure as input and replace the placeholders with the corrosponding dynamic values. Click on Add an action inside the second loop to proceed.
{
"item": "<Item field value Name>",
"price": "<Item field value Price>",
"quantity": "<Item field value Quantity>",
"total_price": "<Item field value Total price>"
}
- From the "Variables" list of actions, select "Append to array variable".
- Select the name of the array and choose "Outputs" of "Compose" as the value. Click on New step outside the loops to proceed.
- From the "Data Operations" list of actions, select "Create CSV table".
- Fill in the information and click on New step to proceed.
- From the "Azure Blob Storage" list of actions, select "Create blob".
- Fill in the information. For "Blob name" use a concat function to append "-items.csv" to the file name for the generated CSV file. For "Blob content", use "Outputs" of the "Create CSV table" action. Save your Logic App to proceed.
- From "Run Trigger", click on "Run".
- Upload the sample receipt "1000-receipt.jpg" to the container in the storage account.
- Wait for the Logic App flow to finish.
- The result will be two CSV files, one for the Receipt information, and one for the Line-items.
For all licensing information refer to LICENSE.