CSV Import Validation

Validation library for CSV files against defined schema. Easy to use and customize. Developed by Logic Spark team.

CSV import validation is a Typescript library that is utilize the function to read CSV data from ExcelJS.

Built With - Getting Started - Usage - License - Acknowledgement

🏗️ Built With

Here is a list of frameworks/libraries used to develop CSV Import Validation

• 
• 
• 
• 
• 

🚀 Getting Started

CSV Import Validation library checks data validation for CSV files against defined schema and returns data and validation messages as an object. In this guide, you will learn what is required before using the library and how to install it quickly. Let's get started!

Prerequisite

Before installing the CSV Import Validation library, please install the most recent Node.js version

• npm

  npm install npm@latest -g

Support Node.js 14.21.0 and above

Installation

Since the example is based on npm, you can use npm to install CSV Import Validation. You can also install via yarn or pnpm.

npm install

npm install csv-import-validation

Other options

Or yarn

yarn add csv-import-validation

Or pnpm

pnpm install csv-import-validation

(back to top)

🔥 Usage

There are 2 main features in the library. One reads CSV and returns an array while another reads CSV and returns as objects with validation messages. For each feature, it can be read via a file path or buffer. The functions are independent and do not need to be called in sequence.

To use the library, you will need to import to your targeted file

import { CsvUtilities } from "csv-import-validation";

Functions that read CSV and return an array

There are two functions, namely:

• function readFileMapArray: read CSV by specifying a file path to the CSV file
• function readBufferMapArray: read CSV from Buffer

const filePath = "files/your-csv-file-name.csv";
const reading = readCsvFileMapArray(filePath);

  const reading = readBufferMapArray(ฺbuffer);

Each function has the same output as below:

[
  [
    "Order ID",
    "Product Name",
    "Customer Name",
    "Quantity",
    "Price",
    "Discount",
    "Total",
    "Region",
    "Category",
    "Discount Rate",
  ],
  [
    "1",
    "Eldon Base for stackable storage shelf, platinum",
    "Muhammed MacIntyre",
    "3",
    "-213.25",
    "38.94",
    "35",
    "Nunavut",
    "Storage & Organization",
    "0.8",
  ],
];

Functions that read CSV and return objects and validations

There are two functions, namely:

• readFileValidator: read CSV by specifying a file path to the CSV file and validate data
• readBufferValidator: read CSV from Buffer and validate data. The validation can be applied to the header name and data type of each column.

If you import with validation, data will only be imported if there is no validation message.

const csvConfig: ValidationConfig = [
  {
    headerName: "Order ID",
    keyName: "orderID",
    type: "number",
    required: true,
    unique: true,
  },
  { headerName: "Product Name", keyName: "productName", type: "string" },
  { headerName: "Customer Name", keyName: "customerName", type: "string" },
  { headerName: "Quantity", keyName: "quantity", type: "number" },
  { headerName: "Price", keyName: "price", type: "number" },
  { headerName: "Discount", keyName: "discount", type: "number" },
  { headerName: "Total", keyName: "total", type: "string" },
  { headerName: "Region", keyName: "region", type: "string" },
  { headerName: "Category", keyName: "category", type: "string" },
  { headerName: "Discount Rate", keyName: "discountRate", type: "number" },
];

const filePath = "files/your-csv-file-name.csv";
const data = await csvUtilities.readFileValidator(filePath, csvConfig);

const csvConfig: ValidationConfig = [
  {
    headerName: "Order ID",
    keyName: "orderID",
    type: "number",
    required: true,
    unique: true,
  },
  { headerName: "Product Name", keyName: "productName", type: "string" },
  { headerName: "Customer Name", keyName: "customerName", type: "string" },
  { headerName: "Quantity", keyName: "quantity", type: "number" },
  { headerName: "Price", keyName: "price", type: "number" },
  { headerName: "Discount", keyName: "discount", type: "number" },
  { headerName: "Total", keyName: "total", type: "string" },
  { headerName: "Region", keyName: "region", type: "string" },
  { headerName: "Category", keyName: "category", type: "string" },
  { headerName: "Discount Rate", keyName: "discountRate", type: "number" },
];
const data = await csvUtilities.readBufferValidator(buffer, csvConfig);

Configuration Parameters

Parameter	Description
headerName	Specify header name of each column to validate with a message
keyName	Specify key to map data of the corresponding column
type	Specify data type of each column (number, string,boolean)
required	Specify if a column value is needed (boolean)
unique	Specify that a column header title cannot duplicate with another column header (boolean)

Each function has the same output. If there is no validation message, the output will return objects of data imported.

{
  "data": [
   {
      orderID: 1,
      productName: 'Eldon Base for stackable storage shelf, platinum',
      customerName: 'Muhammed MacIntyre',
      quantity: 3,
      price: -213.25,
      discount: 38.94,
      total: '35',
      region: 'Nunavut',
      category: 'Storage & Organization',
      discountRate: 0.8
    },
    {
      orderID: 2,
      productName: '1.7 Cubic Foot Compact "Cube" Office Refrigerators',
      customerName: 'Barry French',
      quantity: 293,
      price: 457.81,
      discount: 208.16,
      total: '68.02',
      region: 'Nunavut',
      category: 'Appliances',
      discountRate: 0.58
    },
  ]
}

If there is validation, the data will not be imported and the output will return locations that need to be fixed before importing again.

{
  "invalidData": [
    {
      "rowIndex": 1,
      "columnIndex": "A",
      "message": "Order ID's type is number."
    },
    {
      "rowIndex": 1,
      "columnIndex": "D",
      "message": "Quantity's type is number."
    },
    {
      rowIndex: 6,
      columnIndex: 'A',
      message: "Order ID's is required."
    },
    {
      "rowIndex": 7,
      "columnIndex": "A",
      "message": "Order ID's type is number."
    },
    {
      rowIndex: 9,
      columnIndex: 'A',
      message: 'Order ID must be unique.'
    }
  ]
}

For more details, please see our demo folder. There are demos for both JavaScript and TypeScript. In the example-files folder, we provide csv files with and without header for your ease of use.

(back to top)

📚 License

Distributed under the MIT license. See LICENSE.txt for more information.

ExcelJS is licensed under the MIT license.

(back to top)

🙏 Acknowledgement

• ExcelJS
• Img Shields

(back to top)