Loading into multiple sql tables from different files

[mkshanid]

Hi.. I am trying to load more than 40 csv files into my sql database. Each file loads into seperate tables. Is there any simple generic way to define etl process instead of writing seperate proccess for each table. I have tried for a generic solution but i am stuck at column mapping part.

I can't use a dynamic colum mapper function inside FlatFileDefinition.Create funtion during CrossApplyTextFile

Also need to know is there any functionality available to rollback the action incase of any failures.

I was looking into the documentation and discussion area but i didn't get an example for multiple table loading.

Also it will be helpfull if you can update the documentation for different connectors. I am looking for SFTP connection sample.

[paillave]

load many tables from many CSV

to load many tables, you will find an example here: https://github.com/paillave/Etl.Net/blob/master/src/Tutorials/Paillave.Etl.Samples/TestImport2.cs

transactions

For transactions, ETL.NET totally relies on .NET capabilities. You can use it of course, but according to my humble opinion, enclose a complete ETL process into a transation might lead to very aggressive locks on many tables (that might even be escalated to the database level), and if not aggressive (depending on the kind of transaction you chose) to something very expensive in term of database server ressources.
Here the issue that talks about Transactions: #292

SFTP and others

For SFTP, and everything else you must have a look here:
https://paillave.github.io/Etl.Net/docs/recipes/dealWithFiles

Documention in general

Full documentation is an ongoing project. ETL.NET is a very large project, and writting a complete documentation for it a daunting duty; but it will come step by step! 😄