SSIS Processing Tips: Bulk Insert many files to a database

In the process of building a set of SSIS packages that would process an immense amount of data, below are some learnings that have been determined in the process. The context of these tips is to build a set of packages that would bulk insert many files into a SQL database. For Each Loop In order to bulk insert many files from the file system to a database, one of the nice options included within SSIS is the inclusion of the “Foreach Loop” Container. An obvious selection would be to use the “Foreach File Enumerator”. After all, it allows you…

Rate this:

SSIS global variables and execute process task

Within SQL Server 2000, if you wanted to execute a command line task (e.g. execute a BCP statement within DTS such as "bcp <arguments>"), a common task would be to use a ActiveX Script Task to create your command line statement.  This command line statement would then be placed within a global variable.  Then a dynamic variable task would be used to take the global variable value and place it within the Arguments property of the Execute Process Task.   For SQL Server 2005, you can use SQL Server Integration Services (SSIS) to do this task much simpler.  You can…

Rate this: