Hands On Training Lab

SQL Server Integration Services 2005

V1.2
..............................................................................................................................................................................................................................................1
1.Hands-on Labs Overview.................................................................................................................................................................................................4
1.Hands-on Labs Overview.................................................................................................................................................................................................4
2.Setup and Congfiguration.................................................................................................................................................................................................5
2.Setup and Congfiguration.................................................................................................................................................................................................5
3.Hands-on Lab I (Short data trouble shooting package)....................................................................................................................................................6
3.Hands-on Lab I (Short data trouble shooting package)....................................................................................................................................................6
3.1. Hands-on Lab Preset.................................................................................................................................................................................................6
3.2. Hands-on Lab Details...............................................................................................................................................................................................6
3.3. Conclusion..............................................................................................................................................................................................................13
3.3.1.Comments.........................................................................................................................................................................................................13
3.3.2.BOL ..................................................................................................................................................................................................................13
4.Hands-on Lab II (Create text files from adventure works table then loop to load them)...............................................................................................14
4.Hands-on Lab II (Create text files from adventure works table then loop to load them)...............................................................................................14
4.1. Hands-on Lab Preset...............................................................................................................................................................................................14
4.2. Hands-on Lab Details.............................................................................................................................................................................................14
4.3. Conclusion..............................................................................................................................................................................................................23
4.3.1.Comments.........................................................................................................................................................................................................23
4.3.2.BOL ..................................................................................................................................................................................................................23
5.Hands-on Lab III (Expand loop to write iteration audit information)............................................................................................................................24
5.Hands-on Lab III (Expand loop to write iteration audit information)............................................................................................................................24
5.1. Hands-on Lab Preset...............................................................................................................................................................................................24
5.2. Hands-on Lab Details.............................................................................................................................................................................................24
5.3. Conclusion..............................................................................................................................................................................................................27
5.3.1.Comments.........................................................................................................................................................................................................27
5.3.2.BOL ..................................................................................................................................................................................................................27
6.Hands-on Lab IV (Open a different application based on day of week)........................................................................................................................28
6.Hands-on Lab IV (Open a different application based on day of week)........................................................................................................................28
Author: Craigg Page 1 of 53 Microsoft Corporation
 6.1. Hands-on Lab Preset...............................................................................................................................................................................................28
6.2. Hands-on Lab Details.............................................................................................................................................................................................28
6.3. Conclusion..............................................................................................................................................................................................................29
6.3.1.Comments.........................................................................................................................................................................................................29
6.3.2.BOL ..................................................................................................................................................................................................................29
7.Hands-on Lab V (Extend LoadApplications to feed it application names with a configuration file)............................................................................30
7.Hands-on Lab V (Extend LoadApplications to feed it application names with a configuration file)............................................................................30
7.1. Hands-on Lab Preset...............................................................................................................................................................................................30
7.2. Hands-on Lab Details.............................................................................................................................................................................................30
7.3. Conclusion..............................................................................................................................................................................................................32
7.3.1.Comments.........................................................................................................................................................................................................32
7.3.2.BOL ..................................................................................................................................................................................................................32
8.Hands-on Lab VI (Error Rows and logging data)...........................................................................................................................................................33
8.Hands-on Lab VI (Error Rows and logging data)...........................................................................................................................................................33
8.1. Hands-on Lab Preset...............................................................................................................................................................................................33
8.2. Hands-on Lab Details.............................................................................................................................................................................................33
8.3. Conclusion..............................................................................................................................................................................................................38
8.3.1.Comments.........................................................................................................................................................................................................38
8.3.2.BOL ..................................................................................................................................................................................................................38
9.Hands-on Lab VII (Deploy packages and execute from Management Studio)..............................................................................................................39
9.Hands-on Lab VII (Deploy packages and execute from Management Studio)..............................................................................................................39
9.1. Hands-on Lab Preset...............................................................................................................................................................................................39
9.2. Hands-on Lab Details.............................................................................................................................................................................................39
9.3. Conclusion..............................................................................................................................................................................................................44
9.3.1.Comments.........................................................................................................................................................................................................44
9.3.2.BOL ..................................................................................................................................................................................................................44
10.Hands-on Lab VIII (Add Execute SQL to truncate table to the LoopAndLoadProductionTable package).................................................................45
10.Hands-on Lab VIII (Add Execute SQL to truncate table to the LoopAndLoadProductionTable package).................................................................45
10.1. Hands-on Lab Preset.............................................................................................................................................................................................45
10.2. Hands-on Lab Details...........................................................................................................................................................................................45
10.2.1.Comments.......................................................................................................................................................................................................48
10.2.2.BOL ................................................................................................................................................................................................................48
11.Appendix A Where to learn more..................................................................................................................................................................................49
11.Appendix A Where to learn more..................................................................................................................................................................................49
12.Appendix B sample Reports for SSIS Log Providers (OnPipeline Rows sent and .....................................................................................................50
12.Appendix B sample Reports for SSIS Log Providers (OnPipeline Rows sent and .....................................................................................................50
12.1. General Log summary, history, results.................................................................................................................................................................50
Author: Craigg Page 2 of 53 Microsoft Corporation
 12.2. Detailed package log history report......................................................................................................................................................................51
12.3. Detailed Pipeline report........................................................................................................................................................................................52
12.4. Error Row Report..................................................................................................................................................................................................53

Author: Craigg Page 3 of 53 Microsoft Corporation

 1. Hands-on Labs Overview

1. Introduction
SQL Server Integration Services (SSIS), the successor to DTS in SQL Server 2005, is an all-new application
that provides a data integration platform from easy-to-use tasks and transforms for the non-developer to a
robust object model supporting creation of custom tasks data transformations. With the SSIS platform you
can create solutions integrating data from non homogenous data sources, cleansing, and aggregation as
well as work flow surrounding the data processing. SSIS goes beyond standard ETL processing (Extract
Transform Load) providing components such as Web Service, XML, WMI tasks, and many more. Add to the
rich list of out of the box components a full object model underneath, and users can create their own tasks,
transformations, data sources/destinations, and Log Providers to suit almost any scenario.

2. Audience and Pre-requisite knowledge

• Users understand the roles and differences between the applications "Business Intelligence Development
Studio" and "SQL Server Management Studio".
• Users have basic understanding of SQL Server Integration Services functionality and for example have a
basic understanding of the difference between "Control Flow" and "Data Flow" as well as know what "Log
providers", "Connection Managers", "Property Expressions" do. I am looking for students to be familiar
with the terms and the general idea of all of the things mentioned above. It’s ok if you have never played
with them, just want terminology and general purpose to be familiar.
• SQL Server Business Intelligence studio is used to create, edit, trouble shoot packages while the command
line tool ‘dtexec.exe’ is what you will execute packages with in production environments.

Someone can gear up on the above topics from the SSIS portal on MSDN.
http://msdn.microsoft.com/SQL/sqlwarehouse/SSIS/default.aspx
Where they can find links to the following two webcasts as well as other webcasts and articles.
• Introducing SQL Server Integration Services for SQL Server 2005 (Level 200)
• TechNet Webcast: Deploying, Managing and Securing Integration Services (Level 300)

3. Scenarios
• This Lab is comprised of several smaller labs to better cover various portions of the SSIS product as well as
provide natural checkpoint for the training such that if a particular exercise cannot be completed they still
participate in later exercises.

Author: Craigg Page 4 of 53 Microsoft Corporation

2. Setup and Congfiguration

I. Installation
• Extract the zip file to the root c:\ so you end up with “C:\_SSIS_Training”.
• Attach the following 2 databases. “SSISLOGGING.mdf” for audit and logging data (tables myfileaudit
and ssis_ErrorRows)and database “SSISTRAINING.mdf” (tables mydescriptions) for data destination.
• We will be using data from the AdventureWorks which ships with SQL 2005 (optional during installation)
•

II. Sql Server Books Online (BOL)

• The BOL topics noted through the doc with the icon above were found with BOL filtered to just search the
“Technology” Integration Services, from the BOL Search Page.
III. Miscellaneous
• You may want to have a soft copy of this manual, which can be handy to have open while you do the
labs, to allow you to copy/paste expressions from the manual to the Visual Studio design environment.

Author: Craigg Page 5 of 53 Microsoft Corporation

 3. Hands-on Lab I (Short data trouble shooting package)

I. Purpose of Hands-on Lab

• The purpose of this lab is to provide a quick sample/best practice on how to trouble shoot data within a
package flow and to address common questions on ‘how to breakpoint and step through data in the
flow’. We will load a contact table as the source data and parse out a portion of the email field, viewing
the data to insure our parse expression is correct before connecting any ‘real’ destinations.
• This exercise will be more detailed than others on exactly what to click and type as a form of
introduction, while the other labs will have more generalized instructions.
• This lab will also allow us to gauge the overall pre-knowledge of the students.

3.1.Hands-on Lab Preset

II. Configuration
• We will be using data from the AdventureWorks sample database which ships with SQL Server 2005.

3.2.Hands-on Lab Details

Discussion Points Actions taken
1 Create a new SSIS project and solution. 1. Open SQL Server Business Intelligence Studio (BIDS).
• During this training we will build several 2. Select ‘Create’ in the ‘Recent Projects’ window.
packages within a single solution. 3. Select ‘Integration Services’ template.
Remember the BI Studio is really Visual 4. Name of “SSISTraining” (this will be the name of the overall
Studio 2005. The solution can contain solution).
several projects (packages) as well as 5. Location of existing folder “C:\_SSIS_Training”.
Reporting services reports, Analysis Services 6. Check the ‘Create Directory For Solution’ box.
projects….. 7. Click ‘Ok’ and the Solution will be created.
• SSIS packages are by default XML files with 8. In the Solution Explorer window Right click the default package
.dtsx extension. The file has a name as well name of ‘Package.dtsx’ and select rename.
as the package has a name ‘property’. 9. Use name “TroubleShootDataInFlow.dtsx” and press
Typically you will want them to be the same. <Enter>. Select yes to renaming the pack object as well.
• The ‘file’ used for the package can have a
different name than the ‘object’ which is the
‘Name’ property within the package.

Author: Craigg Page 6 of 53 Microsoft Corporation

 2 Add a Data Flow Task to the package 1. From the Toolbox, Double-click or drag the ‘Data Flow Task’
• The data flow task is a separate to the Control Flow canvas.
task that is the overall data 2. Double-Click the Data Flow Task OR click the Data Flow tab to
‘pipeline’. open the task.
3 Add and configure and OLE DB 1. From the Toolbox Double-click or drag the OLE DB Source to
source adapter to the data flow. the Data Flow Task.
• The Connection manager handles 2. Double-Click the OLE DB Source to Open it.
the low level hand shake sort of 3. In the top ‘Provider’ drop down list we want to leave it at
exchange with your data source “OLE DB\SQL Native Client” but drop the list so you can see all
while the source adapter handles the choices that ship out of the box.
the data itself, such as which of the 4. Click the ‘New’ Button to add a new Connection Manager.
fields to actually use. 5. Click ‘New’ in the ‘Configure OLE DB Connection Manager’
dialog.
6. Enter your computer name, for example‘DBIL405’.
7. Select Database name ‘AdventureWorks’.
8. Click ‘Test Connection’ to insure the connection works then
‘ok’ to the test, Connection Manager, and Configure
Connection Manager dialogs.
9. Leave Data Access Mode as ‘Table or View’.
10. In the box below, select the table [Person].[Contact].
11. NOTE: In the left pane click ‘Columns’. If you wanted to
remove some columns from the flow you can uncheck them
now to minimize data flowing down. Leave all columns
Checked for our lab We will look at error rows in a later lab.
12. Click OK.

4 Add a Derived Column Transform to 1. From the Toolbox, Double-click or drag a ‘Derived Column’
the dataflow Transform to the Data Flow Canvas.
• Double-Clicking will automatically 2. If the transform is not connected to the OLE DB Source, then
connect the new object to the click on the OLE DB Source, click the Green Output arrow and
previous object, assuming its still connect it to the Derived Column Transform.
selected. If not select the upstream
object and drag the green ‘output’
arrow to the object below.

Author: Craigg Page 7 of 53 Microsoft Corporation

 • We want to parse out a portion of the
EmailAddress, the characters left of
the “@”. We will start by just
“deriving” what character position #
the “@” symbol is at. 3. Double-click to open the Derived column transform and expand
• The Derived Column transform allows the String Functions folder on the right.
us to ‘create’ or derive new columns 4. Drag the ‘FindString’ Function down to the first row’s
from existing columns and variables. ‘Expression’ column and note the other columns populate with
• Note: the arrows inside the ‘Data defaults for you.
Flow Task’ deal with data and error 5. Widen the Derived column transform window so you can see
outputs while in the Control flow they the whole FindString Expression. Note the whole expression is
are precedent constraints (success, Red and will stay that way until we enter all needed fields.
failure, complete). We look a bit at 6. Expand the ‘Columns’ folder on the left, and drag
each later, but see the Suggested ‘EmailAddress’ to the <<Character Expression>> field inside
BOL topics to learn more. the FindString expression.
• The Precedent constraints (green, 7. Click on the <<String>> field and replace it with "@"
red arrows) are one of the ways you (including the double quotes). So, what string are we looking
can control the work flow. For for inside our character expression.
example if a task Fails (red) you can 8. Click on the <<Occurrence>> field and replace with a 1 (no
flow to one task (say SendMail). quotes). So, find the first occurrence of the string, inside the
character expression.
9. Click the empty expression row below, the FindString
expression should turn black as all fields are entered and of
the correct data type. For example try putting “ “ around the
1, which implies string rather than numeric, then click below
and see the text turn red..
5 Add a DataReaderDest Destination 1. From the Toolbox Double-click the DataReaderDest
to the dataflow 2. Set the ReadTimeOut property to 0. This is not required as
• This destination can be used without much as it makes testing easier\faster.
any configuration changes, in effect
creating a ‘null’ destination. The
data goes nowhere but we can
trouble shoot any issue in our
derived column upstream before

Author: Craigg Page 8 of 53 Microsoft Corporation

 adding any actual destinations like a
SQL table or flat file.
• Because the DataReader destination
is really caching the data until some
other reader application pulls the
data, the destination has a timeout
so allow time for the reader
application to pull the data. Because
we only want to trouble shoot the
data, we will set the timeout to 0 and
execution will complete faster.
• DataReader is one of the data classes
in ADO.net. So, the SSIS data
streams into the SSIS DataReader
Destination, and you can have some
other application\component which is
designed to look for\connect (hence
the timeout property) using the
common DataReader Interfaces. The
reader application can fetch the data
(read-only) and in the order it was
sent down the SSIS Dataflow task.

6 Add a Data Viewer to the data path 1. Double click the Green Data Flow path between the Derived
between the Derived Column and column and destination.
DataReader Dest 2. In the Data Flow Path editor click ‘Data Viewers’ on the left.
• The viewer shows 1 ‘buffer’ of data 3. Click the ‘Add’ button below and leave the default ‘Grid’ as the
vs you specifying a # of rows. A type of viewer.
buffer is part of the dataflow 4. Select the ‘Grid’ tab at the top and note by default, all columns
architecture. See BOL topic will be displayed in the viewer. Lets leave that for now.
suggestions at bottom. 5. Click ‘ok’ and ‘ok’ again.
• Imagine using a data viewer to
trouble shoot bad data…your in the
viewer, see the issue and can hit the
Author: Craigg Page 9 of 53 Microsoft Corporation
 ‘Copy Data’ button to get data to the
Windows clipboard which you can
now paste into and fire off an email
to your support team or data vendor.

7 Visually your package will look

like….
• Note the little icon indicating a Data
Viewer is on the path.

8 Execute the Package 1. Save the package using the icon or from the “File” menu
• You can add more than 1 viewer. choose “save selected items”.
Perhaps 1 full grid and another that 2. Click the button on the toolbar or select “Debug Start
is only 2 keys fields, and perhaps a Debugging” menu option.
graph. 3. The package should execute and open the Data Viewer.
• Data viewers are only ‘functional’ 4. In the Data Viewer scroll all the way to the right and note the
while in BIDS and do nothing and derived column we created is there and should contain a
affect nothing when the package is number indicating the location of the “@” symbol per row.
executed at the command line with 5. Note the objects stay yellow while the data viewer is attached
dtexec.exe and open. The objects will turn green when all the buffers
• If someone asks ‘how to trouble have been viewed and\or you close the viewer and execution
shoot data a row at a time’, the data can complete.
viewer is the answer. Again, you 6. This data set is small and not as easy to test the full data
cannot control the number of rows viewer functionality but you can use the green arrow to
returned, its based on 1 ‘buffer’ advance the data viewer to the next buffer. Detach will
which is a unit of measure in the ‘unhook’ the viewer from the dataflow and the package will run

Author: Craigg Page 10 of 53 Microsoft Corporation

 SSIS memory manager. The number data through until you click ‘Attach’ again (the ‘Detach’
of rows in a single buffer depends on changes values)
the column types/length as well as
the number of column’s flowing down
the data path.
7. Once the package has completed (green) you can restart
execution with the button or stop execution with the
button, or choose Stop and Restart from the Debug menu.
9 Add another derived column to parse 1. If you have not already, stop execution so you an edit the
the @ package further.
• Note: if you add fields upstream after 2. Open the Derived Column transform and click the first empty
the data viewer is added you will need row, expression field.
to re-configure the data viewer to 3. Build the following expression to parse the email name. You
include the new columns, or just can copy/paste from below or construct the nested expression
delete it and re-add the whole viewer. with the editor. Note the FindString function is used as the
• You can attaché more than 1 data <<Length>> field of the substring function.
viewer to a data path. They are SUBSTRING(EmailAddress,1,(FINDSTRING(EmailAddress,"
automatically linked such that if click @",1) - 1))
on data rows on one viewer, the 4. Again, click below the expression when you are done and
associated data points are selected on verify it turns black, indicating all fields are complete and there
the graph viewer, for example. are no syntax errors.
5. Click OK to close the derived column transform

10 Edit the Data Viewer 1. Double click the Green Constraint data path between the
• To include only our 2 derived Derived column and destination.
columns and the original email 2. In the Dataflow path editor click ‘Data Viewers’ on the left
field. So “EmailAddress”, “Derived 3. The existing Grid viewer should be selected on the right
column1”, and “DerivedColumn2”. 4. Click the ‘Configure’ button at the bottom
5. This time, lets only have the 3 relevant columns in the viewer
so remove all fields but the 3 we want using the < button or
remove the all with << and then add back the 3 we want with
>
6. Click OK to close the data view dialog
7. Click OK to close the Data Path Editor dialog

Author: Craigg Page 11 of 53 Microsoft Corporation

 Execute the Package 1. Save the package using the icon or from the “File” menu
choose “save selected items”.
2. Click the button in the toolbar or select “Debug Start
Debugging” menu option.
3. The package should execute and open the Data Viewer.
4. Verify the derived column is correct, the portion of the email
address to the left of the “@” symbol.

Author: Craigg Page 12 of 53 Microsoft Corporation

 3.3.Conclusion

3.3.1.Comments
• It pays to verify expressions, parsing, and inbound data in general before pushing it further down your flow.
Discovering a mistake at the end just means that much more to edit on the way back up. This is be design
overall, because each component in the flow holds meta data about the objects its dealing with as inputs
and outputs. Because each component can do so much ‘locally’ with its known meta data and because the
wide variety of transforms which do a wide variety of things to the local meta data, its not realistic to have
changes made to one component automatically reflected up or down the flow.
• (Another Example of a Great platform. ) As an example of how great a platform SSIS is, lets look at how
there are 3 basic levels to parsing….1) This lab did some basic parsing with the Derived column transform,
which can handle rather sophisticated logic. However if you need very advanced parsing and per row
sniffing, your best off using the 2) Script Component, which will allow you to use VB.net code, yet it’s aware
of buffers and rows in the data flow task. The script component, with a bit of advanced tweaking can allow
you to have one inbound row you parse in some way, to result in more than 1 outbound row. 3) for even
more advanced handling or perhaps just to ease re-use, there is the Data Flow API and you can write your
own custom transform for sophisticated logic OR just to make re-use easier because the custom transform
can be available from the Toolbox

3.3.2.BOL
• “How to: Add a Data Viewer to a Data Flow”
• “Debugging Data Flow”
• “Precedence Constraints”
• “Creating a Transformation Component with Asynchronous Outputs”

Author: Craigg Page 13 of 53 Microsoft Corporation

 4. Hands-on Lab II (Create text files from adventure works table then loop to load them)

III. Purpose of Hands-on Lab

• The purpose is show and example of the conditional split transform, For Each Looping container, .property
expressions to dynamically alter execution, and the execute SQL task with parameters to insert data into a
table.
• Nice portable example of some transformations as well as file looping. Assumes the user has Adventure works
installed.

4.1.Hands-on Lab Preset

A. Configuration
• We will be using data from the AdventureWorks sample database.
B. Pre-stage
• Stop execution of the previous package.
• Close the previous package if it’s still open.

4.2.Hands-on Lab Details

Discussion Points Actions taken
1 Create a new SSIS package in the 1. In the Solution Explorer right click the “SSIS Packages” folder\node
existing solution and choose “New SSIS Package”
2. Right-Click the new package and re-name it
“LoopAndLoadProductionTable.dtsx”
3.
2 Create a Dataflow task and OLDE DB 1. With the ‘Control Flow’ canvas active, go to the Tool Box, Add and
Source to load the [production] table. open a Data Flow task to the new package.
• We could just select the table name from 2. Add and open an OLE DB Source to Data Flow design Canvas.
the list but this underscores the Source 3. Create a new Connection Manager to your server and the ‘Adventure
can use TSQL as well. Works’ database.
4. Use Data Access mode of “SQL Command” and use the following
SELECT * FROM Production.ProductDescription
5. Click the ‘Parse Query’ button on the right to verify your syntax

Author: Craigg Page 14 of 53 Microsoft Corporation

 6. Try the ‘Preview’ button to see a portion of the result set your SQL
command will return.
7. Click OK to close the OLE DB Source
3 Add a Conditional Split transform to 1. From the Toolbox, Add, Connect, and open a Conditional split
split the data stream based on ranges transform.
of values in the ProductionID column 2. In the Conditional Split Transformation Editor, create 4 outputs (rows
in the Conditional Split UI). The ‘Conditions’ are just expressions like
we built with the derived column. Use the values in the table below.
Again, you can type it all in, copy/paste from the soft copy of the
manual or use the ‘Columns’ in the left pane, such as dragging down
(to the condition row) 2 copies of [ProductDescriptionID] and then
manually editing the remainder of the expression.
NOTE: you can type one then copy/paste the syntax to the next row
and edit as needed.

4 Or Output Condition
• So we will create the 4 conditions in de Name
the Conditional Split, which creates 4 r
outputs.
1 1to1000 ProductDescriptionID > 0 &&
• The next step(s) we create
ProductDescriptionID < 1001
downstream processing for each of
2 1001to1 ProductDescriptionID > 1000 &&
the outputs.
500 ProductDescriptionID < 1501
3 1501to1 ProductDescriptionID > 1500 &&
800 ProductDescriptionID < 1801
4 over180 ProductDescriptionID > 1800
1
5 1. Click OK to close the Conditional Split
6 Add a Flat File Destination 1. From the Toolbox add a Flat File Destination
• We are creating downstream 2. When the destination component is connected to the Conditional Split,
processing for 1 of the 4 outputs a dialog will ask you to decide which output to use. For this first
output select “1to1000” from the previous table.
• We want each output from the
3. Open the Flat File Destination and note the box “overwrite data in
conditional split to end/create a the file” is checked so each execution will result in a unique dataset
file. (vs. cumulative).
4. Click ‘New’ to add a new connection manager of type ‘Delimited’.
5. Use “ProductDescriptions1” for both Connection Manager name and
description of the connection manager.
6. For filename enter
“C:\_SSIS_Training\LoopFiles\ProductDescriptions1.txt”
7. In the window ‘Flat File Connection manager editor” Check the box
“column names in the first data row’

Author: Craigg Page 15 of 53 Microsoft Corporation

 8. Check the box “Unicode” which disables the ‘Code Page’ field.
9. Enter the pipe symbol “|” (<Shift>+<\>) as the text qualifier **
We want to do this because some of the descriptions in the source
table contain double quotes, commas, and semi-colons which can
throw off normal delimiter parsing.
10. Click the Advanced page. The column “ProductDescriptionID”
should be highlighted.
11. Change the “Text Qualifier” property to False.
12. Click the ‘Description’ Field, set the OutPutColumnWidth to 400.
13. Click the “Row Guid” column Change the “Text Qualifier” property to
False
14. Click the “Modified Date” column Change the “Text Qualifier”
property to False (we only want our “|” text qualifier to be used for
the Description field\data.)
15. Click OK to close the Connection Manager dialog.
16. In the still open Flat File Destination Editor click ‘Mappings’ on the
left. You should see the source and destination columns automatically
mapped. This is one reason to trouble shoot data first, like we did in
the first example.
17. Click OK to close the Flat File Destination
18. Note the connections listed in the Connection manager window
7 Execute the package to insure it runs. 1. Save the package
2. Execute the package to insure we have something similar to this
image.
3. NOTE: If the wrong package executes, you need to change the
default object in the solution by right clicking the desired package
name (“LoopAndLoadProductionTable.dtsx”) in the solution explorer
and choose “Set As Startup Object’

8 Add 3 more destinations for the other 1. Add a new Flat File Destination first then link it to the
3 outputs
upstream Conditional Split, by clicking on the Conditional
Split and note you get another green data path to
Author: Craigg Page 16 of 53 Microsoft Corporation
 use\drag to the new Flat File Destination.
2. Each flat file destination needs to use a unique file
connection manager and point to\create a unique
filename. See the table below
3. Be sure to configure the other 3 Connection Managers as
we did the first one .e.g. check the Unicode box, add the
pipe “|” text qualifier, Check box for column names in first
• row, On the Advanced page…Length of Description field
400, and only the Description field has “Text Qualifier” set
to TRUE.
4.
9 Path Name Filename to use in Connection
/Description Mgr
1001to1 ProductDescriptio C:\_SSIS_Training\LoopFiles\Produ
500 ns2 ctDescriptions2.txt
1501to1 ProductDescriptio C:\_SSIS_Training\LoopFiles\Produ
800 ns3 ctDescriptions3.txt
over180 ProductDescriptio C:\_SSIS_Training\LoopFiles\Produ
1 ns4 ctDescriptions4.txt
1 Visually your package will look like….
• You may want to play with the auto
layout option under Format>>Auto
Layout>>Diagram menu option

1 Execute the package to insure it runs. 1. Save the package

• When the package executes, you 2. Execute the package to insure it runs successfully.
can look in the progress pane, near 3. NOTE: If the wrong package executes, you need to change the
the bottom and note the rows default object in the solution by right clicking the desired package
written per destination, such as name (“LoopAndLoadProductionTable.dtsx”) in the solution explorer,
[DTS.Pipeline] Information: and choosing “set as startup object”.
"component "Flat File Destination" 4. Browse to the destination folder C:\_SSIS_Training\LoopFiles
(78)" wrote 88 rows. (that was defined in the connection managers) to see the files.

Author: Craigg Page 17 of 53 Microsoft Corporation

 [DTS.Pipeline] Information: 5. Double-click one of the files to open it in notepad and note our pipe
"component "Flat File Destination 1" “|” delimiter.
(97)" wrote 173 rows.
1 Add and configure a For Each 1. Stop execution of the package if you have not done so already.
container 2. Make the Control Flow canvas (vs. Data Flow) active.
• We will now loop over the files we 3. From the Toolbox, add a ‘ForEach Loop‘ container, connect it to the
just created above and ‘load’ them. Data Flow Task and open it
• The container will return the full 4. On the “Collection” page use the default enumerator type of
file name into a mapped variable. “Foreach File Enumerator”. So it will loop once per file in the specified
• A variable scoped to the package folder.
(which is a container too) is more 5. For “Folder” enter “C:\_SSIS_Training\LoopFiles” (no quotes)
or less a global variable and visible 6. Leave the default “Fully Qualified” for “Retrieve File name”
to containers below it. 7. Leave the ‘Files’ to “*.*” so we grab all files in the folder.
• NOTE: variables are case sensitive 8. Change to the “Variable Mappings” page
so for now just use all lower case 9. In the ‘Variable’ column, drop the list down and choose Add a new
to keep things easier to variable that is scoped to the top most container, which is the
remember/trouble shoot. package itself.
10. Name the variable “myfilenamefull” , leave NameSpace
“user”,Value Type of “String”, and leave Value empty.
11. Click “OK” to close the ‘add variable’ dialog.
12. Click “OK” to close the For Each Loop Editor.

1 Add a new user variable to hold row 1. From the “SSIS” menu choose “Variables”. You should see the
counts “myfilenamefull” already there.
• Later, inside the DataFlow task we will
map RowCount transform to this variable, 2. Click the Add new variable button . You may want to widen the
in effect storing the row count of the Data variables window.
Flow Path. 3. Name should be “myrowcount”
• The scope of a variable cannot be 4. Scope should be the top most, of the package itself, so
changed. If you notice the scope is “Loopandloadproductiontable”.
wrong, delete the existing variable and 5. Data Type of int32
create a new one. If you are trying to 6. Value can be left at 0
create a package level variable, click the
Control Flow canvas, (not any of the
containers on it) then when you create a
new variable it is assumed you are
creating one at the package level.
1 Add a dataflow task 1. From the Toolbox add a Data Flow Task, to the inside of the loop
• The task will be processed once per container, and open the task. You need to drag and drop, the easier
iteration of the loop. Therefore in our double-click method does not work when adding objects to the inside
case for each file in the folder the of a container.

Author: Craigg Page 18 of 53 Microsoft Corporation

 dataflow task will be executed.
Visually your package will look like….

1 Add Flat File Source and Connection 1. Open the Data Flow Task
Manager 2. From the Toolbox add a Flat File Source to the Data Flow Task, and
• We define a single and specific file in this open the Flat file Source.
step, it could be to any of our existing files. 3. Click new to create a new Flat File Connection Manager
After this step we will define a Property 4. For “Description” and “Connection Manager Name” enter
Expression on the new “LoadProductDescriptions”
LoadProductDescriptions connection 5. For the “File Name” point to our first file
manager. The Property Expression will “C:\_SSIS_Training\LoopFiles\ProductDescriptions1.txt” (no
dynamically alter our connection manager to quotes)
load a different file per iteration of the loop. 6. Check the box “Unicode”
7. Use the default type of Delimited
8. Enter the pipe symbol “|” as the text qualifier ** We want to do
this because some of the descriptions contain double quotes,
commas, and semi-colons which can throw off normal delimiter
parsing.
9. Check the box “Column names in the first data row’
10. Click the “Columns” Page and see a preview of the data
11. Click the “Advanced” page. The column “ProductDescriptionID”
should be highlighted.
12. Change the “Text Qualifier” property to False.
13. Click the ‘Description’ Field, and set the OutPutColumnWidth to 400,
and Change the “Text Qualifier” property to True.
14. Click the column “Row Guid” and change the “Text Qualifier” property
to False.
15. Click the column “Modified Date” and change the “Text Qualifier”
property to False. We only want our “|” text qualifier for the
Description field\data.
16. Click OK to close the Connection Manager dialog.
17. Click OK to close the “Flat File Source Editor”.

Author: Craigg Page 19 of 53 Microsoft Corporation

 1 Add Column to hold name of the file 1. With the Flat File source selected, view the properties window and
processed.. 2. Set the “FileNameColumnName” property to “mysourcefilename”
• This will add a new column to our data 3. After you hit enter a warning icon may appear on the transform. If
flow, containing the filename we are you hover your mouse over the transform, a tool tip will mention a
processing, to each data row. Nice for meta data error.
auditing but obviously adds more data…
• This is a very useful feature/property but
a bit hidden in that it’s not part of the
advanced editor (when you double click 4. We need to refresh the meta data. Right click the source and choose
the Flat File Source). You can only edit “Advanced Editor”.
the property from the properties 5. Select the ‘Refresh’ button at the bottom and click OK close the
Window., which is why we had the source source.
‘selected’ but not ‘opened’.
• Note the advanced editor allows
powerful control of the input and output
of objects. Very useful when you need to
go back and edit an existing package and
tweak meta data in the middle of a flow.
1 Add a row count transform 1. Add, Connect, and open a Row Count transform to the dataflow
• To capture the # of rows processed to a 2. Enter “myrowcount” in the Variable Name property. (the variable
variable. we created earlier).
• Anomaly with the rowcount transform is 3. Click “ok” to close the Row Count transform.
you have to manually type in variable
names, it will not allow you to pick from
a list. So, remember variable
names\usage of are case sensitive.
1 Add Audit Transform 1. Add, Connect, and open an Audit Transform to the dataflow
• To add in useful meta data to our data 2. In the first blank row, click the “Audit Type” column and select
stream for capturing in log data, such “Execution Instance GUID”
as package name, and start time. 3. Note the name is automatically filled in for you.
4. Now keep adding audit types in order until you have all 9 added, so 9
rows, as seen in image below.

Author: Craigg Page 20 of 53 Microsoft Corporation

 5. Click “OK” to close the Audit transform
1 Add OLE DB Destination 1. Add, Connect, and open an OLE DB Destination
• To store the rows, with all of the audit 2. Click “New” to create a new connection manager to your server, and
information. database “SSISTRAINING”.
3. After you close the Connection manager dialog, select ‘Data Access
Mode’ of ‘Table or View’
4. Name of existing table “mydescriptions”
5. Click the “Mappings” page to insure your fields have mapped. You
might want to resort the either of the 2 table field boxes to make it
easier to verify the mappings. Click the top by ‘Name’ to sort
6. Click “OK” to close the OLE DB destination.
NOTE: if you receive a message similar to the following, verify the
Database SSISTRAINING was attached (use SQL Server Management
Studio).
• Test connection failed because of an error in initializing provider. Login
failed for user 'REDMOND\craigg'.
• Cannot open database "SSISTRAINING" requested by the login. The login
failed.

Author: Craigg Page 21 of 53 Microsoft Corporation

 Visually your package will look like….
NOTE: you can add ‘annotations’ (labels)
to your flow to help with instructions. To
add an annotation, right-click the design
surface and choose ‘Add Aannotation’

1 Execute the Package to insure it runs 1. Save the package

• We will loop 4 items, one for each file 2. Have the Dataflow task open so we can watch it execute, watch the
but we are loading the same file over number of rows when it executes.
and over? Why?? 3. Execute the package to insure it runs successfully.
4. ** Did you notice the row counts were all the same? Perhaps 88?
5. You can verify in the Progress Pane, looking for the rows information
such as the following, repeated 4 times, with the same ’88 rows’
[Flat File Source [1]] Information: The total number of data rows
processed for file "C:\_SSIS_Training\LoopFiles\ProductDescriptions1.txt"
is 88.

2 Modify Connection String to 1. Stop execution of the package if you have not done so already.
dynamically change (via property 2. In the Connection manager window select (but not open) the
expression) with loop iterations… “LoadProductDescriptions” connection manager. We want to view its
• Remember the variable “myfilenamefull” properties in the property sheet not the editor window. (**)
we created earlier. We need that to feed
3. If the property sheet is not already visible in the right side of your
our connection string per iteration of the
loop, via a property expression. screen click the button or choose “properties” from the View
• We will build another property expression
in the LoadApplications sample. menu.
• “User::” indicates the NameSpace.
4. In the properties pane click in the empty row for “Expressions” and

then click the ellipse button

Author: Craigg Page 22 of 53 Microsoft Corporation

 5. Choose the “ConnectionString” property and click the ellipse button
for the ‘Expression” column to go into the expression builder by
clicking the ellipse button
6. Expand the variables folder, and drag the myfilenamefull variable
down to the expression, which will end up looking like the following.
7. @[User::myfilenamefull]
8. Click OK to close the expression builder and then OK again to close
the Property Expression Editor window.
9. (**) As a side/extra exercise, go to the ‘Package Explorer’ tab
(same level as Control Flow and Data Flow) and find the expression
you entered before.
2 Execute the Package to insure it runs 1. Save the package
• We will loop 4 times, one for each file. 2. Have the Dataflow task open so we can watch it execute, watch the
Now we should see 4 different row count number of rows when it executes.
results. 3. Execute the package to insure it runs successfully.
4. You should see 4 different row counts this time. You can verify in the
Progress Pane, looking for the rows
[Flat File Source [1]] Information: The total number of data rows
processed for file "C:\_SSIS_Training\LoopFiles\ProductDescriptions4.txt"
is 206.

4.3.Conclusion

4.3.1.Comments
• One of the most common uses for Property Expressions is for dynamic Connection Strings.

4.3.2.BOL
• “Foreach Loop Container”
• “How to: Create a Property Expression
• ”Advanced Integration Services Expressions”
• “Using Property Expressions to Specify Data Flow Property Values”

Author: Craigg Page 23 of 53 Microsoft Corporation

 5. Hands-on Lab III (Expand loop to write iteration audit information)

C. Purpose of Hands-on Lab

• Provide an example of capturing execution information such as the number of rows processed, and push
it into a DB with Execute SQL.

5.1.Hands-on Lab Preset

D. Pre-stage
• Uses the package “LoopAndLoadProductionTable.dtsx” built in the previous lab.
• Stop execution of the previous package, “LoopAndLoadProductionTable.dtsx” if it’s still running.

5.2.Hands-on Lab Details

Discussion Points Actions taken

Author: Craigg Page 24 of 53 Microsoft Corporation

 1 Add and configure an Execute SQL 1. Stop Execution of or open the package “LoopAndLoadProductionTable.dtsx”
task to follow the DataFlow task… 2. While viewing the Control Flow (vs Data Flow)….
• Which will be used to write per 3. Add, Connect, and open an Execute SQL task. This is to follow/be attached to the
iteration information such as file name 2nd dataflow task, the one inside the For Each container.
and row count, to a table 4. Click on the “General” page
5. Leave the default “Connection Type” as “OLE DB”
6. For the “Connection” drop the list and choose “<New Connection>”
7. Click “new” to create a new connection manager to your server “DBIL405”, and
database “SSISLOGGING”. Verify with the Test Connection button.
8. Click OK to close the Connection Manager window.
9. Click on the “Parameter Mapping” page
10. Add 5 parameters, per the image below. Note, Order is not important but the
parameter names must match the variable names shown.

11. Click on the “General” page

12. Leave the default for “SQL Source Type” as “Direct Input”
13. Type copy/paste the following into the “SQL Statement” property
14. INSERT INTO myfileaudit
(packagename,packageid,sourcefilename,executionid,starttime,[rowcount])
VALUES (?,?,?,?,?,?)
15. Note: There is a ‘Parse Query’ button which can help verify your syntax in
many cases, but if you click it with our current package you will likely get a
message of “The query failed to parse. Parameter Information cannot be
derived from SQL statements…”. This is by design in that at design time the
SQL parser cannot retrieve/fill-in parameter values to replace the “?” place
holders.
16. Click “OK” to close the Execute SQL Task.

Author: Craigg Page 25 of 53 Microsoft Corporation

 2 Visually your package will look like….

3 Execute the Package to insure it 1. Save the package

runs 2. Execute the package to insure it runs successfully.

4 View the results of the audit data… 1. Open SQL Management Studio, and connect to your SQL server
“DBIL405”
2. Expand the Databases Folder, down to the database
SSISLOGGING, and table “myfileaudit”.
3. Right click the table name “myfileaudit” and choose “Open Table”

4. You should see data rows, with a row count column matching the
values we visually see.
5. You can keep re-running the packages and refresh the query in
SQL Management Studio with and see rows build up. Of
course the row counts match as we are re-running the same files
Author: Craigg Page 26 of 53 Microsoft Corporation
 but the ExecutionID and StartTime should Differ for each
execution.

5.3.Conclusion

5.3.1.Comments
• While we earlier used the audit transform to capture extra package data per row of execution (so Data Flow
level auditing), you might want to also capture data at the Control Flow level, especially when there are loops.
You can now see this is very easy to do with an execute SQL and the various/handy system variables.

5.3.2.BOL
• “Execute SQL Task”
• “How to: Map Query Parameters to Variables in an Execute SQL Task”

Author: Craigg Page 27 of 53 Microsoft Corporation

6. Hands-on Lab IV (Open a different application based on day of week)

E. Purpose of Hands-on Lab

• The purpose is to show an example of property expressions in a fringe operation, using the Execute
Process task to open other applications.
• Its also a very short exercise to walk through in case we are short on time or a student was not able to
get the previous labs to execute.

6.1.Hands-on Lab Preset

6.2.Hands-on Lab Details

Discussion Points Actions taken
1 Create a new SSIS package in the 1. Save and close all other open packages.
existing solution 2. In the Solution Explorer right click the “SSIS Packages” and choose “New SSIS
Package”.
3. Rights-Click the new package name and rename it “LoadAppliations.dtsx”.
4. Click ‘Yes’ to rename the package object as well.

2 Add Execute process task 1. Add and open an Execute Process Task to the new package.
5. Using property expressions we will 2. Click the “Process” page
configure the single task to open a 3. Enter “notepad.exe” for the ‘Executable’ property.
different application based on the day 4. Click the “Expressions” page
of the week. Open either notepad.exe
5. In the right pane click in the empty row for “Expressions” and then press the
or mspaint.exe depending on day of
ellipse button
week. Using a property Expression on
6. Choose the “Executable” property and either copy/paste the following
the 'executable' property.
expression or press the other ellipse button to go into the expression builder
6. Sunday=1, Monday=2, Tuesday=3,
and build this yourself. Its good practice to build and you can test this
4=Wednesday, 5=Thursday…
expression.
7. DATEPART("weekday", GETDATE()) ==5?"notepad.exe":"mspaint.exe"
8. Click “OK” as needed (2-3 times).
Author: Craigg Page 28 of 53 Microsoft Corporation
 9. Close the Execute Process task.

3 Execute the Package to insure it 1. Save the package, and execute.

runs 2. IF the current day is Thursday (5) then Windows Notepad will open, otherwise
Paint will.
3. NOTE: If the wrong package executes, you need to change the default object
in the solution by right clicking the desired package name
(“LoadApplications.dtsx.dtsx”) in the solution explorer
4. Note: The Execute Process task is still yellow and the package is still
considered ‘running’ until you close the application that opened, then the task
will turn green.

6.3.Conclusion

6.3.1.Comments
• Property Expressions are a very powerful feature. One of the most common uses is for dynamic connection
manger changes like one of the previous labs. Another is for the Send Mail Task. For example the following
expression is used in a property expression on the Subject property of a SendMail task...the message will
arrive with the name, start, duration information in the email subject, very handy! You can extended it and
add in a variable that is populated with a Row Count transform, then without even opening the message
you can see who, what, when.

"PExpression-->Package: (" + @[System::PackageName] + ") Started:"+ (DT_WSTR, 30)

@[System::StartTime] + " Duration:" + (DT_WSTR,10) (DATEDIFF( "ss", @[System::StartTime] ,
GETDATE() )) + " seconds"

6.3.2.BOL
• “Using Property Expressions in Packages”
• “Execute Process Task
• “Execute Process Task Editor (General Page)”

Author: Craigg Page 29 of 53 Microsoft Corporation

 7. Hands-on Lab V (Extend LoadApplications to feed it application names with a configuration file)

F. Purpose of Hands-on Lab

• The purpose is to show an example of Configuration file usage.
• The configuration file will feed values to variables on package load, which are then mapped into the
Property Expression instead of the hard coded executable names such as mspaint.exe

7.1.Hands-on Lab Preset

• Requires the previously built package “LoadApplications.dtsx”

7.2.Hands-on Lab Details

Discussion Points Actions taken
1 Add Variables to package 1. Open the existing “LoadApplications.dtsx” Package.
2. From the “SSIS” menu choose ‘Variables’
3. Create a string variable named “app1” with a ‘Value’ of Notepad.exe. The
Scope should be the package container, so ‘LoadApplications’.
4. Create a 2nd string variable names “app2” with a ‘Value’ of mspaint.exe. The
Scope should be the package container, so ‘LoadApplications’.

2 Edit the Execute process task 1. Open the Execute Process Task.
• Using property expressions we will 2. Click the “Expressions” page on the left.
configure the single task to open a 3. Expand the ‘Expressions’ list on the right, you should see our existing
different application based on the day of expression for the Executable property
the week. Open either notepad.exe or 4. DATEPART("weekday", GETDATE()) ==5?"notepad.exe":"mspaint.exe"
mspaint.exe depending on day of week.
Using a property Expression on the 5. Either manually edit or click the ellipse button to edit the expression
'executable' property. 6. Replace “notepad.exe” with @app1 and mspaint.exe with @app2 (no quote) so
• Sunday=1, Monday=2, Tuesday=3.. the expression looks like…
7. DATEPART("weekday", GETDATE()) ==5?@app1:@app2
8. Click OK as needed
9. Close the task
3 Execute to insure the package 1. Save the package, and execute
behaves as it did before
4 Now add an xml configuration to 1. Stop Execution\Debugging if you have not done so already.
Author: Craigg Page 30 of 53 Microsoft Corporation
 the package 2. From the SSIS menu choose “Package Configurations”.
• After you select the type of configuration 3. Click the “Enable Package Configurations” check box.
you want, you select what properties of 4. Click the Add button which will launch the configuration wizard.
which objects are added to the 5. Click Next
configuration file. 6. Leave the radio button Specify configuration settings directly’ selected.
• After the configuration is added, the 7. Choose an XML Configuration file as the type.
SSIS execution engine knows to look at 8. Enter C:\_SSIS_Training\myconfig.dtsConfig as the configuration filename.
the contents of the file, during the initial 9. Click Save.
load of the package, mapping the values 10. Click Next.
in the file to our variables. 11. Scroll up in the objects list and find our variables app1 and app2. NOTE: If you
do not see the variables listed under the package container, you likely had the
scope incorrect when you created them. Go back to the variables window and
verify the scope.

12. For each, we want to drill down to and check the box for the ‘Value’ property.
13. Click “Next” and you will see the confirmation page of the type and contents of
the configuration.
14. Click “Finish” and Close the Configuration organizer.

•
5 Review and edit the configuration 1. You want to edit the config file, C:\_SSIS_Training\myconfig.dtsConfig
file contents 2. You could use notepad.exe (you will want to choose ‘word wrap from format
menu)
3. Start>Run C:\_SSIS_Training\myconfig.dtsConfig
4. Or use BI Studio itself, by going to the File Menu>>Open >> File

We want to find the two tag sets for

<ConfiguredValue>notepad.exe</ConfiguredValue>
Which should be associated with Variables[User::app1]
<ConfiguredValue>mspaint.exe</ConfiguredValue>
Which should be associated with Variables[User::app2]
5. And change the first to Calc.exe and the other to dtexecui.exe
6. Save and close the file

6 Execute the Package 1. Save the package, and execute

2. To Insure there are no errors. What application loaded?

Author: Craigg Page 31 of 53 Microsoft Corporation

 3. Stop package Execution and Go find the app1 and app2 variables (SSIS menu and
choose variables) and see what their ‘value’ property now is

7.3.Conclusion

7.3.1.Comments
• You can have more than one configuration in a package, they are executed in the order you see them in
the configuration organizer.
• A common practice could be to use SQL Configurations from a central DB. With SQL Configurations you
can have more than 1 package from more than 1 server all using the central configuration table.

7.3.2.BOL
• “Package Configurations”
• “Creating Package Configurations”

Author: Craigg Page 32 of 53 Microsoft Corporation

8. Hands-on Lab VI (Error Rows and logging data)

G. Purpose of Hands-on Lab

• The purpose is to show an error flow from within the Data flow. Also an example of using a
MultiFlatFileConnection manager rather than a normal one.
• We will also add a SQL Log Provider

8.1.Hands-on Lab Preset

8.2.Hands-on Lab Details

Discussion Points Actions taken
1 Create a new SSIS package in the 1. In the Solution Explorer right click the “SSIS Packages” and choose “New SSIS
existing solution Package”
2. Right Click the new package name and re-name it “ErrorFlow.dtsx”, answer
‘Yes’ to rename the package object as well.

2 Add a DataFlow Task and Connection 1. Add and open a Data Flow Task
Mgr 2. In the Connection Manager window, right click and choose “New Connection”
3. Then Select the “MultiFlatFile” connection manager and click ADD
4. For the Name and Description enter “badrows”
5. For the Filename, browse to the folder
C:\_SSIS_Training\SourceDataErrorRows
6. The folder should contain 3 files with the name like “bad_data1.txt”
7. Pick one of the them and click “Open”
8. Click the ‘Columns’ tab and you should see some of the data, and you will
likely note the first column is a numeric field and some have XX making the
rows ‘bad’

Author: Craigg Page 33 of 53 Microsoft Corporation

 9. Click the ‘Advanced’ page and change the data for column0 to “four-byte
unsigned integer [DT_UI4]”
10. Click ‘OK’ You should see the new connection manager in the Connection
Managers window.
3 Add a Flat File Source 1. Add and open a Flat File Source and choose the “badrows” connection
• Which can use a FlatFile or manager.
MultiFlatFile connection manager 2. Click ‘OK’ to close the Source
3. Add and connect a Data Reader Destination; again we are using it as a ‘null
destination’.

4 Test the package execution 1. Save the package, and execute.

• Does it Fail? If we look at the 2. NOTE: If the wrong package executes, you need to change the default object
progress pane we can see indications in the solution by right clicking the desired package name (“ErrorFlow.dtsx”) in
of data type problem for a column and the solution explorer).
a row
5 Now add an error flow 1. From the Toolbox, Add another Data reader Destination to the package
• This will allow us to redirect the few 2. Click on the Flat File Source and connect the RED line to the new data reader
bad rows and continue processing the destination
rest of the rows 3. The “Configure Error Output” Dialog should appear.
4. For “column0” change the error column from “Fail Component” to “Redirect
Row”.
5. Click OK to close the configure error output
6.
6 Execute 1. Save the package, and execute.
• Users can extend this simple example 2. NOTE: If the wrong package executes, you need to change the default object
such that their error flow includes and in the solution by right clicking the desired package name
Audit Transform and (“LoadAppliations.dtsx.dtsx”) in the solution explorer.
“FileNameColumnName” property of the
Source connection, to then pass along 3. You should see a few rows flow down the error path. (note how many do and
good trouble shooting data to a central how many flow down the green Success path.
table
• In an Appendix you can see samples of
Author: Craigg Page 34 of 53 Microsoft Corporation
 Reporting Services reports that were built
on top of a centralized Error Rows table.

4. Add a Grid Data viewer to the Error path and re-execute to view the data.
5. ErrorCode users will be able to look up in Books Online.
6. The ErrorColumn matches the ID you can see in the Advanced editor of the
flatfile Source.
7. To see the Advanced editor, (stop execution) right click the flat file source and
choose ‘Advanced Editor’.
8. Then click the “Input and Output Parameters’ tab.
9. Expand the “Flat File Source Output”
10. Then expand the “Output Columns”
11. Click on column0 and the ID should match the one in the error rows

12. Click OK to close the Advanced Editor.

7 Modify the Connection Mgr to 1. Open the BadRows connection manger.

process all of the files 2. Change the filename to include the * wild card.
Author: Craigg Page 35 of 53 Microsoft Corporation
 • The * wild card WITH the multi-flat file 3. So from
connection manger, will process all files 4. C:\_SSIS_Training\SourceDataErrorRows\bad_data1.txt
that match the pattern. 5. To
6. C:\_SSIS_Training\SourceDataErrorRows\*.txt
8 Execute 1. Save the package, and execute.
2. You should see more rows for both the success and error flows.
9 Add Log Provider and details 1. Stop execution if you have not already
• SSIS has a fixed table format for the 2. Go to the SSIS menu and choose Logging
logging data. You can specify which SQL 3. Select the log provider type “SSIS Log Provider for SQL Server”,
Server and which database you want and click ADD
your data written to (via the OLE DB 4. Under Configuration select <New Connection..> unless there
Connection Manager you select). is already one for database SSISLOGGING. If not, create on.
• The first time Log data is written, SSIS 5. Once your connection is selected/created you need to ‘check’
Will automatically create a table called which containers should log data.
“sysdtslog90” 6. In the Left, containers window, Check the Error Flow (package)
• You can choose which log provider(s) per and then check the Name of the provider you created
container. 7. Then check the Data Flow task and again check the Name of the
provider you created

8. Now click the Details tab. You can select which log entries you
want. For now just check the ‘Events’ box at the top which will
pick up all the log entries. Do it for each item in the Containers
window, so select the contain name on the left, then ‘Events’
check box on the right.

Author: Craigg Page 36 of 53 Microsoft Corporation

 9. Click on the Advanced button and note you can select which
‘fields’ you want. For now leave the default of all fields selected.
10.NOTE: you can also Load and Save templates of the log entries
for use later.
11.Click OK to close the Logging window

1 Execute 1. Save the package, and execute.

0 2. Visually to you, it should execute the same but now we will have
rows written to our logging table.

1 View the results of the logging data… 1. Open SQL Management Studio, and connect to your SQL server
• In an Appendix you can see samples
“DBIL405”
of Reporting Services reports that
were built on top of a centralized Log 2. Expand the Databases Folder, down to the database
provider data. SSISLOGGING, and table “sysdtslog90”.. If you cannot find the
table there and you are sure you have executed the package, go
back the logging screen and verify the settings of the chosen
connection manager. Perhaps you selected one other than
SSISLOGGING and therefore the table was created in a different
DB.
3. Right click the table name “sysdtslog90” and choose “Open
Table”

4. You should see logging rows. Note there are very useful fields
like executionid which allows you to differentiate multiple
executions of the same package.

Author: Craigg Page 37 of 53 Microsoft Corporation

 5. You can keep re-running the package and refresh the query
with and see rows build up. Of course the row counts match
as we are re-running the same files but the ExecutionID and
StartTime should Differ for each execution

8.3.Conclusion

8.3.1.Comments
• So now we have seen 2 different approaches to processing multiple file, using a loop container and a
multi-flat file connection manager. One is not more ‘correct’ than the other, just depends on the
application. With a large number of files the loop structure approach would take longer because it needs
to start/close the data flow engine each time vs. the other approach where only 1 data flow is used.
However, the loop approach does provider more flexibility in the control flow, for example you can take
action after each file

8.3.2.BOL
• “Using Error Outputs”
• “Integration Services Log Providers”
• “Implementing Logging in Packages”

Author: Craigg Page 38 of 53 Microsoft Corporation

9. Hands-on Lab VII (Deploy packages and execute from Management Studio)

H. Purpose of Hands-on Lab

• The purpose is to show an example of deploying packages with the SSIS deployment utilities.
• This process essentially just copies files to a desired location, which could be SQL or file system share.

9.1.Hands-on Lab Preset

• Uses Solution and packages created in previous labs. You do not need them all to complete this exercise but if
you did not complete the configuration file exercise of the LoadApplications package, the configuration
information\exercise in this lab will not apply.

9.2.Hands-on Lab Details

Discussion Points Actions taken
1 Configure Deployment for the
1. Close all open packages in your Solution
solution.
2. From the “Project” Menu choose “SSISTraining Properties”, the
• Specify where to build the deployment
name of your solution. (it might say just ‘properties’ if you have
file set and turn on the set creation to
added more than 1 project to the solution)
occur when the solution is built.
• “AllowConfigurationChanges” controls if
the Installation wizard will prompt for
changes to values currently in a
configuration file, during the installation.

Author: Craigg Page 39 of 53 Microsoft Corporation

 3. Click the ‘Deployment Utility’ tab
4. Set the “AllowConfigurationChanges” to True
5. Set “CreateDeploymentUtility” to True
6. Leave the “Deployment Output Path” set to ‘bin\Deployment’. This
will be a subfolder under our solution
7. Click OK to close the “SSISTraining Property Pages”

2 Build the solution and Deployment 1. From the Build Menu choose “Build SSISTraining”
files 2. If you look in the output window you should see results similar to
• When you ‘build’ the solution SQL
the following screen shot. If the Build menu is not visible goto
Development Studio will create the
deployment file set which includes all menu View>>Other Windows>>Output.
of the packages from the Solution,
configuration files that are associated
with packages, as well as any files you
may have in the “Miscellaneous” folder
of your solution. This is a handy want

Author: Craigg Page 40 of 53 Microsoft Corporation

 to insure a readme file is deployed
with your package

3 Review the Deployment File Set 1. Go view the folder where our files were gathered
• Users can then copy\move the file set C:\_SSIS_Training \SSISTraining\bin\
to where they want to run deployment 2. You will see our packages (*.dtsx), the configuration file we created, and a
from. That machine needs to have manifest file that is used to perform the actual deployment.
SSIS installed to deploy, or else it will
not recognize the manifest file.
4 Install Package (Deploy) to 1. In file explorer double click the manifest file
another folder on the same “SSISTraining.SSISDeploymentManifest”
machine 2. In the Package installation wizard, Choose File System deployment.
• From the desired deployment machine, a 3. Click Next
can now run the manifest file and deploy 4. Use the default file path
(copy) packages to any file share or SQL 5. C:\Program Files\Microsoft SQL Server\90\DTS\Packages\SSISTraining
server where they have permissions. 6. Click Next
• Keep in mind the packages can only 7. The Installer will note it has enough information to install, click Next
execute on machines that have SSIS 8. The install will run a bit then should pause at a “Configure Packages” Screen’
installed (dtexec.exe) 9. For now just leave the existing values, calc.exe and dtexecui.exe, click Next
10. This is screen would allow you to change current configuration values as you
install. For example if you had configured connection managers, you can change
each installation for appropriate server names, without physically altering the
package itself.
11. Note on the ‘Finish’ Screen the log information about the Installation.
12. Click ‘Finish’ to complete and close the installation.
5 Execute Package using 1. You can browse to the folder where you deployed and double click a package to
DTEXECUI.exe open DTEXECUI.exe for that package.
2. You can use DTEXECUI to execute the package directly (Execute button on
bottom) or build a command line for use in a batch file, agent, other process.
3. When you execute via DTEXECUI.EXE you will see a progress window
(below)similar to what you see in SQL BI Studio. When you execute directly with
dtexec.exe from a command prompt, you will not see that window though
dtexec.exe supports many switches including console reporting. See help topic
suggestions at the end of this lab.

Author: Craigg Page 41 of 53 Microsoft Corporation

 6 Deploy to SQL Sever 1. Run the Installation wizard again (double click the manifest file)
• The Installation wizard will not convert 2. This time choose “SQL Server Deployment”
an XML configuration file to SQL 1. Choose your local server “(local)” or the name if the server, and
configuration when you install the windows authentication.
package to SQL sever, by design. 2. Click ‘Next’
3. The next screen is regarding package dependency files
(configuration, miscellaneous). Even though the package is going
into SQL, the configuration files still need a file location default
should be fine. Click ‘Next’
4. Click ‘Next’ on the ‘Confirm Installation’ screen.
5. Click After the ‘enough information’ wizard you will see the screen
for editing configuration values. You can experiment with other
execution file name if you like. OR just leave what’s there and
click Next.

Author: Craigg Page 42 of 53 Microsoft Corporation

 6. And Finish on the Finish Installation screen.

7 Open SQL Management Studio to 1. Open SQL Server Management Studio, connect to your local SQL
see and execute the packages you box with the initial connection dialog.
just installed to SQL 2. Once in Mgt Studio, connect to you local SQL Server Integration
Services, server. One way is to double click the name in the
‘Registered Servers’ pane

8 Find and execute package 1. Expand the Stored packages folder down to your packages in
“LoadApplications” MSDB

2. Right Click the “LoadApplications” package and choose “Run

Package”
3. DTEXECUI.exe runs and is pre-populated with the package
information.
4. Execute the package using the Execute button on the bottom.
5. If we left the package configuration file as is, another instance of
dtexecUI will likely load (it was the value of “app2”)
6. Before you close dtexecui (or whichever application loaded), go to

Author: Craigg Page 43 of 53 Microsoft Corporation

 the “Running Packages” node in SQL Management Studio. You
should see your package listed there.
7. If you click the running package name and then the “Report”
button in the right pane, you will see details about the currently
executing package, such as Package name, Start Time….

9.3.Conclusion

9.3.1.Comments
• The 2nd level of Folders found in the SQL Mgt Studio ‘Stored Packages folder’ can be controlled by the
user via a configuration file used by the SSIS Windows Service. The default file installed, ships with
folder names “MSDB” and “Maintenance plans” but the user can create whatever XML configuration
(service configuration not package configuration) file they like, and then modify a registry key to tell the
SSIS service, on Service start, what file to load and where its located. See BOL topics.

9.3.2.BOL
• “Creating a Deployment Utility”
• “How to: Create an Integration Services Package Deployment Utility”
• “Installing Packages”
• “How to: Run a Package Using the DTExecUI Utility “
• “Command Prompt Utilities (SSIS) “
• “Configuring the Integration Services Service”
• “Scheduling Package Execution in SQL Server Agent”
•

Author: Craigg Page 44 of 53 Microsoft Corporation

 10.Hands-on Lab VIII (Add Execute SQL to truncate table to the LoopAndLoadProductionTable package)

I. Purpose of Hands-on Lab

• The purpose is to show an example of the Execute SQL task to truncate the table ‘mydescriptions’ prior
to loading of new data
• Because SSIS is a platform, you can easily create packages that focus Database maintenance such as
truncate, drop, attach, move. The “Maintenance Plans“ feature found in the tree [your server
name]\Management folder of SQL Management Studio, uses SSIS tasks under the hood.
•

10.1.Hands-on Lab Preset

• Uses the package LoopAndLoadProductionTable.dtsx that was built earlier in the lab

10.2.Hands-on Lab Details

Discussion Points Actions taken
1 Verify number of rows in the table 1. We want some data in the table ‘mydescriptions’ so execute the
‘mydescriptions’ ‘LoopAndLoadProductionTable.dtsc’ package several times prior to
executing this lab.
2. Use SQL Management Studio to find the table ‘mydescriptions’
in the ‘SSISTRAINING’ database, and verify the number of rows in
the table (right click the table name and select ‘Open Table’)
2 Add and configure an Execute SQL 1. While viewing the Control Flow (vs Data Flow)
task 2. Add, Connect, and open an Execute SQL task. This is to precede
• This will clear all the rows in the the 1st dataflow task
‘mydescriptions’ table 3. Leave the default “Connection Type” as “OLE DB”
4. For the “Connection” drop the list and choose the connection
manager for the SSISTRAINING Database
5. Leave the default for “SQL Source Type” as “Direct Input”
6. Type copy/paste the following into the “SQL Statement” property.
This is a straight forward SQL statement and a good one for using
the ‘Parse Query’ button to validate your syntax.
truncate table mydescriptions
7. Click “OK” to close the Execute SQL Task.
Author: Craigg Page 45 of 53 Microsoft Corporation
 3 Visually your package will look
like….

4
1. Save the package
2. EXECUTE
3. Verify the number of rows now in the ‘mydescriptions’ table
4. NOTE: If the wrong package executes, you need to change the
default object in the solution by right clicking the desired package
name (“LoadAppliations.dtsx.dtsx”) in the solution explorer.

Author: Craigg Page 46 of 53 Microsoft Corporation

 5 Disable the new Task
• In the Control Flow, right click the New
Execute SQL task and choose ‘Disable’.
Notice the task turns gray (see image).
Now when you execute the package
the execute SQL is ignored.
• THIS IS A GOOD DESIGN TIP\Practice
in some scenarios to have little
maintenance like tasks built into your
package but disabled. Then they are
there for maintenance/trouble shooting
as needed
.
Now you should be able to run the package several times in a row, see
the number of rows increasing in the table, then enable the task and
disable the others to verify the table is clean.
** Out of class, little extra work. Add a Sequence container to the
package, it will contain your first DataFow task, ForEach Loop, and the
Inserting Execute SQL task…which makes it easy to disable all of the at
once (by disable the sequence container). Great if you want to disable
everything but the ‘truncating’ Execute SQL, use it to truncate all of the
data, then reverse what is enabled/disabled and continue on.

Author: Craigg Page 47 of 53 Microsoft Corporation

 10.2.1.Comments
•

10.2.2.BOL
• “Maintenance Tasks “
• “Execute SQL Task “

Author: Craigg Page 48 of 53 Microsoft Corporation

 11.Appendix A Where to learn more

• The SSIS portal on MSDN. Lots of great information including white papers, Webcasts, recommended books.
http://msdn.microsoft.com/SQL/sqlwarehouse/SSIS/default.aspx

• SSIS Support Forum on MSDN (moving away from newsgroups)

http://forums.microsoft.com/msdn/showforum.aspx?forumid=80

• Of course the excellent SQL Server Books Online which ship with the product. You\customers can also download
a separate copy. Handy for initial investigations when they want some details on specific features but are not
ready yet to install and play with the product.
http://www.microsoft.com/downloads/details.aspx?FamilyId=F0D182C1-C3AA-4CAC-B45C-
BD15D9B072B7&displaylang=en

Author: Craigg Page 49 of 53 Microsoft Corporation

12.Appendix B sample Reports for SSIS Log Providers (OnPipeline Rows sent and

These are just examples of Reporting Services Reports you can create based on the data. SSIS is a data integration
platform that includes various ways to produce detailed ‘instance data’ (Logging, Error Rows, and Audit Information in
flow) which customers can pull together in whatever way best suits them. One reason why there is not a
detailed/fixed support console like application. Every customers needs are different and we provide the data. The
examples here were built in SQL 2005 Reporting Services and will be available at some point for customers,
downloadable or via a RS report pack.

12.1.General Log summary, history, results.

Report showing summary of package executions results/history across machines. This is supported by all packages
executing on various machines all using the same SQL Log provider destination, providing central log collection and
reporting.

Author: Craigg Page 50 of 53 Microsoft Corporation

 12.2.Detailed package log history report
A More detailed report including a graph of execution instance vs execution duration.

Author: Craigg Page 51 of 53 Microsoft Corporation

 12.3.Detailed Pipeline report
This report is low level, based on the even OnPipelinRowsSent event that can be logged (if you turn it on in the
log provider window).
The report is showing the number of rows that passed through the data paths and components for dataflow
task, per execution. A customer may find this information very useful to review over time and may want to
create a data warehouse with these sorts of results

Author: Craigg Page 52 of 53 Microsoft Corporation

 12.4.Error Row Report
The following report is against a central table containing data from Error Row data from Data Flow Tasks.

Author: Craigg Page 53 of 53 Microsoft Corporation