Data Flow ->> Script Component

时间:2022-09-17 08:57:35

和Control Flow中的Script Task非常类似,不同的是Script Component是Per-Row的执行类型。打个比方,在Script Component中加入两个Output的字段,Script中针对每一行可以输出不同的值给这两个Output字段。Script Component要求你指定它是Source、Destination和Transformation中的哪一种。Source只有Input,Destination和Transformation可以有Input和Output,不同的是Destination的Input不能同时作为Output,而Transfomation两者都可以有。

--------------------------------------- Update 10/13/2015 --------------------------------------------------------------------------------------------

今天刚好有一个场景需要实现,具体就是要更新某张表的字段,因为表名是动态的,希望用Data Flow数据库上游生成的值更新。这个实现过程其实可以通过把数据保存到一个文件或者数据库表中再join的,但是因为表名是不确定的,如果用SQL实现就必须用动态SQ。还有另一个办法就是用Script Component。就像下图。通过加载、筛选再输出表字段的结果集给Fuzzy lookup,生成similarity和confidence给下游作为输入更新表的字段。这里有两个Script Component。

Data Flow ->> Script Component

input component

#region Help:  Introduction to the Script Component
/* The Script Component allows you to perform virtually any operation that can be accomplished in
* a .Net application within the context of an Integration Services data flow.
*
* Expand the other regions which have "Help" prefixes for examples of specific ways to use
* Integration Services features within this script component. */
#endregion #region Namespaces
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using Microsoft.SqlServer.Dts.Runtime.Wrapper;
using System.Data.SqlClient;
#endregion /// <summary>
/// This is the class to which to add your code. Do not change the name, attributes, or parent
/// of this class.
/// </summary>
[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
{
#region Help: Using Integration Services variables and parameters
/* To use a variable in this script, first ensure that the variable has been added to
* either the list contained in the ReadOnlyVariables property or the list contained in
* the ReadWriteVariables property of this script component, according to whether or not your
* code needs to write into the variable. To do so, save this script, close this instance of
* Visual Studio, and update the ReadOnlyVariables and ReadWriteVariables properties in the
* Script Transformation Editor window.
* To use a parameter in this script, follow the same steps. Parameters are always read-only.
*
* Example of reading from a variable or parameter:
* DateTime startTime = Variables.MyStartTime;
*
* Example of writing to a variable:
* Variables.myStringVariable = "new value";
*/
#endregion #region Help: Using Integration Services Connnection Managers
/* Some types of connection managers can be used in this script component. See the help topic
* "Working with Connection Managers Programatically" for details.
*
* To use a connection manager in this script, first ensure that the connection manager has
* been added to either the list of connection managers on the Connection Managers page of the
* script component editor. To add the connection manager, save this script, close this instance of
* Visual Studio, and add the Connection Manager to the list.
*
* If the component needs to hold a connection open while processing rows, override the
* AcquireConnections and ReleaseConnections methods.
*
* Example of using an ADO.Net connection manager to acquire a SqlConnection:
* object rawConnection = Connections.SalesDB.AcquireConnection(transaction);
* SqlConnection salesDBConn = (SqlConnection)rawConnection;
*
* Example of using a File connection manager to acquire a file path:
* object rawConnection = Connections.Prices_zip.AcquireConnection(transaction);
* string filePath = (string)rawConnection;
*
* Example of releasing a connection manager:
* Connections.SalesDB.ReleaseConnection(rawConnection);
*/
#endregion #region Help: Firing Integration Services Events
/* This script component can fire events.
*
* Example of firing an error event:
* ComponentMetaData.FireError(10, "Process Values", "Bad value", "", 0, out cancel);
*
* Example of firing an information event:
* ComponentMetaData.FireInformation(10, "Process Values", "Processing has started", "", 0, fireAgain);
*
* Example of firing a warning event:
* ComponentMetaData.FireWarning(10, "Process Values", "No rows were received", "", 0);
*/
#endregion /// <summary>
/// This method is called once, before rows begin to be processed in the data flow.
///
/// You can remove this method if you don't need to do anything here.
/// </summary>
/// IDTSConnectionManager100 connMgr;
SqlConnection sqlConn;
SqlDataReader sqlReader; public override void AcquireConnections(object Transaction)
{
connMgr = this.Connections.StageDB;
sqlConn = (SqlConnection)connMgr.AcquireConnection(null); } public override void PreExecute()
{
SqlCommand cmd = new SqlCommand("SELECT src_column_name FROM Stage.dbo.[" + this.Variables.varvalidtablename + "] Where src_column_name is not null and trg_column_name is null", sqlConn);
sqlReader = cmd.ExecuteReader();
/*
* Add your code here
*/
} /// <summary>
/// This method is called after all the rows have passed through this component.
///
/// You can delete this method if you don't need to do anything here.
/// </summary>
public override void PostExecute()
{
sqlReader.Close();
/*
* Add your code here
*/
} public override void CreateNewOutputRows()
{
/*
Add rows by calling the AddRow method on the member variable named "<Output Name>Buffer".
For example, call MyOutputBuffer.AddRow() if your output was named "MyOutput".
*/
while (sqlReader.Read())
{
{
MyOutputBuffer.AddRow();
MyOutputBuffer.SrcColumnName = sqlReader.GetString();
}
}
} public override void ReleaseConnections()
{ connMgr.ReleaseConnection(sqlConn); } }

destination script component

#region Help:  Introduction to the Script Component
/* The Script Component allows you to perform virtually any operation that can be accomplished in
* a .Net application within the context of an Integration Services data flow.
*
* Expand the other regions which have "Help" prefixes for examples of specific ways to use
* Integration Services features within this script component. */
#endregion #region Namespaces
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using Microsoft.SqlServer.Dts.Runtime.Wrapper;
using System.Data.SqlClient;
#endregion /// <summary>
/// This is the class to which to add your code. Do not change the name, attributes, or parent
/// of this class.
/// </summary>
[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
{
#region Help: Using Integration Services variables and parameters
/* To use a variable in this script, first ensure that the variable has been added to
* either the list contained in the ReadOnlyVariables property or the list contained in
* the ReadWriteVariables property of this script component, according to whether or not your
* code needs to write into the variable. To do so, save this script, close this instance of
* Visual Studio, and update the ReadOnlyVariables and ReadWriteVariables properties in the
* Script Transformation Editor window.
* To use a parameter in this script, follow the same steps. Parameters are always read-only.
*
* Example of reading from a variable or parameter:
* DateTime startTime = Variables.MyStartTime;
*
* Example of writing to a variable:
* Variables.myStringVariable = "new value";
*/
#endregion #region Help: Using Integration Services Connnection Managers
/* Some types of connection managers can be used in this script component. See the help topic
* "Working with Connection Managers Programatically" for details.
*
* To use a connection manager in this script, first ensure that the connection manager has
* been added to either the list of connection managers on the Connection Managers page of the
* script component editor. To add the connection manager, save this script, close this instance of
* Visual Studio, and add the Connection Manager to the list.
*
* If the component needs to hold a connection open while processing rows, override the
* AcquireConnections and ReleaseConnections methods.
*
* Example of using an ADO.Net connection manager to acquire a SqlConnection:
* object rawConnection = Connections.SalesDB.AcquireConnection(transaction);
* SqlConnection salesDBConn = (SqlConnection)rawConnection;
*
* Example of using a File connection manager to acquire a file path:
* object rawConnection = Connections.Prices_zip.AcquireConnection(transaction);
* string filePath = (string)rawConnection;
*
* Example of releasing a connection manager:
* Connections.SalesDB.ReleaseConnection(rawConnection);
*/
#endregion #region Help: Firing Integration Services Events
/* This script component can fire events.
*
* Example of firing an error event:
* ComponentMetaData.FireError(10, "Process Values", "Bad value", "", 0, out cancel);
*
* Example of firing an information event:
* ComponentMetaData.FireInformation(10, "Process Values", "Processing has started", "", 0, fireAgain);
*
* Example of firing a warning event:
* ComponentMetaData.FireWarning(10, "Process Values", "No rows were received", "", 0);
*/
#endregion IDTSConnectionManager100 connMgr;
SqlConnection sqlConn;
SqlCommand sqlCmd;
SqlParameter sqlParam; /// <summary>
/// This method is called once, before rows begin to be processed in the data flow.
///
/// You can remove this method if you don't need to do anything here.
/// </summary>
/// public override void AcquireConnections(object Transaction)
{ connMgr = this.Connections.StageDB;
sqlConn = (SqlConnection)connMgr.AcquireConnection(null); } public override void PreExecute()
{
sqlCmd = new SqlCommand("DECLARE @str NVARCHAR(30); SET @str = @trg_column_name + ':' + CAST(@similarity AS NVARCHAR(10)) + ':' + CAST(@confidence AS NVARCHAR(10));" +
" UPDATE Stage.dbo.[" + this.Variables.varvalidtablename + "]" +
" SET src_similarity_confidence_with_trg = ISNULL(src_similarity_confidence_with_trg,'') + '{' + @str + '}' WHERE src_column_name = @src_column_name", sqlConn);
sqlParam = new SqlParameter("@similarity", SqlDbType.Float);
sqlCmd.Parameters.Add(sqlParam);
sqlParam = new SqlParameter("@confidence", SqlDbType.Float);
sqlCmd.Parameters.Add(sqlParam);
sqlParam = new SqlParameter("@src_column_name", SqlDbType.NVarChar, );
sqlCmd.Parameters.Add(sqlParam);
sqlParam = new SqlParameter("@trg_column_name", SqlDbType.NVarChar, );
sqlCmd.Parameters.Add(sqlParam);
/*
* Add your code here
*/
} /// <summary>
/// This method is called after all the rows have passed through this component.
///
/// You can delete this method if you don't need to do anything here.
/// </summary>
public override void PostExecute()
{
base.PostExecute();
/*
* Add your code here
*/
} /// <summary>
/// This method is called once for every row that passes through the component from Input0.
///
/// Example of reading a value from a column in the the row:
/// string zipCode = Row.ZipCode
///
/// Example of writing a value to a column in the row:
/// Row.ZipCode = zipCode
/// </summary>
/// <param name="Row">The row that is currently passing through the component</param>
public override void Input0_ProcessInputRow(Input0Buffer Row)
{
sqlCmd.Parameters["@similarity"].Value = Row.Similarity;
sqlCmd.Parameters["@confidence"].Value = Row.Confidence;
sqlCmd.Parameters["@src_column_name"].Value = Row.SrcColumnName;
sqlCmd.Parameters["@trg_column_name"].Value = Row.SrcColumnName; sqlCmd.ExecuteNonQuery();
} public override void ReleaseConnections()
{ connMgr.ReleaseConnection(sqlConn); } }

Data Flow ->> Script Component的更多相关文章

  1. Data Flow -&gt&semi;&gt&semi; Source -&gt&semi;&gt&semi; Error Output -&gt&semi;&gt&semi; Error &amp&semi; Truncation&colon; Ignore Failure&comma; Redirect Now&comma; Fail Component

    Ignore Failure: 当该字段遇到错误时,字段值被设为NULL Redirect Now: 把该行输出到SSIS的Source组件的红色输出线,这时红色输出线应该连接一个可以接受结果集的组件 ...

  2. Script component 用法

    在SSIS中,可以使用C#编写脚本,这是十分激动人心的事,能够使用C#代码,使得Script Component无所不能. 第一部分:组件简介Script Component 有三种类型:Source ...

  3. 使用Script Component源处理不规则平面文件

    微软 BI 系列随笔 - SSIS 2012 高级应用 - Script Component处理不规则平面文件 场景介绍 在使用SSIS从平面文件导入源数据时,最常遇到的是以下两种情况: 导入规则的平 ...

  4. 微软BI 之SSIS 系列 - 使用 Script Component Destination 和 ADO&period;NET 解析不规则文件并插入数据

    开篇介绍 这一篇文章是 微软BI 之SSIS 系列 - 带有 Header 和 Trailer 的不规则的平面文件输出处理技巧 的续篇,在上篇文章中介绍到了对于这种不规则文件输出的处理方式.比如下图中 ...

  5. SSIS -&gt&semi;&gt&semi; Control Flow And Data Flow

    In the Control Flow, the task is the smallest unit of work, and a task requires completion (success, ...

  6. 微软BI 之SSIS 系列 - 通过 ROW&lowbar;NUMBER 或 Script Component 为数据流输出添加行号的方法

    开篇介绍 上午在天善回答看到这个问题 - SSIS 导出数据文件,能否在第一列增加一个行号,很快就帮助解决了,方法就是在 SQL 查询的时候加一个 ROW_NUMBER() 就可以了. 后来想起在两年 ...

  7. &lbrack;转&rsqb;How to handle Failed Rows in a Data Flow

    本文转自:http://www.rad.pasfu.com/index.php?/archives/23-How-to-handle-Failed-Rows-in-a-Data-Flow.html s ...

  8. Data Flow的Error Output

    一,在Data Flow Task中,对于Error Row的处理通过Error Output Tab配置的. 1,操作失败的类型:Error(Conversion) 和 Truncation. 2, ...

  9. SSIS Data Flow 的 Execution Tree 和 Data Pipeline

    一,Execution Tree 执行树是数据流组件(转换和适配器)基于同步关系所建立的逻辑分组,每一个分组都是一个执行树的开始和结束,也可以将执行树理解为一个缓冲区的开始和结束,即缓冲区的整个生命周 ...

随机推荐

  1. 使用自定义标签模拟jstl的&lt&semi;c&colon;for each&gt&semi;标签

    一.自定义标签的基本编写 下面编写一个自定义标签,它可以输出当前的时间. 1.编写标签类 类可以通过继承SimpleTagSupport类实现一个标签类编写.父类为我们提供了一些编写自定义标签的快捷的 ...

  2. java 顺序表

    想看看java版的数据结构,了解一下树的一些操作,写了个顺序表熟悉一下 package com.sqlist; /** * @author xiangfei * 定义一个顺序表 * */ public ...

  3. Linux计时体系结构

    [Linux操作系统分析]定时测量——RTC,TSC,PIT,jiffies,计时体系结构,延迟函数   1 基本概念 定时机制连同一些更可见的内核活动(如检查超时)来驱使进程切换. 两种主要的定时测 ...

  4. PCL点云库中怎样读取指定的PCD文件,又一次命名,处理后保存到指定目录

    我一直想把处理后的pcd文件重命名,然后放到指定的目录,尝试了好久最终做到了: 比方我想读取  "table_scene_lms400.pcd" 把它进行滤波处理,重命名为 &qu ...

  5. 翻译:SELECT INTO语句(已提交到MariaDB官方手册)

    本文为mariadb官方手册:SELECT INTO的译文. 原文:https://mariadb.com/kb/en/selectinto/我提交到MariaDB官方手册的译文:https://ma ...

  6. DOM元素查找

    一.DOM是document的缩写,他是操作html文档的方法 二.常用查找元素的方法 直接 1.document.getElementById('标签的id')   在html中标签的id是不允许重 ...

  7. PHP 判断字符串括号是否匹配

    <?php function aa($str) { $temp = array(); for ($i = 0; $i < strlen($str); $i++) { $t = $str[$ ...

  8. octave画心形曲线

    octave是gnu出品和matlab兼容的科学计算软件,具有体积小,兼容性好,免费的优点. 心形曲线是根据函数:( x2 + y2 -1 )3 - x2y3=0 改编而成. clear all; c ...

  9. HDU 1166 - 敌兵布阵 - &lbrack;线段树&rsqb;&lbrack;树状数组&rsqb;

    题目链接:http://acm.hdu.edu.cn/showproblem.php?pid=1166 Time Limit: 2000/1000 MS (Java/Others) Memory Li ...

  10. Windows OS系统变量

    %userprofile% C:\Users\Administrator\ %windir% C:\Windows\