If I have the option of using .Net and can do data transformations just fine in .Net, when would I need SSIS? Is there a certain task that SSIS would be better for? Are the added benefits of transparency worth it? Is it just what I am more comfortable with? What are the best practices for determining this?
如果我可以使用。net,并且可以在。net中进行数据转换,那么什么时候需要SSIS呢?SSIS有什么更好的任务吗?透明度带来的额外好处值得吗?这就是我更喜欢的吗?确定这一点的最佳实践是什么?
11 个解决方案
#1
39
good question.
好问题。
if the amount of data transfer huge? are you processing multiple data files and need transactions (both at file system level and database level)? are you dealing with multiple data sources at different locations (for eg ftp, local file system, database)?
如果数据传输量巨大?您是否正在处理多个数据文件并需要事务(在文件系统级和数据库级)?您是否正在处理不同位置的多个数据源(例如ftp、本地文件系统、数据库)?
if answers to above are yes then go ahead with ssis. basically .net is cool with small data import / export jobs, but when you have anything more complex, ssis is a definite winner
如果以上的答案是肯定的,那么请继续进行ssis。基本上。net对于小的数据导入/导出任务来说很酷,但是当你有更复杂的任务时,ssis绝对是赢家
the other thing which i look at is - is it worth writing .net code when everything is available inside ssis. (dont mistake me - i love coding) however, anything you code, you need to maintain :-)
我看到的另一件事是,在ssis中所有内容都可用时,它是否值得编写。net代码。(不要误会我-我喜欢编码)然而,任何你编码的东西,你需要保持:
#2
18
I think project time/budget constraints and the use of a standard tool are some of the biggest arguments for using SSIS. Creating an SSIS package is most of the times way faster than trying to code something similar in .NET.
我认为项目时间/预算限制和标准工具的使用是使用SSIS的最大理由。创建一个SSIS包的速度比在。net中编写类似的代码快得多。
But with that said, it seems like SSIS have a lot of pain points that sometimes might invalidate this argument. It did for me when developing a solution that needed to run in different environments at many different clients. SSIS simply looked too painful the more I evaluated it for the project. A properly architected .NET solution is easier to deploy, more reliable, more flexible, easier to understand and can also achieve very good performance.
但话虽如此,SSIS似乎有很多痛点,有时可能会推翻这个论点。它为我开发了一个解决方案,需要在不同的客户端运行不同的环境。SSIS看起来太痛苦了,我对这个项目进行了评估。一个架构良好的。net解决方案更容易部署,更可靠,更灵活,更容易理解,也能获得很好的性能。
IMHO: consider using SSIS for projects that you only need to deploy to one or maybe two in house SQL Server environments. Otherwise, the .NET approach will quickly become more appealing.
IMHO:考虑在只需要部署到house SQL Server环境中的一个或两个项目中使用SSIS。否则,. net方法将很快变得更有吸引力。
#3
5
I guess it depends on what you are doing. SSIS is very powerful, just like old DTS. If you are loading lots of items and expect to have constant change, I would go SSIS all the way. If you are looking to load only a few items and it’s for lots of customers, I would put it in code. I prefer SSIS for in house ETL processes, but I use .Net at client shops when I need to load data from a legacy system into a SQL database. Now as I stated before if you have a lot of transformations and lots of different data silos to load, I think you would be crazy to do this in .Net and I would go SSIS. If you have only a few items to load and it’s for a single application and may be installed as part of an application at various clients, I would go .Net all the way. Just my 2 cents.
我想这取决于你在做什么。SSIS非常强大,就像老DTS一样。如果你装载了大量的项目并且期望有恒定的变化,我会一直使用SSIS。如果你想只加载一些东西,而且它是给很多客户的,我会把它写在代码里。我更喜欢SSIS for in house ETL进程,但是当我需要从遗留系统加载数据到SQL数据库时,我在客户端商店使用。net。就像我之前说的,如果你有很多的转换和很多不同的数据竖井要加载,我认为你在。net中做这个会很疯狂,我会去SSIS。如果您只有几个项目要加载,并且它是针对单个应用程序的,并且可以作为不同客户端的应用程序的一部分安装,那么我将一直使用。net。那只是我的个人意见。
#4
4
I have a lot of experience with SSIS from small projects to large, complex ETL. Without going into the details, this is my guidance for you:
从小型项目到大型、复杂的ETL,我在SSIS有很多经验。以下是我对你的指导,不谈细节。
-
If you are a DBA and you are not familiar with .NET, or if you are a developer quite familiar with SSIS, then you can use SSIS for small, simple, fairly straightforward extract, transform, load (ETL) tasks.
如果您是DBA,并且不熟悉. net,或者是非常熟悉SSIS的开发人员,那么您可以将SSIS用于小型、简单、相当简单的提取、转换、加载(ETL)任务。
-
SSIS is very quirky and there are many pitfalls, gotchas, and what might be considered outright bugs. It is extremely powerful if you are intimately familiar.
SSIS是非常古怪的,有很多陷阱,陷阱,以及可能被认为是完全错误的东西。如果你非常熟悉它,它会非常强大。
-
C# now has TPL Dataflow. Simple performance tests put it ahead of SSIS. (eg http://mymemoryleaks.blogspot.cz/2013/10/ssis-vs-tpldataflow.html)
c#现在有TPL数据流。简单的性能测试使它领先于SSIS。(如http://mymemoryleaks.blogspot.cz/2013/10/ssis-vs-tpldataflow.html)
-
If you want to do anything beyond trivial, and if you can use .NET skills, use .NET instead of SSIS.
如果你想要做一些无关紧要的事情,如果你能使用。net技能,那就使用。net而不是SSIS。
#5
4
My arguments for not using SSIS are:
我不使用SSIS的理由是:
-
Design greenfield products so that they have RESTful data feeds for reporting and extraction built-in to the project plan and budget, preferably to a standard like OData so that other tools can plug right in.
设计greenfield产品,使其具有用于报告和提取的RESTful数据提要,并将其嵌入到项目计划和预算中,最好是针对OData这样的标准,以便其他工具可以直接插入。
-
Data feeds should pull and transform from upstream systems and feeds on demand; such that schedule tasks, configuration of scheduled tasks, task runner VMs and staff to run all this unreliable scheduling stuff is negated.
数据提要应根据需要从上游系统和提要中提取和转换;这样,调度任务、计划任务的配置、任务运行程序vm和运行所有这些不可靠调度内容的人员就被否定了。
-
RESTful data feeds leverage HTTP caching.
RESTful数据feed利用HTTP缓存。
-
Feeds/services/APIs can be moved to elastic-scale cloud easily.
可以很容易地将提要/服务/ api移动到弹性大小的云中。
-
SSIS requires finding people with SSIS skills that enjoy doing that stuff for weeks. In my experience, finding and retaining SSIS developers is hard and expensive and the people found tend to be sub-par.
SSIS需要找到那些具有SSIS技能的人,他们喜欢在几周内完成这些工作。根据我的经验,寻找和保留SSIS的开发人员是非常困难和昂贵的,而且人们发现他们往往是低于标准的。
-
SSIS doesn't work well with source control and collaborative work.
SSIS不能很好地与源代码控制和协作工作配合。
-
SSIS doesn't lend itself well to code reuse, unlike microservices and traditional code libraries.
与微服务和传统的代码库不同,SSIS不太适合代码重用。
-
SSIS doesn't version easily, unlike a REST service.
与REST服务不同,SSIS不容易版本化。
-
SSIS doesn't lend itself to modular designs and continuous deployment of many small changes, it tends to be large-batch with scary releases.
SSIS不支持模块化设计和持续部署许多小的更改,它倾向于大量发布可怕的版本。
-
SSIS promotes the use of stored-procedures which places a lot of demand on SQL which is the hot-spot. Favour designs that place demands on a scaleable, stateless middle tier.
SSIS促进了存储过程的使用,这使得对SQL的大量需求成为热点。支持对可扩展的、无状态的中间层进行需求的设计。
-
The tooling is clunky and unreliable.
该工具笨重且不可靠。
-
You're at the mercy of Microsoft's roadmap for SSIS.
你得看微软SSIS的路线图了。
-
Consider writing to tables/services that support analysis, reporting and views as soon as the data comes into the application; see CQRS and other application architecture patterns.
考虑在数据进入应用程序后立即写信给支持分析、报告和视图的表/服务;参见CQRS和其他应用程序体系结构模式。
-
Never use Excel as a data source; train employees.
绝不使用Excel作为数据源;培训员工。
-
Code is king.
代码为王。
Ultimately, I see SSIS as a relic of Enterprise IT. I like to ask, "Would Google use SSIS?" How else can the problem be solved? Think outside the box.
最后,我认为SSIS是企业IT的遗物。我想问,“谷歌会使用SSIS吗?”这个问题还能怎么解决呢?跳出固有思维模式。
#6
2
SSIS has many built in ways of doing transformations from different data sources and you can string them together in a way that makes it very customizeable. They have built in optimizations that make them fast.
SSIS有许多构建方式,用于从不同数据源进行转换,您可以将它们以一种非常可定制的方式连接在一起。他们进行了优化,使它们运行得更快。
You can also use .NET to make your own custom transformations to take advantage of the speed and repeatability of an SSIS job.
您还可以使用。net进行自定义转换,以利用SSIS工作的速度和可重复性。
#7
2
Bit late to answer this question but I hope it worth,
回答这个问题有点晚,但我希望它值得,
SSIS is often misunderstood when compared to programming language. SSIS is a framework whereas C# is a language on .NET Framework. I have extensive experience in handling & developing large data warehousing solution using (MSBI suite) and also had developed large websites (ASP.NET) - so I can't be biased.
与编程语言相比,SSIS经常被误解。SSIS是一个框架,而c#是一个关于。net框架的语言。我在使用MSBI套件处理和开发大型数据仓库解决方案方面有丰富的经验,也开发过大型网站(ASP.NET)——所以我不会有偏见。
SSIS if not used properly can decrease performance by par. SSIS packages have three kind of transformation:
如果使用不当,SSIS会降低par的性能。SSIS包有三种转换:
- Blocking transformation - which can only pass the data when above transformation is completed fetching all the rows and completed required calculation on it.
- 阻塞转换——当完成上述转换时,只能传递数据,获取所有行并完成所需的计算。
- Semi-Blocking transformation - Which can pass partial data
- 半阻塞转换—可以传递部分数据
- Non-Blocking - which process the row as soon as it is ready
- 非阻塞—一旦行准备好就处理它
SSIS works exceptionally good with non blocking transformation with proper setting on control flow and data flow. I have used it on larger (over 2 TB of data warehouse) and I can guarantee that it was the fastest load experience. You can check Microsoft blog about We Loaded 1TB in 30 Minutes with SSIS, and So Can You
SSIS在控制流和数据流正确设置的非阻塞转换中工作得非常好。我在更大的(超过2 TB的数据仓库)上使用过它,我可以保证它是最快的负载体验。你可以在Microsoft blog上查看我们在30分钟内装载了1TB的SSIS,你也可以
I agree that SSIS degraded performance when dealing with blocking transformation and they should be carried by T-SQL whenever required.
我同意SSIS在处理阻塞转换时降低了性能,并且应该在需要时使用T-SQL。
Coming to C#, I accept that SSIS uses .NET framework and data provider to accomplish task. But C#, as a language is bit more logical and must be treated to deal with business logic. For example, If we have to run exe with different parameter based on condition, you can write a package which will consider parameters and then logically decide what parameter needs to be passed to run an exe file. It would be lengthy process to do that in SSIS while I can do that easily in C# because logical thing can be easily done in language instead of a framework.
谈到c#,我承认SSIS使用。net框架和数据提供程序来完成任务。但是c#作为一种语言更符合逻辑,必须处理业务逻辑。例如,如果我们必须基于条件运行带有不同参数的exe,您可以编写一个将考虑参数的包,然后逻辑地决定运行exe文件需要传递什么参数。在SSIS中这样做将会是一个漫长的过程,而我在c#中可以很容易地做到这一点,因为逻辑事情可以很容易地在语言中而不是框架中完成。
Now the point here is what is more convenient approach to solve your problem statement. SSIS is a sure winner loading large amount of records loading data from source to destination while C# is perfect for writing logic. Even if you like C#, I won't recommend you to choose for doing ETL (Extract Transform Load) operation on large data warehouse systems.
这里的重点是,什么是更方便的方法来解决你的问题陈述。SSIS是一个可靠的赢家,加载大量的记录从源到目的地加载数据,而c#是编写逻辑的最佳方式。即使您喜欢c#,我也不建议您选择在大型数据仓库系统上执行ETL(提取转换负载)操作。
#8
1
I think main advantage is defining the entire programming construct visually. Any one look at the SSIS package is it pretty much self explainer. The tight integration with the SSIS with SQL allows you to be part of SQL for back up scheduling and huge plus.
我认为主要的优势是可视化地定义整个编程结构。任何看过SSIS包的人都知道它是自我解释的。与SSIS与SQL的紧密集成使您能够成为SQL的一部分,以支持调度和巨大的优势。
As every one explained if you are doing the lot of data manipulation it is good tool. It is free if you have SQL you all set to go and very easy to learn with VS 2008 BIDS
正如每个人所解释的,如果你在做大量的数据操作,这是一个很好的工具。它是免费的,如果你有SQL,你将会很容易地学习与VS 2008投标。
#9
0
Day-to-Day Tasks , which are used by a SSIS Deveoper and are relatively easy as compared to .Net can include :
SSIS Deveoper使用的日常任务,与.Net相比相对容易,其中包括:
Data Comparison between the tables.
表之间的数据比较。
Conditional Splitting,data blocking the data on the basis of some logic.
条件分裂,数据以某种逻辑为基础的数据阻塞。
Data Conversion,look up , merge , unionall , relatively easy to use.
数据转换,查找,合并,银联,使用相对方便。
File Handling(Modifying , validations).
文件处理(修改、验证)。
Error Handing , Email Alerts.
错误处理,电子邮件提醒。
Containers , FOR/FOReach loops are easy to use.
容器,FOR/FOReach循环很容易使用。
Posting data on web services is easy using the WebService task.
在web服务上发布数据很容易使用web服务任务。
Checkpointing,Re-runablity of the data loads is easy to handle.
检查、重新运行数据负载很容易处理。
Debugging is easy in ssis - can be done on conatiner lever , package level.
调试在ssis中很容易——可以在conatiner杠杆、包级上完成。
Scripting can also be done , if the task is not available. Also , you can customize your own tasks
如果任务不可用,也可以编写脚本。此外,您还可以自定义自己的任务
#10
0
Whatever folks say in previous answers are correct but I think that the most important aspect of using SSIS instead of coding is to have easy maintenance process and also a reusable product.
无论人们在前面的回答中说什么都是正确的,但是我认为使用SSIS而不是编码的最重要的方面是拥有简单的维护过程和可重用的产品。
#11
0
SSIS is great for BI applications, you can manipulate the data on Stage Table and than make avaiable on DataWarehouse tables to be used for BI.
SSIS非常适合于BI应用程序,您可以操作Stage表上的数据,而不是使DataWarehouse表上的数据成为可用的BI。
I can connect on SAP, Oracle to get employee information and make avaiable on PowerBI, QlikView, etc...
我可以通过SAP, Oracle来获取员工信息,并在PowerBI, QlikView等…
Its a nice tool if you know where and why use it. Use ir because its cool you will have troubles.
如果你知道在哪里和为什么使用它,它是一个很好的工具。使用红外线,因为它很酷,你会遇到麻烦。
#1
39
good question.
好问题。
if the amount of data transfer huge? are you processing multiple data files and need transactions (both at file system level and database level)? are you dealing with multiple data sources at different locations (for eg ftp, local file system, database)?
如果数据传输量巨大?您是否正在处理多个数据文件并需要事务(在文件系统级和数据库级)?您是否正在处理不同位置的多个数据源(例如ftp、本地文件系统、数据库)?
if answers to above are yes then go ahead with ssis. basically .net is cool with small data import / export jobs, but when you have anything more complex, ssis is a definite winner
如果以上的答案是肯定的,那么请继续进行ssis。基本上。net对于小的数据导入/导出任务来说很酷,但是当你有更复杂的任务时,ssis绝对是赢家
the other thing which i look at is - is it worth writing .net code when everything is available inside ssis. (dont mistake me - i love coding) however, anything you code, you need to maintain :-)
我看到的另一件事是,在ssis中所有内容都可用时,它是否值得编写。net代码。(不要误会我-我喜欢编码)然而,任何你编码的东西,你需要保持:
#2
18
I think project time/budget constraints and the use of a standard tool are some of the biggest arguments for using SSIS. Creating an SSIS package is most of the times way faster than trying to code something similar in .NET.
我认为项目时间/预算限制和标准工具的使用是使用SSIS的最大理由。创建一个SSIS包的速度比在。net中编写类似的代码快得多。
But with that said, it seems like SSIS have a lot of pain points that sometimes might invalidate this argument. It did for me when developing a solution that needed to run in different environments at many different clients. SSIS simply looked too painful the more I evaluated it for the project. A properly architected .NET solution is easier to deploy, more reliable, more flexible, easier to understand and can also achieve very good performance.
但话虽如此,SSIS似乎有很多痛点,有时可能会推翻这个论点。它为我开发了一个解决方案,需要在不同的客户端运行不同的环境。SSIS看起来太痛苦了,我对这个项目进行了评估。一个架构良好的。net解决方案更容易部署,更可靠,更灵活,更容易理解,也能获得很好的性能。
IMHO: consider using SSIS for projects that you only need to deploy to one or maybe two in house SQL Server environments. Otherwise, the .NET approach will quickly become more appealing.
IMHO:考虑在只需要部署到house SQL Server环境中的一个或两个项目中使用SSIS。否则,. net方法将很快变得更有吸引力。
#3
5
I guess it depends on what you are doing. SSIS is very powerful, just like old DTS. If you are loading lots of items and expect to have constant change, I would go SSIS all the way. If you are looking to load only a few items and it’s for lots of customers, I would put it in code. I prefer SSIS for in house ETL processes, but I use .Net at client shops when I need to load data from a legacy system into a SQL database. Now as I stated before if you have a lot of transformations and lots of different data silos to load, I think you would be crazy to do this in .Net and I would go SSIS. If you have only a few items to load and it’s for a single application and may be installed as part of an application at various clients, I would go .Net all the way. Just my 2 cents.
我想这取决于你在做什么。SSIS非常强大,就像老DTS一样。如果你装载了大量的项目并且期望有恒定的变化,我会一直使用SSIS。如果你想只加载一些东西,而且它是给很多客户的,我会把它写在代码里。我更喜欢SSIS for in house ETL进程,但是当我需要从遗留系统加载数据到SQL数据库时,我在客户端商店使用。net。就像我之前说的,如果你有很多的转换和很多不同的数据竖井要加载,我认为你在。net中做这个会很疯狂,我会去SSIS。如果您只有几个项目要加载,并且它是针对单个应用程序的,并且可以作为不同客户端的应用程序的一部分安装,那么我将一直使用。net。那只是我的个人意见。
#4
4
I have a lot of experience with SSIS from small projects to large, complex ETL. Without going into the details, this is my guidance for you:
从小型项目到大型、复杂的ETL,我在SSIS有很多经验。以下是我对你的指导,不谈细节。
-
If you are a DBA and you are not familiar with .NET, or if you are a developer quite familiar with SSIS, then you can use SSIS for small, simple, fairly straightforward extract, transform, load (ETL) tasks.
如果您是DBA,并且不熟悉. net,或者是非常熟悉SSIS的开发人员,那么您可以将SSIS用于小型、简单、相当简单的提取、转换、加载(ETL)任务。
-
SSIS is very quirky and there are many pitfalls, gotchas, and what might be considered outright bugs. It is extremely powerful if you are intimately familiar.
SSIS是非常古怪的,有很多陷阱,陷阱,以及可能被认为是完全错误的东西。如果你非常熟悉它,它会非常强大。
-
C# now has TPL Dataflow. Simple performance tests put it ahead of SSIS. (eg http://mymemoryleaks.blogspot.cz/2013/10/ssis-vs-tpldataflow.html)
c#现在有TPL数据流。简单的性能测试使它领先于SSIS。(如http://mymemoryleaks.blogspot.cz/2013/10/ssis-vs-tpldataflow.html)
-
If you want to do anything beyond trivial, and if you can use .NET skills, use .NET instead of SSIS.
如果你想要做一些无关紧要的事情,如果你能使用。net技能,那就使用。net而不是SSIS。
#5
4
My arguments for not using SSIS are:
我不使用SSIS的理由是:
-
Design greenfield products so that they have RESTful data feeds for reporting and extraction built-in to the project plan and budget, preferably to a standard like OData so that other tools can plug right in.
设计greenfield产品,使其具有用于报告和提取的RESTful数据提要,并将其嵌入到项目计划和预算中,最好是针对OData这样的标准,以便其他工具可以直接插入。
-
Data feeds should pull and transform from upstream systems and feeds on demand; such that schedule tasks, configuration of scheduled tasks, task runner VMs and staff to run all this unreliable scheduling stuff is negated.
数据提要应根据需要从上游系统和提要中提取和转换;这样,调度任务、计划任务的配置、任务运行程序vm和运行所有这些不可靠调度内容的人员就被否定了。
-
RESTful data feeds leverage HTTP caching.
RESTful数据feed利用HTTP缓存。
-
Feeds/services/APIs can be moved to elastic-scale cloud easily.
可以很容易地将提要/服务/ api移动到弹性大小的云中。
-
SSIS requires finding people with SSIS skills that enjoy doing that stuff for weeks. In my experience, finding and retaining SSIS developers is hard and expensive and the people found tend to be sub-par.
SSIS需要找到那些具有SSIS技能的人,他们喜欢在几周内完成这些工作。根据我的经验,寻找和保留SSIS的开发人员是非常困难和昂贵的,而且人们发现他们往往是低于标准的。
-
SSIS doesn't work well with source control and collaborative work.
SSIS不能很好地与源代码控制和协作工作配合。
-
SSIS doesn't lend itself well to code reuse, unlike microservices and traditional code libraries.
与微服务和传统的代码库不同,SSIS不太适合代码重用。
-
SSIS doesn't version easily, unlike a REST service.
与REST服务不同,SSIS不容易版本化。
-
SSIS doesn't lend itself to modular designs and continuous deployment of many small changes, it tends to be large-batch with scary releases.
SSIS不支持模块化设计和持续部署许多小的更改,它倾向于大量发布可怕的版本。
-
SSIS promotes the use of stored-procedures which places a lot of demand on SQL which is the hot-spot. Favour designs that place demands on a scaleable, stateless middle tier.
SSIS促进了存储过程的使用,这使得对SQL的大量需求成为热点。支持对可扩展的、无状态的中间层进行需求的设计。
-
The tooling is clunky and unreliable.
该工具笨重且不可靠。
-
You're at the mercy of Microsoft's roadmap for SSIS.
你得看微软SSIS的路线图了。
-
Consider writing to tables/services that support analysis, reporting and views as soon as the data comes into the application; see CQRS and other application architecture patterns.
考虑在数据进入应用程序后立即写信给支持分析、报告和视图的表/服务;参见CQRS和其他应用程序体系结构模式。
-
Never use Excel as a data source; train employees.
绝不使用Excel作为数据源;培训员工。
-
Code is king.
代码为王。
Ultimately, I see SSIS as a relic of Enterprise IT. I like to ask, "Would Google use SSIS?" How else can the problem be solved? Think outside the box.
最后,我认为SSIS是企业IT的遗物。我想问,“谷歌会使用SSIS吗?”这个问题还能怎么解决呢?跳出固有思维模式。
#6
2
SSIS has many built in ways of doing transformations from different data sources and you can string them together in a way that makes it very customizeable. They have built in optimizations that make them fast.
SSIS有许多构建方式,用于从不同数据源进行转换,您可以将它们以一种非常可定制的方式连接在一起。他们进行了优化,使它们运行得更快。
You can also use .NET to make your own custom transformations to take advantage of the speed and repeatability of an SSIS job.
您还可以使用。net进行自定义转换,以利用SSIS工作的速度和可重复性。
#7
2
Bit late to answer this question but I hope it worth,
回答这个问题有点晚,但我希望它值得,
SSIS is often misunderstood when compared to programming language. SSIS is a framework whereas C# is a language on .NET Framework. I have extensive experience in handling & developing large data warehousing solution using (MSBI suite) and also had developed large websites (ASP.NET) - so I can't be biased.
与编程语言相比,SSIS经常被误解。SSIS是一个框架,而c#是一个关于。net框架的语言。我在使用MSBI套件处理和开发大型数据仓库解决方案方面有丰富的经验,也开发过大型网站(ASP.NET)——所以我不会有偏见。
SSIS if not used properly can decrease performance by par. SSIS packages have three kind of transformation:
如果使用不当,SSIS会降低par的性能。SSIS包有三种转换:
- Blocking transformation - which can only pass the data when above transformation is completed fetching all the rows and completed required calculation on it.
- 阻塞转换——当完成上述转换时,只能传递数据,获取所有行并完成所需的计算。
- Semi-Blocking transformation - Which can pass partial data
- 半阻塞转换—可以传递部分数据
- Non-Blocking - which process the row as soon as it is ready
- 非阻塞—一旦行准备好就处理它
SSIS works exceptionally good with non blocking transformation with proper setting on control flow and data flow. I have used it on larger (over 2 TB of data warehouse) and I can guarantee that it was the fastest load experience. You can check Microsoft blog about We Loaded 1TB in 30 Minutes with SSIS, and So Can You
SSIS在控制流和数据流正确设置的非阻塞转换中工作得非常好。我在更大的(超过2 TB的数据仓库)上使用过它,我可以保证它是最快的负载体验。你可以在Microsoft blog上查看我们在30分钟内装载了1TB的SSIS,你也可以
I agree that SSIS degraded performance when dealing with blocking transformation and they should be carried by T-SQL whenever required.
我同意SSIS在处理阻塞转换时降低了性能,并且应该在需要时使用T-SQL。
Coming to C#, I accept that SSIS uses .NET framework and data provider to accomplish task. But C#, as a language is bit more logical and must be treated to deal with business logic. For example, If we have to run exe with different parameter based on condition, you can write a package which will consider parameters and then logically decide what parameter needs to be passed to run an exe file. It would be lengthy process to do that in SSIS while I can do that easily in C# because logical thing can be easily done in language instead of a framework.
谈到c#,我承认SSIS使用。net框架和数据提供程序来完成任务。但是c#作为一种语言更符合逻辑,必须处理业务逻辑。例如,如果我们必须基于条件运行带有不同参数的exe,您可以编写一个将考虑参数的包,然后逻辑地决定运行exe文件需要传递什么参数。在SSIS中这样做将会是一个漫长的过程,而我在c#中可以很容易地做到这一点,因为逻辑事情可以很容易地在语言中而不是框架中完成。
Now the point here is what is more convenient approach to solve your problem statement. SSIS is a sure winner loading large amount of records loading data from source to destination while C# is perfect for writing logic. Even if you like C#, I won't recommend you to choose for doing ETL (Extract Transform Load) operation on large data warehouse systems.
这里的重点是,什么是更方便的方法来解决你的问题陈述。SSIS是一个可靠的赢家,加载大量的记录从源到目的地加载数据,而c#是编写逻辑的最佳方式。即使您喜欢c#,我也不建议您选择在大型数据仓库系统上执行ETL(提取转换负载)操作。
#8
1
I think main advantage is defining the entire programming construct visually. Any one look at the SSIS package is it pretty much self explainer. The tight integration with the SSIS with SQL allows you to be part of SQL for back up scheduling and huge plus.
我认为主要的优势是可视化地定义整个编程结构。任何看过SSIS包的人都知道它是自我解释的。与SSIS与SQL的紧密集成使您能够成为SQL的一部分,以支持调度和巨大的优势。
As every one explained if you are doing the lot of data manipulation it is good tool. It is free if you have SQL you all set to go and very easy to learn with VS 2008 BIDS
正如每个人所解释的,如果你在做大量的数据操作,这是一个很好的工具。它是免费的,如果你有SQL,你将会很容易地学习与VS 2008投标。
#9
0
Day-to-Day Tasks , which are used by a SSIS Deveoper and are relatively easy as compared to .Net can include :
SSIS Deveoper使用的日常任务,与.Net相比相对容易,其中包括:
Data Comparison between the tables.
表之间的数据比较。
Conditional Splitting,data blocking the data on the basis of some logic.
条件分裂,数据以某种逻辑为基础的数据阻塞。
Data Conversion,look up , merge , unionall , relatively easy to use.
数据转换,查找,合并,银联,使用相对方便。
File Handling(Modifying , validations).
文件处理(修改、验证)。
Error Handing , Email Alerts.
错误处理,电子邮件提醒。
Containers , FOR/FOReach loops are easy to use.
容器,FOR/FOReach循环很容易使用。
Posting data on web services is easy using the WebService task.
在web服务上发布数据很容易使用web服务任务。
Checkpointing,Re-runablity of the data loads is easy to handle.
检查、重新运行数据负载很容易处理。
Debugging is easy in ssis - can be done on conatiner lever , package level.
调试在ssis中很容易——可以在conatiner杠杆、包级上完成。
Scripting can also be done , if the task is not available. Also , you can customize your own tasks
如果任务不可用,也可以编写脚本。此外,您还可以自定义自己的任务
#10
0
Whatever folks say in previous answers are correct but I think that the most important aspect of using SSIS instead of coding is to have easy maintenance process and also a reusable product.
无论人们在前面的回答中说什么都是正确的,但是我认为使用SSIS而不是编码的最重要的方面是拥有简单的维护过程和可重用的产品。
#11
0
SSIS is great for BI applications, you can manipulate the data on Stage Table and than make avaiable on DataWarehouse tables to be used for BI.
SSIS非常适合于BI应用程序,您可以操作Stage表上的数据,而不是使DataWarehouse表上的数据成为可用的BI。
I can connect on SAP, Oracle to get employee information and make avaiable on PowerBI, QlikView, etc...
我可以通过SAP, Oracle来获取员工信息,并在PowerBI, QlikView等…
Its a nice tool if you know where and why use it. Use ir because its cool you will have troubles.
如果你知道在哪里和为什么使用它,它是一个很好的工具。使用红外线,因为它很酷,你会遇到麻烦。