使用SSIS,asp.NET和wget函数的混合来提取,转换和加载数据到网页

时间:2021-12-23 16:36:29

This is more of a question to check feasibility.

这是检查可行性的更多问题。

Ultimately what I'd like to do is:

最终我想做的是:

1) Create an asp.NET page with one string parameter field

1)创建一个带有一个字符串参数字段的asp.NET页面

2) Have that parameter field pass to an SSIS package that looks at (for simplicity let's say an XML data page with the parameter variable being in the URL)

2)将该参数字段传递给查看的SSIS包(为简单起见,假设一个XML数据页面,参数变量在URL中)

3) Have that SSIS package take the data from that variable XML page

3)让SSIS包从该变量XML页面获取数据

4) Load the data into a cache table

4)将数据加载到缓存表中

5) Display the live data (would the live data also need to be put into a 'live table'?) on the next page to be used for in-depth analysis (using crystal or component art or similar)

5)在下一页上显示实时数据(实时数据是否也需要放入“实时表格?”)以用于深入分析(使用水晶或组件艺术或类似)

From an end user perspective they're logging onto a website, typing in a specific string (let's say carrots) and then SSIS goes away and gets carrot data from our third party XML source and gives a detailed analysis of carrots on the next page with the data gathered.

从最终用户的角度来看,他们登录到一个网站,输入一个特定的字符串(比方说胡萝卜),然后SSIS消失并从我们的第三方XML源获取胡萝卜数据,并在下一页上提供胡萝卜的详细分析。收集的数据。

1 个解决方案

#1


1  

There are multiple obstacles You may encounter using SSIS in this scenario:

在这种情况下,使用SSIS可能会遇到多种障碍:

  1. SSIS packages require constant metadata which means that your source (fetched XML) and destination (table) should not change underlying metadata - XML should have exactly the same schema and destination table columns should not be altered, also.
  2. SSIS包需要常量元数据,这意味着您的源(获取的XML)和目标(表)不应更改底层元数据 - XML应具有完全相同的模式,并且也不应更改目标表列。
  3. Passign data from SSIS package to ASP.NET page is tricky: storing data in 'live table' (tricky to differentiate row id to fetch from table)
  4. 将数据从SSIS包传递到ASP.NET页面很棘手:将数据存储在“实时表”中(难以将行ID区分为从表中获取)
  5. Executing SSIS package requires permissions
  6. 执行SSIS包需要权限
  7. How to store connection strings in SSIS package in safe manner
  8. 如何以安全的方式在SSIS包中存储连接字符串
  9. asp.net user (impersonated or not) has to have permissions to execute package and then SSIS connection managers with Integrated Security login might fail
  10. asp.net用户(模拟与否)必须具有执行包的权限,然后具有集成安全性登录的SSIS连接管理器可能会失败
  11. if You decide to execute package thru job, asp.net user must have permissions to execute package (then use proxy/credentials for job to set appropriate permissions inside SSIS package)
  12. 如果您决定通过作业执行包,则asp.net用户必须具有执行包的权限(然后使用代理/凭证作业来在SSIS包内设置适当的权限)
  13. is this list long enough!? :)
  14. 这个清单足够长了!? :)

I strongly suggest to avoid SSIS in this simple scenario because it adds additional overhead layer for implementation and maintenance. Why don't You write .NET code to fetch XML and process it. You can even skip storing results to 'live table' this way.

我强烈建议在这种简单的场景中避免使用SSIS,因为它为实现和维护增加了额外的开销层。为什么不编写.NET代码来获取XML并对其进行处理。您甚至可以通过这种方式跳过将结果存储到“live table”的过程。

#1


1  

There are multiple obstacles You may encounter using SSIS in this scenario:

在这种情况下,使用SSIS可能会遇到多种障碍:

  1. SSIS packages require constant metadata which means that your source (fetched XML) and destination (table) should not change underlying metadata - XML should have exactly the same schema and destination table columns should not be altered, also.
  2. SSIS包需要常量元数据,这意味着您的源(获取的XML)和目标(表)不应更改底层元数据 - XML应具有完全相同的模式,并且也不应更改目标表列。
  3. Passign data from SSIS package to ASP.NET page is tricky: storing data in 'live table' (tricky to differentiate row id to fetch from table)
  4. 将数据从SSIS包传递到ASP.NET页面很棘手:将数据存储在“实时表”中(难以将行ID区分为从表中获取)
  5. Executing SSIS package requires permissions
  6. 执行SSIS包需要权限
  7. How to store connection strings in SSIS package in safe manner
  8. 如何以安全的方式在SSIS包中存储连接字符串
  9. asp.net user (impersonated or not) has to have permissions to execute package and then SSIS connection managers with Integrated Security login might fail
  10. asp.net用户(模拟与否)必须具有执行包的权限,然后具有集成安全性登录的SSIS连接管理器可能会失败
  11. if You decide to execute package thru job, asp.net user must have permissions to execute package (then use proxy/credentials for job to set appropriate permissions inside SSIS package)
  12. 如果您决定通过作业执行包,则asp.net用户必须具有执行包的权限(然后使用代理/凭证作业来在SSIS包内设置适当的权限)
  13. is this list long enough!? :)
  14. 这个清单足够长了!? :)

I strongly suggest to avoid SSIS in this simple scenario because it adds additional overhead layer for implementation and maintenance. Why don't You write .NET code to fetch XML and process it. You can even skip storing results to 'live table' this way.

我强烈建议在这种简单的场景中避免使用SSIS,因为它为实现和维护增加了额外的开销层。为什么不编写.NET代码来获取XML并对其进行处理。您甚至可以通过这种方式跳过将结果存储到“live table”的过程。