During our work as web developer for a meteorological company, we are faced with the same task over and over again: Get some files from somewhere (FTP/Web/directory/mail) and import the contained data to a database.
在我们作为气象公司的Web开发人员工作期间,我们一遍又一遍地面对同样的任务:从某个地方获取一些文件(FTP / Web /目录/邮件)并将包含的数据导入数据库。
Of course the file format is never the same, the databases are always designed differently, countless special cases have to be handled, etc, etc.
当然,文件格式永远不会相同,数据库的设计总是不同的,必须处理无数的特殊情况等等。
So now I'm planning an importing framework for exactly this kind of work. Since we're all experienced PHP developers and the current scripts are either PHP or Perl, we'll stick with PHP as scripting language.
所以现在我正在为这种工作计划一个导入框架。由于我们都是经验丰富的PHP开发人员,当前脚本是PHP或Perl,因此我们将坚持使用PHP作为脚本语言。
- A data getter will fetch the file from the source, open it and store the content into a string variable. (Don't worry, PHP will get enough memory from us.)
- The data handler will do the complicated work to convert the string into some kind of array.
- The array will be saved to the database or written to a new file or whatever we're supposed to do with it.
数据获取器将从源获取文件,打开它并将内容存储到字符串变量中。 (别担心,PHP会从我们那里获得足够的内存。)
数据处理程序将执行复杂的工作以将字符串转换为某种数组。
该数组将保存到数据库或写入新文件或我们应该用它做什么。
Along with this functionality there will be some common error handling, log writing and email reporting.
除此功能外,还会有一些常见的错误处理,日志编写和电子邮件报告。
The idea is to use a collection of classes (Some getter-classes, a lot of specialised handlers, some writer classes).
这个想法是使用一组类(一些getter-classes,许多专门的处理程序,一些writer类)。
My question: How do I practically organize these classes in a working script? Do I invent some kind of meta language which will be interpreted and the the classes are called accordingly? Or just provide some simple interfaces these classes have to implement and the my users (like I said: Experienced PHP developers) will write small PHP scripts loading these classes?
我的问题:我如何在工作脚本中实际组织这些类?我是否发明了某种将被解释的元语言,并相应地调用了这些类?或者只是提供一些这些类必须实现的简单接口,我的用户(比如我说过:经验丰富的PHP开发人员)会编写加载这些类的小PHP脚本吗?
The second version almost certainly offers the biggest flexiblity and extensibility.
第二个版本几乎可以肯定提供最大的灵活性和可扩展性。
Do you have any other ideas concerning such an undertaking?
你对这样的事业还有其他想法吗?
3 个解决方案
#1
Working in a similar environment of dozens of different external data formats that need to be im- and exported, I can recommend to at least try and get them to unify the data formats. We had some success by developing tools that help others outside our company to transform their data into our format. We also gave them the source code, for free.
在需要进行导入和导出的许多不同外部数据格式的类似环境中工作,我建议至少尝试让它们统一数据格式。我们通过开发工具帮助我们公司以外的其他人将数据转换为我们的格式,从而获得了一些成功。我们还免费提供了源代码。
Some others are now transforming their data for us using our tools, and if they change their format, it is them that changes the transformation tool. One cause of a headache less for us.
其他一些人正在使用我们的工具为我们转换他们的数据,如果他们改变了他们的格式,他们就会改变转换工具。对我们来说头痛的原因之一。
In one case it even lead to another company switching to the file format our systems use internally. Granted, it is only one case, but I consider it a first step on a long road ;-)
在一种情况下,它甚至导致另一家公司切换到我们的系统内部使用的文件格式。当然,这只是一个案例,但我认为这是漫长道路上的第一步;-)
#2
I suggest borrowing concepts from Data Transformation Services (DTS). You could have data sources and data sinks, import tasks, transformation tasks and so on.
我建议从数据转换服务(DTS)借用概念。您可以拥有数据源和数据接收器,导入任务,转换任务等。
#3
Is there a reason why defining a standard web service wouldn't work here? Then you can supply the data in a standard format, returning a SOAP error (possibly populated by a field in the input document) if there's a fault.
有没有理由为什么定义标准Web服务在这里不起作用?然后,您可以以标准格式提供数据,如果出现故障,则返回SOAP错误(可能由输入文档中的字段填充)。
It's potentially more limited than Pavel's suggestion (or would require more up-front design), but might be something worth considering.
它可能比Pavel的建议更有限(或者需要更多的前期设计),但可能值得考虑。
#1
Working in a similar environment of dozens of different external data formats that need to be im- and exported, I can recommend to at least try and get them to unify the data formats. We had some success by developing tools that help others outside our company to transform their data into our format. We also gave them the source code, for free.
在需要进行导入和导出的许多不同外部数据格式的类似环境中工作,我建议至少尝试让它们统一数据格式。我们通过开发工具帮助我们公司以外的其他人将数据转换为我们的格式,从而获得了一些成功。我们还免费提供了源代码。
Some others are now transforming their data for us using our tools, and if they change their format, it is them that changes the transformation tool. One cause of a headache less for us.
其他一些人正在使用我们的工具为我们转换他们的数据,如果他们改变了他们的格式,他们就会改变转换工具。对我们来说头痛的原因之一。
In one case it even lead to another company switching to the file format our systems use internally. Granted, it is only one case, but I consider it a first step on a long road ;-)
在一种情况下,它甚至导致另一家公司切换到我们的系统内部使用的文件格式。当然,这只是一个案例,但我认为这是漫长道路上的第一步;-)
#2
I suggest borrowing concepts from Data Transformation Services (DTS). You could have data sources and data sinks, import tasks, transformation tasks and so on.
我建议从数据转换服务(DTS)借用概念。您可以拥有数据源和数据接收器,导入任务,转换任务等。
#3
Is there a reason why defining a standard web service wouldn't work here? Then you can supply the data in a standard format, returning a SOAP error (possibly populated by a field in the input document) if there's a fault.
有没有理由为什么定义标准Web服务在这里不起作用?然后,您可以以标准格式提供数据,如果出现故障,则返回SOAP错误(可能由输入文档中的字段填充)。
It's potentially more limited than Pavel's suggestion (or would require more up-front design), but might be something worth considering.
它可能比Pavel的建议更有限(或者需要更多的前期设计),但可能值得考虑。