Talend - 基于从列表中获取的值的URI

I need to feed a collection in MongoDB based on values obtained from webservices that provide JSON, but I'm having trouble mounting the job because the URI of some of the web services rely on values that can be obtained from other webservices. For example, the URI http://172.31.15.180:80/ws/getAgenciasUF/52 provides a JSON with a collection, in this format:

我需要根据从提供JSON的web服务获得的值在MongoDB中提供一个集合,但我在安装作业时遇到问题,因为某些Web服务的URI依赖于可以从其他Web服务获取的值。例如,URI http://172.31.15.180:80/ws/getAgenciasUF/52以这种格式提供带有集合的JSON:

{ "COD_AGENCIA", "521800300", "NAME", "PORANGATU", "UORG": "902", "INTRA_MUNICIPAL": "0"},
{ "COD_AGENCIA", "521830000", "NAME", "HOLD", "UORG": "904", "INTRA_MUNICIPAL": "0"}
...
(20 other values)
...

Through this webservice, I could have insert in a MongoDB collection using the tREST and tExtractJSONFields components. However, there is another webservice whose URI is http://172.31.15.180:80/ws/getCidadesPorAgencia/521800300, where the latter value is one of COD_AGENCIA available in JSON above. That is, if I read the COD_AGENCIA up and put in some component that iterate on these values and shoot x times the second URI, varying only the code, I could get all the values needed to feed another collection MongoDB.

通过这个web服务,我可以使用tREST和tExtractJSONFields组件插入MongoDB集合。但是,还有另一个web服务,其URI是http://172.31.15.180:80/ws/getCidadesPorAgencia/521800300,其中后一个值是上面JSON中可用的COD_AGENCIA之一。也就是说,如果我读取COD_AGENCIA并放入一些迭代这些值的组件并拍摄x次第二个URI,只改变代码,我就可以得到提供另一个集合MongoDB所需的所有值。

Using the ESB TOS 6.2.1, I tried to interconnect a Trest a tExtractJSONField and this to a tRESTRequest, like this:

使用ESB TOS 6.2.1,我尝试将Trest和tExtractJSONField互连到tRESTRequest,如下所示:

but I received the following error:

但我收到以下错误:

[statistics] connecting to socket on port 3587
[statistics] connected

[WARN ]: org.eclipse.jetty.util.component.AbstractLifeCycle - FAILED SelectChannelConnector@172.31.15.180:80: java.net.BindException: Cannot assign requested address: bind
java.net.BindException: Cannot assign requested address: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)
at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)
at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:187)
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:316)
at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:265)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at org.eclipse.jetty.server.Server.doStart(Server.java:293)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at org.apache.cxf.transport.http_jetty.JettyHTTPServerEngine.addServant(JettyHTTPServerEngine.java:472)
at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.activate(JettyHTTPDestination.java:175)
at org.apache.cxf.transport.AbstractObservable.setMessageObserver(AbstractObservable.java:53)
at org.apache.cxf.binding.AbstractBindingFactory.addListener(AbstractBindingFactory.java:95)
at org.apache.cxf.jaxrs.JAXRSBindingFactory.addListener(JAXRSBindingFactory.java:88)
at org.apache.cxf.endpoint.ServerImpl.start(ServerImpl.java:123)
at org.apache.cxf.jaxrs.JAXRSServerFactoryBean.create(JAXRSServerFactoryBean.java:206)
at bdogo.teste_0_1.teste$Thread4RestServiceProviderEndpoint.run(teste.java:791)

[WARN ]: org.eclipse.jetty.util.component.AbstractLifeCycle - FAILED org.eclipse.jetty.server.Server@13df084: java.net.BindException: Cannot assign requested address: bind
java.net.BindException: Cannot assign requested address: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)
at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)
at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:187)
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:316)
at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:265)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at org.eclipse.jetty.server.Server.doStart(Server.java:293)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at org.apache.cxf.transport.http_jetty.JettyHTTPServerEngine.addServant(JettyHTTPServerEngine.java:472)
at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.activate(JettyHTTPDestination.java:175)
at org.apache.cxf.transport.AbstractObservable.setMessageObserver(AbstractObservable.java:53)
at org.apache.cxf.binding.AbstractBindingFactory.addListener(AbstractBindingFactory.java:95)
at org.apache.cxf.jaxrs.JAXRSBindingFactory.addListener(JAXRSBindingFactory.java:88)
at org.apache.cxf.endpoint.ServerImpl.start(ServerImpl.java:123)
at org.apache.cxf.jaxrs.JAXRSServerFactoryBean.create(JAXRSServerFactoryBean.java:206)
at bdogo.teste_0_1.teste$Thread4RestServiceProviderEndpoint.run(teste.java:791)

[ERROR]: org.apache.cxf.transport.http_jetty.JettyHTTPServerEngine - Could not start Jetty server on port 80: Cannot assign requested address: bind
org.apache.cxf.service.factory.ServiceConstructionException
at org.apache.cxf.jaxrs.JAXRSServerFactoryBean.create(JAXRSServerFactoryBean.java:219)
at bdogo.teste_0_1.teste$Thread4RestServiceProviderEndpoint.run(teste.java:791)

Caused by: org.apache.cxf.interceptor.Fault: Could not start Jetty server on port 80: Cannot assign requested address: bind
at org.apache.cxf.transport.http_jetty.JettyHTTPServerEngine.addServant(JettyHTTPServerEngine.java:483)
at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.activate(JettyHTTPDestination.java:175)
at org.apache.cxf.transport.AbstractObservable.setMessageObserver(AbstractObservable.java:53)
at org.apache.cxf.binding.AbstractBindingFactory.addListener(AbstractBindingFactory.java:95)
at org.apache.cxf.jaxrs.JAXRSBindingFactory.addListener(JAXRSBindingFactory.java:88)
at org.apache.cxf.endpoint.ServerImpl.start(ServerImpl.java:123)
at org.apache.cxf.jaxrs.JAXRSServerFactoryBean.create(JAXRSServerFactoryBean.java:206)
... 1 more

Caused by: java.net.BindException: Cannot assign requested address: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)
at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)
at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:187)
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:316)
at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:265)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at org.eclipse.jetty.server.Server.doStart(Server.java:293)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at org.apache.cxf.transport.http_jetty.JettyHTTPServerEngine.addServant(JettyHTTPServerEngine.java:472)
... 7 more

If I eliminate tRESTRequest and link the tExtractJSONField directly to tLog this error doesn't happen, and the output of tExtract is listed on the console. The use of this tRESTRequest component (which is new to me) is that I seem to be something wrong. It is so even that it should be used (see figure below)?

如果我消除了tRESTRequest并将tExtractJSONField直接链接到tLog,则不会发生此错误,并且控制台上会列出tExtract的输出。使用这个tRESTRequest组件(对我来说是新的)是我似乎有点不对劲。甚至应该使用它(见下图)?

Note that there is a warning in tExtractJSONFields (the text says: "This component has not enough Row type outputs"). The following figures shows as the configuration of the components was made.

请注意,tExtractJSONFields中有一个警告(文本说:“此组件没有足够的行类型输出”)。下图显示了组件的配置。

Could help me on how to configure it from values received from tExtractJSONFields? What am I doing wrong? There is another way to get the desired result ?

可以帮助我如何从tExtractJSONFields收到的值配置它?我究竟做错了什么?还有另一种方法可以获得理想的结果吗?

1 个解决方案

#1

Your approach is fine, you only seem to lack a few components.

你的方法很好,你似乎只缺少一些组件。

Job design

Get list of COD_AGENCIA

获取COD_AGENCIA列表

The components tREST and tExtractJSONFields are well suited for this.

组件tREST和tExtractJSONFields非常适合这种情况。

At this point you should have a list of all the values you got with the first REST call.

此时,您应该拥有第一个REST调用所获得的所有值的列表。

Now for the connection between the first and the second call.

现在进行第一次和第二次呼叫之间的连接。

Get list of getCidadesPorAgencia per COD_AGENCIA

获取每个COD_AGENCIA的getCidadesPorAgencia列表

Here, you could again use a tREST component and maybe a tExtractJSONFields. Also, here would be the spot where to use tMongoDBOutput.

在这里,您可以再次使用tREST组件,也可以使用tExtractJSONFields。此外,这里将是使用tMongoDBOutput的地方。

To connect both of those requests, use a tFlowToIterate component after the first tExtractJSONFields.

要连接这两个请求,请在第一个tExtractJSONField之后使用tFlowToIterate组件。

Add a key value for COD_AGENCIA and connect the second tREST with Iterate. Now in the second tREST you will have access to a global variable which you just named as the key value. Use this in the call, e.g.: "http://172.31.15.180:80/ws/getCidadesPorAgencia/" + globalMap.get("var_agencia")"

为COD_AGENCIA添加一个键值,并将第二个tREST与Iterate连接。现在,在第二个tREST中,您将可以访问一个全局变量,您只需将其命名为键值。在通话中使用它,例如:“http://172.31.15.180:80/ws/getCidadesPorAgencia/”+ globalMap.get(“var_agencia”)“

Now you should be able to loop through every agency and get all the cities connected to it.

现在,您应该能够遍历每个代理商,并将所有城市连接到它。

Reason

What you did was a great idea but the wrong connector. OnComponentOk waits until the connected component runs through without an error. Then the next component will be started. No data will be transmitted. And no row iteration will happen, which seems to be key here.

你做了什么是一个好主意,但错误的连接器。 OnComponentOk等待直到连接的组件运行而没有错误。然后将启动下一个组件。不会传输任何数据。并且不会发生行迭代,这似乎是关键。

#1