使用Quartz Worker线程拆分log4j输出

I'm working on an application that consists of an overall Quartz-based scheduler and "CycledJob" run using CronTriggers. The purpose of the application is to process inputs from different email inboxes based on the source country.

我正在开发一个应用程序,它包含一个基于Quartz的整体调度程序和使用CronTriggers运行的“CycledJob”。该应用程序的目的是根据源国家处理来自不同电子邮件收件箱的输入。

Based on the country that it comes in from (i.e. US, UK, FR, etc.) the application triggers one job thread to run each country's processing cycle, so there would be a UK Worker thread, one for US, France, etc. When formatting the output to log4j, I'm using the thread parameter, so it emits [ApplicationName_Worker-1], [ApplicationName_Worker-2] etc. Try as I might, I can't find a way to name the threads since they're pulled out of Quartz's Thread Pools. Although I could possibly go so far as to extend Quartz, I'd like to work out a different solution instead of messing with the standard library.

根据它所来自的国家(即美国,英国,法国等),该应用程序触发一个工作线程来运行每个国家的处理周期,因此将有一个英国工人线程,一个用于美国,法国等。将输出格式化为log4j时,我正在使用thread参数,因此它会发出[ApplicationName_Worker-1],[ApplicationName_Worker-2]等。尽可能尝试,我找不到一种方法来命名线程,因为它们'重新退出Quartz的Thread Pools。虽然我可能会扩展Quartz,但我想制定一个不同的解决方案,而不是搞乱标准库。

Here's the problem: When using log4j, I'd like to have all log items from the US thread output to a US only file, likewise for each of the country threads. I don't care if they stay in one unified ConsoleAppender, the FileAppender split is what I'm after here. I already know how to specify multiple file appenders and such, my issue is I can't differentiate based on country. There are 20+ classes within the application that can be on the execution chain, very few of which I want to burden with the knowledge of passing an extra "context" parameter through EVERY method... I've considered a Strategy pattern extending a log4j wrapper class, but unless I can let every class in the chain know which thread it's on to parameterize the logger call, that seems impossible. Without being able to name the thread also creates a challenge (or else this would be easy!).

这就是问题所在:当使用log4j时,我希望将来自US线程输出的所有日志项输出到仅限US的文件,同样适用于每个国家/地区线程。我不在乎他们是否留在一个统一的ConsoleAppender中,FileAppender拆分就是我在这里所追求的。我已经知道如何指定多个文件追加器等,我的问题是我无法根据国家区分。应用程序中有20多个类可以在执行链上,我想通过每个方法传递一个额外的“上下文”参数的知识很少...我已经考虑过扩展一个策略模式log4j包装类,但除非我能让链中的每个类知道它所在的哪个线程参数化记录器调用,这似乎是不可能的。无法命名线程也会产生挑战(否则这很容易!)。

So here's the question: What would be a suggested approach to allow many subordinate classes in an application that are each used for every different thread to process the input know that they are within the context of a particular country thread when they are logging?

所以这里有一个问题:什么是一个建议的方法,允许应用程序中的许多从属类,每个用于每个不同的线程处理输入,知道它们在记录时是在特定国家线程的上下文中?

Good luck understanding, and please ask clarifying questions! I hope someone is able to help me figure out a decent way to tackle this. All suggestions welcome.

祝你好运,请提出澄清问题!我希望有人能够帮助我找到解决这个问题的正确方法。欢迎所有建议。

4 个解决方案

#1

At the top of each country's processing thread, put the country code into Log4j's mapped diagnostic context (MDC). This uses a ThreadLocal variable so that you don't have to pass the country up and down the call stack explicitly. Then create a custom filter that looks at the MDC, and filters out any events that don't contain the current appender's country code.

在每个国家/地区的处理线程的顶部,将国家/地区代码放入Log4j的映射诊断上下文(MDC)。这使用ThreadLocal变量,因此您不必显式地在调用堆栈中上下传递国家/地区。然后创建一个查看MDC的自定义过滤器,并过滤掉任何不包含当前appender国家/地区代码的事件。

In your Job:

在你的工作中:

...
public static final String MDC_COUNTRY = "com.y.foo.Country";
public void execute(JobExecutionContext context)
  /* Just guessing that you have the country in your JobContext. */
  MDC.put(MDC_COUNTRY, context.get(MDC_COUNTRY));
  try {
    /* Perform your job here. */
    ...
  } finally {
    MDC.remove(MDC_COUNTRY);
  }
}
...

Write a custom Filter:

编写自定义过滤器:

package com.y.log4j;

import org.apache.log4j.spi.LoggingEvent;

/**
 * This is a general purpose filter. If its "value" property is null, 
 * it requires only that the specified key be set in the MDC. If its 
 * value is not null, it further requires that the value in the MDC 
 * is equal.
 */
public final class ContextFilter extends org.apache.log4j.spi.Filter {

  public int decide(LoggingEvent event) {
    Object ctx = event.getMDC(key);
    if (value == null)
      return (ctx != null) ? NEUTRAL : DENY;
    else
      return value.equals(ctx) ? NEUTRAL : DENY;
  }

  private String key;
  private String value;

  public void setContextKey(String key) { this.key = key; }
  public String getContextKey() { return key; }
  public void setValue(String value) { this.value = value; }
  public String getValue() { return value; }

}

In your log4j.xml:

在你的log4j.xml中:

<appender name="fr" class="org.apache.log4j.FileAppender">
  <param name="file" value="france.log"/>
  ...
  <filter class="com.y.log4j.ContextFilter">
    <param name="key" value="com.y.foo.Country" />
    <param name="value" value="fr" />
  </filter>
</appender>

#2

I wish I could be a bit more helpful than this, but you may want to investigate using some filters? Perhaps your logging could output the country code and you could match your filter based on that?

我希望我能比这更有帮助,但你可能想调查使用一些过滤器?也许您的日志记录可以输出国家/地区代码,您可以根据它来匹配您的过滤器?

A StringMatchFilter should probably be able to match it for you.

StringMatchFilter可能应该能够为您匹配它。

Couldn't get the below address to work properly as a link, but if you look at it, it has some stuff on separate file logging using filters.

无法获得以下地址作为链接正常工作,但如果你看一下它,它有一些东西在使用过滤器的单独文件记录。

http://mail-archives.apache.org/mod_mbox/logging-log4j-user/200512.mbox/<1CC26C83B6E5AA49A9540FAC8D35158B01E2968E@pune.kaleconsultants.com > (just remove the space before the >)

http://mail-archives.apache.org/mod_mbox/logging-log4j-user/200512.mbox/<1CC26C83B6E5AA49A9540FAC8D35158B01E2968E@pune.kaleconsultants.com>(只需删除>之前的空格)

http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/spi/Filter.html

#3

Why not just call Thread.setName() when your job starts to set the name the Thread? If there is an access problem, configure quartz to use your own thread pool.

为什么不在作业开始设置Thread的名称时调用Thread.setName()?如果存在访问问题,请将quartz配置为使用您自己的线程池。

#4

I may be completely off base on my understanding of what you are attempting to accomplish, but I will take a stab at the solution. It sounds like you want a separate log file for each country for which you are processing email. Based on that understanding, here is a possible solution:

我可能完全不基于我对你想要完成的事情的理解,但我会采取一种解决方案。听起来您需要为要处理电子邮件的每个国家/地区单独记录日志文件。基于这种理解,这是一个可能的解决方案:

Set up an appender in your log4j configuration for each country for which you wish to log separately (US example provided):

在log4j配置中为您希望单独记录的每个国家/地区设置一个appender(提供美国示例):

log4j.appender.usfile=org.apache.log4j.FileAppender

log4j.appender.usfile.File=us.log

log4j.appender.usfile.layout=org.apache.log4j.PatternLayout

log4j.appender.usfile.layout.ConversionPattern=%m%n
Create a logger for each country and direct each of them to the appropriate appender (US example provided):

为每个国家/地区创建一个记录器,并将它们分别引导到相应的appender(提供的美国示例):

log4j.logger.my-us-logger=debug,usfile
In your code, create your Logger based on the country for which the email is being processed:

在您的代码中,根据正在处理电子邮件的国家/地区创建Logger:

Logger logger = Logger.getLogger("my-us-logger");

Logger logger = Logger.getLogger(“my-us-logger”);
Determine how you will accomplish step 3 for the subsequent method calls. You could repeat step 3 in each class/method; or you could modify the method signatures to accept a Logger as input; or you could possibly use ThreadLocal to pass the Logger between methods.

确定如何完成后续方法调用的步骤3。您可以在每个类/方法中重复步骤3;或者您可以修改方法签名以接受Logger作为输入;或者您可以使用ThreadLocal在方法之间传递Logger。

Extra info: If you do not want the log statements going to parent loggers (e.g. the rootLogger), you can set their additivity flags to false (US example provided):

额外信息:如果您不希望日志语句转到父记录器(例如rootLogger),则可以将其可加性标志设置为false(提供美国示例):

log4j.additivity.my-us-logger=false

#1