内部IT环境中日志文件的最佳位置

时间:2021-10-30 10:19:35

All of my users are a short walk down the hall, and all of my programs run on workstations on the same LAN. Some years ago, I had the staff write the log files for all of their programs to a shared folder hierarchy, naming each log file after the machine name in a sub-directory named after the app.

我的所有用户都在大厅里走了一小段路,我的所有程序都在同一局域网上的工作站上运行。几年前,我让工作人员将所有程序的日志文件写入共享文件夹层次结构,将每个日志文件命名为以app命名的子目录中的机器名称。

But this arrangement wasn't that great, since if the file server went down then none of the programs anywhere could write logs. Yet keeping logs local to each workstation would make it a pain in the ass to read them whenever we had to debug a problem.

但是这种安排并不是很好,因为如果文件服务器出现故障,那么任何地方的程序都无法写入日志。然而,每当我们不得不调试问题时,将日志保存在每个工作站的本地会使得阅读它们变得很麻烦。

We tried making a DNS alias for the logging fileserver so we could point it to a backup machine when necessary, but DNS aliases don't work with Windows file shares.

我们尝试为日志记录文件服务器创建DNS别名,以便我们可以在必要时将其指向备份计算机,但DNS别名不适用于Windows文件共享。

Putting the path to the shared log folder in each program isn't great--even if it's field configurable--because we have dozens of programs on dozens of machines.

将路径放在每个程序的共享日志文件夹中并不是很好 - 即使它是可现场配置的 - 因为我们在几十台机器上有几十个程序。

We've also looked into using Microsoft's distributed file system, but the price is ridiculous.

我们也研究过使用微软的分布式文件系统,但价格太荒谬了。

I'd like a way to gather the logging for many programs on the local network into one place so I can tail and analyze them without paying a visit to the remote machine. We use .Net for all our programs.

我想要一种方法将本地网络上的许多程序的日志记录收集到一个地方,这样我就可以在不访问远程计算机的情况下进行尾随和分析。我们为所有程序使用.Net。

Edit: I'd like to avoid setting up a file-share on each user's workstation, or solutions that trawl for logs each night, since I want to be able to read fresh logs on demand, or moments after a problem is reported.

编辑:我想避免在每个用户的工作站上设置文件共享,或者每天晚上搜索日志的解决方案,因为我希望能够按需读取新的日志,或者在报告问题后立即读取。

10 个解决方案

#1


1  

You can use MSMQ. Write your logs to a MSMQ queue, and then have a service that picks these logs up and puts them in a database or out to a file in a central location if you want. It wouldn't be instantaneous, but you could tell it to run whenever you want to get new log entries. Plus it would be reliable since it uses MSMQ.

您可以使用MSMQ。将您的日志写入MSMQ队列,然后获得一个服务,该服务可以选择这些日志并将其放入数据库中,或者根据需要将其放入*位置的文件中。它不会是即时的,但是只要您想获得新的日志条目,就可以告诉它运行。此外,它使用MSMQ是可靠的。

#2


3  

In an IT environment which I worked in a few years back, we had each machine write it's log files locally and wipe them every five days. The server would log in every night to grab the latest logs from each machine. If the server went down, it would just grab two days worth of logs from everyone. If a client went down, it's log could be grabbed the next day as well.

在我几年前工作的IT环境中,我们让每台机器在本地写入日志文件并每隔五天擦除一次。服务器每晚都会登录以从每台机器获取最新的日志。如果服务器出现故障,它只会从每个人那里获取两天的日志。如果客户端发生故障,那么第二天也可以抓住它的日志。

#3


1  

Our shop has had good experience with storing logs in a database table. You can schedule cleanup as a database job whenever you feel like it, to prevent it from getting too large.

我们的商店在将日志存储在数据库表中方面具有良好的经验。您可以随时将清理计划为数据库作业,以防止它变得太大。

#4


1  

We use the approach of logging to a known location locally which is shared. Then it's easy to have a second process pull these logs for later processing (dump to database, collect and archive etc). If the collection process dies, then nothing is really is affected.

我们使用在本地登录到共享的已知位置的方法。然后很容易让第二个进程拉出这些日志以便以后处理(转储到数据库,收集和存档等)。如果收集过程死亡,那么什么都不会受到影响。

Make sure you don't have a central point of failure. I've seen examples of this that used a central logging database that all apps depended on and when it went down everything else did too. Not clever.

确保没有中心故障点。我已经看到过这样的例子,它使用了一个所有应用程序所依赖的*日志记录数据库,当它崩溃时其他所有内容都是如此。不聪明。

#5


0  

Can you have the log files writtin to peoples local machine and then have a script that pulls them to a common file server as a nightly batch job?

您是否可以将日志文件写入本地计算机,然后使用脚本将它们作为夜间批处理作业提取到公共文件服务器?

#6


0  

What about having your application's logger as a subscriber service (using remoting). Then implement LogServers which subscribe to the application. When the app has something to log, it sends it to the subscribers. If there is a problem, such as the subscriber not responding/dead and you have no working subscribers, write locally but sound an alarm. E.g. a multi-machine version of the TraceListener and Debug classes.

如何将应用程序的记录器作为订户服务(使用远程处理)。然后实现订阅该应用程序的LogServers。当应用程序有记录内容时,会将其发送给订阅者。如果存在问题,例如订户没有响应/死,并且您没有正在工作的订户,则在本地写入但发出警报。例如。 TraceListener和Debug类的多机器版本。

You'd almost certainly want the application to send log messages from a different thread to the rest of your application.

您几乎肯定希望应用程序将日志消息从不同的线程发送到您的应用程序的其余部分。

#7


0  

DFS is included with Windows Server, and you can regedit to enable NetBIOS with a CNAME.

DFS包含在Windows Server中,您可以使用CNAME来重新启用NetBIOS。

I've also tried Splunk for pulling and analyzing log files, but it's a bit of a CPU hog and I couldn't spare the horsepower. Otherwise, though, it was a pretty nice product.

我也尝试过Splunk来提取和分析日志文件,但它有点像CPU,我无法放弃马力。否则,它是一个非常好的产品。

#8


0  

We wrote a single application that hosts other applications (plug-in architecture). When an exception occurs, the necessary information is written to an XML file on the user's machine (which is named <USERNAME>.xml). Either upon our request or when the application exits, it sends the XML to a web service where it writes the files to a location where we can go and check them.

我们编写了一个托管其他应用程序的单个应用程序(插件架构)。发生异常时,必要的信息将写入用户计算机上的XML文件(名为 .xml)。根据我们的请求或应用程序退出时,它会将XML发送到Web服务,然后将文件写入我们可以去的位置并检查它们。

If the web service is down, it's no big deal since the XML file is still on the user's machine and will get uploaded the next time they run the application.

如果Web服务已关闭,那么由于XML文件仍在用户的计算机上并且下次运行应用程序时会上传,因此没什么大不了的。

#9


0  

write to a database also (like CALM) so you only have to troll the log files when the database was down

也写入数据库(如CALM),因此您只需在数据库关闭时拖动日志文件

#10


0  

I agree with craigb's comment. Creating additional single points of failure should be avoided in a LAN environment.

我同意克雷格的评论。在LAN环境中应避免创建其他单点故障。

My preference is to have the clients log locally, and then copy these log files to a central location. In this case if your log server falls over then there won't be any serious outage. Additionally, sometimes writing a log file to a network can be performance bottleneck.

我的首选是让客户端本地登录,然后将这些日志文件复制到一个中心位置。在这种情况下,如果您的日志服务器崩溃,那么将不会有任何严重的中断。此外,有时将日志文件写入网络可能是性能瓶颈。

Another issue with the central log server is if one application misbehaves, it can take down all the apps. I've seen countless bugs, which when encountered cause the program to enter an infinite loop and spew logs messages until the device fills up.

*日志服务器的另一个问题是,如果一个应用程序行为不当,它可能会删除所有应用程序。我遇到过无数的错误,当遇到这些错误时会导致程序进入无限循环并且会记录消息,直到设备填满为止。

If you really, want the central logging, for compliance reasons or something else, I would either write to a network drive, or write to the socket in the fashion of syslogd. Since syslogd is quite standard, you shouldn't have any trouble setting one up on a server, and there's plenty of example code and libraries to write to it. One can even configure a hardware load balance to sit in front of two or more syslogd servers, providing higher availability of the service.

如果您真的想要*日志记录,出于合规性原因或其他原因,我会写入网络驱动器,或者以syslogd的方式写入套接字。由于syslogd非常标准,因此在服务器上设置一个是不容易的,并且有很多示例代码和库可以写入它。甚至可以将硬件负载平衡配置为位于两个或更多syslogd服务器之前,从而提供更高的服务可用性。

#1


1  

You can use MSMQ. Write your logs to a MSMQ queue, and then have a service that picks these logs up and puts them in a database or out to a file in a central location if you want. It wouldn't be instantaneous, but you could tell it to run whenever you want to get new log entries. Plus it would be reliable since it uses MSMQ.

您可以使用MSMQ。将您的日志写入MSMQ队列,然后获得一个服务,该服务可以选择这些日志并将其放入数据库中,或者根据需要将其放入*位置的文件中。它不会是即时的,但是只要您想获得新的日志条目,就可以告诉它运行。此外,它使用MSMQ是可靠的。

#2


3  

In an IT environment which I worked in a few years back, we had each machine write it's log files locally and wipe them every five days. The server would log in every night to grab the latest logs from each machine. If the server went down, it would just grab two days worth of logs from everyone. If a client went down, it's log could be grabbed the next day as well.

在我几年前工作的IT环境中,我们让每台机器在本地写入日志文件并每隔五天擦除一次。服务器每晚都会登录以从每台机器获取最新的日志。如果服务器出现故障,它只会从每个人那里获取两天的日志。如果客户端发生故障,那么第二天也可以抓住它的日志。

#3


1  

Our shop has had good experience with storing logs in a database table. You can schedule cleanup as a database job whenever you feel like it, to prevent it from getting too large.

我们的商店在将日志存储在数据库表中方面具有良好的经验。您可以随时将清理计划为数据库作业,以防止它变得太大。

#4


1  

We use the approach of logging to a known location locally which is shared. Then it's easy to have a second process pull these logs for later processing (dump to database, collect and archive etc). If the collection process dies, then nothing is really is affected.

我们使用在本地登录到共享的已知位置的方法。然后很容易让第二个进程拉出这些日志以便以后处理(转储到数据库,收集和存档等)。如果收集过程死亡,那么什么都不会受到影响。

Make sure you don't have a central point of failure. I've seen examples of this that used a central logging database that all apps depended on and when it went down everything else did too. Not clever.

确保没有中心故障点。我已经看到过这样的例子,它使用了一个所有应用程序所依赖的*日志记录数据库,当它崩溃时其他所有内容都是如此。不聪明。

#5


0  

Can you have the log files writtin to peoples local machine and then have a script that pulls them to a common file server as a nightly batch job?

您是否可以将日志文件写入本地计算机,然后使用脚本将它们作为夜间批处理作业提取到公共文件服务器?

#6


0  

What about having your application's logger as a subscriber service (using remoting). Then implement LogServers which subscribe to the application. When the app has something to log, it sends it to the subscribers. If there is a problem, such as the subscriber not responding/dead and you have no working subscribers, write locally but sound an alarm. E.g. a multi-machine version of the TraceListener and Debug classes.

如何将应用程序的记录器作为订户服务(使用远程处理)。然后实现订阅该应用程序的LogServers。当应用程序有记录内容时,会将其发送给订阅者。如果存在问题,例如订户没有响应/死,并且您没有正在工作的订户,则在本地写入但发出警报。例如。 TraceListener和Debug类的多机器版本。

You'd almost certainly want the application to send log messages from a different thread to the rest of your application.

您几乎肯定希望应用程序将日志消息从不同的线程发送到您的应用程序的其余部分。

#7


0  

DFS is included with Windows Server, and you can regedit to enable NetBIOS with a CNAME.

DFS包含在Windows Server中,您可以使用CNAME来重新启用NetBIOS。

I've also tried Splunk for pulling and analyzing log files, but it's a bit of a CPU hog and I couldn't spare the horsepower. Otherwise, though, it was a pretty nice product.

我也尝试过Splunk来提取和分析日志文件,但它有点像CPU,我无法放弃马力。否则,它是一个非常好的产品。

#8


0  

We wrote a single application that hosts other applications (plug-in architecture). When an exception occurs, the necessary information is written to an XML file on the user's machine (which is named <USERNAME>.xml). Either upon our request or when the application exits, it sends the XML to a web service where it writes the files to a location where we can go and check them.

我们编写了一个托管其他应用程序的单个应用程序(插件架构)。发生异常时,必要的信息将写入用户计算机上的XML文件(名为 .xml)。根据我们的请求或应用程序退出时,它会将XML发送到Web服务,然后将文件写入我们可以去的位置并检查它们。

If the web service is down, it's no big deal since the XML file is still on the user's machine and will get uploaded the next time they run the application.

如果Web服务已关闭,那么由于XML文件仍在用户的计算机上并且下次运行应用程序时会上传,因此没什么大不了的。

#9


0  

write to a database also (like CALM) so you only have to troll the log files when the database was down

也写入数据库(如CALM),因此您只需在数据库关闭时拖动日志文件

#10


0  

I agree with craigb's comment. Creating additional single points of failure should be avoided in a LAN environment.

我同意克雷格的评论。在LAN环境中应避免创建其他单点故障。

My preference is to have the clients log locally, and then copy these log files to a central location. In this case if your log server falls over then there won't be any serious outage. Additionally, sometimes writing a log file to a network can be performance bottleneck.

我的首选是让客户端本地登录,然后将这些日志文件复制到一个中心位置。在这种情况下,如果您的日志服务器崩溃,那么将不会有任何严重的中断。此外,有时将日志文件写入网络可能是性能瓶颈。

Another issue with the central log server is if one application misbehaves, it can take down all the apps. I've seen countless bugs, which when encountered cause the program to enter an infinite loop and spew logs messages until the device fills up.

*日志服务器的另一个问题是,如果一个应用程序行为不当,它可能会删除所有应用程序。我遇到过无数的错误,当遇到这些错误时会导致程序进入无限循环并且会记录消息,直到设备填满为止。

If you really, want the central logging, for compliance reasons or something else, I would either write to a network drive, or write to the socket in the fashion of syslogd. Since syslogd is quite standard, you shouldn't have any trouble setting one up on a server, and there's plenty of example code and libraries to write to it. One can even configure a hardware load balance to sit in front of two or more syslogd servers, providing higher availability of the service.

如果您真的想要*日志记录,出于合规性原因或其他原因,我会写入网络驱动器,或者以syslogd的方式写入套接字。由于syslogd非常标准,因此在服务器上设置一个是不容易的,并且有很多示例代码和库可以写入它。甚至可以将硬件负载平衡配置为位于两个或更多syslogd服务器之前,从而提供更高的服务可用性。