RandomAccessFile实时读取大文件（转）

时间：2021-02-09 20:14:08

    最近有一个银行数据漂白系统,要求操作人员在页面调用远端Linux服务器的shell,并将shell输出的信息保存到一个日志文件,前台页面要实时显示日志文件的内容.这个问题难点在于如何判断哪些数据是新增加的,通过查看JDK 的帮助文档,java.io.RandomAccessFile
可以解决这个问题.为了模拟这个问题,编写LogSvr和 LogView类,LogSvr不断向mock.log日志文件写数据,而 LogView则实时输出日志变化部分的数据.

代码1:日志产生类

import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.Writer;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
/**
 *<p>title: 日志服务器</p>
 *<p>Description: 模拟日志服务器</p>
 *<p>CopyRight: CopyRight (c) 2010</p>
 *<p>Company: 99bill.com</p>
 *<p>Create date: 2010-6-18</P>
 *@author Tank Zhang<tank.zhang@99bill.com>
 *@version v0.1 2010-6-18
 */
public class LogSvr {
    
    private SimpleDateFormat dateFormat = 
        new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");

    /**
     * 将信息记录到日志文件
     * @param logFile 日志文件
     * @param mesInfo 信息
     * @throws IOException 
     */
    public void logMsg(File logFile,String mesInfo) throws IOException{
        if(logFile == null) {
            throw new IllegalStateException("logFile can not be null!");
        }
        Writer txtWriter = new FileWriter(logFile,true);
        txtWriter.write(dateFormat.format(new Date()) +"\t"+mesInfo+"\n");
        txtWriter.flush();
    }
    
    public static void main(String[] args) throws Exception{
        
        final LogSvr logSvr = new LogSvr();
        final File tmpLogFile = new File("mock.log");
        if(!tmpLogFile.exists()) {
            tmpLogFile.createNewFile();
        }
        //启动一个线程每5秒钟向日志文件写一次数据
        ScheduledExecutorService exec = 
            Executors.newScheduledThreadPool(1);
        exec.scheduleWithFixedDelay(new Runnable(){
            public void run() {
                try {
                    logSvr.logMsg(tmpLogFile, " 99bill test !");
                } catch (IOException e) {
                    throw new RuntimeException(e);
                }
            }
        }, 0, 5, TimeUnit.SECONDS);
    }
}

代码2:显示日志的类

import java.io.File;   
import java.io.IOException;   
import java.io.RandomAccessFile;   
import java.util.concurrent.Executors;   
import java.util.concurrent.ScheduledExecutorService;   
import java.util.concurrent.TimeUnit;   
  
public class LogView {   
    private long lastTimeFileSize = 0;  //上次文件大小   
    /**  
     * 实时输出日志信息  
     * @param logFile 日志文件  
     * @throws IOException  
     */  
    public void realtimeShowLog(File logFile) throws IOException{   
        //指定文件可读可写   
        final RandomAccessFile randomFile = new RandomAccessFile(logFile,"rw");   
        //启动一个线程每10秒钟读取新增的日志信息   
        ScheduledExecutorService exec =    
            Executors.newScheduledThreadPool(1);   
        exec.scheduleWithFixedDelay(new Runnable(){   
            public void run() {   
                try {   
                    //获得变化部分的   
                    randomFile.seek(lastTimeFileSize);   
                    String tmp = "";   
                    while( (tmp = randomFile.readLine())!= null) {   
                        System.out.println(new String(tmp.getBytes("ISO8859-1")));   
                    }   
                    lastTimeFileSize = randomFile.length();   
                } catch (IOException e) {   
                    throw new RuntimeException(e);   
                }   
            }   
        }, 0, 1, TimeUnit.SECONDS);   
    }   
       
    public static void main(String[] args) throws Exception {   
        LogView view = new LogView();   
        final File tmpLogFile = new File("mock.log");   
        view.realtimeShowLog(tmpLogFile);   
    }   
  
}

执行LogSvr类,LogSvr类会启动一个线程,每5秒钟向mock.log日志文件写一次数据,然后再执行LogView类,LogView每隔1秒钟读一次,如果数据有变化则输出变化的部分.

 

结果输出:

2010-06-19 17:25:54  99bill test !
2010-06-19 17:25:59  99bill test !
2010-06-19 17:26:04  99bill test !
2010-06-19 17:26:09  99bill test !
2010-06-19 17:26:14  99bill test !
2010-06-19 17:26:19  99bill test !

http://sunnylocus.iteye.com/blog/694666?page=2#comments

如果读取的日志文件，会被重命令或打开的操作。在使用RandomAccessFile每次读取后,执行close操作即可



在实习的公司碰到一个古怪的需求：在一台服务器上写日志文件，每当日志文件写到一定大小时，比如是1G，会将这个日志文件改名成另一个名字，并新建一个与原文件名相同的日志文件，再往这个新建的日志文件里写数据；要求写一个程序能实时地读取日志文件中的内容，并且不能写日志操作、重命名操作。

首先，这个问题在windows下几乎无解，因为一个程序打开了一个文件，再要对文件重命名是不可能的；而在Linux下，可以得到完美解决。因为Linux的文件系统有别于windows，Linux不使用文件名，而是使用inode号码来识别文件。关于inode的详细介绍参看理解inode。

Linux下文件夹下创建文件个数是有限的，即inode数是有限定的

更新：在windows下实时读取也是可行的

RandomAccessFile类中seek方法可以从指定位置读取文件，可以用来实现文件实时读取。JDK文档对RandomAccessFile的介绍

Instances of this class support both reading and writing to a random access file. A random access file behaves like a large array of bytes stored in the file system. There is a kind of cursor, or index into the implied array, called the file pointer; input operations read bytes starting at the file pointer and advance the file pointer past the bytes read. If the random access file is created in read/write mode, then output operations are also available; output operations write bytes starting at the file pointer and advance the file pointer past the bytes written. Output operations that write past the current end of the implied array cause the array to be extended.

在每一次读取后，close一下就不会影响重命名操作了。

seek

public void seek(long pos)
          throws IOException
Sets the file-pointer offset, measured from the beginning of this file, at which the next read or write occurs. The offset may be set beyond the end of the file. Setting the offset beyond the end of the file does not change the file length. The file length will change only by writing after the offset has been set beyond the end of the file.

 

Parameters:
pos - the offset position, measured in bytes from the beginning of the file, at which to set the file pointer.
Throws:
IOException - if pos is less than 0 or if an I/O error occurs.

getFilePointer

public long getFilePointer()
                    throws IOException
Returns the current offset in this file.

 

Returns:
the offset from the beginning of the file, in bytes, at which the next read or write occurs.
Throws:
IOException - if an I/O error occurs.


import java.io.File;

import java.io.IOException;
import java.io.RandomAccessFile;
import java.text.SimpleDateFormat;
import java.util.Date;

public class LogReader implements Runnable {
    private File logFile = null;
    private long lastTimeFileSize = 0; // 上次文件大小
    private static SimpleDateFormat dateFormat = new SimpleDateFormat(
            "yyyy-MM-dd HH:mm:ss");

    public LogReader(File logFile) {
        this.logFile = logFile;
        lastTimeFileSize = logFile.length();
    }

    /**
     * 实时输出日志信息
     */
    public void run() {
        while (true) {
            try {
                RandomAccessFile randomFile = new RandomAccessFile(logFile, "r");
                randomFile.seek(lastTimeFileSize);
                String tmp = null;
                while ((tmp = randomFile.readLine()) != null) {
                    System.out.println(dateFormat.format(new Date()) + "\t"
                            + tmp);
                }
                lastTimeFileSize = randomFile.length();
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
            try {
                Thread.sleep(500);
            } catch (InterruptedException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }
    }

}

http://www.cnblogs.com/en-heng/p/3926708.html


标签：

相关文章