如何使用Java从Internet下载和保存文件?

There is an online file (such as http://www.example.com/information.asp) I need to grab and save to a directory. I know there are several methods for grabbing and reading online files (URLs) line-by-line, but is there a way to just download and save the file using Java?

有一个在线文件(如http://www.example.com/inform.asp)需要抓取并保存到目录中。我知道有几种方法可以逐行抓取和读取在线文件(url)，但是有没有一种方法可以使用Java下载和保存文件?

20 个解决方案

#1

Give Java NIO a try:

尝试Java NIO:

URL website = new URL("http://www.website.com/information.asp");
ReadableByteChannel rbc = Channels.newChannel(website.openStream());
FileOutputStream fos = new FileOutputStream("information.html");
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);

Using transferFrom() is potentially much more efficient than a simple loop that reads from the source channel and writes to this channel. Many operating systems can transfer bytes directly from the source channel into the filesystem cache without actually copying them.

使用transferFrom()可能比从源通道读取并写入该通道的简单循环要高效得多。许多操作系统可以直接将字节从源通道传输到文件系统缓存，而不必实际复制它们。

Check more about it here.

在这里多检查一下。

Note: The third parameter in transferFrom is the maximum number of bytes to transfer. Integer.MAX_VALUE will transfer at most 2^31 bytes, Long.MAX_VALUE will allow at most 2^63 bytes (larger than any file in existence).

注意:transferFrom中的第三个参数是要传输的最大字节数。整数。MAX_VALUE转移最多2 ^ 31个字节,长。MAX_VALUE最多允许2 ^ 63字节(比任何文件存在)。

#2

Use apache commons-io, just one line code:

使用apache common -io，只需一行代码:

FileUtils.copyURLToFile(URL, File)

#3

Simpler nio usage:

简单的nio用法:

URL website = new URL("http://www.website.com/information.asp");
try (InputStream in = website.openStream()) {
    Files.copy(in, target, StandardCopyOption.REPLACE_EXISTING);
}

#4

public void saveUrl(final String filename, final String urlString)
        throws MalformedURLException, IOException {
    BufferedInputStream in = null;
    FileOutputStream fout = null;
    try {
        in = new BufferedInputStream(new URL(urlString).openStream());
        fout = new FileOutputStream(filename);

        final byte data[] = new byte[1024];
        int count;
        while ((count = in.read(data, 0, 1024)) != -1) {
            fout.write(data, 0, count);
        }
    } finally {
        if (in != null) {
            in.close();
        }
        if (fout != null) {
            fout.close();
        }
    }
}

You'll need to handle exceptions, probably external to this method.

您将需要处理异常，可能是该方法的外部异常。

#5

Downloading a file requires you to read it, either way you will have to go through the file in some way. Instead of line by line, you can just read it by bytes from the stream:

下载一个文件需要你阅读它，无论哪种方式，你都必须以某种方式浏览这个文件。不是逐行逐行，而是从流中读取字节数:

BufferedInputStream in = new BufferedInputStream(new URL("http://www.website.com/information.asp").openStream())
    byte data[] = new byte[1024];
    int count;
    while((count = in.read(data,0,1024)) != -1)
    {
        out.write(data, 0, count);
    }

#6

When using Java 7+ use the following method to download a file from the Internet and save it to some directory:

使用Java 7+时，使用以下方法从Internet下载文件并保存到某个目录:

private static Path download(String sourceURL, String targetDirectory) throws IOException
{
    URL url = new URL(sourceURL);
    String fileName = sourceURL.substring(sourceURL.lastIndexOf('/') + 1, sourceURL.length());
    Path targetPath = new File(targetDirectory + File.separator + fileName).toPath();
    Files.copy(url.openStream(), targetPath, StandardCopyOption.REPLACE_EXISTING);

    return targetPath;
}

Documentation here.

这里的文档。

#7

This answer is almost exactly like selected answer but with two enhancements: it's a method and it closes out the FileOutputStream object:

这个答案几乎与所选答案完全相同，但有两个增强:一个是方法，它关闭FileOutputStream对象:

    public static void downloadFileFromURL(String urlString, File destination) {    
        try {
            URL website = new URL(urlString);
            ReadableByteChannel rbc;
            rbc = Channels.newChannel(website.openStream());
            FileOutputStream fos = new FileOutputStream(destination);
            fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
            fos.close();
            rbc.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

#8

import java.io.*;
import java.net.*;

public class filedown {
    public static void download(String address, String localFileName) {
        OutputStream out = null;
        URLConnection conn = null;
        InputStream in = null;

        try {
            URL url = new URL(address);
            out = new BufferedOutputStream(new FileOutputStream(localFileName));
            conn = url.openConnection();
            in = conn.getInputStream();
            byte[] buffer = new byte[1024];

            int numRead;
            long numWritten = 0;

            while ((numRead = in.read(buffer)) != -1) {
                out.write(buffer, 0, numRead);
                numWritten += numRead;
            }

            System.out.println(localFileName + "\t" + numWritten);
        } 
        catch (Exception exception) { 
            exception.printStackTrace();
        } 
        finally {
            try {
                if (in != null) {
                    in.close();
                }
                if (out != null) {
                    out.close();
                }
            } 
            catch (IOException ioe) {
            }
        }
    }

    public static void download(String address) {
        int lastSlashIndex = address.lastIndexOf('/');
        if (lastSlashIndex >= 0 &&
        lastSlashIndex < address.length() - 1) {
            download(address, (new URL(address)).getFile());
        } 
        else {
            System.err.println("Could not figure out local file name for "+address);
        }
    }

    public static void main(String[] args) {
        for (int i = 0; i < args.length; i++) {
            download(args[i]);
        }
    }
}

#9

It's an old question but here's an elegant JDK-only solution:

这是一个古老的问题，但这里有一个优雅的只有jdk的解决方案:

public static void download(String url, String fileName) throws Exception {
    try (InputStream in = URI.create(url).toURL().openStream()) {
        Files.copy(in, Paths.get(fileName));
    }
}

Concise, readable, properly closed resources leveraging nothing but the core JDK and language features.

简洁、可读、适当关闭的资源只利用核心的JDK和语言特性。

#10

Personally, I've found Apache's HttpClient to be more than capable of everything I've needed to do with regards to this. Here is a great tutorial on using HttpClient

就我个人而言，我发现Apache的HttpClient比我需要做的一切都要强大。这里有一个关于使用HttpClient的很棒的教程

#11

This is another java7 variant based on Brian Risk's answer with usage of try-with statement:

这是另一个基于Brian Risk的答案并使用try-with语句的java7变体:

public static void downloadFileFromURL(String urlString, File destination) throws Throwable {

      URL website = new URL(urlString);
      try(
              ReadableByteChannel rbc = Channels.newChannel(website.openStream());
              FileOutputStream fos = new FileOutputStream(destination);  
              ){
          fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
      }

  }

#12

There are many elegant and efficient answers here. But the conciseness can make us lose some useful information. In particular, one often does not want to consider a connection error an Exception, and one might want to treat differently some kind of network-related errors - for example, to decide if we should retry the download.

这里有许多优雅而高效的答案。但是简洁会让我们失去一些有用的信息。特别是，人们通常不希望将连接错误视为异常，而可能希望对某些与网络相关的错误进行不同的处理——例如，决定是否应该重试下载。

Here's a method that does not throw Exceptions for network errors (only for truly exceptional problems, as malformed url or problems writing to the file)

这里有一个方法，它不会为网络错误抛出异常(仅针对真正异常的问题，如格式错误的url或写入文件的问题)

/**
 * Downloads from a (http/https) URL and saves to a file. 
 * Does not consider a connection error an Exception. Instead it returns:
 *  
 *    0=ok  
 *    1=connection interrupted, timeout (but something was read)
 *    2=not found (FileNotFoundException) (404) 
 *    3=server error (500...) 
 *    4=could not connect: connection timeout (no internet?) java.net.SocketTimeoutException
 *    5=could not connect: (server down?) java.net.ConnectException
 *    6=could not resolve host (bad host, or no internet - no dns)
 * 
 * @param file File to write. Parent directory will be created if necessary
 * @param url  http/https url to connect
 * @param secsConnectTimeout Seconds to wait for connection establishment
 * @param secsReadTimeout Read timeout in seconds - trasmission will abort if it freezes more than this 
 * @return See above
 * @throws IOException Only if URL is malformed or if could not create the file
 */
public static int saveUrl(final Path file, final URL url, 
  int secsConnectTimeout, int secsReadTimeout) throws IOException {
    Files.createDirectories(file.getParent()); // make sure parent dir exists , this can throw exception
    URLConnection conn = url.openConnection(); // can throw exception if bad url
    if( secsConnectTimeout > 0 ) conn.setConnectTimeout(secsConnectTimeout * 1000);
    if( secsReadTimeout > 0 ) conn.setReadTimeout(secsReadTimeout * 1000);
    int ret = 0;
    boolean somethingRead = false;
    try (InputStream is = conn.getInputStream()) {
        try (BufferedInputStream in = new BufferedInputStream(is); OutputStream fout = Files
                .newOutputStream(file)) {
            final byte data[] = new byte[8192];
            int count;
            while((count = in.read(data)) > 0) {
                somethingRead = true;
                fout.write(data, 0, count);
            }
        }
    } catch(java.io.IOException e) { 
        int httpcode = 999;
        try {
            httpcode = ((HttpURLConnection) conn).getResponseCode();
        } catch(Exception ee) {}
        if( somethingRead && e instanceof java.net.SocketTimeoutException ) ret = 1;
        else if( e instanceof FileNotFoundException && httpcode >= 400 && httpcode < 500 ) ret = 2; 
        else if( httpcode >= 400 && httpcode < 600 ) ret = 3; 
        else if( e instanceof java.net.SocketTimeoutException ) ret = 4; 
        else if( e instanceof java.net.ConnectException ) ret = 5; 
        else if( e instanceof java.net.UnknownHostException ) ret = 6;  
        else throw e;
    }
    return ret;
}

#13

There is an issue with simple usage of:

有一个问题的简单用法是:

org.apache.commons.io.FileUtils.copyURLToFile(URL, File) 

if you need to download and save very large files, or in general if you need automatic retries in case connection is dropped.

如果您需要下载并保存非常大的文件，或者通常如果您需要在连接被删除时自动重试的话。

What I suggest in such cases is Apache HttpClient along with org.apache.commons.io.FileUtils. For example:

在这种情况下，我建议使用Apache HttpClient和org.apache.commons.io.FileUtils。例如:

GetMethod method = new GetMethod(resource_url);
try {
    int statusCode = client.executeMethod(method);
    if (statusCode != HttpStatus.SC_OK) {
        logger.error("Get method failed: " + method.getStatusLine());
    }       
    org.apache.commons.io.FileUtils.copyInputStreamToFile(
        method.getResponseBodyAsStream(), new File(resource_file));
    } catch (HttpException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
    method.releaseConnection();
}

#14

It's possible to download the file with with Apache's HttpComponents instead of Commons-IO. This code allows you to download a file in Java according to its URL and save it at the specific destination.

可以使用Apache的httpcomponent(而不是common - io)下载文件。此代码允许您根据Java的URL下载文件并将其保存到特定的目的地。

public static boolean saveFile(URL fileURL, String fileSavePath) {

    boolean isSucceed = true;

    CloseableHttpClient httpClient = HttpClients.createDefault();

    HttpGet httpGet = new HttpGet(fileURL.toString());
    httpGet.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0");
    httpGet.addHeader("Referer", "https://www.google.com");

    try {
        CloseableHttpResponse httpResponse = httpClient.execute(httpGet);
        HttpEntity fileEntity = httpResponse.getEntity();

        if (fileEntity != null) {
            FileUtils.copyInputStreamToFile(fileEntity.getContent(), new File(fileSavePath));
        }

    } catch (IOException e) {
        isSucceed = false;
    }

    httpGet.releaseConnection();

    return isSucceed;
}

In contrast to the single line of code:

与单行代码相比:

FileUtils.copyURLToFile(fileURL, new File(fileSavePath),
                        URLS_FETCH_TIMEOUT, URLS_FETCH_TIMEOUT);

this code will give you more control over a process and let you specify not only time outs but User-Agent and Referer values, which are critical for many web-sites.

这段代码将使您对一个过程有更多的控制，并且让您不仅指定超时，还指定用户代理和引用值，这对于许多web站点来说是非常重要的。

#15

To summarize (and somehow polish and update) previous answers. The three following methods are practically equivalent. (I added explicit timeouts because I think they are a must, nobody wants a download to freeze forever when the connection is lost.)

总结(以某种方式改进和更新)以前的答案。以下三种方法实际上是等价的。(我添加了显式超时，因为我认为它们是必须的，当连接丢失时，没有人希望下载到永久冻结。)

public static void saveUrl1(final Path file, final URL url,
   int secsConnectTimeout, int secsReadTimeout)) 
    throws MalformedURLException, IOException {
    // Files.createDirectories(file.getParent()); // optional, make sure parent dir exists
    try (BufferedInputStream in = new BufferedInputStream(
       streamFromUrl(url, secsConnectTimeout,secsReadTimeout)  );
        OutputStream fout = Files.newOutputStream(file)) {
        final byte data[] = new byte[8192];
        int count;
        while((count = in.read(data)) > 0)
            fout.write(data, 0, count);
    }
}

public static void saveUrl2(final Path file, final URL url,
   int secsConnectTimeout, int secsReadTimeout))  
    throws MalformedURLException, IOException {
    // Files.createDirectories(file.getParent()); // optional, make sure parent dir exists
    try (ReadableByteChannel rbc = Channels.newChannel(
      streamFromUrl(url, secsConnectTimeout,secsReadTimeout) 
        );
        FileChannel channel = FileChannel.open(file,
             StandardOpenOption.CREATE, 
             StandardOpenOption.TRUNCATE_EXISTING,
             StandardOpenOption.WRITE) 
        ) {
        channel.transferFrom(rbc, 0, Long.MAX_VALUE);
    }
}

public static void saveUrl3(final Path file, final URL url, 
   int secsConnectTimeout, int secsReadTimeout))  
    throws MalformedURLException, IOException {
    // Files.createDirectories(file.getParent()); // optional, make sure parent dir exists
    try (InputStream in = streamFromUrl(url, secsConnectTimeout,secsReadTimeout) ) {
        Files.copy(in, file, StandardCopyOption.REPLACE_EXISTING);
    }
}

public static InputStream streamFromUrl(URL url,int secsConnectTimeout,int secsReadTimeout) throws IOException {
    URLConnection conn = url.openConnection();
    if(secsConnectTimeout>0) conn.setConnectTimeout(secsConnectTimeout*1000);
    if(secsReadTimeout>0) conn.setReadTimeout(secsReadTimeout*1000);
    return conn.getInputStream();
}

I don't find significant differences, all seem right to me. They are safe and efficient. (Differences in speed seem hardly relevant - I write 180Mb from local server to a SSD disk in times that fluctuate around 1.2 to 1.5 segs). They don't require external libraries. All work with arbitrary sizes and (to my experience) HTTP redirections.

我没有发现明显的差别，我觉得一切都是对的。它们既安全又高效。(速度上的差异似乎无关紧要——我从本地服务器写入180Mb到SSD磁盘的时间在1.2到1.5 segs之间波动)。它们不需要外部库。所有的工作都使用任意大小和(以我的经验)HTTP重定向。

Additionally, all throw FileNotFoundException if the resource is not found (error 404, typically), and java.net.UnknownHostException if the DNS resolution failed; other IOException correspond to errors during transmission.

此外，如果没有找到资源，所有人都会抛出FileNotFoundException(通常是error 404)，如果DNS解析失败，则会抛出java.net.UnknownHostException;其他IOException对应传输期间的错误。

(Marked as community wiki, feel free to add info or corrections)

(标记为社区维基，请随意添加信息或更正)

#16

public class DownloadManager {

    static String urls = "[WEBSITE NAME]";

    public static void main(String[] args) throws IOException{
        URL url = verify(urls);
        HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        InputStream in = null;
        String filename = url.getFile();
        filename = filename.substring(filename.lastIndexOf('/') + 1);
        FileOutputStream out = new FileOutputStream("C:\\Java2_programiranje/Network/DownloadTest1/Project/Output" + File.separator + filename);
        in = connection.getInputStream();
        int read = -1;
        byte[] buffer = new byte[4096];
        while((read = in.read(buffer)) != -1){
            out.write(buffer, 0, read);
            System.out.println("[SYSTEM/INFO]: Downloading file...");
        }
        in.close();
        out.close();
        System.out.println("[SYSTEM/INFO]: File Downloaded!");
    }
    private static URL verify(String url){
        if(!url.toLowerCase().startsWith("http://")) {
            return null;
        }
        URL verifyUrl = null;

        try{
            verifyUrl = new URL(url);
        }catch(Exception e){
            e.printStackTrace();
        }
        return verifyUrl;
    }
}

#17

You can do this in 1 line using netloader for Java:

您可以在一行中使用netloader进行Java:

new NetFile(new File("my/zips/1.zip"), "https://example.com/example.zip", -1).load(); //returns true if succeed, otherwise false.

#18

If you are behind a proxy, you can set the proxies in java program as below:

如果您支持代理，可以在java程序中设置如下代理:

        Properties systemSettings = System.getProperties();
        systemSettings.put("proxySet", "true");
        systemSettings.put("https.proxyHost", "https proxy of your org");
        systemSettings.put("https.proxyPort", "8080");

If you are not behind a proxy, don't include the lines above in your code. Full working code to download a file when you are behind a proxy.

如果您不在代理后面，请不要在代码中包含上面的行。在代理后下载文件的完整工作代码。

public static void main(String[] args) throws IOException {
        String url="https://raw.githubusercontent.com/bpjoshi/fxservice/master/src/test/java/com/bpjoshi/fxservice/api/TradeControllerTest.java";
        OutputStream outStream=null;
        URLConnection connection=null;
        InputStream is=null;
        File targetFile=null;
        URL server=null;
        //Setting up proxies
        Properties systemSettings = System.getProperties();
            systemSettings.put("proxySet", "true");
            systemSettings.put("https.proxyHost", "https proxy of my organisation");
            systemSettings.put("https.proxyPort", "8080");
            //The same way we could also set proxy for http
            System.setProperty("java.net.useSystemProxies", "true");
            //code to fetch file
        try {
            server=new URL(url);
            connection = server.openConnection();
            is = connection.getInputStream();
            byte[] buffer = new byte[is.available()];
            is.read(buffer);

                targetFile = new File("src/main/resources/targetFile.java");
                outStream = new FileOutputStream(targetFile);
                outStream.write(buffer);
        } catch (MalformedURLException e) {
            System.out.println("THE URL IS NOT CORRECT ");
            e.printStackTrace();
        } catch (IOException e) {
            System.out.println("Io exception");
            e.printStackTrace();
        }
        finally{
            if(outStream!=null) outStream.close();
        }
    }

#19

There is method $.fetch() in underscore-java library.

在underscore-java库中有一个方法$.fetch()。

pom.xml:

pom.xml:

  <groupId>com.github.javadev</groupId>
  <artifactId>underscore</artifactId>
  <version>1.31</version>

Code example:

代码示例:

import com.github.underscore.lodash.$;

public class Download {
    public static void main(String ... args) {
        String text = $.fetch("https://*.com/questions"
        + "/921262/how-to-download-and-save-a-file-from-internet-using-java").text();
    }
}

#20

Below is the sample code to download movie from internet with java code:

下面是用java代码从internet下载电影的示例代码:

URL url = new 
URL("http://103.66.178.220/ftp/HDD2/Hindi%20Movies/2018/Hichki%202018.mkv");
    BufferedInputStream bufferedInputStream = new  BufferedInputStream(url.openStream());
    FileOutputStream stream = new FileOutputStream("/home/sachin/Desktop/test.mkv");


    int count=0;
    byte[] b1 = new byte[100];

    while((count = bufferedInputStream.read(b1)) != -1) {
        System.out.println("b1:"+b1+">>"+count+ ">> KB downloaded:"+new File("/home/sachin/Desktop/test.mkv").length()/1024);
        stream.write(b1, 0, count);
    }