使用bash脚本从在线数据库下载文件

I want to download some files from an online database, but it does not allow me to download all the files at once. Instead it offers to download a file for a searched keyword. Because I have more than 20000 keywords, it's not feasible for me. For example, I want to download whole information about miRNA-mRNA interaction from SarBase, but it does not offer an option to download all of them at once. I wonder, how can I download it by writing some scripts. Can anybody help me?

我想从在线数据库下载一些文件，但它不允许我一次下载所有文件。相反，它提供下载搜索关键字的文件。因为我有超过20000个关键字，所以对我来说不可行。例如，我想从SarBase下载有关miRNA-mRNA相互作用的全部信息，但它不提供同时下载所有这些信息的选项。我想知道，如何通过编写一些脚本来下载它。有谁能够帮助我？

2 个解决方案

#1

Make a file called getdb.sh.

创建一个名为getdb.sh的文件。

#!/bin/bash
echo "Download keywords in kw.txt."
for kw in $(cat kw.txt)
do
    curl http://www.mirbase.org/cgi-bin/get_seq.pl?acc=$kw > $kw.txt
done

Create another file called kw.txt:

创建另一个名为kw.txt的文件：

MI0000342
MI0000343
MI0000344

Then run this

然后运行它

$ chmod +x getdb.sh
$ ./getdb.sh

Download keywords in kw.txt.

$ ls -1 *.txt
kw.txt
MI0000342.txt
MI0000343.txt
MI0000344.txt

#2

another way

其他方式

cat kw.txt |xargs -i curl -o {}.txt http://www.mirbase.org/cgi-bin/get_seq.pl?acc={}

#1