Downloading all available CSV & KML files from the specific website

I am trying to use python to automate the process of downloading all the available CSV and KML files from data.gov.sg. However, we have gotten the "HTTP Error 403: Forbidden" error message. We used to get a robots.txt error which has been solved. Is there anything wrong with our coding below?

我正在尝试使用python自动化从data.gov.sg下载所有可用的CSV和KML文件的过程。但是，我们收到了“HTTP Error 403：Forbidden”错误消息。我们曾经得到一个已经解决的robots.txt错误。我们的编码在下面有什么问题吗？

import mechanize
from time import sleep
br = mechanize.Browser()

br.open('https://data.gov.sg/')

f=open("source.html","w")
f.write(br.response().read()) 
f.close()

filetypes=[".csv",".kml"] 
myfiles=[]
for l in br.links(): 
    for t in filetypes:
        if t in str(l): 
            myfiles.append(l)


def downloadlink(l):
    f=open(l.text,"w") 
    br.click_link(l)
    f.write(br.response().read())
    f.close()
    print l.text," has been downloaded"
#br.back()

for l in myfiles:
    sleep(1) 
    downloadlink(l)

1 个解决方案

#1

HTTP 403 error http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.4 means that you have been forbidden access, either you need to be authorised to access this, or the server administrator has blocked you from accessing it and sending a 403 response as the notification.

HTTP 403错误http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.4表示您已被禁止访问，您需要被授权访问它，或者服务器管理员已被阻止您从访问它并发送403响应作为通知。

So there is nothing wrong with your code (although you appear to have lost indentation) that I can see that would cause this issue.

所以你的代码没有任何问题（尽管你似乎已经丢失了缩进），我可以看到这会导致这个问题。