Python3 BigQuery或谷歌云Python通过HTTP代理。

How to route BigQuery client calls through HTTP Proxy ?

如何通过HTTP代理路由BigQuery客户端调用?

Before Posting this, I tried following but it is still not routing through http proxy. And the Google Cloud service credentials are set through shell environment variable GOOGLE_APPLICATION_CREDENTIALS

在发布此消息之前，我尝试了以下步骤，但仍然没有通过http代理进行路由。而谷歌云服务凭据通过shell环境变量GOOGLE_APPLICATION_CREDENTIALS设置。

import httplib2
import socks
import google.auth

credentials, _ = google.auth.default()
http_client = httplib2.Http(proxy_info = httplib2.ProxyInfo(socks.PROXY_TYPE_HTTP, 'someproxy', 80));

bigquery_client = bigquery.Client(credentials=credentials, _http=http_client)

Outgoing traffic ( 172.217.x.x belongs to googleapis.com ) not routing through HTTP Proxy ,

即将离任的交通(172.217.x。x属于googleapis.com，而不是通过HTTP代理进行路由，

$ netstat -nputw
Local Address           Foreign Address
x.x.x.x                 172.217.6.234:443       SYN_SENT

EDIT: Found the solution, Posted it as answer.

编辑:找到解决方案，将其作为答案发布。

2 个解决方案

#1

The only way I found to create these credentials was by setting it directly in my os environment.

我发现创建这些凭据的惟一方法是直接将其设置在操作系统环境中。

Supposing that you have your json credential file already, maybe this will work for you:

假设您已经有了json凭据文件，可能这将对您有用:

import httplib2
import socks
import os
import google.auth

os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'path/to/your/credentials_file.json'

credentials, _ = google.auth.default()
http_client = httplib2.Http(proxy_info = httplib2.ProxyInfo(socks.PROXY_TYPE_HTTP, 'someproxy', 80))
bigquery_client = bigquery.Client(_http=http_client, credentials=credentials)

#2

Answering the question myself as I found the reason/solution.

当我找到原因/解决方案时，我自己回答这个问题。

Reason:

原因:

google-cloud-python library uses httplib2, As of this writing httplib2 has two code bases for python 2 and python 3. The Python 3 version of httplib2 is not implemented with socks/proxy support. Please refer to httplib2's repo#init_py.

google-cloud python库使用httplib2，就像这篇文章一样，httplib2在python 2和python 3中有两个代码库。Python 3版本的httplib2并没有使用socks/代理支持实现。请参考httplib2的repo#init_py。

Work Around:

解决:

There is a discussion to move google-cloud-python from httplib2 to urllib3, but in the mean time one can use httplib2shim

有一种讨论将谷歌云python从httplib2移到urllib3，但同时也可以使用httplib2shim。

import google.auth
import httplib2shim
import google_auth_httplib2

// More declarative way exists, but left for simplicity
os.environ["HTTP_PROXY"] = "someproxy:80"
os.environ["HTTPS_PROXY"] = "someproxy:80"
http_client = httplib2shim.Http()
credentials, _ = google.auth.default()

# IMO, Following 2 lines should be done at the google-cloud-python
# This exposes client speicific logic, and it already does that
credentials = google.auth.credentials.with_scopes_if_required
              (credentials, bigquery.Client.SCOPE)
authed_http = google_auth_httplib2.AuthorizedHttp(credentials,http_client)

bigquery_client = bigquery.Client(credentials=credentials, _http=authed_http)

#1