在Ubuntu 16.04.3 LTS上安裝Python 3.6.3的脫坑指南

02-01

心血來潮,想要裝一個Python+Tensorflow玩一玩,於是從官網下了一個Python3.6.3的源代碼包,解壓,configure,make,make install,一切正常.

直到pip3 install tensorflow-gpu,出錯了.

pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Collecting tensorflow-gpu Could not fetch URL https://pypi.python.org/simple/tensorflow-gpu/: There was a problem confirming the ssl certificate: Cant connect to HTTPS URL because the SSL module is not available. - skipping

最坑的是,StackOverflow到處都是錯誤的教程,不外乎讓你安裝libssl-dev,或者是在CentOS 下面安裝openssl-devel,或者修改Setup. 本人親測,這種方法對於標題所述的環境(再加上openssl 1.1.0g)沒有作用. 後來經過一晚上的debug,四方尋找,終於從各種信息中拼湊出合適的方案.

直接debug pip.main(["install","tensorflow-gpu"])的執行過程,終於找到了關鍵的代碼:

# If we dont have TLS enabled, then WARN if anyplace were lookingn# relies on TLS.nif not HAS_TLS:n for link in itertools.chain(self.index_urls, self.find_links):n parsed = urllib_parse.urlparse(link)n if parsed.scheme == "https":n logger.warning(n "pip is configured with locations that require "n "TLS/SSL, however the ssl module in Python is not "n "available."n )n breakn

而HAS_LTS的定義是

try:n import ssl # noqan HAS_TLS = Truenexcept ImportError:n HAS_TLS = Falsen

於是我手動執行import ssl,果然出錯了

Traceback (most recent call last):n import ssln File "/.../ssl.py", line 97, in <module>n import _ssl # if we cant import it, let the error propagatenImportError: No module named _ssl n

原來是沒有安裝_ssl模塊.但是如何安裝呢?網上有教程說看源代碼的根目錄下面的Setup.py文件.於是,裡面有這麼幾行:

# Detect SSL support for the socket module (via _ssl)n search_for_ssl_incs_in = [n /usr/local/ssl/include,n /usr/contrib/ssl/include/n ]n ssl_incs = find_file(openssl/ssl.h, inc_dirs,n search_for_ssl_incs_inn )n if ssl_incs is not None:n krb5_h = find_file(krb5.h, inc_dirs,n [/usr/kerberos/include])n if krb5_h:n ssl_incs += krb5_hn ssl_libs = find_library_file(self.compiler, ssl, lib_dirs,n [/usr/local/ssl/lib,n /usr/contrib/ssl/lib/,n ])nn if (ssl_incs is not None andn ssl_libs is not None):n exts.append(Extension(_ssl, [_ssl.c],n include_dirs=ssl_incs,n library_dirs=ssl_libs,n libraries=[ssl, crypto],n depends=[socketmodule.h]), )n else:n missing.append(_ssl)n

去/usr/local/ssl下面找了一下,根本沒有include這個文件夾.問題很有可能出在這裡.我又去翻了openssl的readme,裡面是這麼說的:

4. If everything tests ok, install OpenSSL with
$ make install # Unix
$ mms install ! OpenVMS
$ nmake install # Windows
This will install all the software components in this directory

tree under PREFIX (the directory given with --prefix or its
default):
Unix:
bin/ Contains the openssl binary and a few other
utility scripts.
include/openssl
Contains the header files needed if you want
to build your own programs that use libcrypto
or libssl.
lib Contains the OpenSSL library files.

lib/engines Contains the OpenSSL dynamically loadable engines.
......

所以說include應該在/usr/local/include或者/usr/include裡面.於是,往search_for_ssl_incs_in裡面加一個/usr/include或者/usr/local/include(前者是系統自帶的openssl的地址),然後按照其他教程的說法把Python-3.6.3/Modules/Setup裡面的這麼幾行的注釋去掉:(按下面所示的這樣修改代碼)

# Socket module helper for socket(2)
_socket socketmodule.c
# Socket module helper for SSL support; you must comment out the other
# socket line above, and possibly edit the SSL variable:
SSL=/usr/local/ssl
_ssl _ssl.c
t-DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl
t-L$(SSL)/lib -lssl -lcrypto

重新make&&sudo make install,之後檢查import ssl是否報錯,如果不報錯應該就能正常使用pip了.(撒花)

至於問我為什麼不用系統自帶的Python 2.7的話,我只有說,我就是看不慣Python 2,來打我呀

另外,Ubuntu 的repo裡面貌似只到Python 3.5....

附教程:

用Python-3.6.3表示源代碼所在的根目錄,首先修改Python-3.6.3/setup.py,將相關部分的代碼改成下面這樣:

search_for_ssl_incs_in = [n /usr/local/ssl/include,n /usr/contrib/ssl/include/,n /usr/local/include,n /usr/includen ]n

然後cd Python-3.6.3,運行

./configure --enable-optimizations --enable-ipv6n

然後修改Python-3.6.3/Modules/Setup,將相關部分改成下面這樣:

# Socket module helper for socket(2)
_socket socketmodule.c
# Socket module helper for SSL support; you must comment out the other

# socket line above, and possibly edit the SSL variable:
SSL=/usr/local/ssl
_ssl _ssl.c
t-DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl
t-L$(SSL)/lib -lssl -lcrypto

然後make.在make的輸出之中如果能夠看到

The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_socket _ssl atexit
pwd time

說明_ssl模塊一定是被成功編譯了的,之後sudo make install,親測有效.

然後如果想要換安裝的DIR和PREFIX,可自便.

然後就能開心的用pip了.不用pip自己手動安裝module實在是很頭疼...