Fix pyopenssl adapter under Python 3

[webknjaz]

It works under Python 2, but not under Python 3.
It's a follow-up for #6

I'm submitting a ...

• bug report
• feature request
• question about the decisions made in the repository

Describe the bug. What is the current behavior?

It raises exception OpenSSL.SSL.WantReadError.

What is the motivation / use case for changing the behavior?
I want consistency in SSL.

To Reproduce

Steps to reproduce the behavior: tox -- --no-cov -s cheroot/test/test_ssl.py

Expected behavior

That it'd work.

Details

 cheroot/test/test_ssl.py::test_ssl_adapters[builtin] ✓                                                          50% █████     
WantReadError()                                                                                                               
Traceback (most recent call last):        
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 1219, in communicate                                    
    req.parse_request()            
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 689, in parse_request                                   
    success = self.read_request_line()        
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 728, in read_request_line                               
    request_line = self.rfile.readline()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 277, in readline                                        
    data = self.rfile.readline(256)         
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 513, in readline                                          
    b = self.read(nreadahead())     
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 492, in nreadahead
    readahead = self.peek(1)      
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 1076, in peek                                           
    return self._peek_unlocked(size)                                                                                          
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 1083, in _peek_unlocked                                   
    current = self.raw.read(to_read)                                                                                           
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)                                                                                             
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1814, in recv_into
    self._raise_ssl_error(self._ssl, result)                       
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1614, in _raise_ssl_error
    raise WantReadError()                                                                        
OpenSSL.SSL.WantReadError
Exception in thread CP Server Thread-13:                                                                                      Traceback (most recent call last):
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 1219, in communicate
    req.parse_request()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 689, in parse_request
    success = self.read_request_line()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 728, in read_request_line
    request_line = self.rfile.readline()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 277, in readline
    data = self.rfile.readline(256)
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 513, in readline
    b = self.read(nreadahead())
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 492, in nreadahead
    readahead = self.peek(1)
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 1076, in peek
    return self._peek_unlocked(size)
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 1083, in _peek_unlocked
    current = self.raw.read(to_read)
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1814, in recv_into
    self._raise_ssl_error(self._ssl, result)
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1614, in _raise_ssl_error
    raise WantReadError()
OpenSSL.SSL.WantReadError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/workers/threadpool.py", line 112, in run                                     conn.communicate()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 1256, in communicate
    self._conditional_error(req, '500 Internal Server Error')
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 1282, in _conditional_error                                 req.simple_response(response)                             
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/server.py", line 1071, in simple_response
    self.conn.wfile.write(EMPTY.join(buf))
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/makefile.py", line 31, in write
    self._flush_unlocked()
  File "/home/wk/src/github/cherrypy/cheroot/cheroot/makefile.py", line 40, in _flush_unlocked
    n = self.raw.write(bytes(self._write_buf))
  File "/home/wk/.pyenv/versions/3.7.0/lib/python3.7/socket.py", line 607, in write
    return self._sock.send(b)
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1729, in send
    self._raise_ssl_error(self._ssl, result)
  File "/home/wk/src/github/cherrypy/cheroot/.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1614, in _raise_ssl_error
    raise WantReadError()
OpenSSL.SSL.WantReadError

Environment

• Cheroot version: master (6.5.2)
• CherryPy version: N/A
• Python version: 3.X
• OS: XXX
• Browser: N/A

Additional context

[webknjaz]

Related:

• Setting socket timeouts breaks handshake pyca/pyopenssl#168 (comment)
• https://pypi.org/project/timeout-decorator/

[webknjaz]

https://github.com/maojingios/MKFQ/blob/master/XX-Net-3.3.1/code/default/python27/1.0/lib/noarch/hyper/ssl_compat.py#L98-L117

[webknjaz]

python-trio/trio#200 (comment)

[webknjaz]

https://github.com/Lukasa/hyper/blob/development/hyper/ssl_compat.py#L114-L115

[webknjaz]

https://bugs.python.org/issue32219

[webknjaz]

https://github.com/python-trio/trio/pull/365/files

[webknjaz]

Fixed.

[webknjaz]

@mar10 could you please try out PyOpenSSL support now? I think I've finally fixed it.

[mar10]

Hi,

gave it a quick test.
The problem was reproducible here: WsgiDAV/3.0.0a8 Cheroot/6.5.3 Python/3.7.0

(wsgidav-9PbuForg) bash-3.2$ python -m OpenSSL.debug
pyOpenSSL: 18.0.0
cryptography: 2.4.2
cffi: 1.11.5
cryptography's compiled against OpenSSL: OpenSSL 1.1.0j  20 Nov 2018
cryptography's linked OpenSSL: OpenSSL 1.1.0j  20 Nov 2018
Pythons's OpenSSL: OpenSSL 1.0.2p  14 Aug 2018
Python executable: /Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/bin/python
Python version: 3.7.0 (default, Jul 26 2018, 22:23:34) 
[Clang 9.1.0 (clang-902.0.39.2)]
Platform: darwin

2018-12-31 12:09:50.866 - INFO    : SSL / HTTPS enabled. Adapter: <class 'cheroot.ssl.pyopenssl.pyOpenSSLAdapter'>
2018-12-31 12:09:50.866 - INFO    : Running WsgiDAV/3.0.0a8 Cheroot/6.5.3 Python/3.7.0
2018-12-31 12:09:50.866 - INFO    : Serving on https://0.0.0.0:8080 ...
WantReadError()
Traceback (most recent call last):
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/cheroot/server.py", line 1242, in communicate
    req.parse_request()
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/cheroot/server.py", line 712, in parse_request
    success = self.read_request_line()
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/cheroot/server.py", line 751, in read_request_line
    request_line = self.rfile.readline()
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/cheroot/server.py", line 300, in readline
    data = self.rfile.readline(256)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_pyio.py", line 513, in readline
    b = self.read(nreadahead())
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_pyio.py", line 492, in nreadahead
    readahead = self.peek(1)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_pyio.py", line 1076, in peek
    return self._peek_unlocked(size)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_pyio.py", line 1083, in _peek_unlocked
    current = self.raw.read(to_read)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1814, in recv_into
    self._raise_ssl_error(self._ssl, result)
  File "/Users/martin/.local/share/virtualenvs/wsgidav-9PbuForg/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1614, in _raise_ssl_error
    raise WantReadError()
OpenSSL.SSL.WantReadError

(looks like issue #132)
Seems you nailed it with the current master:

(wsgidav-9PbuForg) bash-3.2$ pip install git+git://github.com/cherrypy/cheroot.git@master
[...]
2018-12-31 12:24:28.789 - INFO    : WsgiDAV/3.0.0a8 Python/3.7.0 Darwin-18.2.0-x86_64-i386-64bit
[...]
2018-12-31 12:24:28.850 - INFO    : SSL / HTTPS enabled. Adapter: <class 'cheroot.ssl.pyopenssl.pyOpenSSLAdapter'>
2018-12-31 12:24:28.851 - INFO    : Running WsgiDAV/3.0.0a8 Cheroot/6.5.4.dev43+gab92fb97 Python/3.7.0
2018-12-31 12:24:28.851 - INFO    : Serving on https://0.0.0.0:8080 ...
2018-12-31 12:24:45.333 - INFO    : 127.0.0.1 - (anonymous) - [2018-12-31 11:24:45] "GET /" elap=0.006sec -> 200 OK

Thanks for keeping up the good work and happy new year 🎉

[webknjaz]

Cool thanks :)