Issue2858

classification
Title: test_ssl failure due to embedding Bouncy Castle JAR
Type: crash Severity: normal
Components: Library Versions: Jython 2.7.2
Milestone: Jython 2.7.2
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: jeff.allen Nosy List: jeff.allen, zyasoft
Priority: high Keywords: test failure causes

Created on 2020-02-03.20:43:46 by jeff.allen, last changed 2020-03-01.15:09:47 by jeff.allen.

Messages
msg12972 (view) Author: Jeff Allen (jeff.allen) Date: 2020-02-03.20:43:46
In a newly-installed v2.7.2b2 or b3, test_ssl fails like this:

test test_ssl_jy failed -- Traceback (most recent call last):
  File "...\inst\Lib\test\test_ssl.py", line 940, in test_load_cert_chain
    ctx.load_cert_chain(CERTFILE_PROTECTED, password=KEY_PASSWORD)

  ...

  File "...\inst\Lib\_sslcerts.py", line 281, in _read_pem_cert_from_data
    key_pair = key_converter.getKeyPair(obj.decryptKeyPair(provider))
PEMException: org.python.bouncycastle.openssl.PEMException: Unable to create OpenSSL PBDKF: PBKDF-OpenSSL SecretKeyFactory not available


This does not happen in the development configuration, nor if I introduce the Bouncy Castle JARs to the class path. It began after upgrading the JARs.

It seems likely the cause is the shading of the JAR that we do when embedding it in jython.jar during the Ant build. We do not do this shading to all the JARs we depend on. Some we name-translate:

    org/python/apache/commons/compress
    org/python/bouncycastle

And some we just copy:

    com/ziclix/python/sql
    jnr/ffi

Options appear to be:

1. hope the cause is within our library and fix it. (But maybe it is a BC feature?)
2. include the library unshaded. (Not known if this would be a problem.)
msg12974 (view) Author: Jeff Allen (jeff.allen) Date: 2020-02-04.22:46:30
This is not to do with shading, since I get the same effect when classes are bundled as org.bouncycastle... .. The Bouncy Castle JAR is signed and our jython.jar is rejected as a security provider, somewhere in the depths of javax.crypto,. The exception then emerges as a "not available".

BC is perhaps following this guidance: https://docs.oracle.com/javase/8/docs/technotes/guides/security/crypto/HowToImplAProvider.html#integritycheck

The checks only apply to certain operations, which are used in the failing test. Options now appear to be:
1. Do not bundle Bouncy Castle at all, and recommend installing it on the class path.
2. Bundle Bouncy Castle (un)shaded, and recommend installing it on the class path if an application needs the checked services.
3. Get our JARs acknowledged as a security provider.

I favour 2, shaded. Most of SSL then still works, judging by the test. When the checked services are required, _sslcerts.py will pick up the unshaded version in preference to the one we bundle. Option 3 sounds difficult: it's a claim of quality I doubt we could get the CA to back.
msg12976 (view) Author: Jim Baker (zyasoft) Date: 2020-02-04.22:54:57
@Jeff, thanks for the clarification. Option #2 - a release notes fix - seems the best approach, given signing vs shading, and the desire to get a timely 2.7.2 release. Longer term, it should also works better for the long term approach for Jython to avoid bundling jars at all.
msg13000 (view) Author: Jeff Allen (jeff.allen) Date: 2020-02-23.17:13:47
Ok. I think I will:

1. Leave BC shaded and bundled as it is.
2. Add something to NEWS about putting the BC JAR on the class path if needed.
3. Extract NEWS New Features to release notes on jython.org.
msg13002 (view) Author: Jeff Allen (jeff.allen) Date: 2020-02-23.21:44:26
Update to NEWS: https://hg.python.org/jython/rev/941001c49dd3
History
Date User Action Args
2020-03-01 15:09:47jeff.allensetstatus: pending -> closed
2020-02-23 21:44:26jeff.allensetmessages: + msg13002
2020-02-23 19:44:39jeff.allensetstatus: open -> pending
resolution: accepted -> wont fix
2020-02-23 17:13:47jeff.allensetmessages: + msg13000
2020-02-04 22:54:57zyasoftsetnosy: + zyasoft
messages: + msg12976
2020-02-04 22:46:30jeff.allensetassignee: jeff.allen
resolution: accepted
messages: + msg12974
title: test_ssl failure due to shading Bouncy Castle JAR -> test_ssl failure due to embedding Bouncy Castle JAR
2020-02-03 20:43:46jeff.allencreate