Title: Jython Class.__subclasses__() does not match Python output (not in load order)
Type: Severity: major
Components: Core Versions: Jython 2.7
Milestone: Jython 2.7.1
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: zyasoft Nosy List: Myles, zyasoft
Priority: high Keywords:

Created on 2016-08-15.17:24:55 by Myles, last changed 2016-09-14.16:15:54 by zyasoft.

msg10893 (view) Author: Myles (Myles) Date: 2016-08-15.17:24:54

I'm using jython-standalone on maven, 2.7.0.

The output of class.__subclasses__() differs between Jython & Python.

In Python subclasses are in load order, in Jython the order is random.

I am attempting to run an application which loads files, then obtains the subclasses of the files by using __subclasses__()[-1]. Because it's not load order this causes the application to horrifically fail.

The application could be modified, but i'm in belief this should be fixed, I haven't looked into how complicated it is myself, so it might not be a walk in the park.
msg10894 (view) Author: Jim Baker (zyasoft) Date: 2016-08-16.00:52:04
Somewhat related to in that we would expect this ordering is done in the same (currently buggy) code path.
msg10895 (view) Author: Jim Baker (zyasoft) Date: 2016-08-16.22:21:11
Here's the relevant backing structure for subclasses in PyType:

    private Set<WeakReference<PyType>> subclasses = Generic.set();

Generic.set() in turn constructs the Set with a HashSet backing; synchronization is used on the subclasses for all usage of this set. So it would seem to be trivial to change this to using a LinkedHashSet instead to maintain original load order per Myles request to get the same behavior as CPython.

This code still will intersect with this other code I mentioned, but at least it's very much contained, with little to no risk I can see in making this change.
msg10898 (view) Author: Myles (Myles) Date: 2016-08-17.17:09:08
The solution that zyasoft (Jim Baker) suggested seems to work fine.

I tested this by changing it to a LinkedHashSet in Generic (May not be the ideal place but for the test I did this).

Then using the interpreter:

>>> Exception.__subclasses__()
[<type 'exceptions.StandardError'>, <type 'exceptions.StopIteration'>, <type 'exceptions.Warning'>, <class 'warnings._OptionError'>, <class 'sre_constants.error'>]

>>> class TestException(Exception):
...     test = 1

>>> Exception.__subclasses__()
[<type 'exceptions.StandardError'>, <type 'exceptions.StopIteration'>, <type 'exceptions.Warning'>, <class 'warnings._OptionError'>, <class 'sre_constants.error'>, <class '__main__.TestException'>]
>>> Exception.__subclasses__()
[<type 'exceptions.StandardError'>, <type 'exceptions.StopIteration'>, <type 'exceptions.Warning'>, <class 'warnings._OptionError'>, <class 'sre_constants.error'>, <class '__main__.TestException'>]

Indicating the order is now retained as previously the items would all be random.
msg10937 (view) Author: Jim Baker (zyasoft) Date: 2016-09-06.03:23:23
So CPython 2.7 doesn't quite have the behavior of LinkedHashSet when the subclasses are GC'ed, as seen when the subclasses are deleted.

But it doesn't seem worth replicating that behavior - class deletion is relatively rare.
msg10938 (view) Author: Jim Baker (zyasoft) Date: 2016-09-06.04:52:55
Fixed as of
Date User Action Args
2016-09-14 16:15:54zyasoftsetstatus: pending -> closed
2016-09-06 04:52:56zyasoftsetstatus: open -> pending
resolution: accepted -> fixed
messages: + msg10938
2016-09-06 03:23:23zyasoftsetmessages: + msg10937
2016-08-24 20:25:27zyasoftsetpriority: high
assignee: zyasoft
resolution: accepted
milestone: Jython 2.7.0 -> Jython 2.7.1
2016-08-17 17:09:09Mylessetmessages: + msg10898
2016-08-16 22:21:11zyasoftsetmessages: + msg10895
2016-08-16 00:52:05zyasoftsetnosy: + zyasoft
messages: + msg10894
2016-08-15 17:25:48Mylessettitle: Jython Class. -> Jython Class.__subclasses__() does not match Python output (not in load order)
2016-08-15 17:24:55Mylescreate