Issue1996

classification
Title: Core slots array out of bounds with multiple inheritance
Type: crash Severity: normal
Components: Core Versions: Jython 2.7
Milestone: Jython 2.7.1
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: stefan.richthofer Nosy List: amak, fwierzbicki, jeff.allen, pjenvey, stefan.richthofer, zyasoft
Priority: normal Keywords:

Created on 2012-12-05.07:53:01 by jeff.allen, last changed 2017-03-28.05:28:11 by zyasoft.

Messages
msg7543 (view) Author: Jeff Allen (jeff.allen) Date: 2012-12-05.07:53:00
I've run across what may be a bug in the core treatment of multiple 
inheritance when the slots array is active in a base.

Indications are that the slots array of only the first base mentioned  
is honoured. I came across this in work on the IO library where the 
exception class is defined (in CPython _pyio and _io):

class UnsupportedOperation(ValueError, IOError):
     pass

Now this class (both the one defined in Python and my Java equivalent 
using Py.makeClass()) causes a java.lang.ArrayIndexOutOfBounds exception as soon as an instance is required. This happens at the point where "errno" is assigned to slots[0] (by the __init__() of EnvironmentError, ancestor of IOError). The type that results in either case has zero slots, and that's where the out-of-bounds error arises.

This JUnit test fails quite nicely too:
     @Test
     public void pythonMyExc2() {
         interp.exec("class MyExc2(ValueError, IOError):\n    pass\n\n");
         try {
             interp.exec("raise MyExc2()");
             fail("MyExc2 not raised when expected");
         } catch (PyException e) {
             assertEquals("<class '__main__.MyExc2'>", e.type.toString());
         }
     }

Put the bases in the other order, and it works: the type you get has 3 
slots. (There's my work-around.)

I've tracked the problematic logic down to PyType.createAllSlots(), 
called from PyType newType(), which only takes notice of the number of 
slots in the first base mentioned (which becomes type.base) rather than 
making some aggregation involving all the bases. This is probably a bug.


*** Philip Jenvey adds (via the jython-dev list):

This is a nasty because:
a) the spec of how slots are handled, especially with inheritance, is pretty obscure. You can maybe get some idea from our test_slots_jy.py
b) the fact that our base exceptions define __slots__ is kind of a hack to simulate CPython's base exception attributes (that are stored as slot-like 'structure members' instead of in the __dict__).

... 

It *seems* like a bug in the slots handling. It'd be nice to reproduce this case without the exceptions as it would prove that (and would be a nice test for test_slots_jy.py). Otherwise maybe it's some strange interaction with how we're defining these exception slots.
msg7544 (view) Author: Philip Jenvey (pjenvey) Date: 2012-12-05.08:46:23
It's actually not slot handling itself. Turns out the problem is that PyType.best_base determines the wrong base type here (of which the slots handling is based off of):

Jython 2.7.0a2+ (j6 qbase qtip tip:f39766cb9cf6, Sep 10 2012, 12:13:45) 
[Java HotSpot(TM) 64-Bit Server VM (Apple Inc.)] on java1.6.0_37
Type "help", "copyright", "credits" or "license" for more information.
>>> class UnsupportedOperation(ValueError, IOError): pass
... 
>>> UnsupportedOperation.__base__
<type 'exceptions.ValueError'>

On CPython 2.5 and 2.7:
Python 2.5.4 (r254:67916, Aug  1 2011, 15:52:49) 
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class UnsupportedOperation(ValueError, IOError): pass
... 
>>> UnsupportedOperation.__base__
<type 'exceptions.IOError'>
msg7545 (view) Author: Philip Jenvey (pjenvey) Date: 2012-12-05.08:52:04
It's very likely our PyType.isSolidBase. It's supposed to simulate what typeobject.c::solid_base/extra_ivars does but it's not that straightforward to match it exactly. PyPy handles this quite a bit differently
msg9927 (view) Author: Jim Baker (zyasoft) Date: 2015-04-23.03:54:55
Related to #2101
msg10431 (view) Author: Jim Baker (zyasoft) Date: 2015-11-02.16:03:12
A number of FIXMEs in #1861 are due to this bug.
msg11159 (view) Author: Stefan Richthofer (stefan.richthofer) Date: 2017-03-03.09:44:32
Fixed as of https://hg.python.org/jython/rev/4884658aa644.
msg11206 (view) Author: Jeff Allen (jeff.allen) Date: 2017-03-08.08:30:18
Nice. I wasn't able to follow the clues Philip Jenvey gave us.

I'll follow up Jim's hint about FIXME-skips referring to this issue, easy in this case as they (all?) helpfully name the issue. There was a thing I did re-ordering the bases too, which is not now quite the same as CPython.
msg11211 (view) Author: Jeff Allen (jeff.allen) Date: 2017-03-09.09:24:51
Skips now removed, and this work-around reversed: https://hg.python.org/jython/file/tip/src/org/python/modules/_io/_jyio.java#l44. Surprised that removing the skips didn't produce test failures. Will push when regrtest is done, later today.
History
Date User Action Args
2017-03-28 05:28:11zyasoftsetstatus: pending -> closed
2017-03-09 09:24:51jeff.allensetmessages: + msg11211
2017-03-08 08:30:18jeff.allensetmessages: + msg11206
2017-03-03 09:44:32stefan.richthofersetstatus: open -> pending
nosy: + stefan.richthofer
messages: + msg11159
assignee: stefan.richthofer
milestone: Jython 2.7.2 -> Jython 2.7.1
resolution: fixed
2015-11-02 16:03:12zyasoftsetmessages: + msg10431
2015-11-02 15:38:18zyasoftsetmilestone: Jython 2.7.1 -> Jython 2.7.2
2015-04-23 03:54:55zyasoftsetnosy: + zyasoft
messages: + msg9927
milestone: Jython 2.7.1
2014-10-05 17:15:22zyasoftsetassignee: fwierzbicki -> (no value)
2013-02-20 00:33:25fwierzbickisetassignee: fwierzbicki
versions: + Jython 2.7, - 2.7a2
2013-02-18 09:29:32amaksetnosy: + amak
2012-12-05 08:52:04pjenveysetmessages: + msg7545
2012-12-05 08:46:23pjenveysetmessages: + msg7544
2012-12-05 07:53:01jeff.allencreate