Issue1521

I discovered this issue while trying some old Python code on Jython
2.5.1 on Mac OS X 10.5.

The following page lists built in Python functions,
http://www.jython.org/docs/library/functions.html

At the end of the page there are four "Non-essential Built-in Functions"
listed:
* apply
* buffer
* coerce
* intern

There are all present in Jython 2.5.0 and 2.5.1 except buffer:

$ ~/jython2.5.1/jython
Jython 2.5.1 (Release_2_5_1:6813, Sep 26 2009, 13:47:54) 
[Java HotSpot(TM) Client VM (Apple Inc.)] on java1.5.0_20
Type "help", "copyright", "credits" or "license" for more information.
>>> apply
<built-in function apply>
>>> coerce
<built-in function coerce>
>>> intern
<built-in function intern>
>>> buffer
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'buffer' is not defined

Note that buffer is still in (C) Python 2.6 and the current
documentation for Python 2.7, and has not been deprecated.

Implementing the buffer protocol for strings, arrays, and buffers would seem to be reasonable for Jython, and necessary for implementing the memoryview type for 2.6. However, it's rarely used and the workaround of 

buffer = str

although inefficient and not quite right, is often just what's needed.

Deferring to 2.6.

Deferring to 2.7

This currently prevents Twisted to run (tested with Jython 2.7 beta).

Monkey patching (in twisted/python/compat.py)

      # workaround for Jython, see: http://bugs.jython.org/issue1521
      import __builtin__
      if not hasattr(__builtin__, 'buffer'):
          def _buffer(object, offset = None, size = None):
             if offset is None:
                offset = 0
             if size is None:
                size = len(object)
             return object[offset:offset+size]
          __builtin__.buffer = _buffer

works, but is less than ideal.

Would you agree to increase the prio of this ticket, so that it gets fixed before Jython 2.7 release?

I've upped the priority - getting Twisted to work with Jython is very important to us. I don't know if this will block a 2.7.0 though - it might get pushed to a 2.7.x just because getting buffer fully implemented may be a very large task (not as bad as it used to be, we have a large part of memoryview implemented now)

I really want to see 2.7.0 out in the next few months at the latest. I don't think it will be a typical .0 release in that I think I will loudly warn that backwards compatibility won't be frozen as it is in normal releases since it is the last of the 2.x line. I expect we'll be learning a bunch by working on 3.x and I want to backport lots of these lessons. I am highly interested in getting Twisted to work on both 2.7 and the future 3.x of Jython.

Thanks for upping prio!

There is at least one more problem with Twisted: with Jython, neither sys.platform nor os.name allows to identify the underlying real OS .. which is necessary for various reasons (event loop etc etc).

Do you consider os.name == 'java' to be a Jython bug or a feature?

If not, is 

import java.lang.System
java.lang.System.getProperty('os.name')

the proper/supported way of getting the real OS?

Here is the corresponding Twisted ticket: http://twistedmatrix.com/trac/ticket/3413#comment:21

Back when it was implemented, os.name being Java made sense as the original Jython was attempting to be OS unaware as Java had been doing at the time. Now that things have changed (we have a growing posix layer for example) it really doesn't make sense anymore, so I think it is now a bug. Unfortunately there is to much history here for changing in 2.x. I plan to make os.name properly point to the real underlying OS in 3.x.

For now we have os._name that points to the real OS. I think we will probably keep that as an alias in 3.x for backwards compatibility.

BTW for twisted, maybe memoryview() could be used instead of buffer? We have pretty good support for memoryview in 2.7b1

Talked to Glyph briefly at PyCon and apparently memoryview isn't the right thing for this. He mentioned a strategy used for Python3 that could probably be used for us too while we get around to possibly doing a buffer. Knowing that something about Python3's buffer makes it unusable for Twisted makes me less sure we want to bother with it though...

Of course I mean I'm not sure we want to bother with buffer() - I really want Twisted working one way or another as soon as we can.

I've been working on this for a few weeks, as jython-dev followers know, but I just noticed we have an actual ticket for it.

There's a reasonably conformant buffer object in the trunk now.

But there are several parts to this work: the buffer API at Java level, which we've had for nearly a year; the memoryview object; the buffer object; and to make other objects in Jython both offer and accept the buffer API where they should. That last part is a big piece of work as there are several object types that should offer the API, and lots of methods where only str (bytes) is accepted now, but Python should allow any object with the buffer API. See for example #2011. On the plus side, there is only one buffer API in Jython, so whatever works with  buffer should work with memoryview.

Users should find that buffer and memoryview work nicely with str and bytearray, and with unicode as a method argument (e.g. u"Hello".endswith(buffer('llo')) returns True).

I'm working on array.array at present, first to make it offer the API, then to accept buffer arguments. array.array raises interesting questions about the buffer API, and about memoryview, when the element implementation type is not byte. For now, I'm just making it work for array.array('b').

BTW: It would be good if the os.name issue and other Twisted blockers could be taken as distinct issues: this is big enough as the "buffer missing" ticket. I'm not up for a whole "get Twisted working" ticket.

Jeff: Luckily other people are thinking specifically about "get Twisted working in Jython" :) -- but this is a nice big part of it!

For the record, I completely agree about working on these things in small chunks :)

Closing this out; test_buffer works, along with the underlying buffer protocol, even though there are related issues with Java integration and more tangential ones about Twisted. But those are separate chunks of work.