Title: solrpy: testscript runs in python, but not jython
Type: Severity: normal
Components: Library Versions: Jython 2.5
Status: open Resolution: accepted
Dependencies: Superseder:
Assigned To: Nosy List: StefanGeissler, jeff.allen, zyasoft
Priority: normal Keywords:

Created on 2013-04-13.18:48:29 by StefanGeissler, last changed 2018-03-07.18:54:28 by jeff.allen.

File name Uploaded Description Edit Remove
jython-problem.txt StefanGeissler, 2013-04-13.18:48:28
msg7993 (view) Author: Stefan Geißler (StefanGeissler) Date: 2013-04-13.18:48:28
Trying to run solrpy with Jython 2.5 and Python 2.7: 

Simple test script behaves ok in Python, but in Jython checking the document's title results in an error. 

Attaching the session with calls to python and jython and version number s and code. 

Test done on Windows 7, 64 bit, java version "1.6.0_27"
msg7994 (view) Author: Jeff Allen (jeff.allen) Date: 2013-04-14.07:28:41
Thank you for trying out solrpy on Jython, and letting us know how you got on.

In order to make this a Jython bug, it will be necessary to discover what Jython is doing wrong. Failure of a particular sequence of operations in solrpy doesn't really pin it down: it could be that solrpy depends on a quirk of CPython, which would be a solrpy fault.

If you have the enthusiasm for it, could you please identify the Python operation that Jython does not do correctly? For example did the add() complete correctly? If not, why not? I see there's a in the solrpy source that might help.

It's also worth pointing out that 2.5.3 is current and there's a 2.5.4 in the wings.
msg7995 (view) Author: Stefan Geißler (StefanGeissler) Date: 2013-04-14.14:54:02
Hi Jeff, 
thanks for the super fast reply.  
I have to say I am new to Jython, (about to migrate from perl) having started by trying out a number of not so trivial Java libraries (Stanford Parser, WEKA, ...) to check whether one can really smoothly use arbitrary Java goodies ... and so far I could. 

I now tried solrpy and used the sample script that was provided with the package: It basically inserts a simple doc into the index and retrieves it with solr again. 

when saying 

  response = s.query('title:lucene')
  for hit in response:
      print hit

then in python I get this

{u'title': [u'Lucene in Action'], u'author': u'Erik Hatcher', u'author_s': u'Erik Hatcher', u'score': 0.15342641, u'_version_': 1432303263573606400L, u'id': u'1'}

which is fact what I also see when inspecting the index with Solr on-board tools

but when running the same code in jython 2.5 I only get

{u'score': 0.15342641}

so in essence the script doesn't exactly fail in jython, but I don't get the same search result.

Not sure what to make of it. But this seems clearly a case where python 
and jython do different things with the same script. Will try the version 2.5.3 in any case. 

Best regards for now,
msg7996 (view) Author: Stefan Geißler (StefanGeissler) Date: 2013-04-14.15:34:02
Update on versions: 

Have tried the same thing with 

same result: the query (see above) returns only the score, not the other fields of the document.
msg8002 (view) Author: Jeff Allen (jeff.allen) Date: 2013-04-28.17:45:31
Hi Stefan. If you're still struggling with this, are you able to isolate this to a particular failure in Jython itself? I'm trying to avoid someone having to learn solrpy.

What you've learned so far suggests maybe this XML response is either not received fully, or not parsed correctly. 

I notice there is a raw_query operation that returns the XML response. Could you post the results of raw_query in the Python and Jython client. If that is the same, we would be able to study in isolation why Jython can't parse it. A difference would suggest an i/o or networking problem.
msg9121 (view) Author: Jim Baker (zyasoft) Date: 2014-10-07.23:34:25
After a little bit of setup (need to pip install solrpy to get the current solr package), I was able to reproduce Stefan's issue using Jython 2.7 trunk against Solr 4.10.1, which is pretty easy to install and run:

$ jython27
{u'score': 0.15342641}

More work to diagnose
msg9122 (view) Author: Jim Baker (zyasoft) Date: 2014-10-07.23:45:47
Not so difficult to look into this. The query returns the following XML:

<?xml version="1.0" encoding="UTF-8"?>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">1</int><lst name="params"><str name="fl">*,score</str><str name="q">title:lucene</str><str name="wt">standard</str><str name="version">2.2</str></lst></lst><result name="response" numFound="1" start="0" maxScore="0.15342641"><doc><str name="author">Erik Hatcher</str><str name="author_s">Erik Hatcher</str><str name="id">1</str><arr name="title"><str>Lucene in Action</str></arr><long name="_version_">1481349698602139648</long><float name="score">0.15342641</float></doc></result>

This is turn parsed by a parser returned by xml.sax.make_parser, with a handler defined by solrpy in solr.core.ResponseContentHandler. Unfortunately, I have not actually used or looked into this ancient tech ;) that supports sax, but that's the proximate point of our failure.
msg9123 (view) Author: Jim Baker (zyasoft) Date: 2014-10-09.00:18:11
My limited testing suggests this problem is not occurring in xml.sax, but instead in solrpy's ResponseContentHandler.endElement and associated classes/functions. I don't see anything in this method that would suggest a problem with running on Jython - it's all straightforward code. Further investigation required.
msg10632 (view) Author: Jim Baker (zyasoft) Date: 2016-01-11.05:05:29
Still present. My test:

# -*- coding: utf-8 -*-
import solr

# create a connection to a solr server
s = solr.SolrConnection('http://localhost:8983/solr')

# add a document to the index
doc = dict(
    title='Lucene in Thought',
    author=['Erik Hatcher', 'Otis Gospodnetić'],
s.add(doc) #, commit=True)

# do a search
response = s.query('title:lucene')
for hit in response.results:
    print hit['title']
msg10733 (view) Author: Jim Baker (zyasoft) Date: 2016-02-09.01:26:30
Problem is still present despite recent updates to XML support in 2.7.1 (hence my testing yet again). Note that it is connection.add(**dict)

Deferring to 2.7.2
msg11759 (view) Author: Jeff Allen (jeff.allen) Date: 2018-03-07.18:54:27
I notice that the result Stefan exhibited, the structures have unicode keys, whereas our argument to query is a byte string. We have made some changes lately to use the default encoding in string comparisons. See
and related issues.

Seems knotty, so I'm removing the promise of a fix in 2.7.2.
Date User Action Args
2018-03-07 18:54:28jeff.allensetmessages: + msg11759
milestone: Jython 2.7.2 ->
2016-02-09 01:26:30zyasoftsetpriority: normal
resolution: accepted
messages: + msg10733
milestone: Jython 2.7.2
2016-01-11 05:05:30zyasoftsetmessages: + msg10632
2014-10-09 00:18:13zyasoftsetmessages: + msg9123
2014-10-07 23:45:48zyasoftsetmessages: + msg9122
2014-10-07 23:34:26zyasoftsetnosy: + zyasoft
messages: + msg9121
2013-04-28 17:45:31jeff.allensetmessages: + msg8002
2013-04-14 15:34:02StefanGeisslersetmessages: + msg7996
2013-04-14 14:54:03StefanGeisslersetmessages: + msg7995
2013-04-14 07:28:42jeff.allensetnosy: + jeff.allen
messages: + msg7994
2013-04-13 18:48:29StefanGeisslercreate