Issue2041

Trying to run solrpy with Jython 2.5 and Python 2.7: 

Simple test script behaves ok in Python, but in Jython checking the document's title results in an error. 

Attaching the session with calls to python and jython and version number s and code. 

Test done on Windows 7, 64 bit, java version "1.6.0_27"

Thank you for trying out solrpy on Jython, and letting us know how you got on.

In order to make this a Jython bug, it will be necessary to discover what Jython is doing wrong. Failure of a particular sequence of operations in solrpy doesn't really pin it down: it could be that solrpy depends on a quirk of CPython, which would be a solrpy fault.

If you have the enthusiasm for it, could you please identify the Python operation that Jython does not do correctly? For example did the add() complete correctly? If not, why not? I see there's a test.test_all.py in the solrpy source that might help.

It's also worth pointing out that 2.5.3 is current and there's a 2.5.4 in the wings.

Hi Jeff, 
thanks for the super fast reply.  
I have to say I am new to Jython, (about to migrate from perl) having started by trying out a number of not so trivial Java libraries (Stanford Parser, WEKA, ...) to check whether one can really smoothly use arbitrary Java goodies ... and so far I could. 

I now tried solrpy and used the sample script that was provided with the package: It basically inserts a simple doc into the index and retrieves it with solr again. 

when saying 

  response = s.query('title:lucene')
  for hit in response:
      print hit

then in python I get this

{u'title': [u'Lucene in Action'], u'author': u'Erik Hatcher', u'author_s': u'Erik Hatcher', u'score': 0.15342641, u'_version_': 1432303263573606400L, u'id': u'1'}

which is fact what I also see when inspecting the index with Solr on-board tools

but when running the same code in jython 2.5 I only get

{u'score': 0.15342641}

so in essence the script doesn't exactly fail in jython, but I don't get the same search result.

Not sure what to make of it. But this seems clearly a case where python 
and jython do different things with the same script. Will try the version 2.5.3 in any case. 

Best regards for now,
Stefan

Update on versions: 

Have tried the same thing with 
Jython-2.5.2
Jython-2.5.4rc1
Jython-2.7b1

same result: the query (see above) returns only the score, not the other fields of the document.

Hi Stefan. If you're still struggling with this, are you able to isolate this to a particular failure in Jython itself? I'm trying to avoid someone having to learn solrpy.

What you've learned so far suggests maybe this XML response is either not received fully, or not parsed correctly. 

I notice there is a raw_query operation that returns the XML response. Could you post the results of raw_query in the Python and Jython client. If that is the same, we would be able to study in isolation why Jython can't parse it. A difference would suggest an i/o or networking problem.

After a little bit of setup (need to pip install solrpy to get the current solr package), I was able to reproduce Stefan's issue using Jython 2.7 trunk against Solr 4.10.1, which is pretty easy to install and run:

$ jython27 test_solr2.py
{u'score': 0.15342641}

More work to diagnose

Not so difficult to look into this. The query returns the following XML:

<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">1</int><lst name="params"><str name="fl">*,score</str><str name="q">title:lucene</str><str name="wt">standard</str><str name="version">2.2</str></lst></lst><result name="response" numFound="1" start="0" maxScore="0.15342641"><doc><str name="author">Erik Hatcher</str><str name="author_s">Erik Hatcher</str><str name="id">1</str><arr name="title"><str>Lucene in Action</str></arr><long name="_version_">1481349698602139648</long><float name="score">0.15342641</float></doc></result>
</response>

This is turn parsed by a parser returned by xml.sax.make_parser, with a handler defined by solrpy in solr.core.ResponseContentHandler. Unfortunately, I have not actually used or looked into this ancient tech ;) that supports sax, but that's the proximate point of our failure.

My limited testing suggests this problem is not occurring in xml.sax, but instead in solrpy's ResponseContentHandler.endElement and associated classes/functions. I don't see anything in this method that would suggest a problem with running on Jython - it's all straightforward code. Further investigation required.

Still present. My test:

# -*- coding: utf-8 -*-
import solr

# create a connection to a solr server
s = solr.SolrConnection('http://localhost:8983/solr')

# add a document to the index
doc = dict(
    id=2,
    title='Lucene in Thought',
    author=['Erik Hatcher', 'Otis Gospodnetić'],
    )
s.add(doc) #, commit=True)

# do a search
response = s.query('title:lucene')
for hit in response.results:
    print hit['title']

Problem is still present despite recent updates to XML support in 2.7.1 (hence my testing yet again). Note that it is connection.add(**dict)

Deferring to 2.7.2

I notice that the result Stefan exhibited, the structures have unicode keys, whereas our argument to query is a byte string. We have made some changes lately to use the default encoding in string comparisons. See
https://hg.python.org/jython/rev/f71e0b2cfaf7
and related issues.

Seems knotty, so I'm removing the promise of a fix in 2.7.2.