Issue2182

Jython relies on "ant" for the build process and developers have to configure their IDEs individually to import/use the project.
I would like to migrate Jython to the standard Maven structure, this would make it much easier to get started (pom.xml can easily be opened out-of-the-box in Eclipse, Netbeans, IntelliJ). We can still use the ant build.xml for the more complicated or "special" parts (it's easy to call ant from Maven).

This is a good idea!

I believe the general consensus is that we will use Gradle instead, especially since it's also easy to use Ant tasks (such as our expose processor for Python types) from Gradle.

Maybe we can do this as the first task for 2.7.2?

Given the recent switch of the repo from hg.python.org to GitHub, we need to get this build change done sooner (2.7.2) than later. See http://sourceforge.net/p/jython/mailman/message/34736089/

Top priority for 2.7.2, so marking accordingly

Also discussed here: https://github.com/jythontools/jython/issues/11

Users will continue to depend on the JARs we currently build using Ant, and the installer, which takes the approach that all dependencies are bundled in javalib (or the standalone JAR). So it is clear that we need to continue to deliver that. Call this the "bundled" product.

I take this issue to be a request that we build (using Gradle) an artifact that can be cited as a dependency by other projects, and cites its dependencies in turn. And this is an enhancement that stands alongside the established, bundled product. I imagine it would be referenced by the co-ordinates "org.python:jython:2.7.2". Is this a reasonable interpretation?

It seems feasible for me to try this, leaving the Ant build exactly as it is and starting from scratch with Gradle. If it fails we haven't broken anything. One would run both builds to release Jython.

There will be a problem keeping the versions of JARs in sync between the two builds, that we will want to solve eventually (I predict), by moving production of both to Gradle: not for 2.7.2, though.

Brief progress report. I have a build.gradle of about 700 lines now and it feels like each line has been re-written about 5 times. Jython is an interesting choice as one's first Gradle project.

I am aiming for the build to produce a JAR that can be placed in a repository and cited in other projects' Maven and Gradle builds. For this I think I need:

1. to include all the Jython-specific material (including the borrowed standard library and other resources), so a bit like the standalone JAR.

2. to exclude all the material that a dependent project could get by itself from the repository (no jar'red jars or their contents), so quite unlike jython.jar or the standalone JAR.

The Gradle build will not (for some time) give us all the facilities of the Ant build, e.g. regrtest or installers. For this reason, it runs as a separate command, with its own build and output directories. Also, the source layout should be as we have it now, although Gradle is quite grumpy about that.

Does anyone think that the specification above is *not* what we should be aiming for?

I can run the compile, expose, stdlib copy, and (some) unit tests, and I get a decent-looking JAR. If you name the target JAR correctly, I can generate version.properties from it, without all that error-prone editing we have now. (I like Gradle better than Ant for this, apart from the fact that someone has already given us an Ant build. Friends don't make friends write XML.)

I have not yet managed to run the Jython compiler Ant task from Gradle (pycompile), so as to get the compiled stdlib in the JAR. What I did for the expose task isn't quite working for me. This is how it's been.

Still to do: finish pycompile, resources a comparison with jython-standalone.jar indicates are still missing, generate a pom, and learn to publish to a repository (if only on my machine).

re "Jython [as] an interesting choice as one first's Grade project": I think this is true for many tools or libraries we have integrated, often it's our first exposure to them as well, and we tend to push their capabilities simultaneously. Glad you made some good progress!

Having two parallel build systems for the time being makes sense to me; the specific initial targets in particular hit what we need for this Gradle build in terms of decoupling external dependencies, and thus playing better in the overall Java ecosystem. It shouldn't be too difficult to get -m test.regrtest running given what you have, even without stdlib building done, but I may be missing somehting in what you are describing.

I've got Gradle building (what looks like) a correct JAR now. The JAR works when substituted for jython-dev.jar in the development environment. Jython gives me a prompt and appears to work, but I can't test it yet.

There are a few wrinkles not due to the Gradle build itself. The regrtest does not run for reasons to do with normalizing paths that include a JAR, and %py.class files are ignored due to #2697. This makes me wonder how things are with the standalone JAR.

I've chosen to exclude the tests from the JAR but I think there a case for building one that includes the test suite. Or one that contains just the test suite.

Our unit tests will need tweaks since some assume a particular layout of "installed Jython" that doesn't apply under dependency management. Many unit tests are disabled at present in the Gradle build.

This seems like a good place to pause. I'm to be holding several bits of unfinished (Jython) work and while this one is gradually yeilding, it's putting up a fight every step of the way. If you want to kick the tyres its here:

https://hg.python.org/jython/rev/921c66481876

I got a bit further with this. I had to hack the POM manually, but I managed to cite jython (published to my .m2 local repository) from a simple Maven project and have create an interpreter. :)

However, I am stuck enough over the POM thing to take the unprecedented step asking for help in the Gradle forums. https://discuss.gradle.org/t/create-pom-from-custom-jar-task/27839 .

If I get past that, there's still a lot to do, and some functional questions like what python.home or sys.excutable means in these circumstances.

I found my own work-around for to avoid manually hacking the POM. Gradle's ratio of questions to knowledgeable answers is high in the forums.

I can now make what I think is the right JAR (may be missing a resource or two) and a JAR of tests separately (https://hg.python.org/jython/rev/ed155b929cf7). I can get aprompt, but not run much as  the import mechanism is broken when using Gradle-built JAR. I ought to be able to run the regression test suite (most of it) and that's not working so far.

It's currently more a question of what to do than how. When running from our JAR, what do python.home or sys.executable mean? Some of our libraries and many of our tests assume a particular layout that works where we normally tests (it's all in ./dist), but not here.

However, this is very like the position an application is in when it uses our standalone JAR, so I'm looking at how that gets its bearings and which tests normally work.

The problems I was having with import were attributable to limitations in stand alone Jython.

I get reasonable success running the regrtest with this JAR, but our tests are not developed to run with the library in a JAR. We do not test the stand-alone JAR this way, and when I try it's about the same. I can attribute all my failures to one of the following:
1. There is no launcher. (Maybe sys.executable should be None, but we've been there.)
2. We do not have namespace packages, so a separate JAR of tests doesn't quite work for e.g. lib2to3.tests.
3. We try to open as a regular file test material that is now a resource in the JAR.

Many of the Java unit tests work, but others attempt to navigate the file tree of ~/dist, which doesn't exist at the point where Gradle runs unit tests.

I've been able to get a trivial demo program to work citing Jython as a dependency when published to my local .m2 cache.

I suggest the level of success in tests means I've got the JAR basically correct. And none of this has changed Jython, so I can't have broken it. I feel less certain that the JAR and POM will work correctly as dependencies in someone else's complex project, and it's that I'd to see tested when we get to a beta release (or maybe an alpha) published via sonatype.

In 2.7.2b1 (dry-run!) I can build a slim JAR containing just the core classes and the standard library. I can make a Jython application using Maven with this as a dependency, and run most of the regression tests (actually slightly more than with the standalone JAR).

So I tentatively call this done, and we'll see what users think.