Issue1672

classification
Title: Compileall does not compile if .pyc file exists
Type: Severity: normal
Components: Versions: Jython 2.7
Milestone:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: zyasoft Nosy List: fwierzbicki, otmarhumbel, pdina, yanne, zyasoft
Priority: high Keywords: patch

Created on 2010-11-01.13:36:26 by yanne, last changed 2015-03-17.13:51:13 by zyasoft.

Files
File name Uploaded Description Edit Remove
test.zip yanne, 2010-11-01.13:36:25 test files to reproduce the bugs
issue1672.patch pdina, 2014-09-06.23:39:08 patch
issue1672.patch pdina, 2014-09-07.01:46:40 patch + test
Messages
msg6215 (view) Author: Janne Härkönen (yanne) Date: 2010-11-01.13:36:24
Unzip attached zip and run python test.py

Afterwards, run 
java -jar jython-standalone-2.5.1.jar -m compileall .

$py.class file is not created for greeter.py because corresponding .pyc file already exists.
msg6219 (view) Author: Oti Humbel (otmarhumbel) Date: 2010-11-02.20:00:32
does the -f (force) flag help?
msg6221 (view) Author: Janne Härkönen (yanne) Date: 2010-11-03.05:00:14
Yes, the -f flag does the job indeed, so there's a workaround.

Still, being new to compileall, it took me a while to figure this out.
msg6222 (view) Author: Oti Humbel (otmarhumbel) Date: 2010-11-03.06:11:40
Thanks, that gives a hint where to look in the code!
msg6223 (view) Author: Janne Härkönen (yanne) Date: 2010-11-03.07:15:39
Sorry, I was actually lazy when creating the report.

This code (from compileall.py) is the culprit:

 56             if tail == '.py': 
 57                 cfile = fullname + (__debug__ and 'c' or 'o') 
 58                 ftime = os.stat(fullname).st_mtime 
 59                 try: ctime = os.stat(cfile).st_mtime 
 60                 except os.error: ctime = 0 
 61                 if (ctime > ftime) and not force: continue 

It seems to me that this is copied from Python stdlib directly and it checks for existence of .pyc/.pyo files when determining whether to compile or not.
msg8536 (view) Author: Jim Baker (zyasoft) Date: 2014-05-22.01:15:42
This issue still exists in compileall.

Target beta 4
msg8953 (view) Author: Jim Baker (zyasoft) Date: 2014-09-05.16:43:36
easy bug for interested devs
msg8955 (view) Author: Paolo Dina (pdina) Date: 2014-09-06.23:37:21
In attach a patch that should fix the issue.

1. it's not clear how (where) to correctly override the stdlib module compileall.py. I moved it from /path/to/jython/lib-python/2.7 to /path/to/jython/Lib and changed the file there. Is it correct?

2. This is the change
- cfile = fullname + (__debug__ and 'c' or 'o')
+ cfile = fullname.replace('.py', '$py.class')

When -O flag will be implemented the patch should also consider __debug__.

3. about tests, I should create a temporary directory structure, prepopulated, and delete it at the end of the test. Should I proceed this way?

Thanks
msg8956 (view) Author: Paolo Dina (pdina) Date: 2014-09-06.23:39:08
uploaded the right file
msg8957 (view) Author: Paolo Dina (pdina) Date: 2014-09-07.01:46:40
added test

See also: https://github.com/paolodina/jython/commit/a829df15fa95943a9b160ba87b8499f7be7bb9a6
msg9606 (view) Author: Jim Baker (zyasoft) Date: 2015-03-10.18:22:51
Fixed as of https://hg.python.org/jython/rev/6c38d2eda569
History
Date User Action Args
2015-03-17 13:51:13zyasoftsetstatus: pending -> closed
2015-03-10 18:22:51zyasoftsetstatus: open -> pending
resolution: accepted -> fixed
messages: + msg9606
2015-01-07 06:18:36zyasoftsetassignee: zyasoft
resolution: accepted
2014-09-07 01:46:41pdinasetfiles: + issue1672.patch
messages: + msg8957
2014-09-06 23:39:08pdinasetfiles: + issue1672.patch
keywords: + patch
messages: + msg8956
2014-09-06 23:38:06pdinasetfiles: - compileall.py
2014-09-06 23:37:22pdinasetfiles: + compileall.py
messages: + msg8955
2014-09-05 16:43:36zyasoftsetmessages: + msg8953
2014-08-23 22:20:07pdinasetnosy: + pdina
2014-05-22 01:15:43zyasoftsetassignee: fwierzbicki -> (no value)
messages: + msg8536
nosy: + zyasoft
versions: + Jython 2.7, - Jython 2.5
2013-02-25 18:16:16fwierzbickisetpriority: high
2013-02-20 19:31:12fwierzbickisetassignee: otmarhumbel -> fwierzbicki
nosy: + fwierzbicki
versions: + Jython 2.5, - 2.5.1
2010-11-03 07:15:41yannesetmessages: + msg6223
2010-11-03 06:11:41otmarhumbelsetmessages: + msg6222
2010-11-03 05:00:15yannesetmessages: + msg6221
2010-11-02 20:00:32otmarhumbelsetassignee: otmarhumbel
messages: + msg6219
nosy: + otmarhumbel
2010-11-01 13:36:26yannecreate