Issue1951

classification
Title: Bytecode Interpreter stack optimization for larger arguments
Type: behaviour Severity: normal
Components: Core Versions: 2.7a2
Milestone:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: pjenvey Nosy List: fwierzbicki, ndrzmansn, pjenvey
Priority: normal Keywords: patch

Created on 2012-07-27.06:09:25 by ndrzmansn, last changed 2012-08-03.20:03:03 by pjenvey.

Files
File name Uploaded Description Edit Remove
array_pystack.diff ndrzmansn, 2012-07-27.06:09:25 patch
benchmark_scripts.tgz ndrzmansn, 2012-07-27.22:06:46 benchmark scripts
Messages
msg7336 (view) Author: Wei Zhang (ndrzmansn) Date: 2012-07-27.06:09:25
This patch is an array version of PyStack in PyBytecode bytecode interpreter.
Although, by default jython runs compiler, bytecode interpreter is still used when unmarshal .pyc files.
Changing PyStack from an ArrayList to array makes it more efficient.
msg7337 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-07-27.17:43:56
Hi Wei Zhang - it would be nice to see a little microbenchmarking code to show that this is indeed speeds things up (and so others can have a try in their particular environments)
msg7339 (view) Author: Wei Zhang (ndrzmansn) Date: 2012-07-27.22:06:46
I used the attached files to test on my machines.

in jython root dir:
tar xvf benchmark_scripts.tgz
cd patch
taskset -c 1,3,5,7 ./jython.py -u -r

-r: run all benchmarks specified in ./benchmarks/.run;
    I chose a couple of ones from Language Shootout Game with various arguments;
-u: unmarshal; It uses ./benchmarks/unmarshal.py to unmarshal pregenerated .pyc files for 
    the same set of benchmarks included in ./benchmarks/.run

The machine I used:
Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz
Mem 64GB
Linux 3.2.0-25-generic Ubuntu

result:
------------------------------------------------------------------------------------------------
ArrayList (old)                                        | Array (new)
------------------------------------------------------------------------------------------------
Unmarshalling ... binarytrees.py 9                      Unmarshalling ... binarytrees.py 9         
2.845                                                   3.016    
Unmarshalling ... binarytrees.py 10                     Unmarshalling ... binarytrees.py 10    
2.618                                                   2.642    
Unmarshalling ... binarytrees.py 11                     Unmarshalling ... binarytrees.py 11    
2.945                                                   2.608    
Unmarshalling ... binarytrees.py 12                     Unmarshalling ... binarytrees.py 12    
4.333                                                   3.607    
Unmarshalling ... binarytrees.py 14                     Unmarshalling ... binarytrees.py 14    
10.724                                                  10.148    
Unmarshalling ... fannkuchredux.py 8                    Unmarshalling ... fannkuchredux.py 8    
2.744                                                   3.022    
Unmarshalling ... fannkuchredux.py 9                    Unmarshalling ... fannkuchredux.py 9    
10.646                                                  10.667    
Unmarshalling ... fasta.py 2000                         Unmarshalling ... fasta.py 2000    
2.952                                                   3.320    
Unmarshalling ... fasta.py 3000                         Unmarshalling ... fasta.py 3000    
3.234                                                   3.023    
Unmarshalling ... fasta.py 4000                         Unmarshalling ... fasta.py 4000    
2.917                                                   3.399    
Unmarshalling ... fasta.py 5000                         Unmarshalling ... fasta.py 5000    
3.246                                                   3.225    
Unmarshalling ... fasta.py 6000                         Unmarshalling ... fasta.py 6000    
2.977                                                   3.502    
Unmarshalling ... mandelbrot.py 200                     Unmarshalling ... mandelbrot.py 200    
3.262                                                   3.295    
Unmarshalling ... mandelbrot.py 300                     Unmarshalling ... mandelbrot.py 300    
4.355                                                   4.378    
Unmarshalling ... mandelbrot.py 400                     Unmarshalling ... mandelbrot.py 400    
5.466                                                   5.156    
Unmarshalling ... mandelbrot.py 500                     Unmarshalling ... mandelbrot.py 500    
7.586                                                   7.193    
Unmarshalling ... mandelbrot.py 600                     Unmarshalling ... mandelbrot.py 600    
9.992                                                   9.145    
Unmarshalling ... meteor.py 400                         Unmarshalling ... meteor.py 400    
6.135                                                   5.728    
Unmarshalling ... meteor.py 500                         Unmarshalling ... meteor.py 500    
7.030                                                   6.765    
Unmarshalling ... meteor.py 600                         Unmarshalling ... meteor.py 600    
7.755                                                   7.505    
Unmarshalling ... meteor.py 700                         Unmarshalling ... meteor.py 700    
8.485                                                   8.280    
Unmarshalling ... meteor.py 800                         Unmarshalling ... meteor.py 800    
8.744                                                   8.723    
Unmarshalling ... nbody.py 40000                        Unmarshalling ... nbody.py 40000    
4.635                                                   4.037    
Unmarshalling ... nbody.py 50000                        Unmarshalling ... nbody.py 50000    
5.056                                                   4.623    
Unmarshalling ... nbody.py 60000                        Unmarshalling ... nbody.py 60000    
6.479                                                   5.054    
Unmarshalling ... nbody.py 70000                        Unmarshalling ... nbody.py 70000    
6.534                                                   5.868    
Unmarshalling ... nbody.py 80000                        Unmarshalling ... nbody.py 80000    
7.400                                                   6.751    
Unmarshalling ... spectralnorm.py 100                   Unmarshalling ... spectralnorm.py 100    
2.552                                                   2.787    
Unmarshalling ... spectralnorm.py 200                   Unmarshalling ... spectralnorm.py 200    
5.180                                                   4.616    
Unmarshalling ... spectralnorm.py 300                   Unmarshalling ... spectralnorm.py 300    
7.843                                                   7.437    
Unmarshalling ... spectralnorm.py 400                   Unmarshalling ... spectralnorm.py 400    
11.694                                                  11.282    
Unmarshalling ... spectralnorm.py 500                   Unmarshalling ... spectralnorm.py 500    
17.548                                                  16.271    
------------------------------------------------------------------------------------------------

It shows that if the supplied arguments are not very small, the array version is noticeably faster.
In PyBytecode, line 46, the comments actually mentions that the field co_stacksize is used to convert PyStack from ArrayList to Array.
Thanks.
msg7342 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-07-31.17:12:02
Great - thanks for the update - I will try to have a look this week.
msg7349 (view) Author: Philip Jenvey (pjenvey) Date: 2012-08-03.20:03:02
applied on default, 7cebb8b5aea7. thanks

nice to meet you & your colleague @ the jvm lang summit =]
History
Date User Action Args
2012-08-03 20:03:03pjenveysetstatus: open -> closed
resolution: fixed
messages: + msg7349
2012-08-03 19:53:01pjenveysetassignee: fwierzbicki -> pjenvey
nosy: + pjenvey
title: PyStack using array -> Bytecode Interpreter stack optimization for larger arguments
2012-07-31 17:12:03fwierzbickisetmessages: + msg7342
2012-07-31 17:11:36fwierzbickisetpriority: normal
assignee: fwierzbicki
2012-07-27 22:06:46ndrzmansnsetfiles: + benchmark_scripts.tgz
messages: + msg7339
2012-07-27 17:43:56fwierzbickisetnosy: + fwierzbicki
messages: + msg7337
2012-07-27 06:09:26ndrzmansncreate