forked from PyTables/PyTables
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathwoody-pentiumIV.txt
More file actions
189 lines (163 loc) · 11.6 KB
/
woody-pentiumIV.txt
File metadata and controls
189 lines (163 loc) · 11.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
This is for Debian woody!
Below are some benchmarking figures obtained while reading and writing
to a file with three tables, each table containing 10000 records. For
reference, the same tests have been repeated using the shelve module
that comes with Python. The tests were conducted on a platform with a
2 GHz AMD Athlon chip, an IDE disk at 4600 rpm, and 256 MB of RAM.
Version 0.2
| medium size records | small size records
| (47 Bytes) | (16 Bytes)
+---------------------------+------------------------------
| rows/s filesize | rows/s filesize
| write read | write read
------------+---------------------------+------------------------------
no compress| |
record | 24400 39000 1184 KB | 32600 52600 506 KB
tupla | 17100 81100 1184 KB | 66666 107142 506 KB
------------+---------------------------+------------------------------
compress | |
record | 22200 37500 494 KB | 31900 51700 94 KB
tupla | 16100 75000 494 KB | 63900 107142 94 KB
------------+---------------------------+------------------------------
Shelve | 25800 14400 2500 KB | 68200 17000 921 KB
New version (15-Jan-2003)
PyTables pre-0.3
Rec length | rows/s | KB/s | rows | filesz | memory |
| write read | write read | | (MB) | (MB) |
------------+-----------------+-----------------+-------+--------+--------+
16 B | 31000 166600 | 480 2600 | 3.e4 | 0.49| 6.5 |
------------+-----------------+-----------------+-------+--------+--------+
56 B | 17300 136000 | 942 7460 | 3.e4 | 1.7 | 7.2 |
------------+-----------------+-----------------+-------+--------+--------+
56 B* | 1560 136000 | 85 7560 | 3.e4 | 1.7 | 7.2 |
------------+-----------------+-----------------+-------+--------+--------+
64 B* | 1540 130000 | 96 8152 | 3.e4 | 1.9 | 7.2 |
------------+-----------------+-----------------+-------+--------+--------+
550 B* | 879 81100 | 472 43500 | 3.e4 | 19 | 7.2 |
------------+-----------------+-----------------+-------+--------+--------+
550 B** | 12000 103000 | 6440 55400 | 3.e5 | 168 | 7.2 |
------------+-----------------+-----------------+-------+--------+--------+
550 B** | 15500 81100 | 8350 43500 | 3.e4 | 19 | 7.2 |
------------+-----------------+-----------------+-------+--------+--------+
550 B**c| 909 1100 | 490 1081 | 3.e4 | 0.76| 8.5 |
------------+-----------------+-----------------+-------+--------+--------+
550 B***| 3600 81100 | 1950 43500 | 3.e4 | 19 | 7.2 |
------------+-----------------+-----------------+-------+--------+--------+
* These are figures obtained with a numarray as part of the record
** The numarray record fields are not set in each iteration
*** Some numarray elements of a record field are changed on each iteration
**c Like ** but with compression (level 1)
New version (10-March-2003)
PyTables pre-0.4
Rec | rows/s | KB/s | rows | filesz | memory |%CP|%CP
length | write read | write read | | (MB) | (MB) |(w)|(r)
--------+-----------------+-----------------+-------+--------+--------+---+----
16 B |434000 469000 | 6800 7300 | 3.e4 | 0.49| 6.5 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
16 Bc |326000 435000 | 5100 6800 | 3.e4 | 0.12| 6.5 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
16 B |663000 728000 | 10400 11400 | 3.e5 | 4.7 | 7.0 | 99|100
--------+-----------------+-----------------+-------+--------+--------+---+----
16 B |679000 797000 | 10600 12500 | 3.e6 | 46.0 | 10.0 | 98| 98
--------+-----------------+-----------------+-------+--------+--------+---+----
16 Bc |452000 663000 | 7100 10400 | 3.e6 | 9.3 | 10.0 | 98| 98
--------+-----------------+-----------------+-------+--------+--------+---+----
16 B |576000 590000 | 9000 9200 | 3.e7 | 458.0 | 11.0 | 78| 76
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B | 3050 380000 | 163 20700 | 3.e4 | 1.7 | 7.2 | 98|100
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B* |194000 340000 | 10600 18600 | 3.e4 | 1.7 | 7.2 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B*c |142000 306000 | 7800 16600 | 3.e4 | 0.3 | 7.2 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B* |273600 589000 | 14800 32214 | 3.e5 | 16.0 | 9.0 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B*c |184000 425000 | 10070 23362 | 3.e5 | 2.7 | 9.7 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B* |203600 649000 | 11100 35500 | 3.e6 | 161.0 | 12.0 | 72| 99
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B* |184000 229000 | 10000 12500 | 1.e7 | 534.0 | 17.0 | 56| 40
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B*np|184000 229000 | 10000 12500 | 1.e7 | 534.0 | 17.0 | 56| 40
--------+-----------------+-----------------+-------+--------+--------+---+----
550 B | 2230 143000 | 1195 76600 | 3.e4 | 19 | 9.4 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
550 B* | 76000 250000 | 40900 134000 | 3.e4 | 19 | 9.4 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
550 B*c | 13900 30000 | 7400 16100 | 3.e4 | 0.7 | 10.0 | 99| 99
--------+-----------------+-----------------+-------+--------+--------+---+----
550 B* | 25400 325000 | 13600 174000 | 3.e5 | 167 | 11.0 | 71| 96
--------+-----------------+-----------------+-------+--------+--------+---+----
550 B* | 18700 28000 | 10000 15100 | 6.e5 | 322 | 13.0 | 76| 9
--------+-----------------+-----------------+-------+--------+--------+---+----
550 B*c | 7300 21000 | 3900 11300 | 6.e5 | 11 | 17.0 | 98| 99
--------+-----------------+-----------------+-------+--------+--------+---+----
* These are figures obtained with a numarray as part of the record
** The numarray record fields are not set in each iteration
c With compression (level 1)
np No psyco optimizations
Shelve
Rec length | rows/s | KB/s | rows | filesz | memory |
| write read | write read | | (MB) | (MB) |
------------+-----------------+-----------------+-------+--------+--------+
16 B | 68200 17000 | 1070 266 | 3.e4 | 0.94| 7.2 |
------------+-----------------+-----------------+-------+--------+--------+
56 B | 25000 14400 | 1367 784 | 3.e4 | 2.5 | 10.6 |
------------+-----------------+-----------------+-------+--------+--------+
56 B* | 2980 2710 | 162 148 | 3.e4 | 7.3 | 33 |
------------+-----------------+-----------------+-------+--------+--------+
64 B* | 2900 2700 | 182 168 | 3.e4 | 7.5 | 33 |
------------+-----------------+-----------------+-------+--------+--------+
550 B* | 1090 1310 | 590 710 | 3.e4 | 58 | 122 |
------------+-----------------+-----------------+-------+--------+--------+
550 B** | 16000 14900 | 2400 1200 | 3.e4 | 2.4 | 10.6 |
------------+-----------------+-----------------+-------+--------+--------+
550 B***| 28000 11900 | 2400 1100 | 3.e4 | 2.5 | 10.6 |
------------+-----------------+-----------------+-------+--------+--------+
* These are figures obtained with a numarray as part of the record
** The nuamrray records are not set on each iteration
*** Some numarray elements of a record field are changed on each iteration
Python cPickle & bsddb3 RECNO with variable length
Rec | Krows/s | MB/s | Krows | filesz | memory |%CP|%CP
length | write read | write read | | (MB) | (MB) |(w)|(r)
--------+-----------------+-----------------+-------+--------+--------+---+----
16 B | 23.0 4.3 | 0.65 0.12 | 30 | 2.3 | 6.0 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
16 B | 22.0 4.3 | 0.60 0.12 | 300 | 24 | 25.0 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B | 12.3 2.0 | 0.68 0.11 | 30 | 5.8 | 6.2 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B | 8.8 2.0 | 0.44 0.11 | 300 | 61 | 6.2 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
Python struct & bsddb3 RECNO with fixed length
Rec | Krows/s | MB/s | Krows | filesz | memory |%CP|%CP
length | write read | write read | | (MB) | (MB) |(w)|(r)
--------+-----------------+-----------------+-------+--------+--------+---+----
16 B | 61 71 | 1.6 1.9 | 30 | 1.0 | 5.0 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
16 B | 56 65 | 1.5 1.8 | 300 | 10 | 5.8 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
16 B | 51 61 | 1.4 1.6 | 3000 | 100 | 6.1 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B | 51 52 | 2.7 2.8 | 30 | 1.8 | 5.8 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B | 18 50 | 1.0 2.7 | 300 | 18 | 6.2 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B | 16 48 | 0.9 2.6 | 1000 | 61 | 6.5 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
PySqlLite
Rec | rows/s | KB/s | rows | filesz | memory |%CP|%CP
length | write read | write read | | (MB) | (MB) |(w)|(r)
--------+-----------------+-----------------+-------+--------+--------+---+----
16 B | 4290 1400000 | 200 48000 | 3.e4 | 1.4 | 5.0 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
16 B | 3660 1030000 | 182 51000 | 3.e5 | 15 | 5.0 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
16 B | 3580 230000 | 192 12380 | 6.e6 | 322 | 5.0 |100| 25
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B | 2990 882000 | 250 76000 | 3.e4 | 2.6 | 5.0 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B | 2900 857000 | 270 80000 | 3.e5 | 28 | 5.0 |100|100
--------+-----------------+-----------------+-------+--------+--------+---+----
56 B | 2900 120000 | 302 13100 | 3.e6 | 314 | 5.0 |100| 11
--------+-----------------+-----------------+-------+--------+--------+---+----