[ZODB-Dev] Python StressSuite leaktester
Christian Robottom Reis
kiko@async.com.br
Thu, 11 Oct 2001 02:59:35 -0300 (BRT)
This message is in MIME format. The first part should be readable text,
while the remaining parts are likely unreadable without MIME-aware tools.
Send mail to mime@docserver.cac.washington.edu for more info.
---1463810461-1619782003-1002779975=:250
Content-Type: TEXT/PLAIN; charset=US-ASCII
Hi there,
I've implemented a Stress testing suite for ORBit-Python that I'd like to
share with the ZODB people; perhaps somebody might want to see what it's
like and use it to find leaks. Be warned: this isn't packaged code that is
ready to run - it's a hack that will need hacking to fit into anybody's
test framework. If you want to use it, email me for more instructions on
what can be done with it; it's a bit convoluted. However, it does find
leaks in software.
That's what I wrote it for: leak finding. So far we've killed all the
leaks that are known in O-P and we're aiming for more coverage to be sure
we're safe. The basic functionality is that of a "normal" unittest
TestSuite: it runs the exact same testcases as defined. However, it runs
them as many times as you like, iterating over and over again, and prints
out a progress report that will leave you mesmerized over the millions if
bits that are going into limbo. You can use the same tests as the normal
testsuite uses, of course, so you can have a simple test script and one
that does stress testing, just by switching the suite used. This is cool
for tinderboxen and whatnot if you have that sort of thing setup, I guess,
though I haven't hacked that in (yet).
This is for Python2.1 unittest only, and it only works with the Linux
/proc format in case somebody wonders. It's hackable. To see it work, just
try running it:
blackjesus% python StressSuite.py
Test name Progress Time
A0_nop [_______________________________] (0.12s)
A1_alloc [_________........oooooooOOOOOOO] (0.28s)
*** Leak in process: 1940.00 Kbytes, 1986 bytes per invocation
A2_alloc_big [_______........oooooooOOOOOOOOO] (0.40s)
*** Leak in process: 8100.00 Kbytes, 8294 bytes per invocation
----------------------------------------------------------------------
Ran 3000 tests in 0.819s
An example run on orbit-python follows(see A2 and A3 start to leak a bit,
and T16 - T18 significantly towards the end of their runs, F.I.):
blackjesus% ./stresstest.sh
Test name Progress Time
A1_trans_accessor_00_set [____________________] (2.28s)
A2_trans_accessor_01_get [___________________.] (4.04s)
A3_trans_accessor_02_call [_____________.......] (1.14s)
A4_bad_accessor_set [____________________] (0.34s)
A5_bad_accessor_get [____________________] (0.34s)
Test name Progress Time
T0_send_string [____________________] (0.59s)
T1_send_long [____________________] (0.58s)
T2_send_float [____________________] (0.60s)
T3_send_octet_int [____________________] (0.57s)
T4_send_octet_char [____________________] (0.60s)
T5_send_octet_string [____________________] (0.44s)
T6_send_object [____________________] (5.02s)
T7_send_noobject [____________________] (0.60s)
T8_send_dummy [____________________] (5.01s)
T9_send_string_seq [____________________] (0.66s)
T10_send_long [____________________] (0.63s)
T11_send_float [____________________] (0.68s)
T12_send_octet_int_seq [____________________] (0.69s)
T13_send_octet_char_seq [____________________] (0.62s)
T14_send_octet_string_seq [____________________] (0.60s)
T15_send_octet_mixed_seq [____________________] (0.61s)
T16_send_object_seq [__...ooOOOOOOOOOOOOO] (40.61s)
*** Leak in server: 200.00 Kbytes, 204 bytes per invocation
*** Leak in client: 180.00 KBytes, 184 bytes per invocation
T17_send_noobject_seq [_______.....oooooOOO] (0.81s)
*** Leak in server: 160.00 Kbytes, 163 bytes per invocation
T18_send_dummy_seq [___..oooOOOOOOOOOOOO] (40.70s)
*** Leak in server: 196.00 Kbytes, 200 bytes per invocation
*** Leak in client: 176.00 KBytes, 180 bytes per invocation
Test name Progress Time
R0_string_return [____________________] (0.61s)
R1_long_return [____________________] (0.61s)
R2_float_return [____________________] (0.62s)
R3_null_return [____________________] (0.59s)
R4_dummy_return [____________________] (4.09s)
R5_object_return [____________________] (4.10s)
R6_noobject_return [____________________] (0.60s)
R7_octet_int_return [____________________] (0.59s)
R8_octet_char_return [____________________] (0.60s)
R9_string_seq_return [____________________] (0.65s)
R10_long_seq_return [____________________] (0.63s)
R11_float_seq_return [____________________] (0.72s)
R12_dummy_seq_return [____.....oooooOO
Well, there you see we have a couple of functions leaking (test R12 is
going to be a big leaker too there). That's actually me sprinking
malloc(8); around the codebase to exemplify and test the suite. The
progress bar updates in realtime and indicates the percentage of runs that
I could verify a leak. If it's > 75% of the runs, we print out a big O to
show the baloon the process turned into.
This is really alpha quality code, but it Works for me (tm). If anybody
would like to use it, it's LGPLed (*sigh*) and currently a part of O-P. If
the license is no good for you, email me about it.
If the ZODB doesn't have something like this, it sure could.
Take care,
--
Christian Reis, Senior Engineer, Async Open Source, Brazil.
http://async.com.br/~kiko/ | [+55 16] 272 3330 | NMFL
---1463810461-1619782003-1002779975=:250
Content-Type: TEXT/PLAIN; charset=US-ASCII; name="StressSuite.py"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.LNX.4.40.0110110259350.250@blackjesus.async.com.br>
Content-Description:
Content-Disposition: attachment; filename="StressSuite.py"
IyEvdXNyL2Jpbi9lbnYgcHl0aG9uDQoNCiMgU3RyZXNzU3VpdGUsIGEgdW5p
dHRlc3QgdGVzdFN1aXRlIGRlc2lnbmVkIGZvciBzdHJlc3Npbmcgc29mdHdh
cmUgYW5kDQojIGNhdGNoaW5nIG1lbW9yeSBsZWFrcy4NCiMNCiMgQ29weXJp
Z2h0IChDKSAyMDAxICBDaHJpc3RpYW4gUmVpcyA8a2lrb0Bhc3luYy5jb20u
YnI+DQojDQojIFRoaXMgbGlicmFyeSBpcyBmcmVlIHNvZnR3YXJlOyB5b3Ug
Y2FuIHJlZGlzdHJpYnV0ZSBpdCBhbmQvb3INCiMgbW9kaWZ5IGl0IHVuZGVy
IHRoZSB0ZXJtcyBvZiB0aGUgR05VIExpYnJhcnkgR2VuZXJhbCBQdWJsaWMN
CiMgTGljZW5zZSBhcyBwdWJsaXNoZWQgYnkgdGhlIEZyZWUgU29mdHdhcmUg
Rm91bmRhdGlvbjsgZWl0aGVyDQojIHZlcnNpb24gMiBvZiB0aGUgTGljZW5z
ZSwgb3IgKGF0IHlvdXIgb3B0aW9uKSBhbnkgbGF0ZXIgdmVyc2lvbi4NCiMg
DQojIFRoaXMgbGlicmFyeSBpcyBkaXN0cmlidXRlZCBpbiB0aGUgaG9wZSB0
aGF0IGl0IHdpbGwgYmUgdXNlZnVsLA0KIyBidXQgV0lUSE9VVCBBTlkgV0FS
UkFOVFk7IHdpdGhvdXQgZXZlbiB0aGUgaW1wbGllZCB3YXJyYW50eSBvZg0K
IyBNRVJDSEFOVEFCSUxJVFkgb3IgRklUTkVTUyBGT1IgQSBQQVJUSUNVTEFS
IFBVUlBPU0UuICBTZWUgdGhlIEdOVQ0KIyBMaWJyYXJ5IEdlbmVyYWwgUHVi
bGljIExpY2Vuc2UgZm9yIG1vcmUgZGV0YWlscy4NCiMgDQojIFlvdSBzaG91
bGQgaGF2ZSByZWNlaXZlZCBhIGNvcHkgb2YgdGhlIEdOVSBMaWJyYXJ5IEdl
bmVyYWwgUHVibGljDQojIExpY2Vuc2UgYWxvbmcgd2l0aCB0aGlzIGxpYnJh
cnk7IGlmIG5vdCwgd3JpdGUgdG8gdGhlIEZyZWUNCiMgRm91bmRhdGlvbiwg
SW5jLiwgNTkgVGVtcGxlIFBsYWNlLCBTdWl0ZSAzMzAsIEJvc3RvbiwgTUEg
IDAyMTExLTEzMDcgVVNBDQoNCmltcG9ydCB1bml0dGVzdCwgb3MsIHN0cmlu
Zywgc3lzLCB0aW1lDQoNCiMgTmVlZHMgdG8gaGF2ZSBjZXJ0YWluIG51bWJl
ciBvZiB2YXJpYWJsZXMgc2V0IGV4dGVybmFsbHkuIE9rYXksIHNvDQojIHRo
aXMgaXMgdWdseS4gU3VlIG1lLiA6KQ0KDQpSVU5TID0gMTAwMA0KTlVNX1NB
TVBMRVMgPSAzMA0KDQpURVNUUyA9IFtdDQpERUJVRyA9IDANCg0KZGVmIHVu
aXEoc2VxKToNCiAgICBkaWN0ID0ge30NCiAgICBmb3IgZSBpbiBzZXE6IGRp
Y3RbZV0gPSAxDQogICAgcmV0dXJuIGRpY3Qua2V5cygpDQoNCmNsYXNzIFNp
bmdsZVN0cmVzc1N0YXRzOg0KICAgIGRlZiBfX2luaXRfXyhzZWxmKToNCiAg
ICAgICAgc2VsZi5teXBpZCA9IHN0cihvcy5nZXRwaWQoKSkNCiAgICAgICAg
c2VsZi5zaXplID0gc2VsZi5nZXRfc2l6ZSgpDQogICAgICAgIHNlbGYuc3Rh
cnRfc2l6ZSA9IHNlbGYuZ2V0X3NpemUoKQ0KICAgICAgICBzZWxmLmxlYWsg
PSAwDQogICAgICAgIHNlbGYuc3RhcnRUaW1lID0gdGltZS50aW1lKCkNCg0K
ICAgIGRlZiBnZXRfc2l6ZShzZWxmKToNCiAgICAgICAgcmV0dXJuIHNlbGYu
Z2V0X3Ztc2l6ZShzZWxmLm15cGlkKQ0KDQogICAgZGVmIGdldF92bXNpemUo
c2VsZiwgcGlkKToNCiAgICAgICAgc2l6ZWggPSBvcGVuKCcvcHJvYy8nK3Bp
ZCsnL3N0YXR1cycpDQogICAgICAgIGxpbmVzID0gc2l6ZWgucmVhZGxpbmVz
KCkNCiAgICAgICAgc2l6ZWguY2xvc2UoKQ0KICAgICAgICBmb3IgbCBpbiBs
aW5lczoNCiAgICAgICAgICAgIGlmIGxbMDo2XSA9PSAiVm1TaXplIjoNCiAg
ICAgICAgICAgICAgICByZXR1cm4gaW50KHN0cmluZy5zcGxpdChsKVsxXSkN
CiAgICAgICAgcmFpc2UgIi9wcm9jLyMvc3RhdHVzIGJyb2tlbiAtIG5vdCBm
b3VuZCINCg0KICAgIGRlZiBfX3N0cl9fKHNlbGYpOg0KICAgICAgICBzZWxm
LnVwZGF0ZSgpDQogICAgICAgIGN1cnIgPSBzZWxmLmdldF9zaXplKCkNCiAg
ICAgICAgZGlmZiA9IGN1cnIgLSBzZWxmLnN0YXJ0X3NpemUNCiAgICAgICAg
dGRpZmYgPSB0aW1lLnRpbWUoKSAtIHNlbGYuc3RhcnRUaW1lDQogICAgICAg
IHN0YXRzID0gIiglMDIuMmZzKSIgJSAoIHRkaWZmICkgDQogICAgICAgIGlm
ICggc2VsZi5sZWFrID4gTlVNX1NBTVBMRVMgLyAyICApOg0KICAgICAgICAg
ICAgc3RhdHMgPSBzdGF0cyArIFwNCiAgICAgICAgICAgICJcbiAgICAqKiog
TGVhayBpbiBwcm9jZXNzOiAlLjJmIEtieXRlcywgJWQgYnl0ZXMgcGVyIGlu
dm9jYXRpb24iICUgXA0KICAgICAgICAgICAgICAgICAgICAoIGRpZmYsIChk
aWZmKjEwMjQpLyAoUlVOUykgKQ0KICAgICAgICByZXR1cm4gc3RhdHMNCg0K
ICAgIGRlZiB1cGRhdGUoc2VsZik6DQogICAgICAgIGN1cnIgPSBzZWxmLmdl
dF92bXNpemUoc2VsZi5teXBpZCkNCiAgICAgICAgIyBkbyB4LXkgZm9yIGN1
cnIgYW5kIHNlbGYuc2l6ZXMgaW4gcGFyYWxsZWwNCiAgICAgICAgZGlmZiA9
IGN1cnIgLSBzZWxmLnNpemUNCiAgICAgICAgc2VsZi5zaXplcyA9IGN1cnIN
Cg0KICAgICAgICAjIEF0dGVudGlvbjogbGVhayBpcyBhIGNvdW50ZXIgYmVj
YXVzZSB3ZSB3YW50IHRvIGNhdGNoDQogICAgICAgICMgcmVwcm9kdWNpYmxl
IGxlYWtzLCBhbmQgbm90IGp1c3QgYSBzaW5nbGUgbGVhayAod2hpY2ggY291
bGQgYmUNCiAgICAgICAgIyBjYXVzZWQgYnkgcHl0aG9uIHN0YXJ0dXAgb3Ig
b25lLXRpbWUgZ3Jvd3RoDQogICAgICAgIGlmIGRpZmY6DQogICAgICAgICAg
ICBzZWxmLmxlYWsgPSBzZWxmLmxlYWsrMQ0KICAgICAgICByZXR1cm4gc2Vs
Zi5sZWFrDQoNCiMgQSBTdHJlc3NTdGF0cyBjbGFzcyB0aGF0IHJlYWRzIGlu
Zm9ybWF0aW9uIGZyb20gL3Byb2MgZm9yIHR3byBwcm9jZXNzZXMsIGENCiMg
Y2xpZW50IGFuZCBhIHNlcnZlci4NCg0KY2xhc3MgQ2xpZW50U2VydmVyU3Ry
ZXNzU3RhdHM6DQoNCiAgICBkZWYgX19pbml0X18oc2VsZik6DQogICAgICAg
IHNlbGYubXlwaWQgPSBzdHIob3MuZ2V0cGlkKCkpDQogICAgICAgICMgZ2V0
IHNlcnZlciBwaWQgZnJvbSAicGlkIiBmaWxlLg0KICAgICAgICBzZWxmLnNl
cnZlcnBpZCA9IG9wZW4oJ3BpZCcpLnJlYWRsaW5lKCkNCiAgICAgICAgc2Vs
Zi5zaXplcyA9IHNlbGYuZ2V0X3NpemUoKQ0KICAgICAgICBzZWxmLnN0YXJ0
X3NpemVzID0gc2VsZi5nZXRfc2l6ZSgpDQogICAgICAgICMgc3RvcmUgbGVh
a3MgaW4gY2xpZW50IGFuZCBzZXJ2ZXINCiAgICAgICAgc2VsZi5sZWFrID0g
WyAwLCAwIF0NCiAgICAgICAgc2VsZi5zdGFydFRpbWUgPSB0aW1lLnRpbWUo
KQ0KDQogICAgZGVmIGdldF9zaXplKHNlbGYpOg0KICAgICAgICByZXR1cm4g
WyBzZWxmLmdldF92bXNpemUoc2VsZi5zZXJ2ZXJwaWQpLA0KICAgICAgICAg
ICAgc2VsZi5nZXRfdm1zaXplKHNlbGYubXlwaWQpIF0NCg0KICAgICMgUmVh
Y2ggZm9yIHRoZSBjcmFjayBwaXBlIQ0KICAgIGRlZiBfX3N0cl9fKHNlbGYp
Og0KICAgICAgICBzZWxmLnVwZGF0ZSgpDQogICAgICAgIGN1cnIgPSBzZWxm
LmdldF9zaXplKCkNCiAgICAgICAgZGlmZiA9IG1hcChsYW1iZGEgeCx5OiB4
IC0geSwgY3Vyciwgc2VsZi5zdGFydF9zaXplcykNCiAgICAgICAgdGRpZmYg
PSB0aW1lLnRpbWUoKSAtIHNlbGYuc3RhcnRUaW1lDQogICAgICAgIHN0YXRz
ID0gIiglMDIuMmZzKSIgJSAoIHRkaWZmICkgDQogICAgICAgIGlmICggc2Vs
Zi5sZWFrWzBdID4gTlVNX1NBTVBMRVMgLyAyICApOg0KICAgICAgICAgICAg
c3RhdHMgPSBzdGF0cyArIFwNCiAgICAgICAgICAgICJcbiAgICAqKiogTGVh
ayBpbiBzZXJ2ZXI6ICUuMmYgS2J5dGVzLCAlZCBieXRlcyBwZXIgaW52b2Nh
dGlvbiIgJSBcDQogICAgICAgICAgICAgICAgICAgICggZGlmZlswXSwgKGRp
ZmZbMF0qMTAyNCkvIChSVU5TKSApDQogICAgICAgIGlmICggc2VsZi5sZWFr
WzFdID4gTlVNX1NBTVBMRVMgLyAyICApOg0KICAgICAgICAgICAgc3RhdHMg
PSBzdGF0cyArIFwNCiAgICAgICAgICAgICJcbiAgICAqKiogTGVhayBpbiBj
bGllbnQ6ICUuMmYgS0J5dGVzLCAlZCBieXRlcyBwZXIgaW52b2NhdGlvbiIg
JSBcDQogICAgICAgICAgICAgICAgICAgICggZGlmZlsxXSwgKGRpZmZbMV0q
MTAyNCkvIChSVU5TKSApDQogICAgICAgIHJldHVybiBzdGF0cw0KDQogICAg
ZGVmIGdldF92bXNpemUoc2VsZiwgcGlkKToNCiAgICAgICAgc2l6ZWggPSBv
cGVuKCcvcHJvYy8nK3BpZCsnL3N0YXR1cycpDQogICAgICAgIGxpbmVzID0g
c2l6ZWgucmVhZGxpbmVzKCkNCiAgICAgICAgc2l6ZWguY2xvc2UoKQ0KICAg
ICAgICBmb3IgbCBpbiBsaW5lczoNCiAgICAgICAgICAgIGlmIGxbMDo2XSA9
PSAiVm1TaXplIjoNCiAgICAgICAgICAgICAgICByZXR1cm4gaW50KHN0cmlu
Zy5zcGxpdChsKVsxXSkNCiAgICAgICAgcmFpc2UgIi9wcm9jLyMvc3RhdHVz
IGJyb2tlbiAtIG5vdCBmb3VuZCINCg0KICAgIGRlZiB1cGRhdGUoc2VsZik6
DQogICAgICAgIGN1cnIgPSBbIHNlbGYuZ2V0X3Ztc2l6ZShzZWxmLnNlcnZl
cnBpZCksDQogICAgICAgICAgICBzZWxmLmdldF92bXNpemUoc2VsZi5teXBp
ZCkgXQ0KICAgICAgICAjIGRvIHgteSBmb3IgY3VyciBhbmQgc2VsZi5zaXpl
cyBpbiBwYXJhbGxlbA0KICAgICAgICBkaWZmID0gbWFwKGxhbWJkYSB4LHk6
IHggLSB5LCBjdXJyLCBzZWxmLnNpemVzKQ0KICAgICAgICBzZWxmLnNpemVz
ID0gY3Vycg0KDQogICAgICAgICMgQXR0ZW50aW9uOiBsZWFrIGlzIGEgY291
bnRlciBiZWNhdXNlIHdlIHdhbnQgdG8gY2F0Y2gNCiAgICAgICAgIyByZXBy
b2R1Y2libGUgbGVha3MsIGFuZCBub3QganVzdCBhIHNpbmdsZSBsZWFrICh3
aGljaCBjb3VsZCBiZQ0KICAgICAgICAjIGNhdXNlZCBieSBweXRob24gc3Rh
cnR1cCBvciBvbmUtdGltZSBncm93dGgNCiAgICAgICAgaWYgZGlmZlswXToN
CiAgICAgICAgICAgIHNlbGYubGVha1swXSA9IHNlbGYubGVha1swXSsxDQog
ICAgICAgIGlmIGRpZmZbMV06DQogICAgICAgICAgICBzZWxmLmxlYWtbMV0g
PSBzZWxmLmxlYWtbMV0rMQ0KDQogICAgICAgIHJldHVybiBzZWxmLmxlYWtb
MF0gKyBzZWxmLmxlYWtbMV0NCg0KIw0KIyBPdmVycmlkZSB0aGlzIGlmIHlv
dSBkb24ndCB3YW50IHRoZSBkZWZhdWx0IEMvUyB0ZXN0Lg0KIw0KDQpTVEFU
UyA9IENsaWVudFNlcnZlclN0cmVzc1N0YXRzDQoNCiMNCiMgVGhlIHN1aXRl
IGl0c2VsZi4NCiMNCg0KY2xhc3MgU3RyZXNzU3VpdGUodW5pdHRlc3QuVGVz
dFN1aXRlKToNCiAgICBkZWYgX19pbml0X18oc2VsZiwgdGVzdHM9KCkgKToN
CiAgICAgICAgdW5pdHRlc3QuVGVzdFN1aXRlLl9faW5pdF9fKHNlbGYsIHRl
c3RzKQ0KDQogICAgIyBUaGlzIGlzIHRvIGdldCB0aGUgdGVzdHMgcnVubmlu
ZyBpbiB0aGUgcmlnaHQgb3JkZXINCiAgICAjIE5hbWVzIGFyZSBvZiB0aGUg
dHlwZSBYIyMgd2hlcmUgWCBpcyBhbnkgbGV0dGVyLg0KICAgIGRlZiBfY21w
X25hbWVzKHNlbGYseCx5KToNCiAgICAgICAgbnggPSBzdHJpbmcuc3BsaXQo
c3RyKHgpLCJfIilbMV0NCiAgICAgICAgbnkgPSBzdHJpbmcuc3BsaXQoc3Ry
KHkpLCJfIilbMV0NCg0KICAgICAgICAjIERvbid0IHRyeSBhbmQgcmVvcmRl
ciB0aGluZ3Mgd2l0aCBicm9rZW4gbmFtZXMNCiAgICAgICAgdHJ5Og0KICAg
ICAgICAgICAgaW50KG54WzE6XSkNCiAgICAgICAgZXhjZXB0IFZhbHVlRXJy
b3I6DQogICAgICAgICAgICByZXR1cm4gMA0KICAgICAgICAgICAgDQogICAg
ICAgIGlmIGludChueFsxOl0pID4gaW50KG55WzE6XSk6DQogICAgICAgICAg
ICByZXR1cm4gMQ0KICAgICAgICBlbHNlOiANCiAgICAgICAgICAgIHJldHVy
biAtMQ0KDQogICAgZGVmIGVjaG8oc2VsZixzdHIpOg0KICAgICAgICBzeXMu
c3Rkb3V0LndyaXRlKHN0cikNCiAgICAgICAgc3lzLnN0ZG91dC5mbHVzaCgp
DQoNCiAgICBkZWYgX19jYWxsX18oc2VsZiwgcmVzdWx0KToNCiAgICANCiAg
ICAgICAgU0FNUExFX0lOVEVSVkFMID0gUlVOUyAvIE5VTV9TQU1QTEVTDQoN
CiAgICAgICAgc2VsZi5lY2hvKCJcbiIpDQogICAgICAgIG4gPSBzdHIoIlRl
c3QgbmFtZSIpDQogICAgICAgIHNlbGYuZWNobyhuKQ0KICAgICAgICBmb3Ig
aSBpbiByYW5nZSgwLDMwIC0gbGVuKG4pKToNCiAgICAgICAgICAgIHNlbGYu
ZWNobygiICIpDQogICAgICAgIG4gPSAiIFByb2dyZXNzIg0KICAgICAgICBz
ZWxmLmVjaG8obikNCiAgICAgICAgZm9yIGkgaW4gcmFuZ2UoMCxOVU1fU0FN
UExFUyArIDQgLSBsZW4obikpOg0KICAgICAgICAgICAgc2VsZi5lY2hvKCIg
IikNCiAgICAgICAgc2VsZi5lY2hvKCIgVGltZSIpDQogICAgICAgIHNlbGYu
ZWNobygiXG5cbiIpDQoNCiAgICAgICAgIyBnZXQgdGhvc2UgdGVzdHMgaW4g
dGhlIHJpZ2h0IG9yZGVyDQogICAgICAgIHNlbGYuX3Rlc3RzLnNvcnQoc2Vs
Zi5fY21wX25hbWVzKQ0KICAgICAgICBmb3IgdGVzdCBpbiBzZWxmLl90ZXN0
czoNCg0KICAgICAgICAgICAgIyBmaWx0ZXIgb3VyIG5vc3RyZXNzIHRlc3Rz
DQogICAgICAgICAgICBpZiBzdHIodGVzdCkuZmluZCgibm9zdHJlc3MiKSAh
PSAtMToNCiAgICAgICAgICAgICAgICBjb250aW51ZQ0KDQogICAgICAgICAg
ICAjIGlmIFRFU1RTIGV4aXN0cywgb25seSBydW4gdGVzdHMgbGlzdGVkDQog
ICAgICAgICAgICBpZiBURVNUUzoNCiAgICAgICAgICAgICAgICBpZiBzdHJp
bmcuc3BsaXQoc3RyKHRlc3QpLCJfIilbMV0gbm90IGluIFRFU1RTOg0KICAg
ICAgICAgICAgICAgICAgICBjb250aW51ZQ0KDQogICAgICAgICAgICBzdGF0
cyA9IFNUQVRTKCkNCiAgICAgICAgICAgICMgTm8gaWRlYSB3aGF0IHRoaXMg
aXMgLSBzdG9sZW4gZnJvbSBUZXN0U3VpdGUNCiAgICAgICAgICAgIGlmIHJl
c3VsdC5zaG91bGRTdG9wOg0KICAgICAgICAgICAgICAgIGJyZWFrDQogICAg
ICAgICAgICAjIEN1dCB1cCBuYW1lIGludG8gc29tZXRoaW5nIG5pY2UNCiAg
ICAgICAgICAgIGZ1bGxuYW1lID0gc3RyaW5nLnNwbGl0KHN0cih0ZXN0KSwi
ICIpWzBdDQogICAgICAgICAgICB0ZXN0bmFtZSA9IHN0cmluZy5zcGxpdChm
dWxsbmFtZSwiXyIsMSlbMV0NCiAgICAgICAgICAgICMgdHJ1bmNhdGUgbmFt
ZSBhbmQgZml4IG9wZW4gcHJvZ3Jlc3MgYnJhY2tldC4NCiAgICAgICAgICAg
IHNlbGYuZWNobyh0ZXN0bmFtZVs6MjhdKQ0KICAgICAgICAgICAgZm9yIGkg
aW4gcmFuZ2UoMCwzMCAtIGxlbih0ZXN0bmFtZVs6MjhdKSk6DQogICAgICAg
ICAgICAgICAgc2VsZi5lY2hvKCIgIikNCiAgICAgICAgICAgIHNlbGYuZWNo
bygiWyIpDQoNCiAgICAgICAgICAgICMgVGhlIHRlc3RpbmcgbG9vcA0KICAg
ICAgICAgICAgZm9yIGkgaW4gcmFuZ2UoMCxSVU5TKToNCiAgICAgICAgICAg
ICAgICBpZiBERUJVRzogcHJpbnQgaQ0KICAgICAgICAgICAgICAgIHRlc3Qo
cmVzdWx0KQ0KICAgICAgICAgICAgICAgIGlmIG5vdCBpICUgKCBTQU1QTEVf
SU5URVJWQUwgKToNCiAgICAgICAgICAgICAgICAgICAgciA9IHN0YXRzLnVw
ZGF0ZSgpDQogICAgICAgICAgICAgICAgICAgIGlmIHIgPiBOVU1fU0FNUExF
UyAqIDAuNzU6DQogICAgICAgICAgICAgICAgICAgICAgICBzZWxmLmVjaG8o
Ik8iKQ0KICAgICAgICAgICAgICAgICAgICBlbGlmIHIgPiBOVU1fU0FNUExF
UyAqIDAuNToNCiAgICAgICAgICAgICAgICAgICAgICAgIHNlbGYuZWNobygi
byIpDQogICAgICAgICAgICAgICAgICAgIGVsaWYgciA+IE5VTV9TQU1QTEVT
ICogMC4yNToNCiAgICAgICAgICAgICAgICAgICAgICAgIHNlbGYuZWNobygi
LiIpDQogICAgICAgICAgICAgICAgICAgIGVsc2U6DQogICAgICAgICAgICAg
ICAgICAgICAgICBzZWxmLmVjaG8oIl8iKQ0KICAgICAgICAgICAgc2VsZi5l
Y2hvKCJdICIrc3RyKHN0YXRzKSsiXG4iKQ0KDQogICAgICAgICAgICAjIENh
dGNoIG11bHRpcGxlIGVycm9ycyBhbmQganVzdCB1c2UgZmlyc3Qgb25lDQog
ICAgICAgICAgICBpZiBsZW4ocmVzdWx0LmVycm9ycykgPiAwOg0KICAgICAg
ICAgICAgICAgIHJlc3VsdC5lcnJvcnMgPSBbIHJlc3VsdC5lcnJvcnNbMF0s
IF0NCiAgICAgICAgICAgIGlmIGxlbihyZXN1bHQuZmFpbHVyZXMpID4gMDoN
CiAgICAgICAgICAgICAgICByZXN1bHQuZmFpbHVyZXMgPSBbIHJlc3VsdC5m
YWlsdXJlc1swXSwgXQ0KDQogICAgICAgIHJldHVybiByZXN1bHQNCg0KaWYg
X19uYW1lX18gPT0gIl9fbWFpbl9fIjoNCg0KICAgIFNUQVRTID0gU2luZ2xl
U3RyZXNzU3RhdHMNCiAgICBSVU5TID0gMTAwMA0KDQogICAgY2xhc3MgRm9v
VGVzdCh1bml0dGVzdC5UZXN0Q2FzZSk6DQogICAgICAgIGhvYXJkID0gW10N
CiAgICAgICAgZGVmIHRlc3RfQTBfbm9wKHNlbGYpOg0KICAgICAgICAgICAg
cGFzcw0KICAgICAgICBkZWYgdGVzdF9BMV9hbGxvYyhzZWxmKToNCiAgICAg
ICAgICAgIHNlbGYuaG9hcmQuYXBwZW5kKG9wZW4oIi9kZXYvemVybyIpLnJl
YWQoMjA0OCkpDQogICAgICAgIGRlZiB0ZXN0X0EyX2FsbG9jX2JpZyhzZWxm
KToNCiAgICAgICAgICAgIHNlbGYuaG9hcmQuYXBwZW5kKG9wZW4oIi9kZXYv
emVybyIpLnJlYWQoODE5MikpDQoNCiAgICBzdWl0ZXMgPSB1bml0dGVzdC5t
YWtlU3VpdGUgKCBGb29UZXN0LCBzdWl0ZUNsYXNzID0gU3RyZXNzU3VpdGUg
KSANCiAgICBzdWl0ZSA9IHVuaXR0ZXN0LlRlc3RTdWl0ZSAoICggc3VpdGVz
LCApICkgDQogICAgcnVubmVyID0gdW5pdHRlc3QuVGV4dFRlc3RSdW5uZXIo
IHZlcmJvc2l0eSA9IDApDQogICAgcnVubmVyLnJ1bihzdWl0ZSkNCg==
---1463810461-1619782003-1002779975=:250--