Your IP : 216.73.216.213
��X> � @ sL d Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l
Z
d d l Z d d l m
Z
m Z d d l m Z d d l m Z d d l m Z d d l m Z d d l m Z d d l m Z d d
l m Z d d l m Z m Z d d l! m" Z" d d
l# m$ Z$ y e% d � Wn e& k
rtd d � Z% Yn Xd d � Z' d d � Z( d d � Z) Gd d � d e* � Z+ Gd d � d e* � Z, d d � Z- d d � Z. e d d � � Z/ e d! d" � � Z0 e d# d$ � � Z1 e d% d& � � Z2 e d' d( � � Z3 d) d* � Z4 e j5 j6 e e � d+ d, � � Z7 e d- d. � � Z8 e d/ d0 � � Z9 e j5 j6 e e � d1 d2 � � Z: e j5 j6 e e � d3 d4 � � Z; d5 d6 � Z< e d7 d8 � � Z= d9 d: � Z> e d; d< � � Z? e d= d> � � Z@ e d? d@ � � ZA e dA dB � � ZB dC dD � ZC d S)Ez
Test the hashing module.
� N)� assert_equal�assert_not_equal)�hash)� filter_args)�Memory)�assert_raises_regex)�env)� setup_module)�teardown_module)�np�
with_numpy)�TransportableException)� PY3_OR_LATER�strc C s | S)N� )�sr r �I/opt/alt/python35/lib/python3.5/site-packages/joblib/test/test_hashing.py�<lambda>' s r c C s | | k r t d � � d S)Nz%r is not lower than %r)�AssertionError)�a�br r r � assert_less* s r c G sW t � } xA t d � D]3 } t j � } | | � | j t j � | � q Wt | � S)z" Time function func on *args.
� )�list�range�time�append�min)�func�args�times�_�t1r r r � time_func1 s
r# c G s> t | | � } t | | � } d t | | � | | } | S)zP Return the relative time between func1 and func2 applied on
*args.
g �?)r# �abs)Zfunc1Zfunc2r Z
time_func1Z
time_func2�
relative_diffr r r �
relative_time< s
r& c @ s e Z d Z d d � Z d S)�Klassc C s | S)Nr )�self�xr r r �fI s zKlass.fN)�__name__�
__module__� __qualname__r* r r r r r' G s r' c @ s( e Z d Z d d � Z d d � Z d S)�KlassWithCachedMethodc C s, t d t d � } | j | j � | _ d S)N�cachedir�dir)r �test_memory_env�cacher* )r( Zmemr r r �__init__O s z KlassWithCachedMethod.__init__c C s | S)Nr )r( r) r r r r* S s zKlassWithCachedMethod.fN)r+ r, r- r3 r* r r r r r. M s r. c c s� d d d d d
d d d d d
d g d d g d d i d d i d d i d t j d g j t d � t d � d d d d i d d d d d d d i i g } xK | D]C } x: | D]2 } t j j t | � t | � k | | k f Vq� Wq� Wd S)z' Smoke test hash on various types.
� � g �?g @y �?r r N�dy �? �?y @ �?)r4 )r4 r4 )r r4 �r r4 )r r4 r7 )�gc�collectr �set�nose�toolsr r )�obj_list�obj1�obj2r r r �test_trival_hashZ s !
r@ c C s� t j t d � � } t j j t | j � t | j � � t j t
d � � } t j t
d � � } t j j t | j � t | j � � d S)Nr �
� )
�io�StringIO�unicoder; r< r r �flush� collections�dequer r �extend)r �a1�a2r r r �test_hash_methodss s
%rL c
c sJ t j j d � } | j d � } | j � } | j � } | d d 7<| | | f } xT | D]L } xC | D]; } t j j t | � t | � k t j | | k � f Vqj Wq] Wd | d | i } d | d | i } t j j t | � t | � f Vd | d | i } t j j
t | � t | � f Vt j j
t | � t | j � f Vd S)z% Test hashing with numpy arrays.
r rA r4 r5 N)rA rA ) r �random� RandomState�
random_sample�copyr; r< r r �allr �T)
�rnd�arr1�arr2Zarr3r= r> r? �d1�d2�d3r r r �test_hash_numpy| s
rY c C sb d d g } t t j d � � } d d � | D� } x' | D] } t j j t | � | � q; Wd S)Nz
datetime64[s]ztimedelta64[D]rA c s s' | ] } t j d d d | �Vq d S)r rA �dtypeN)r �arange)�.0rZ r r r � <genexpr>� s z,test_numpy_datetime_array.<locals>.<genexpr>)r r r[ r; r< r )�dtypesZa_hash�arrays�arrayr r r �test_numpy_datetime_array� s
ra c C s� t j t j d � j d � d d �d d � d d � d d � f } t j | � } t j j t | � t | � � t j | � } t j j t | � t | � � d S) Nip �� r5 r �order�Fr4 )rb r5 r )
r �asarrayr[ �reshape�ascontiguousarrayr; r< r r �asfortranarray)r r �cr r r � test_hash_numpy_noncontiguous� s ( rj c c s� t j d d � } zq t j | d d
d d �} t j | � } x@ d D]8 } t j j t | d | �t | d | �k | f VqF WWd
d t � k r� ~ t
j � y t j
| � Wn: t k
r� } z t j d k s� | � WYd
d
} ~ Xn XXd
S)zS Check that memmap and arrays hash identically if coerce_mmap is
True.
�prefixZjoblib_test_hash_memmap_�shaperA �modezw+FT� coerce_mmapN�m�nt)rA rA )FT)�tempfile�mktempr �memmapre r; r< r r �localsr8 r9 �os�unlink�OSError�name)�filenamero r rn �er r r �test_hash_memmap� s"
r{ c s� t j d k r t j � � t j j d � } | j d � } t t d � rW t j � n t
� � f d d � } t | t | � } t
| d � d t | | � } t t | | | f � } d t | | � | | } t
| d � d
S) a Check the performance of hashing numpy arrays:
In [22]: a = np.random.random(1000000)
In [23]: %timeit hashlib.md5(a).hexdigest()
100 loops, best of 3: 20.7 ms per loop
In [24]: %timeit hashlib.md5(pickle.dumps(a, protocol=2)).hexdigest()
1 loops, best of 3: 73.1 ms per loop
In [25]: %timeit hashlib.md5(cPickle.dumps(a, protocol=2)).hexdigest()
10 loops, best of 3: 53.9 ms per loop
In [26]: %timeit hash(a)
100 loops, best of 3: 20.8 ms per loop
�win32r i@B � getbufferc s t j � | � � j � S)N)�hashlib�md5� hexdigest)r) )r} r r r � s z-test_hash_numpy_performance.<locals>.<lambda>g333333�?r g �?N)�sys�platformr; �SkipTestr rM rN rO �hasattrr} �
memoryviewr& r r r# r$ )rS r Zmd5_hashr% Z time_hashlibZ time_hashr )r} r �test_hash_numpy_performance� s
r� c C sS t � } t � } t j j t t | j g d � � t t | j g d � � � d S)z~ Make sure that calling the same method on two different instances
of the same class does resolve to the same hashes.
r4 N)r4 )r4 )r' r; r< r r r r* )r r r r r �test_bound_methods_hash� s !r� c C sY t � } t � } t j j t t | j j g d � � t t | j j g d � � � d S)z� Make sure that calling the same _cached_ method on two different
instances of the same class does resolve to the same hashes.
r4 N)r4 )r4 )r. r; r< r r r r* r )r r r r r � test_bound_cached_methods_hash� s $r� c C ss t j d d � t d � D� d t �} t j d d � t d � D� d t �} t j j t | � t | � � d S)z< Make sure that ndarrays with dtype `object' hash correctly.c S s g | ] } t j | � � q Sr )r r[ )r\ �ir r r �
<listcomp>
s z*test_hash_object_dtype.<locals>.<listcomp>� rZ c S s g | ] } t j | � � q Sr )r r[ )r\ r� r r r r� s N)r r` r �objectr; r< r r )r r r r r �test_hash_object_dtype s ((r� c C sA t j d � } t j d � } t j j t | � t | � � d S)Ng @g @)r �float64r; r< r r )r r r r r �test_numpy_scalar s r� c C s� t � } d d g d d g d d g d d g d d g d d g d d g d d g d
d g d d g d d g d
d g d d g i
} | j | � } | j | � } t j j t | � t | � � d S)Nz#s12069__c_maps.nii.gz�! z#s12158__c_maps.nii.gzz#s12258__c_maps.nii.gzz#s12277__c_maps.nii.gzz#s12300__c_maps.nii.gzz#s12401__c_maps.nii.gzz#s12430__c_maps.nii.gzz#s13817__c_maps.nii.gzz#s13903__c_maps.nii.gzz#s13916__c_maps.nii.gzz#s13981__c_maps.nii.gzz#s13982__c_maps.nii.gzz#s13983__c_maps.nii.gz)r. r* r; r< r r )�kr6 r r r r r �test_dict_hash s$ r� c C s} t � } t d d d d d d d d d d
d d d
g
� } | j | � } | j | � } t j j t | � t | � � d S)Nz#s12069__c_maps.nii.gzz#s12158__c_maps.nii.gzz#s12258__c_maps.nii.gzz#s12277__c_maps.nii.gzz#s12300__c_maps.nii.gzz#s12401__c_maps.nii.gzz#s12430__c_maps.nii.gzz#s13817__c_maps.nii.gzz#s13903__c_maps.nii.gzz#s13916__c_maps.nii.gzz#s13981__c_maps.nii.gzz#s13982__c_maps.nii.gzz#s13983__c_maps.nii.gz)r. r: r* r; r< r r )r� r r r r r r �
test_set_hash8 s" r� c C s_ d } | d i } | d i } t j t j | � � } t t | | g � t | | g � � d S)N�foo�bar)�pickle�loads�dumpsr r )�stringr r ri r r r � test_stringR s
r� c C sn t j d t j f d t j f g � } | } t j t j | � � } t t | | g � t | | g � � d S)N�f1�f2) r rZ �uint�int32r� r� r� r r )r r ri r r r �
test_dtype\ s 'r� c # s� t j d � � d d d! � f d d � t d � D� d
d d t d t � g d
d d d" d d
g i g } d d d d d d d g d d d d d d d g i } t r� d n d } | | } x1 t | | � D] \ } } t t | � | f Vq� Wd S)#N�* zThis is a string to hashu C'est lété�@� �1� í c s g | ] } � j � � q Sr )rM )r\ r! )�rngr r r� p s z-test_hashes_stay_the_same.<locals>.<listcomp>� r �abcz the messageZabcde�{ Zsadfasi' r5 �py2Z 80436ada343b0d79a99bfd8883a96e45Z 2ff3a25200eb6219f468de2640913c2dZ 50d81c80af05061ac4dcdc2d5edee6d6Z 536af09b66a087ed18b515acc17dc7fcZ 123ffc6f13480767167e171a8e1f6f4aZ fc9314a39ff75b829498380850447047�py3Z 71b3f47df22cb19431d85d92d0b230b2Z 2d8d189e9b2b0b2e384d93c868c0e576Z e205227dd82250871fa25aa0ec690aa3Z 9e4e9bf9b91890c9734a6111a35e6633Z 6065a3c48e842ea5dee2cfd0d6820ad6Z aeda150553d4bb5c69f0e69d51b0e2ef�3~��)r� r� r� i���) rM �Randomr r
�
ValueErrorr �zipr r )� to_hash_list�
expected_dict�py_version_str�
expected_list�to_hash�expectedr )r� r �test_hashes_stay_the_sameg s0
r� c C sM t j j d � } | j d � } t j | � } t t | � t | � � d S)Nr rA )rA rA )r rM rN rO rh r r )r� Zarr_cZarr_fr r r �Atest_hashes_are_different_between_c_and_fortran_contiguous_arrays� s r� c C s t t j d � � d S)Nr )r r r` r r r r �
test_0d_array� s r� c C s2 t t t j d � � t t j d g � � � d S)Nr )r r r r` r r r r �)test_0d_and_1d_array_hashing_is_different� s r� c
# s� t j j d � � � j d% d d d d �j d � t � f d d � t d � D� � � f d
d � t d � D� d& � j d
d � j d � d � j d d d �j d � � j d � j d � g i t j d d d �j d' � t j
t j d d d �j d( � � t j d d d �j d) � d d � d d � f g } d d d d d d d d g d d d d d! d" d# d$ g i } t r�d n d } | | } x1 t | | � D] \ } } t
t | � | f Vq�Wd S)*Nr� i� �high�size�2 z<i8c 3 s' | ] } � j d � j d � Vq d S)r z<f4N)�randn�astype)r\ r! )r� r r r] � s z?test_hashes_stay_the_same_with_numpy_objects.<locals>.<genexpr>r� c s( g | ] } � j d � j d � � q S)r z<f4)r� r� )r\ r! )r� r r r� � s z@test_hashes_stay_the_same_with_numpy_objects.<locals>.<listcomp>i
r z<f4r rA � �d rZ r5 r� Z 80f2387e7752abbda2658aafed49e086Z 0d700f7f25ea670fd305e4cd93b0e8cdZ 83a2bdf843e79e4b3e26521db73088b9Z 63e0efd43c0a9ad92a07e8ce04338dd3Z 03fef702946b602c852b8b4e60929914Z 07074691e90d7098a85956367045c81eZ d264cf79f353aa7bbfa8349e3df72d8fr� Z 10a6afc379ca2708acfbaef0ab676eabZ 988a7114f337f381393025911ebc823bZ c6809f4b97e35f2fa0ee8d653cbd025cZ b3ad17348e32728a7eb9cda1e7ede438Z 927b3e6b0b6a037e8e035bda134e0b05Z 108f6ee98e7db19ea2006ffd208f4bf1Z bd48ccaaff28e16e6badee81041b7180i���i��)rA rA )rA rA )rA rA )r rM rN �randintr� �tupler r� r[ rf rh r r� r r )r� r� r� r� r� r� r )r� r �,test_hashes_stay_the_same_with_numpy_objects� s: !" $7
r� c C s& d d � } t t j d t | � d S)Nc S s d S)Nr� r r r r r �
non_picklable� s z2test_hashing_pickling_error.<locals>.non_picklablezPicklingError while hashing)r r� �
PicklingErrorr )r� r r r �test_hashing_pickling_error� s r� )D�__doc__r; r r~ rq ru r� r8 rC rG r� rM �
nose.toolsr r Zjoblib.hashingr Zjoblib.func_inspectr Z
joblib.memoryr Zjoblib.testingr Zjoblib.test.test_memoryr r1 r Ztest_memory_setup_funcr
Ztest_memory_teardown_funcZjoblib.test.commonr r Zjoblib.my_exceptionsr
Zjoblib._compatr rE � NameErrorr r# r& r� r' r. r@ rL rY ra rj r{ r� r� r< �
with_setupr� r� r� r� r� r� r� r� r� r� r� r� r� r r r r �<module> sh
+
! !!
$
7