FM / __pycache_

/usr/local/lib/python3.6/site-packages/datasets/__pycache__/

/usr/local/lib/python3.6/site-packages/datasets/__pycache__

Name	Size	Mode	Actions
arrow_dataset.cpython-36.pyc	189850	0644	edit dl rm
arrow_reader.cpython-36.pyc	22483	0644	edit dl rm
arrow_writer.cpython-36.pyc	22110	0644	edit dl rm
builder.cpython-36.pyc	52779	0644	edit dl rm
combine.cpython-36.pyc	5508	0644	edit dl rm
config.cpython-36.pyc	5181	0644	edit dl rm
dataset_dict.cpython-36.pyc	83319	0644	edit dl rm
data_files.cpython-36.pyc	30153	0644	edit dl rm
fingerprint.cpython-36.pyc	18979	0644	edit dl rm
info.cpython-36.pyc	17359	0644	edit dl rm
inspect.cpython-36.pyc	18583	0644	edit dl rm
iterable_dataset.cpython-36.pyc	62932	0644	edit dl rm
keyhash.cpython-36.pyc	3341	0644	edit dl rm
load.cpython-36.pyc	62922	0644	edit dl rm
metric.cpython-36.pyc	23217	0644	edit dl rm
naming.cpython-36.pyc	2571	0644	edit dl rm
search.cpython-36.pyc	31339	0644	edit dl rm
splits.cpython-36.pyc	22522	0644	edit dl rm
streaming.cpython-36.pyc	4119	0644	edit dl rm
table.cpython-36.pyc	79612	0644	edit dl rm
__init__.cpython-36.pyc	2282	0644	edit dl rm

Edit: /usr/local/lib/python3.6/site-packages/datasets/__pycache__/keyhash.cpython-36.pyc (3341B)

3 <%Eg��@sddZddlZddlmZeeeefed�dd�ZGdd�de�Z Gd d �d e�Z Gdd�d�ZdS) a Hashing function for dataset keys using `hashlib.md5` Requirements for the hash function: - Provides a uniformly distributed hash from random space - Adequately fast speed - Working with multiple input types (in this case, `str`, `int` or `bytes`) - Should be platform independent (generates same hash on different OS and systems) The hashing function provides a unique 128-bit integer hash of the key provided. The split name is being used here as the hash salt to avoid having same hashes in different splits due to same keys �N)�Union)� hash_data�returncCsLt|t�r|St|t�r&|jdd�}nt|t�r:t|�}nt|��|jd�S)z| Returns the input hash_data in its bytes form Args: hash_data: the hash salt/key to be converted to bytes �\�/zutf-8)� isinstance�bytes�str�replace�int�InvalidKeyError�encode)r�r�:/usr/local/lib/python3.6/site-packages/datasets/keyhash.py� _as_bytes%s rcs eZdZdZ�fdd�Z�ZS)rz6Raises an error when given key is of invalid datatype.csDd|_d|�dt|��|_d|_t�j|j�|j�|j��dS)Nz7 FAILURE TO GENERATE DATASET: Invalid key type detectedz Found Key z of type z- Keys should be either str, int or bytes type)�prefix�type�err_msg�suffix�super�__init__)�selfr)� __class__rrr?szInvalidKeyError.__init__)�__name__� __module__�__qualname__�__doc__r� __classcell__rr)rrr<srcs"eZdZdZd�fdd� Z�ZS)�DuplicatedKeysErrorz(Raise an error when duplicate key found.�cs�||_||_||_d|_t|�dkr>ddj|��d|��|_n.ddj|dd��dt|�d�d|��|_|rxd|nd |_t�j |j�|j�|j��dS) Nz3Found multiple examples generated with the same key�z The examples at index z, z have the key z... (z more) have the key � r) �key�duplicate_key_indices�fix_msgr�len�joinrrrr)rr"r#r$)rrrrIs.zDuplicatedKeysError.__init__)r)rrrrrrrr)rrrFsrc@s8eZdZdZed�dd�Zeeeefed�dd�Z dS) � KeyHasherz,KeyHasher class for providing hash using md5)� hash_saltcCstjt|��|_dS)N)�hashlib�md5r� _split_md5)rr(rrrrYszKeyHasher.__init__)r"rcCs*|jj�}t|�}|j|�t|j�d�S)z�Returns 128-bits unique hash of input key Args: key: the input key to be hashed (should be str, int or bytes) Returns: 128-bit int hash key�)r+�copyr�updater� hexdigest)rr"r*Zbyte_keyrrr�hash\s zKeyHasher.hashN) rrrrr rrrrr0rrrrr'Vsr')rr)�typingrr rrr� Exceptionrrr'rrrr�s