/usr/local/lib/python3.6/site-packages/transformers/utils/__pycache__
Edit: /usr/local/lib/python3.6/site-packages/transformers/utils/__pycache__/generic.cpython-36.pyc (10493B)
3
Eg\& @ s" d Z ddlZddlmZmZ ddlmZ ddlmZ ddl m
Z
ddl m Z m
Z
mZmZ ddlZdd lmZmZmZmZ G d
d d eZd d
Zdd Zdd Zdd Z dd Z dd Z dd Z dd Z G d d d eZ!G d d d e
Z"G d d! d!e"Z#G d"d# d#e"Z$G d$d% d%Z%d&d' Z&dS )(z
Generic utilities
N) OrderedDictUserDict) ExitStack)fields)Enum)AnyContextManagerListTuple )is_flax_availableis_tf_availableis_torch_availableis_torch_fx_proxyc @ s e Zd ZdZdddZdS )cached_propertyz
Descriptor that mimics @property but caches output in member variable.
From tensorflow_datasets
Built-in in functools from Python 3.8.
Nc C sX |d kr | S | j d kr tdd| j j }t||d }|d krT| j |}t||| |S )Nzunreadable attributeZ __cached_)fgetAttributeError__name__getattrsetattr)selfobjZobjtypeattrcached r D/usr/local/lib/python3.6/site-packages/transformers/utils/generic.py__get__' s
zcached_property.__get__)N)r
__module__ __qualname____doc__r r r r r r s r c C s t | r dS t r*ddl}t| |jr*dS t rHddl}t| |jrHdS t rxddlj }ddl
m } t| |j |frxdS t| t
j S )zl
Tests if `x` is a `torch.Tensor`, `tf.Tensor`, `jaxlib.xla_extension.DeviceArray` or `np.ndarray`.
Tr N)Tracer)r r torch
isinstanceTensorr
tensorflowr jax.numpynumpyZjax.corer ndarraynp)xr! tfjnpr r r r is_tensor5 s
r, c C s t | tjS )N)r" r( r' )r) r r r _is_numpyP s r- c C s dd l }t| |jS )Nr )r! r" r# )r) r! r r r _is_torchT s r. c C s dd l }t| |jS )Nr )r! r" Zdevice)r) r! r r r _is_torch_deviceZ s r/ c C s dd l }t| |jS )Nr )r$ r" r# )r) r* r r r _is_tensorflow` s r0 c C s dd l j}t| |jS )Nr )r% r& r" r' )r) r+ r r r _is_jaxf s
r1 c C s t | ttfr dd | j D S t | ttfr
q s z to_py_obj..c S s g | ] }t |qS r )r2 )r3 or r r
s s z to_py_obj..N)r" dictr itemslisttupler
r0 r& tolistr r. detachcpur r1 r( asarrayr' number)r r r r r2 l s r2 c C s t | ttfr dd | j D S t | ttfr8tj| S t rNt | rN| j
S t rlt | rl| j
j j
S t rt| rtj| S | S dS )zc
Convert a TensorFlow tensor, PyTorch tensor, Numpy array or python list to a Numpy array.
c S s i | ]\}}t ||qS r )to_numpy)r3 r4 r5 r r r r6 s z to_numpy..N)r" r9 r r: r; r< r( arrayr
r0 r& r r. r> r? r r1 r@ )r r r r rB s
rB c sn e Zd ZdZdd Zdd Zdd Zdd Zd
d Zd d
Z fddZ
fddZ e e
dddZ ZS ) ModelOutputa
Base class for all model outputs as dataclass. Has a `__getitem__` that allows indexing by integer or slice (like a
tuple) or strings (like a dictionary) that will ignore the `None` attributes. Otherwise behaves like a regular
python dictionary.
You can't unpack a `ModelOutput` directly. Use the [`~utils.ModelOutput.to_tuple`] method to convert it to a tuple
before.
c s t }t|s"t jj dtdd |dd D sNt jj dt |d j}t fdd|dd D }|ot| rdt |t
r|j }d}n*yt |}d}W n t
k
r d }Y nX |rJx|D ]j}t |ttf st|d
k st |d t rP t |d |d |d d k r|d |d < qW n|d k r| |d j< n0x.|D ]&}t |j}|d k rj| |j< qjW d S ) Nz has no fields.c s s | ]}|j d kV qd S )N)default)r3 fieldr r r s z,ModelOutput.__post_init__..r z. should not have more than one required field.r c 3 s | ]}t |jd kV qd S )N)r name)r3 rF )r r r rG s TF )r len
ValueError __class__r allr rH r, r" r9 r: iter TypeErrorr; r< strr ) r Z class_fieldsZ first_fieldZother_fields_are_noneiteratorZfirst_field_iteratorelementrF r5 r )r r
__post_init__ s>
zModelOutput.__post_init__c O s t d| jj dd S )Nz$You cannot use ``__delitem__`` on a z
instance.) ExceptionrL r )r argskwargsr r r __delitem__ s zModelOutput.__delitem__c O s t d| jj dd S )Nz#You cannot use ``setdefault`` on a z
instance.)rT rL r )r rU rV r r r
setdefault s zModelOutput.setdefaultc O s t d| jj dd S )Nz You cannot use ``pop`` on a z
instance.)rT rL r )r rU rV r r r pop s zModelOutput.popc O s t d| jj dd S )Nz You cannot use ``update`` on a z
instance.)rT rL r )r rU rV r r r update s zModelOutput.updatec C s4 t |tr$dd | j D }|| S | j | S d S )Nc S s i | ]\}}||qS r r )r3 r4 r5 r r r r6 s z+ModelOutput.__getitem__..)r" rP r: to_tuple)r r4 Z
inner_dictr r r __getitem__ s
zModelOutput.__getitem__c s4 || j kr"|d k r"t j|| t j|| d S )N)keyssuper __setitem__ __setattr__)r rH value)rL r r r` s zModelOutput.__setattr__c s t j|| t j|| d S )N)r^ r_ r` )r keyra )rL r r r_ s zModelOutput.__setitem__)returnc s t fdd j D S )za
Convert self to a tuple containing all the attributes/keys that are not `None`.
c 3 s | ]} | V qd S )Nr )r3 r4 )r r r rG s z'ModelOutput.to_tuple..)r< r] )r r )r r r[ s zModelOutput.to_tuple)r r r r rS rW rX rY rZ r\ r` r_ r
r r[
__classcell__r r )rL r rD s , rD c @ s e Zd ZdZedd ZdS ) ExplicitEnumzC
Enum with more explicit error message for missing values.
c C s( t | d| j dt| jj d S )Nz is not a valid z, please select one of )rK r r; _value2member_map_r] )clsra r r r _missing_ s zExplicitEnum._missing_N)r r r r classmethodrh r r r r re s re c @ s e Zd ZdZdZdZdZdS )PaddingStrategyz
Possible values for the `padding` argument in [`PreTrainedTokenizerBase.__call__`]. Useful for tab-completion in an
IDE.
longest
max_lengthZ
do_not_padN)r r r r ZLONGESTZ
MAX_LENGTHZ
DO_NOT_PADr r r r rj s rj c @ s e Zd ZdZdZdZdZdZdS )
TensorTypez
Possible values for the `return_tensors` argument in [`PreTrainedTokenizerBase.__call__`]. Useful for
tab-completion in an IDE.
ptr* r( ZjaxN)r r r r ZPYTORCHZ
TENSORFLOWZNUMPYZJAXr r r r rm s
rm c @ s2 e Zd ZdZee dddZdd Zdd Zd S )
ContextManagersz
Wrapper for `contextlib.ExitStack` which enters a collection of context managers. Adaptation of `ContextManagers`
in the `fastcore` library.
)context_managersc C s || _ t | _d S )N)rp r stack)r rp r r r __init__ s zContextManagers.__init__c C s x| j D ]}| jj| qW d S )N)rp rq
enter_context)r Zcontext_managerr r r __enter__ s zContextManagers.__enter__c O s | j j|| d S )N)rq __exit__)r rU rV r r r ru # s zContextManagers.__exit__N) r r r r r r rr rt ru r r r r ro s ro c C sn | j }|jdr tj| j}n$|jdr6tj| j}n tj| j}d|krZdd |jD S dd |jD S dS )zq
Find the labels used by a given model.
Args:
model_class (`type`): The class of the model.
ZTFZFlaxZQuestionAnsweringc S s g | ]}d |ks|dkr|qS )labelstart_positions
end_positions)rw rx r )r3 pr r r r8 6 s z find_labels..c S s g | ]}d |kr|qS )rv r )r3 ry r r r r8 8 s N)r
startswithinspect signaturecall__call__Zforward
parameters)Z model_classZ
model_namer| r r r find_labels' s
r )'r r{ collectionsr r
contextlibr Z dataclassesr enumr typingr r r r
r& r( Z import_utilsr r
r r propertyr r, r- r. r/ r0 r1 r2 rB rD re rj rm ro r r r r r s. `