/usr/local/lib/python3.6/site-packages/transformers/onnx/__pycache__
Edit: /usr/local/lib/python3.6/site-packages/transformers/onnx/__pycache__/features.cpython-36.pyc (10214B)
3
Eg> @ s8 d dl mZmZ d dlmZmZmZmZmZm Z ddl
m Z m Z m
Z
mZmZ ddlmZ ddlmZ ddlmZ ddlmZ dd lmZ dd
lmZ dd l m Z dd l m Z dd
l m!Z! ddl"m#Z# ddl$m%Z% ddl&m'Z' ddl(m)Z) ddl*m+Z+ ddl,m-Z- ddl.m/Z/ ddl0m1Z1 ddl2m3Z3 ddl4m5Z5 ddl6m7Z7 ddl8m9Z9 ddl:m;Z; ddl
m?Z? e=j@eAZBe rd d lCmDZDmEZEmFZFmGZGmHZHmIZImJZJmKZKmLZL e rd d lCmMZMmNZNmOZOmPZPmQZQmRZRmSZSmTZT e re reBjUd d!d"eVee? eeVee ge?f f d#d$d%ZWG d&d' d'ZXd!S )( )partialreduce)CallableDictOptionalTupleTypeUnion )PretrainedConfigPreTrainedModelTFPreTrainedModelis_tf_availableis_torch_available)AlbertOnnxConfig)BartOnnxConfig)BeitOnnxConfig)BertOnnxConfig)BlenderbotOnnxConfig)BlenderbotSmallOnnxConfig)CamembertOnnxConfig)DistilBertOnnxConfig)ElectraOnnxConfig)FlaubertOnnxConfig)GPT2OnnxConfig)GPTNeoOnnxConfig)GPTJOnnxConfig)IBertOnnxConfig)LayoutLMOnnxConfig)M2M100OnnxConfig)MarianOnnxConfig)MBartOnnxConfig)RobertaOnnxConfig) T5OnnxConfig)
ViTOnnxConfig)XLMRobertaOnnxConfig)logging )
OnnxConfig) AutoModelAutoModelForCausalLM AutoModelForImageClassificationAutoModelForMaskedLMAutoModelForMultipleChoice AutoModelForQuestionAnsweringAutoModelForSeq2SeqLM"AutoModelForSequenceClassification AutoModelForTokenClassification) TFAutoModelTFAutoModelForCausalLMTFAutoModelForMaskedLM TFAutoModelForMultipleChoice TFAutoModelForQuestionAnsweringTFAutoModelForSeq2SeqLM$TFAutoModelForSequenceClassification!TFAutoModelForTokenClassificationzThe ONNX export features are only supported for PyTorch or TensorFlow. You will not be able to export models without one of these libraries installed.N)onnx_config_cls)supported_featuresr: returnc G s` | dkrt di }xF|D ]>}d|krF|jdd}t| j|d||< qt| j|d||< qW |S )a_
Generate the mapping between supported the features and their corresponding OnnxConfig for a given model.
Args:
*supported_features: The names of the supported features.
onnx_config_cls: The OnnxConfig class corresponding to the model.
Returns:
The dictionary mapping a feature to an OnnxConfig constructor.
Nz#A OnnxConfig class must be providedz
-with-past )task)
ValueErrorreplacer Z with_pastZfrom_model_config)r: r; mappingfeaturer> rC D/usr/local/lib/python3.6/site-packages/transformers/onnx/features.pysupported_features_mapping> s
rE c @ s e Zd Zi Zi Ze r.eeee e
e e e
ed Ze rJeeeeeeeedZedddddededd d
d d d
dded edd d
d d d
dded eddd
ddde deddddde deddd
ddde deddddde deddd
ddde dedd d d
d
d e!dedd d d
e"deddd
ddde#dedd d d
e$deddd
ddde%dedd d
d dde&dedd d
d dde'dedd d
d de(dedddde)deddd
ddde*dedde+dedde,dedd d
d d d
e-dedd d
d d d
e.ddZ/e0e1dd dd e/j2 D Z3e4d(e5e6e5 e7e5e8e9ge:f f dddZ;e4e5e5dddZd d d!Z?e4d*e5e5e5e5e@eAeBf d"d#d$ZCe4d+e@eAeBf e5eDe5e8f d%d&d'ZEdS ),FeaturesManager) defaultz masked-lmz causal-lmz
seq2seq-lmzsequence-classificationztoken-classificationzmultiple-choicezquestion-answeringzimage-classification)rG z masked-lmz causal-lmz
seq2seq-lmzsequence-classificationztoken-classificationzmultiple-choicezquestion-answeringrG z masked-lmzsequence-classificationztoken-classificationzquestion-answering)r: zdefault-with-pastz causal-lmzcausal-lm-with-pastz
seq2seq-lmzseq2seq-lm-with-pastzimage-classification)ZalbertZbartZmbartZbertZibertZ camembertZ
distilbertZflaubertZmarianzm2m-100ZrobertaZt5z xlm-robertaZgpt2zgpt-jzgpt-neoZlayoutlmZelectraZvitZbeitZ
blenderbotzblenderbot-smallc C s | |B S )NrC )s1s2rC rC rD ' s zFeaturesManager.c c s | ]}|j V qd S )N)keys).0vrC rC rD ' s zFeaturesManager.N)
model_type
model_namer< c C sX | j } | tjkrN|r&| d| dn| }t| dttjj d| dtj| S )a
Tries to retrieve the feature -> OnnxConfig constructor map from the model type.
Args:
model_type (`str`):
The model type to retrieve the supported features for.
model_name (`str`, *optional*):
The name attribute of the model object, only used for the exception message.
Returns:
The dictionary mapping each feature to a corresponding OnnxConfig constructor.
z ()z is not supported yet. Only z' are supported. If you want to support z) please propose a PR or open up an issue.)lowerrF _SUPPORTED_MODEL_TYPEKeyErrorlistrK )rO rP Zmodel_type_and_model_namerC rC rD %get_supported_features_for_model_type) s
"z5FeaturesManager.get_supported_features_for_model_type)rB r< c C s | j ddS )Nz
-with-pastr= )r@ )rB rC rC rD feature_to_taskC s z FeaturesManager.feature_to_task) frameworkc C sP | dkrt d| dn2| dkr4t r4tdn| dkrLt rLtddS ) z
Validates if the framework requested for the export is both correct and available, otherwise throws an
exception.
pttfzAOnly two frameworks are supported for ONNX export: pt or tf, but z was provided.zOCannot export model to ONNX using PyTorch because no PyTorch package was found.zUCannot export model to ONNX using TensorFlow because no TensorFlow package was found.N)rY rZ )r? r RuntimeErrorr )rX rC rC rD _validate_framework_choiceG s
z*FeaturesManager._validate_framework_choicerY )rB rX r< c C sX t j| }t j| |dkr$t j}nt j}||krPtd| dtt jj || S )ad
Attempts to retrieve an AutoModel class from a feature name.
Args:
feature (`str`):
The feature required.
framework (`str`, *optional*, defaults to `"pt"`):
The framework to use for the export.
Returns:
The AutoModel class corresponding to the feature.
rY zUnknown task: z. Possible values are )rF rW r\ _TASKS_TO_AUTOMODELS_TASKS_TO_TF_AUTOMODELSrT rU values)rB rX r> Ztask_to_automodelrC rC rD get_model_class_for_featureV s
z+FeaturesManager.get_model_class_for_feature)rB modelrX cache_dirr< c C sb t j| |}y|j||d}W n> tk
r\ |dkrH|j|d|d}n|j|d|d}Y nX |S )a
Attempts to retrieve a model from a model's name and the feature to be enabled.
Args:
feature (`str`):
The feature required.
model (`str`):
The name of the model to export.
framework (`str`, *optional*, defaults to `"pt"`):
The framework to use for the export.
Returns:
The instance of the model.
)rb rY T)Zfrom_tfrb )Zfrom_ptrb )rF r` Zfrom_pretrainedOSError)rB ra rX rb Z model_classrC rC rD get_model_from_featureq s z&FeaturesManager.get_model_from_feature)ra rB r< c C sd | j jjdd}t| dd}tj||d}||krNt| j j d| d| | j jtj| | fS )aR
Check whether or not the model has the requested features.
Args:
model: The model to export.
feature: The name of the feature to check if it is available.
Returns:
(str) The type of the model (OnnxConfig) The OnnxConfig instance holding the model export properties.
_-namer= )rP z doesn't support feature z. Supported values are: )configrO r@ getattrrF rV r? rS )ra rB rO rP Zmodel_featuresrC rC rD check_supported_model_or_raise s z.FeaturesManager.check_supported_model_or_raise)N)rY )rY N)rG )F__name__
__module__ __qualname__r] r^ r r) r, r* r/ r0 r1 r- r. r+ r r2 r4 r3 r7 r8 r9 r5 r6 rE r r r! r r r r r r r r" r# r% r r r r r r$ r r r rS sortedr r_ ZAVAILABLE_FEATURES staticmethodstrr r r r r( rV rW r\ r r` r r r
rd r rj rC rC rC rD rF Y sr
"
rF )Y functoolsr r typingr r r r r r r= r r r
r r Z
models.albertr Z models.bartr Z models.beitr Z models.bertr Zmodels.blenderbotr Zmodels.blenderbot_smallr Zmodels.camembertr Zmodels.distilbertr Zmodels.electrar Zmodels.flaubertr Z models.gpt2r Zmodels.gpt_neor Z models.gptjr Z models.ibertr Zmodels.layoutlmr Zmodels.m2m_100r Z
models.marianr Z models.mbartr! Zmodels.robertar" Z models.t5r# Z
models.vitr$ Zmodels.xlm_robertar% utilsr& rh r(
get_loggerrk loggerZtransformers.models.autor) r* r+ r, r- r. r/ r0 r1 r2 r3 r4 r5 r6 r7 r8 r9 warningrp rE rF rC rC rC rD sJ
, (