/usr/local/lib/python3.6/site-packages/transformers/commands/__pycache__
Edit: /usr/local/lib/python3.6/site-packages/transformers/commands/__pycache__/serving.cpython-36.pyc (6748B)
3
Eg[ @ s2 d dl mZmZ d dlmZmZmZ ddlmZm Z m
Z
ddl m Z ddl
mZ yLd dlmZmZmZ d d lmZ d d
lmZ d d lmZ d d lmZ d
ZW n( e e fk
r e Zdd ZdZY nX e j dZ edddZ!G dd deZ"G dd deZ#G dd deZ$G dd d eZ%G d d d eZ&d S ) )ArgumentParser Namespace)AnyListOptional )Pipelineget_supported_taskspipeline)logging )BaseTransformersCLICommand)BodyFastAPI
HTTPException)APIRoute) BaseModel) JSONResponse)runTc O s d S )N )xyr r G/usr/local/lib/python3.6/site-packages/transformers/commands/serving.pyr " s r Fztransformers-cli/serving)argsc C s: t | j| jr| jnd| j| j| jd}t|| j| j| j S )z~
Factory function used to instantiate serving server from provided command line arguments.
Returns: ServeCommand
N)taskmodelconfig tokenizerdevice)
r
r r r r r ServeCommandhostportworkers)r Znlpr r r serve_command_factory+ s
r# c @ s e Zd ZU dZedS )ServeModelInfoResultz"
Expose model information
N)__name__
__module__ __qualname____doc__dictinfosr r r r r$ ; s
r$ c @ s& e Zd ZU dZee eee dS )ServeTokenizeResultz
Tokenize result model
N)
r% r& r' r( r strtokensr int
tokens_idsr r r r r+ C s
r+ c @ s e Zd ZU dZedS )ServeDeTokenizeResultz!
DeTokenize result model
N)r% r& r' r( r, textr r r r r0 L s
r0 c @ s e Zd ZU dZedS )ServeForwardResultz
Forward result model
N)r% r& r' r( r outputr r r r r2 T s
r2 c @ s e Zd ZeedddZeeeedddZ dd Z
d d
Z e d d d
e dd d
fee
dddZe d d d
e dd d
e d d d
fee e
e
dddZe d d d
fddZd S )r )parserc C s | j ddd}|jdtt dd |jdtdd d
|jd td d
d
|jdtddd
|jdtdd |jdtdd |jdtdd |jdtd dd
|jtd dS ) z
Register this command to argparse so it's available for the transformer-cli
Args:
parser: Root parser to register command-specific arguments
ZservezFCLI tool to run inference requests through REST and GraphQL endpoints.)helpz--taskz The task to run the pipeline on)typechoicesr5 z--host localhostz$Interface the server will listen on.)r6 defaultr5 z--porti" z Port the serving will listen to.z --workersr zNumber of http workersz--modelz%Model's name or path to stored model.)r6 r5 z--configz,Model's config name or path to stored model.z --tokenizerzTokenizer name to use.z--devicezSIndicate the device to run onto, -1 indicates CPU, >= 0 indicates GPU (default: -1))funcN)
add_parser add_argumentr, r r. set_defaultsr# )r4 Z serve_parserr r r register_subcommand] s&
z ServeCommand.register_subcommand)r
r r! r" c C s || _ || _|| _|| _ts&tdnttjd| d| tt d| j
t t dgdt d| j
tt dgdt d | jtt dgdt d
| jtt dgdgd d | _d S )
NzUsing serve command requires FastAPI and uvicorn. Please install transformers with [serving]: pip install "transformers[serving]".Or install FastAPI and uvicorn separately.zServing model over :/GET)Zresponse_modelresponse_classmethodsz /tokenizePOSTz /detokenizez/forwardiX )Zroutestimeout) _pipeliner r! r" _serve_dependencies_installed RuntimeErrorloggerinfor r
model_infor$ r tokenizer+
detokenizer0 forwardr2 _app)selfr
r r! r" r r r __init__| sD
zServeCommand.__init__c C s t | j| j| j| jd d S )N)r r! r" )r rP r r! r" )rQ r r r r s zServeCommand.runc C s t t| jjjdS )N)r* )r$ varsrG r r )rQ r r r rL s zServeCommand.model_infoNT)embedF)
text_input
return_idsc C sx y:| j jj|}|r.| j jj|}t||dS t|dS W n8 tk
rr } z tddt|ddW Y dd}~X nX dS )z
Tokenize the provided input and eventually returns corresponding tokens id: - **text_input**: String to
tokenize - **return_ids**: Boolean flags indicating if the tokens have to be converted to their integer
mapping.
)r- r/ )r- i )r error) status_codedetailN)rG r rM Zconvert_tokens_to_idsr+ Exceptionr r, )rQ rU rV Z
tokens_txtr/ er r r rM s zServeCommand.tokenize)r/ skip_special_tokenscleanup_tokenization_spacesc C s\ y | j jj|||}td|dS tk
rV } z tddt|ddW Y dd}~X nX dS )a:
Detokenize the provided tokens ids to readable text: - **tokens_ids**: List of tokens ids -
**skip_special_tokens**: Flag indicating to not try to decode special tokens - **cleanup_tokenization_spaces**:
Flag indicating to remove all leading/trailing spaces and intermediate ones.
rW )r r1 i )r rX )rY rZ N)rG r decoder0 r[ r r, )rQ r/ r] r^ Z decoded_strr\ r r r rN s
zServeCommand.detokenizec sf t |dkrtg g dS y| j|}t|dS tk
r` } ztddt|iW Y dd}~X nX dS )zF
**inputs**: **attention_mask**: **tokens_type_ids**:
r )r3 Z attention)r3 i rX N)lenr2 rG r[ r r, )rQ inputsr3 r\ r r r rO s
zServeCommand.forward)r% r& r' staticmethodr r? r r, r. rR r rL r boolrM r rN rO r r r r r \ s 2&
"
r N)'argparser r typingr r r Z pipelinesr r r
utilsr rW r
Zfastapir r r Zfastapi.routingr Zpydanticr Zstarlette.responsesr Zuvicornr rH ImportErrorAttributeErrorobject
get_loggerrJ r# r$ r+ r0 r2 r r r r r
s,