/usr/local/lib64/python3.6/site-packages/torch/cuda/__pycache__
Edit: /usr/local/lib64/python3.6/site-packages/torch/cuda/__pycache__/graphs.cpython-36.pyc (15811B)
3
EgI @ s d dl Z d dlZddlmZ eejdsHedejjd< edejjd< d dlmZ d dlm Z d d
Z
G d d d ejjZ G d
d de Z
dd ZdS ) N ) _dummy_typeZ_CudaStreamBase
_CUDAGraph_graph_pool_handle)r )r c C s t S )z
Returns an opaque token representing the id of a graph memory pool.
See :ref:`Graph memory management
`.
.. warning::
This API is in beta and may change in future releases.
)r r r =/usr/local/lib64/python3.6/site-packages/torch/cuda/graphs.pygraph_pool_handle s r c sj e Zd ZdZ fddZ fddZd fdd Z fd d
Z fd d Z fd
dZ fddZ
Z S ) CUDAGraphzw
Wrapper around a CUDA graph.
.. warning::
This API is in beta and may change in future releases.
c s t t| j| S )N)superr __new__)cls) __class__r r r $ s zCUDAGraph.__new__c s t t| j d S )N)r
r __init__)self)r
r r r ' s zCUDAGraph.__init__Nc s, |dkrt t| j nt t| j| dS )aX
Begins capturing CUDA work on the current stream.
Typically, you shouldn't call ``capture_begin`` yourself.
Use :class:`~torch.cuda.graph` or :func:`~torch.cuda.make_graphed_callables`,
which call ``capture_begin`` internally.
Arguments:
pool (optional): Token (returned by :func:`~torch.cuda.graph_pool_handle` or
:meth:`other_Graph_instance.pool()`) that hints this graph may share memory
with the indicated pool. See :ref:`Graph memory management`.
N)r
r
capture_begin)r pool)r
r r r * s zCUDAGraph.capture_beginc s t t| j dS )aP
Ends CUDA graph capture on the current stream.
After ``capture_end``, ``replay`` may be called on this instance.
Typically, you shouldn't call ``capture_end`` yourself.
Use :class:`~torch.cuda.graph` or :func:`~torch.cuda.make_graphed_callables`,
which call ``capture_end`` internally.
N)r
r capture_end)r )r
r r r > s zCUDAGraph.capture_endc s t t| j dS )z?
Replays the CUDA work captured by this graph.
N)r
r replay)r )r
r r r I s zCUDAGraph.replayc s t t| j dS )zD
Deletes the graph currently held by this instance.
N)r
r reset)r )r
r r r O s zCUDAGraph.resetc s t t| j S )z
Returns an opaque token representing the id of this graph's memory pool.
This id can optionally be passed to another graph's ``capture_begin``,
which hints the other graph may share the same memory pool.
)r
r r )r )r
r r r U s zCUDAGraph.pool)N) __name__
__module__ __qualname____doc__r r r r r r r
__classcell__r r )r
r r s r c @ s. e Zd ZdZdZd ddZdd Zdd ZdS )
grapha
Context-manager that captures CUDA work into a :class:`torch.cuda.CUDAGraph`
object for later replay.
See :ref:`CUDA Graphs ` for a general introduction,
detailed use, and constraints.
Arguments:
cuda_graph (torch.cuda.CUDAGraph): Graph object used for capture.
pool (optional): Opaque token (returned by a call to :func:`~torch.cuda.graph_pool_handle()` or
:meth:`other_Graph_instance.pool()`) hinting this graph's capture
may share memory from the specified pool. See :ref:`Graph memory management`.
stream (torch.cuda.Stream, optional): If supplied, will be set as the current stream in the context.
If not supplied, ``graph`` sets its own internal side stream as the current stream in the context.
.. note::
For effective memory sharing, if you pass a ``pool`` used by a previous capture and the previous capture
used an explicit ``stream`` argument, you should pass the same ``stream`` argument to this capture.
.. warning::
This API is in beta and may change in future releases.
Nc C sl | j jd krtjj | j _|d kr&f n|f| _|d k r:|n| j j| _| jd k sRttjj| j| _ || _
d S )N) r
default_capture_streamtorchcudaStreamr Zcapture_streamAssertionErrorstream
stream_ctx
cuda_graph)r r" r r r r r r w s zgraph.__init__c C s8 t jj tj t jj | jj | jj | j
d S )N) r r synchronizegcZcollectZ empty_cacher! __enter__r" r r )r r r r r% s
zgraph.__enter__c C s | j j | jj||| d S )N)r" r r! __exit__)r exc_type exc_value tracebackr r r r& s
zgraph.__exit__)NN)r r r r r r r% r& r r r r r ^ s
r c s& d}t | ts d}| f} fxt| D ]\}}t |tjjrt|jdkrjt|jdkrjt|j dksrt
d#t dd |j D st
d%t d d |D s*t
d&q*W dd D }dd | D fddt
t| D }dd t
t| D }dd t
t| D }t } tjj tjjtjj xt| |D ]v\}
}} xdt
dD ]X} |
| }
t |
tjrx|
fn|
}
tjj|
tdd | D tdd |
D ddd}qXW ~
~qDW W dQ R X tjj g }g }xtt| |D ]d\}
}}tjj|| d |
| }
W dQ R X t |
tjr6|jd |
f}
n
|jd |j|
qW g }g }xtt|t|t|t D ]\} }}}t dd |D st
dtd d |D }tjj|| d* tjj|td d | D |ddd}W dQ R X g }d}x8| D ]0}|jr$|j|| |d 7 }n
|jd q W t|}|j| |j| qxW tt|}tt|}d d }g } xt| D ]\} }
||| || | || || || || || || } t |
tjjr d!d" } | |
|
j | |
j |
_ | j|
n
| j| qW |r | d S t| S )'a
Accepts callables (functions or :class:`nn.Module`\ s)
and returns graphed versions.
Each graphed callable's forward pass runs its source callable's
forward CUDA work as a CUDA graph inside a single autograd node.
The graphed callable's forward pass also appends
a backward node to the autograd graph. During backward, this node runs the
callable's backward work as a CUDA graph.
Therefore, each graphed callable should be a drop-in replacement for its source callable
in an autograd-enabled training loop.
See :ref:`Partial-network capture` for detailed use and constraints.
If you pass a tuple of several callables, their captures will use the same memory pool.
See :ref:`Graph memory management` for when this is appropriate.
Arguments:
callables (torch.nn.Module or Python function, or tuple of these): Callable or callables to graph.
See :ref:`Graph memory management` for when passing a tuple of callables
is appropriate. If you pass a tuple of callables, their order in the tuple must be the same order
they'll run in the live workload.
sample_args (tuple of Tensors, or tuple of tuples of Tensors): Samples args for each callable.
If a single callable was passed, ``sample_args`` must be a single tuple of argument Tensors.
If a tuple of callables was passed, ``sample_args`` must be tuple of tuples of argument Tensors.
.. note::
The ``requires_grad`` state of each Tensor in ``sample_args`` must match the state
that's expected for the corresponding real input in the training loop.
.. warning::
This API is in beta and may change in future releases.
.. warning::
``sample_args`` for each callable must be a tuple of Tensors. Other types and keyword args
are not allowed.
.. warning::
Returned callables do not support higher order differentiation (e.g., double backward).
.. warning::
In any :class:`~torch.nn.Module` passed to :func:`~make_graphed_callables`, only parameters
may be trainable. Buffers must have ``requires_grad=False``.
.. warning::
After you pass a :class:`torch.nn.Module` through :func:`~make_graphed_callables`,
you may not add or remove any of that Module's parameters or buffers.
.. warning::
:class:`torch.nn.Module`\s passed to :func:`~torch.cuda.make_graphed_callables` must not have module hooks
registered on them at the time they are passed. However, registering hooks on modules *after* passing them
through :func:`~torch.cuda.make_graphed_callables` is allowed.
.. warning::
When running a graphed callable, you must pass its arguments in the same order and format
they appeared in that callable's ``sample_args``.
.. warning::
All Tensor outputs of graphed callables must require grad.
FTr z_Modules must not have hooks registered at the time they are passed. However, registering hooks zHon modules after passing them through make_graphed_callables is allowed.c s s | ]}|j d kV qdS )FN)
requires_grad).0br r r s z)make_graphed_callables..z+In any :class:`~torch.nn.Module` passed to zY:func:`~make_graphed_callables`, only parameters may be trainable. All buffers must have z``requires_grad=False``.c s s | ]}t |tjV qd S )N)
isinstancer Tensor)r+ argr r r r- s z In the beta API, sample_args z[for each callable must be a tuple of Tensors. Other types and keyword args are not allowed.c S s g | ] }t |qS r )len)r+ argsr r r
s z*make_graphed_callables..c S s* g | ]"}t |tjjr"t|j nf qS r )r. r nnModuletuple
parameters)r+ cr r r r3 s c s g | ]}| | qS r r )r+ i)per_callable_module_params sample_argsr r r3 s c S s g | ]}t jj qS r )r r r )r+ _r r r r3 s c S s g | ]}t jj qS r )r r r )r+ r< r r r r3 s c s s | ]}|j r|V qd S )N)r* )r+ r9 r r r r- s c s s | ]}t j|V qd S )N)r
empty_like)r+ or r r r- s )outputsinputsZ grad_outputsZ only_inputsZ allow_unusedN)r c s s | ] }|j V qd S )N)r* )r+ r? r r r r- , s z/Outputs of graphed callables must require grad.c s s | ]}t j|V qd S )N)r r> )r+ r? r r r r- - s c s s | ]}|j r|V qd S )N)r* )r+ r9 r r r r- 1 s r c
s8 G fdddt jj fdd} | S )Nc s@ e Zd ZefddZeejjj fddZ dS )zOmake_graphed_callables..make_graphed_autograd_function..Graphedc sd x:t D ].}| j || j kr
| j|| q
W j ttsRttdd D S )Nc s s | ]}|j V qd S )N)detach)r+ r? r r r r- ] s zjmake_graphed_callables..make_graphed_autograd_function..Graphed.forward..)rangedata_ptrcopy_r r. r6 r )ctxrA r9 ) fwd_graph
len_user_argsstatic_input_surfacestatic_outputsr r forwardU s zWmake_graphed_callables..make_graphed_autograd_function..Graphed.forwardc sp xFt |D ]8\}}|d kr*|d ksDtq |j |j kr |j| q W j tts^ttdd D S )Nc s s" | ]}|d k r|j n|V qd S )N)rB )r+ r, r r r r- n s zkmake_graphed_callables..make_graphed_autograd_function..Graphed.backward..)zipr rD rE r r. r6 )rF Zgradsggrad) bwd_graphstatic_grad_inputsstatic_grad_outputsr r backward_ s zXmake_graphed_callables..make_graphed_autograd_function..Graphed.backwardN)
r r r staticmethodrK r autogradfunctionZonce_differentiablerR r )rO rG rH rP rQ rI rJ r r GraphedT s
rV c s j | }r|d S |S )Nr )apply) user_argsout)rV
module_paramsoutput_was_tensorr r functionalizedp s zVmake_graphed_callables..make_graphed_autograd_function..functionalized)r rT ZFunction)
rG rO rZ rH r[ rI rJ rQ rP r\ r )
rV rO rG rH rZ r[ rP rQ rI rJ r make_graphed_autograd_functionK s $ z>make_graphed_callables..make_graphed_autograd_functionc s fdd}|S )Nc s j kr| S | S d S )N)training)rX )funcgraph_training_stategraphedorig_fwdr r new_fwd s
zEmake_graphed_callables..make_graphed_forward..new_fwdr )r_ r` ra rb rc r )r_ r` ra rb r make_graphed_forward s z4make_graphed_callables..make_graphed_forwardzModules must not have hooks registered at the time they are passed. However, registering hooks on modules after passing them through make_graphed_callables is allowed.zIn any :class:`~torch.nn.Module` passed to :func:`~make_graphed_callables`, only parameters may be trainable. All buffers must have zIn any :class:`~torch.nn.Module` passed to :func:`~make_graphed_callables`, only parameters may be trainable. All buffers must have ``requires_grad=False``.zxIn the beta API, sample_args for each callable must be a tuple of Tensors. Other types and keyword args are not allowed.) r. r6 rL r r4 r5 r1 Z_backward_hooksZ_forward_hooksZ_forward_pre_hooksr allbuffersrC r r r# r r r/ rT rN r appendreversedr* list enumerater^ rK ) Z callablesr; Zjust_one_callabler8 r2 Zper_callable_len_user_argsZ"per_callable_static_input_surfacesZ
fwd_graphsZ
bwd_graphsZmempoolr_ rI r< r@ Z grad_inputsZper_callable_static_outputsZ per_callable_output_was_tensorrG Z per_callable_static_grad_outputsZ per_callable_static_grad_inputsrJ rO rZ rQ rP Zgrad_idxr0 r] retr9 ra rd r )r: r; r make_graphed_callables s ?
,
/
rl )r$ r _utilsr hasattr_C__dict__Ztorch._Cr r r r objectr rl r r r r s A<