/usr/local/lib64/python3.6/site-packages/torch/cuda/__pycache__
Edit: /usr/local/lib64/python3.6/site-packages/torch/cuda/__pycache__/memory.cpython-36.pyc (21717B)
3
EgGX @ sV d dl Z d dlZd dlZd dlmZmZmZ d dlZddlm Z m
Z
m Z d dl m
Z
dd Zejdd Zd7ee
ef d
d d Zd
d Zd8ee
ef ddddZddddZd9ee
ef eeef dddZd:ee
ef eeef dddZd;ee
ef ddddZd
ee
ef ddd d Z d?ee
ef edd!d"Z d@ee
ef edd#d$Z dAee
ef edd%d&Z dBee
ef edd'd(Z dCee
ef edd)d*Z!dDee
ef edd+d,Z"d-d. Z#dEee
ef e$ed0d1d2Z%dFee
ef edd3d4Z&dGee
ef edd5d6Z'dS )H N)AnyDictUnion )is_initialized_get_device_index
_lazy_init)Devicec C s t tjj S )N)r torch_CZ_cuda_cudaHostAllocator r r =/usr/local/lib64/python3.6/site-packages/torch/cuda/memory.py_host_allocator
s r c
c s& t jj z
d V W d t jj X d S )N)r
r Z_cuda_lock_mutexZ_cuda_unlock_mutexr r r r
_free_mutex s
r )devicec
C s| |dkrt jj }t|}|dkr.t jj|}t|t jjjrD|j}t|t sVt
dt jj | t j j
| |S Q R X dS )a Performs a memory allocation using the CUDA memory allocator.
Memory is allocated for a given device and a stream, this
function is intended to be used for interoperability with other
frameworks. Allocated memory is released through
:func:`~torch.cuda.caching_allocator_delete`.
Args:
size (int): number of bytes to be allocated.
device (torch.device or int, optional): selected device. If it is
``None`` the default CUDA device is used.
stream (torch.cuda.Stream or int, optional): selected stream. If is ``None`` then
the default stream for the selected device is used.
.. note::
See :ref:`cuda-memory-management` for more details about GPU memory
management.
NzsInvalid type for stream argument, must be `torch.cuda.Stream` or `int` representing a pointer to a exisiting stream)r
cudacurrent_devicer Zcurrent_stream
isinstancestreamsZStreamZ cuda_streamint TypeErrorr r Z$_cuda_cudaCachingAllocator_raw_alloc)sizer streamr r r
caching_allocator_alloc s
r c C s t jj| dS )a Deletes memory allocated using the CUDA memory allocator.
Memory allocated with :func:`~torch.cuda.caching_allocator_alloc`.
is freed here. The associated device and stream are tracked inside
the allocator.
Args:
mem_ptr (int): memory address to be freed by the allocator.
.. note::
See :ref:`cuda-memory-management` for more details about GPU memory
management.
N)r
r Z%_cuda_cudaCachingAllocator_raw_delete)Zmem_ptrr r r
caching_allocator_delete: s r )r returnc C sb t |dkrtjj }t|}t| ts2td| dk sB| dkrPtdj | tj
j | | dS )a Set memory fraction for a process.
The fraction is used to limit an caching allocator to allocated memory on a CUDA device.
The allowed value equals the total visible memory multiplied fraction.
If trying to allocate more than the allowed value in a process, will raise an out of
memory error in allocator.
Args:
fraction(float): Range: 0~1. Allowed memory equals total_memory * fraction.
device (torch.device or int, optional): selected device. If it is
``None`` the default CUDA device is used.
.. note::
In general, the total available free memory is less than the total capacity.
Nz3Invalid type for fraction argument, must be `float`r r z.Invalid fraction value: {}. Allowed range: 0~1) r r
r r r r floatr
ValueErrorformatr Z_cuda_setMemoryFraction)fractionr r r r
set_per_process_memory_fractionK s
r )r c C s t rtjj dS )a Releases all unoccupied cached memory currently held by the caching
allocator so that those can be used in other GPU application and visible in
`nvidia-smi`.
.. note::
:func:`~torch.cuda.empty_cache` doesn't increase the amount of GPU
memory available for PyTorch. However, it may help reduce fragmentation
of GPU memory in certain cases. See :ref:`cuda-memory-management` for
more details about GPU memory management.
N)r r
r Z_cuda_emptyCacher r r r
empty_cachef s r! c s8 g fdd t | d} d| j tjS )aX Returns a dictionary of CUDA memory allocator statistics for a
given device.
The return value of this function is a dictionary of statistics, each of
which is a non-negative integer.
Core statistics:
- ``"allocated.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``:
number of allocation requests received by the memory allocator.
- ``"allocated_bytes.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``:
amount of allocated memory.
- ``"segment.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``:
number of reserved segments from ``cudaMalloc()``.
- ``"reserved_bytes.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``:
amount of reserved memory.
- ``"active.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``:
number of active memory blocks.
- ``"active_bytes.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``:
amount of active memory.
- ``"inactive_split.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``:
number of inactive, non-releasable memory blocks.
- ``"inactive_split_bytes.{all,large_pool,small_pool}.{current,peak,allocated,freed}"``:
amount of inactive, non-releasable memory.
For these core statistics, values are broken down as follows.
Pool type:
- ``all``: combined statistics across all memory pools.
- ``large_pool``: statistics for the large allocation pool
(as of October 2019, for size >= 1MB allocations).
- ``small_pool``: statistics for the small allocation pool
(as of October 2019, for size < 1MB allocations).
Metric type:
- ``current``: current value of this metric.
- ``peak``: maximum value of this metric.
- ``allocated``: historical total increase in this metric.
- ``freed``: historical total decrease in this metric.
In addition to the core statistics, we also provide some simple event
counters:
- ``"num_alloc_retries"``: number of failed ``cudaMalloc`` calls that
result in a cache flush and retry.
- ``"num_ooms"``: number of out-of-memory errors thrown.
The caching allocator can be configured via ENV to not split blocks larger than a
defined size (see Memory Management section of the Cuda Semantics documentation).
This helps avoid memory framentation but may have a performance
penalty. Additional outputs to assist with tuning and evaluating impact:
- ``"max_split_size"``: blocks above this size will not be split.
- ``"oversize_allocations.{current,peak,allocated,freed}"``:
number of over-size allocation requests received by the memory allocator.
- ``"oversize_segments.{current,peak,allocated,freed}"``:
number of over-size reserved segments from ``cudaMalloc()``.
Args:
device (torch.device or int, optional): selected device. Returns
statistics for the current device, given by :func:`~torch.cuda.current_device`,
if :attr:`device` is ``None`` (default).
.. note::
See :ref:`cuda-memory-management` for more details about GPU memory
management.
c sV t |trDt| dkr | d7 } x2|j D ]\}} | | | q(W nj| |f d S )Nr .)r dictlenitemsappend)prefixobjkv)_recurse_add_to_resultresultr r
r+ s
z,memory_stats.._recurse_add_to_result)r )memory_stats_as_nested_dictsort collections OrderedDict)r statsr )r+ r, r
memory_statsu s F
r3 c C s" t s
i S t| dd} tjj| S )zNReturns the result of :func:`~torch.cuda.memory_stats` as a nested dictionary.T)optional)r r r
r Z_cuda_memoryStats)r r r r
r. s r. c C s t | dd} tjj| S )a~ Resets the "accumulated" (historical) stats tracked by the CUDA memory allocator.
See :func:`~torch.cuda.memory_stats` for details. Accumulated stats correspond to
the `"allocated"` and `"freed"` keys in each individual stat dict, as well as
`"num_alloc_retries"` and `"num_ooms"`.
Args:
device (torch.device or int, optional): selected device. Returns
statistic for the current device, given by :func:`~torch.cuda.current_device`,
if :attr:`device` is ``None`` (default).
.. note::
See :ref:`cuda-memory-management` for more details about GPU memory
management.
T)r4 )r r
r Z!_cuda_resetAccumulatedMemoryStats)r r r r
reset_accumulated_memory_stats s r5 c C s t | dd} tjj| S )a Resets the "peak" stats tracked by the CUDA memory allocator.
See :func:`~torch.cuda.memory_stats` for details. Peak stats correspond to the
`"peak"` key in each individual stat dict.
Args:
device (torch.device or int, optional): selected device. Returns
statistic for the current device, given by :func:`~torch.cuda.current_device`,
if :attr:`device` is ``None`` (default).
.. note::
See :ref:`cuda-memory-management` for more details about GPU memory
management.
T)r4 )r r
r Z_cuda_resetPeakMemoryStats)r r r r
reset_peak_memory_stats s r6 c C s t jdt t| dS )a Resets the starting point in tracking maximum GPU memory occupied by
tensors for a given device.
See :func:`~torch.cuda.max_memory_allocated` for details.
Args:
device (torch.device or int, optional): selected device. Returns
statistic for the current device, given by :func:`~torch.cuda.current_device`,
if :attr:`device` is ``None`` (default).
.. warning::
This function now calls :func:`~torch.cuda.reset_peak_memory_stats`, which resets
/all/ peak memory stats.
.. note::
See :ref:`cuda-memory-management` for more details about GPU memory
management.
zytorch.cuda.reset_max_memory_allocated now calls torch.cuda.reset_peak_memory_stats, which resets /all/ peak memory stats.)r )warningswarn
FutureWarningr6 )r r r r
reset_max_memory_allocated s r: c C s t jdt t| dS )a Resets the starting point in tracking maximum GPU memory managed by the
caching allocator for a given device.
See :func:`~torch.cuda.max_memory_cached` for details.
Args:
device (torch.device or int, optional): selected device. Returns
statistic for the current device, given by :func:`~torch.cuda.current_device`,
if :attr:`device` is ``None`` (default).
.. warning::
This function now calls :func:`~torch.cuda.reset_peak_memory_stats`, which resets
/all/ peak memory stats.
.. note::
See :ref:`cuda-memory-management` for more details about GPU memory
management.
zvtorch.cuda.reset_max_memory_cached now calls torch.cuda.reset_peak_memory_stats, which resets /all/ peak memory stats.)r )r7 r8 r9 r6 )r r r r
reset_max_memory_cached s r; c C s t | djddS )a` Returns the current GPU memory occupied by tensors in bytes for a given
device.
Args:
device (torch.device or int, optional): selected device. Returns
statistic for the current device, given by :func:`~torch.cuda.current_device`,
if :attr:`device` is ``None`` (default).
.. note::
This is likely less than the amount shown in `nvidia-smi` since some
unused memory can be held by the caching allocator and some context
needs to be created on GPU. See :ref:`cuda-memory-management` for more
details about GPU memory management.
)r zallocated_bytes.all.currentr )r3 get)r r r r
memory_allocated0 s r= c C s t | djddS )a Returns the maximum GPU memory occupied by tensors in bytes for a given
device.
By default, this returns the peak allocated memory since the beginning of
this program. :func:`~torch.cuda.reset_peak_memory_stats` can be used to
reset the starting point in tracking this metric. For example, these two
functions can measure the peak allocated memory usage of each iteration in a
training loop.
Args:
device (torch.device or int, optional): selected device. Returns
statistic for the current device, given by :func:`~torch.cuda.current_device`,
if :attr:`device` is ``None`` (default).
.. note::
See :ref:`cuda-memory-management` for more details about GPU memory
management.
)r zallocated_bytes.all.peakr )r3 r< )r r r r
max_memory_allocatedB s r> c C s t | djddS )a Returns the current GPU memory managed by the caching allocator in bytes
for a given device.
Args:
device (torch.device or int, optional): selected device. Returns
statistic for the current device, given by :func:`~torch.cuda.current_device`,
if :attr:`device` is ``None`` (default).
.. note::
See :ref:`cuda-memory-management` for more details about GPU memory
management.
)r zreserved_bytes.all.currentr )r3 r< )r r r r
memory_reservedX s
r? c C s t | djddS )a Returns the maximum GPU memory managed by the caching allocator in bytes
for a given device.
By default, this returns the peak cached memory since the beginning of this
program. :func:`~torch.cuda.reset_peak_memory_stats` can be used to reset
the starting point in tracking this metric. For example, these two functions
can measure the peak cached memory amount of each iteration in a training
loop.
Args:
device (torch.device or int, optional): selected device. Returns
statistic for the current device, given by :func:`~torch.cuda.current_device`,
if :attr:`device` is ``None`` (default).
.. note::
See :ref:`cuda-memory-management` for more details about GPU memory
management.
)r zreserved_bytes.all.peakr )r3 r< )r r r r
max_memory_reservedh s r@ c C s t jdt t| dS )z4Deprecated; see :func:`~torch.cuda.memory_reserved`.zGtorch.cuda.memory_cached has been renamed to torch.cuda.memory_reserved)r )r7 r8 r9 r? )r r r r
memory_cached~ s rA c C s t jdt t| dS )z8Deprecated; see :func:`~torch.cuda.max_memory_reserved`.zOtorch.cuda.max_memory_cached has been renamed to torch.cuda.max_memory_reserved)r )r7 r8 r9 r@ )r r r r
max_memory_cached s rB c C s
t jj S )a) Returns a snapshot of the CUDA memory allocator state across all devices.
Interpreting the output of this function requires familiarity with the
memory allocator internals.
.. note::
See :ref:`cuda-memory-management` for more details about GPU memory
management.
)r
r Z_cuda_memorySnapshotr r r r
memory_snapshot s
rC F)r abbreviatedr c C s t | dd} t| d}dd }dd }dd |fd
d |fd d
|fdd|fdd|fdd|fdd|fdd|fg}g }|jdd |jd |jdd |jd |jdd |jd x|D ]\}}} |jdd d |fg}
| s|
jd3 |
jd4 d5\} } }
}x|
D ]\}}|d$ | d$ }||d% }||d& }||d' }||d( }| d#krn|} |} |}
|}|jd)j|| || | || | ||
| || q W qW d*d+|fd,d-|fg}x|D ]\}}} |jdd |d$ }||d% }||d& }||d' }||d( }|jd)j|| ||| ||| ||| || qW |jdd d.| d/}x&|j D ]\}}|||jd$d< qjW d0d1j|jf | d2 S )6a Returns a human-readable printout of the current memory allocator
statistics for a given device.
This can be useful to display periodically during training, or when
handling out-of-memory exceptions.
Args:
device (torch.device or int, optional): selected device. Returns
printout for the current device, given by :func:`~torch.cuda.current_device`,
if :attr:`device` is ``None`` (default).
abbreviated (bool, optional): whether to return an abbreviated summary
(default: False).
.. note::
See :ref:`cuda-memory-management` for more details about GPU memory
management.
T)r4 )r c S sX ddddddg}|d }x2|dd D ]"}|d k r4P |}| d
} |d
}q&W d j | |S )
NzB ZKBZMBZGBZTBZPBr r i i z{:7d} {}i )r )szZpref_szprefixesr'
new_prefixr r r
_format_size s z$memory_summary.._format_sizec S sR dddg}|d }x2|dd D ]"}|d k r.P |}| d } |d }q W dj | |S )
N KMr r i i z {:7d} {} iq )r )ZcntZpref_cntrF r' rG r r r
_format_count s
z%memory_summary.._format_countZallocated_byteszAllocated memoryZ active_bytesz
Active memoryZreserved_byteszGPU reserved memoryZinactive_split_byteszNon-releasable memoryZ
allocationZ Allocationsactivez
Active allocssegmentzGPU reserved segmentsZinactive_splitzNon-releasable allocs=K z= {_:16} PyTorch CUDA memory summary, device ID {device:<17d} -zX {_:9} CUDA OOMs: {num_ooms:<12d} | {_:6} cudaMalloc retries: {num_alloc_retries:<8d} zK Metric | Cur Usage | Peak Usage | Tot Alloc | Tot Freed all
large_pool from large pool
small_pool from small poolNr" currentpeak allocatedfreedz {:<21} | {} | {} | {} | {} Zoversize_allocationszOversize allocationsZoversize_segmentszOversize GPU segmentsr- )_r |z|
|z|
)rS rT )rU rV )NNNN)r r3 r& r r% replacejoin)r rD r2 rH rL Zmetrics_to_displaylinesZ
metric_keyZ metric_name formatterZ
submetricsZcurrent_prefvalZ peak_prefvalZallocated_prefvalZ
freed_prefvalZ
submetric_keyZsubmetric_namer' rW rX rY rZ Zfmt_dictr) r* r r r
memory_summary s~
ra c C s y ddl }W n tk
r dS X ddl m} y |j W n |k
rN dS X t| dd} |j| }|j|}g }|jd| t|dkr|jd x4|D ],}|j d }|jd |j
d d
|dd qW dj |S )a Returns a human-readable printout of the running processes
and their GPU memory use for a given device.
This can be useful to display periodically during training, or when
handling out-of-memory exceptions.
Args:
device (torch.device or int, optional): selected device. Returns
printout for the current device, given by :func:`~torch.cuda.current_device`,
if :attr:`device` is ``None`` (default).
r Nz.pynvml module not found, please install pynvml)NVMLError_DriverNotLoadedz-cuda driver can't be loaded, is cuda enabled?T)r4 zGPU:zno processes are runningi zprocess z>10dz uses z>12.3fz MB GPU memory
i ) pynvmlModuleNotFoundErrorrb ZnvmlInitr ZnvmlDeviceGetHandleByIndexZ$nvmlDeviceGetComputeRunningProcessesr& r$ Z
usedGpuMemorypidr^ )r rd rb handleZprocsr_ pZmemr r r
list_gpu_processes s(
"ri c C s* | dkrt jj } t| } t jj j| S )a Returns the global free and total GPU memory occupied for a given
device using cudaMemGetInfo.
Args:
device (torch.device or int, optional): selected device. Returns
statistic for the current device, given by :func:`~torch.cuda.current_device`,
if :attr:`device` is ``None`` (default).
.. note::
See :ref:`cuda-memory-management` for more
details about GPU memory management.
N)r
r r r ZcudartZcudaMemGetInfo)r r r r
mem_get_info9 s
rj )NN)N)N)N)N)N)N)N)N)N)N)N)N)N)NF)N)N)(r0
contextlibr7 typingr r r r
r- r r r Z torch.typesr r contextmanagerr r r r r r! strr3 r. r5 r6 r: r; r= r> r? r@ rA rB rC boolra ri rj r r r r
s8 ""X"
|"