/usr/local/lib64/python3.6/site-packages/torch/ao/quantization/__pycache__
Edit: /usr/local/lib64/python3.6/site-packages/torch/ao/quantization/__pycache__/quantize.cpython-36.pyc (18901B)
3
Eg^ @ sJ d dl Z d dlZd dlZd dlZd dljZd dljjZd dl m
Z
d dl m Z m
Z
mZmZmZmZmZ d dlmZmZ d dlmZmZmZmZ dd Zd/d d
Z d0d d Z d
d Z dd Z d1ddZ dd Z!dd Z"d2ddZ#dd Z$d d Z%d3d d Z&dej'ddfd d!Z(d4d"d#Z)d5d$d%Z*d6d'd(Z+d7d)d*Z,d+d, Z-d8d-d.Z.dS )9 N) _FusedModule))get_default_dynamic_quant_module_mappings(get_default_static_quant_module_mappings get_default_qat_module_mappings$get_default_qconfig_propagation_listno_observer_set _has_special_act_post_process _get_special_act_post_process) DeQuantStub QuantWrapper) add_module_to_qconfig_obs_ctrdefault_dynamic_qconfigfloat16_dynamic_qconfig!float_qparams_weight_only_qconfigc C s t | tjjpt | tjjS )N)
isinstancetorch quantizationZ ObserverBaseZFakeQuantizeBase)module r J/usr/local/lib64/python3.6/site-packages/torch/ao/quantization/quantize.pyis_activation_post_process s r c
C s |dkrt }|jt| |}|j||}t| d|}tjjj|| t|| }|| _x8| j D ],\}}|rx|d | n|} t
||||| q`W dS )ay This is a helper function for `propagate_qconfig_`
Args:
module: input module
qconfig_dict: dictionary that maps from name of submodule to quantization
configuration
allow_list: list of quantizable modules
qconfig_parent: quantization config of parent module, we will fallback to
this config when there is no specified config for current
module
prefix: corresponding prefix of the current module, used as key in
qconfig_dict
Return:
None, module is modified inplace with qconfig attached
Nqconfig.) r gettypegetattrr r r Zassert_valid_qconfigr named_children_propagate_qconfig_helper)
r qconfig_dict
allow_listZqconfig_parentprefixZmodule_qconfigZqconfig_with_device_checknamechild
module_prefixr r r r s
r c C s |dkr i }t | || dS )a Propagate qconfig through the module hierarchy and assign `qconfig`
attribute on each leaf module
Args:
module: input module
qconfig_dict: dictionary that maps from name or type of submodule to
quantization configuration, qconfig applies to all submodules of a
given module unless qconfig for the submodules are specified (when
the submodule already has qconfig attribute)
Return:
None, module is modified inplace with qconfig attached
N)r )r r r r r r propagate_qconfig_D s r% c C s
| j |S )z3Forward hook that calls observer on the output
)activation_post_process)selfinputoutputr r r _observer_forward_hookV s r* c C s t | dstd| jtS )Nr& zFExpect activation_post_process attribut already attached to the module)hasattrAssertionErrorregister_forward_hookr* )r r r r %register_activation_post_process_hook[ s r. c s |dkrt }|dkri } dkr`t| }t|dksDtdj|t|dkr\tt|nd d dddd d fd d
}x
| j D ]\}}t|t j
t j gkrĈ|r|j |_
qt|tr|r|| qt| rt|} ||| q|dk r*t||kr*|r|| q|r~t||kr~|t| j|}
t| ||
|t| t kr||
qt||| | qW t| jdkrt| tjj rt| |kr|| dS )
a Add observer for the leaf child of the module.
This function insert observer module to all leaf child module that
has a valid qconfig attribute.
Args:
module: input module with qconfig attributes for all the leaf modules that we want to quantize
device: parent device, if any
non_leaf_module_list: list of non-leaf modules we want to add observer
Return:
None, module is modified inplace with added observer modules and forward_hooks
N zSadd_observer_ only works with cpu or single-device CUDA modules, but got devices {}r c S s, |d kr| j n| }|d k r(|j| |S )N)
activationto)r devicespecial_act_post_processr0 r r r get_activation_post_process} s
z2add_observer_.
.get_activation_post_processc S s t | do| jd k S )Nr )r+ r )mr r r needs_observation s z(add_observer_..needs_observationc sH | rDt | t rD| jd| j | t| }| jj|jdd dS )zg Adds an activation post process module and register
a post hook that calls the module
r& F)lastN)r r
Z
add_moduler r. _forward_hooks move_to_endid)r5 r3 handle)r2 r4 r6 r r insert_activation_post_process s
z5add_observer_..insert_activation_post_process)N)N)r get_unique_devices_lenr, formatnextiterr r nnqZFloatFunctionalZ QFunctionalr r& r r r r
from_floatsetattrr
add_observer__modulesr nnZ
Sequential) r qconfig_propagation_listZnon_leaf_module_listr2 custom_module_class_mappingdevicesr< r" r# r3 Zobserved_childr )r2 r4 r6 r rE ` sF
"rE c C s$ dd | j D dd | j D B S )Nc S s h | ]
}|j qS r )r2 ).0pr r r s z&get_unique_devices_..c S s h | ]
}|j qS r )r2 )rK rL r r r rM s )
parametersbuffers)r r r r r= s r= c C sN t | jdkr&t| dr&| jr&t| S x"| j D ]\}}t|| j|< q0W | S )a{ Wrap the leaf child module in QuantWrapper if it has a valid qconfig
Note that this function will modify the children of module inplace and it
can return a new module which wraps the input module as well.
Args:
module: input module with qconfig attributes for all the leaf modules
that we want to quantize
Return:
Either the inplace modified module with submodules wrapped in
`QuantWrapper` based on qconfig or a new `QuantWrapper` module which
wraps the input module, the latter case only happens when the input
module is a leaf module and we want to quantize it.
r r )r> rF r+ r r r add_quant_dequant)r r" r# r r r rP s
rP Fc C s t jjd |dkri }|jdi }|s2tj| } |}|dkrDt }t| dd tdd | j D spt
j d t | |||d | S ) a Prepares a copy of the model for quantization calibration or quantization-aware training.
Quantization configuration should be assigned preemptively
to individual submodules in `.qconfig` attribute.
The model will be attached with observer or fake quant modules, and qconfig
will be propagated.
Args:
`model`: input model to be modified in-place
`inplace`: carry out model transformations in-place, the original module is mutated
`allow_list`: list of quantizable modules
`observer_non_leaf_module_list`: list of non-leaf modules we want to add observer
`prepare_custom_config_dict`: customization configuration dictionary for prepare function
.. code-block:: python
# Example of prepare_custom_config_dict:
prepare_custom_config_dict = {
# user will manually define the corresponding observed
# module class which has a from_float class method that converts
# float custom module to observed custom module
"float_to_observed_custom_module_class": {
CustomModule: ObservedCustomModule
}
}
z!quantization_api.quantize.prepareNZ%float_to_observed_custom_module_class)r c s s | ]}t |d o|jV qdS )r N)r+ r )rK r5 r r r s zprepare..zNone of the submodule got qconfig applied. Make sure you passed correct configuration through `qconfig_dict` or by assigning the `.qconfig` attribute directly on submodules)rI )
r _C_log_api_usage_oncer copydeepcopyr r% anymoduleswarningswarnrE )modelinplacer observer_non_leaf_module_listZprepare_custom_config_dictrI rH r r r prepare s
r] c C sl t | dr t| jr t| d t }x(| jj D ]\}}|tkr0|j| q0W x|D ]}| jj | qTW d S )Nr& )
r+ r r& delattrsetr8 itemsr* addpop)r Zhandle_ids_to_removeZ handle_idhook_fnr r r _remove_activation_post_process s
rd c C s4 x| j D ] }t| q
W t| dr(| `t| dS )zClean up the qconfig left in the module so that new qconfig can be
propagated.
Args:
module: module to be cleaned up
r N)children_remove_qconfigr+ r rd )r r# r r r rf s
rf c C s\ t jjd |dkrt }|s(tj| } | j t| dd || f| t| |dd | S )a Quantize the input float model with post training static quantization.
First it will prepare the model for calibration, then it calls
`run_fn` which will run the calibration step, after that we will
convert the model to a quantized model.
Args:
model: input float model
run_fn: a calibration function for calibrating the prepared model
run_args: positional arguments for `run_fn`
inplace: carry out model transformations in-place, the original module is mutated
mapping: correspondence between original module types and quantized counterparts
Return:
Quantized model.
z"quantization_api.quantize.quantizeNT)r[ ) r rR rS r rT rU evalr] convert)rZ run_fnrun_argsmappingr[ r r r quantize# s
rl c C sB t jjd |dkr|t jkrHtjttjttjttj ttj
ttj ti}q|t j kr|tjt
tjt
tjt
tj t
tj
t
tj t
i}q|t jkrtjti}qtdj|n\t|tr|t jkrt}n.|t j krt
}n |t jkrt}ntdt|tt|tj|}|dkrt }|s t j | } | j t | | t | |dd | S )a} Converts a float model to dynamic (i.e. weights-only) quantized model.
Replaces specified modules with dynamic weight-only quantized versions and output the quantized model.
For simplest usage provide `dtype` argument that can be float16 or qint8. Weight-only quantization
by default is performed for layers with large weights size - i.e. Linear and RNN variants.
Fine grained control is possible with `qconfig` and `mapping` that act similarly to `quantize()`.
If `qconfig` is provided, the `dtype` argument is ignored.
Args:
model: input model
qconfig_spec: Either:
- A dictionary that maps from name or type of submodule to quantization
configuration, qconfig applies to all submodules of a given
module unless qconfig for the submodules are specified (when the
submodule already has qconfig attribute). Entries in the dictionary
need to be QConfigDynamic instances.
- A set of types and/or submodule names to apply dynamic quantization to,
in which case the `dtype` argument is used to specify the bit-width
inplace: carry out model transformations in-place, the original module is mutated
mapping: maps type of a submodule to a type of corresponding dynamically quantized version
with which the submodule needs to be replaced
z*quantization_api.quantize.quantize_dynamicNzTDon't know how to quantize with default settings for {}. Provide full qconfig pleasez.Unknown dtype specified for quantize_dynamic: T)r[ )!r rR rS qint8rG ZLinearr
ZLSTMZGRUZLSTMCellZRNNCellZGRUCellZfloat16r Zquint8Z EmbeddingBagr
ValueErrorr? r r_ RuntimeErrorstrdictzip itertoolsrepeatr rT rU rg r% rh )rZ Z qconfig_specZdtyperk r[ Zdefault_qconfigr r r quantize_dynamic? sJ
ru c C s^ t jjd |dkrt }|s(tj| } t| dd t| |ddd t| t |j
dd | S )a
Prepares a copy of the model for quantization calibration or
quantization-aware training and converts it to quantized version.
Quantization configuration should be assigned preemptively
to individual submodules in `.qconfig` attribute.
Args:
model: input model to be modified in-place
mapping: dictionary that maps float modules to quantized modules to be
replaced.
inplace: carry out model transformations in-place, the original module
is mutated
z%quantization_api.quantize.prepare_qatN)r TF)rk r[ remove_qconfig)r\ r[ ) r rR rS r rT rU r% rh r] r_ values)rZ rk r[ r r r prepare_qat s
rx c C sL t jjd |stj| } | j t| dd || f| t| dd | S )ag Do quantization aware training and output a quantized model
Args:
model: input model
run_fn: a function for evaluating the prepared model, can be a
function that simply runs the prepared model or a training
loop
run_args: positional arguments for `run_fn`
Return:
Quantized model.
z&quantization_api.quantize.quantize_qatT)r[ )r rR rS rT rU Ztrainrx rh )rZ ri rj r[ r r r quantize_qat s
ry Tc C s: t jjd |stj| } t| |d|d |r6t| | S )ag Converts submodules in input module to a different module according to `mapping`
by calling `from_float` method on the target module class. And remove qconfig at the
end if remove_qconfig is set to True.
Args:
`module`: prepared and calibrated module
`mapping`: a dictionary that maps from source module type to target
module type, can be overwritten to allow swapping user defined
Modules
`inplace`: carry out model transformations in-place, the original module
is mutated
`convert_custom_config_dict`: custom configuration dictionary for convert function
.. code-block:: python
# Example of convert_custom_config_dict:
convert_custom_config_dict = {
# user will manually define the corresponding quantized
# module class which has a from_observed class method that converts
# observed custom module to quantized custom module
"observed_to_quantized_custom_module_class": {
ObservedCustomModule: QuantizedCustomModule
}
}
z!quantization_api.quantize.convertT)r[ convert_custom_config_dict)r rR rS rT rU _convertrf )r rk r[ rv rz r r r rh s
rh c
C s |dkrt }|dkri }|jdi }|s4tj| } i }xJ| j D ]>\}}t|t rpt||krpt||d| t |||||< qBW x |j
D ]\}} | | j |< qW | S )a Converts submodules in input module to a different module according to `mapping`
by calling `from_float` method on the target module class
Args:
module: input module
mapping: a dictionary that maps from source module type to target
module type, can be overwritten to allow swapping user defined
Modules
inplace: carry out model transformations in-place, the original module
is mutated
NZ)observed_to_quantized_custom_module_classT) r r rT rU r r r r r{ swap_moduler` rF )
r rk r[ rz rI Zreassignr" modkeyvaluer r r r{ s"
r{ c C s | }t | dr| jdk rd}t| |kr@|t| j| }d}n"t| |krb|t| j| }d}|rx | jj D ]}|j| qrW x$| jj D ]}|t k r|j
| qW t | }t |dkst
dj|t |dkrtt|nd}|r|j| |S )a Swaps the module if it has a quantized counterpart and it has an
`observer` attached.
Args:
mod: input module
mapping: a dictionary that maps from nn module to nnq module
Return:
The corresponding quantized module of `mod`
r NFTr/ zQswap_module only works with cpu or single-device CUDA modules, but got devices {}r )r+ r r Z
from_observedrC Z_forward_pre_hooksrw Zregister_forward_pre_hookr8 r* r- r= r> r, r? r@ rA r1 ) r} rk rI new_modZswappedZ pre_hook_fnrc rJ r2 r r r r| s.
r| c C s^ dd }t | dr$| j|||d < x4| j D ](\}}|rF||| n|}t||| q.W dS )a, Traverse the modules and save all observers into dict.
This is mainly used for quantization accuracy debug
Args:
mod: the top module we want to save all observers
prefix: the prefix for the current module
target_dict: the dictionary used to save all the observers
c S s | dkr | S | d S )Nr r r )r! r r r
get_prefix= s z%get_observer_dict..get_prefixr& N)r+ r& r get_observer_dict)r} target_dictr! r r" r# r$ r r r r 5 s
r )NNr )NN)NNNN)FNNN)NF)NF)F)NFTN)NFN)r )/rT rs rX r Ztorch.nnrG Ztorch.nn.quantizedZ quantizedrB Ztorch.nn.intrinsicr Z(torch.quantization.quantization_mappingsr r r r r r r Ztorch.ao.quantization.stubsr
r Ztorch.quantization.qconfigr r
r r r r r% r* r. rE r= rP r] rd rf rl rm ru rx ry rh r{ r| r r r r r sD
$
$
R
6
M
&
$*