/usr/local/lib64/python3.6/site-packages/torch/distributed/__pycache__
Edit: /usr/local/lib64/python3.6/site-packages/torch/distributed/__pycache__/rendezvous.cpython-36.pyc (7958B)
3
Eg!
@ s4 yd dl mZmZ W n ek
r0 edY nX d dlZd dlZd dlZd dlmZ d dl m
Z
m Z m Z d dl
jZd dlmZmZmZmZ ddlmZ i ad d
Zd"eeed d d
Zdd Z edddZ e dddZ edddZ efeedddZ!efeeddd Z"ed e! ed e" ed e dS )# )urlparse
urlunparsezEurllib cannot be found, urlparse from python2 is no longer supported.N) timedelta)DictOptionalUnion) FileStore PrefixStoreStoreTCPStore )default_pg_timeoutc C s" | t krtdj| |t | < dS )a Registers a new rendezvous handler.
Before we can run collective algorithms, participating processes
need to find each other and exchange information to be able to
communicate. We call this process rendezvous.
The outcome of the rendezvous process is a triplet containing a
shared key/value store, the rank of the process, and the total
number of participating processes.
If none of the bundled rendezvous methods apply to your execution
environment you can opt to register your own rendezvous handler.
Pick a unique name and use the URL scheme to identify it when
calling the `rendezvous()` function.
Args:
scheme (str): URL scheme to identify your rendezvous handler.
handler (function): Handler that is invoked when the
`rendezvous()` function is called with a URL that uses
the corresponding scheme. It must be a generator function
that yields the triplet.
z/Rendezvous handler for {}:// already registeredN)_rendezvous_handlers RuntimeErrorformat)schemehandler r H/usr/local/lib64/python3.6/site-packages/torch/distributed/rendezvous.pyregister_rendezvous_handler s r )urlrank
world_sizec K s* t | tjs tdjt| | t |tjs:tdj|t |tjsTtdj|t| }|dksn|dk rt dd t
d |j j dD }d|krd |kst
d
j| d |dkr||d< |dkr||d < |jd jdjd
d |j D d}t|} |jtkrtdj|jt|j | f|S )Nz `url` must be a string. {}: {}z `rank` must be an integer. {}z#`world_size` must be an integer. {}r c s s | ]}|j d V qdS )=N)split).0pairr r r
D s z rendezvous..&r r zEThe url: {url} has node-specific arguments(rank, world_size) already.)r z{}c S s g | ]\}}d j ||qS )z{}={})r )r kvr r r
R s z rendezvous..)queryz No rendezvous handler for {}://r# r# r# )
isinstancesixZstring_classesr r typenumbersIntegralr dictfilterr" r AssertionError_replacejoinitemsr r r )r r r kwargsresultZ
query_dictr r r
rendezvous6 s0
r1 c C s t d| S )Nz+Error initializing torch.distributed using )
ValueError)msgr r r _rendezvous_error\ s r4 )r c k s dd }t | }|j}tjdkrPdd l}|j|j }|jj|}|rPtjj |}|s\|dt
dd t d |j j
dD }d |kr|d
d |kr|d t|d }t|d } t|| }
|
|| fV td
d S )Nc S s t d| S )Nzfile:// rendezvous: )r4 )r3 r r r _errora s z(_file_rendezvous_handler.._errorwin32r z path missingc s s | ]}|j d V qdS )r N)r )r r r r r r s s z+_file_rendezvous_handler..r r zrank parameter missingr z world size parameter missingz3Unable to perform rerendezvous using file:// method)r pathsysplatformurllib.requestnetlocrequest url2pathnameosnormpathr) r* r" r intr r ) r r/ r5 r0 r7 urllib full_pathr" r r storer r r _file_rendezvous_handler` s*
rD )returnc C s t jjdd tdkS )NZ TORCHELASTIC_USE_AGENT_STORET)r> environgetstrr r r r _torchelastic_use_agent_store s rI c C sP t r0tjd }t| ||d|}td| |S |dk}t| ||||ddS dS )a
Smartly creates a c10d Store object on ``rank`` based on whether
we need to re-use agent store. The TCPStore server is assumed to be hosted
on ``hostname:port``.
If ``torchelastic_use_agent_store()`` is ``True``, then it is assumed that
the agent leader (node rank 0) hosts the TCPStore server (for which the
endpoint is specified by the given ``hostname:port``). Hence
ALL ranks will create and return a TCPStore client (e.g. ``start_daemon=False``).
If ``torchelastic_use_agent_store()`` is ``False``, then rank 0 will host
the TCPStore (with multi-tenancy) and it is assumed that rank 0's hostname
and port are correctly passed via ``hostname`` and ``port``. All
non-zero ranks will create and return a TCPStore client.
ZTORCHELASTIC_RESTART_COUNTFz/worker/attempt_r T)Z multi_tenantN)rI r> rF r r )hostnameportr r timeoutattemptZ tcp_storeZ start_daemonr r r _create_c10d_store s
rN )r rL c k s dd }t | }|js |dtdd td |jjdD }d|krN|dd |kr^|d
t|d }t|d }|jd k stt |j|j|||}|||fV t
d d S ) Nc S s t d| S )Nztcp:// rendezvous: )r4 )r3 r r r r5 s z'_tcp_rendezvous_handler.._errorzport number missingc s s | ]}|j d V qdS )r N)r )r r r r r r s z*_tcp_rendezvous_handler..r r zrank parameter missingr z world size parameter missingz3Unable to perform re-rendezvous using tcp:// method) r rK r) r* r" r r@ rJ r+ rN r ) r rL r/ r5 r0 r" r r rC r r r _tcp_rendezvous_handler s rO c + s dd fdd t t d fdd }t| }tdd td |jjd
D }d |krft|d }n t|d }d
|krt|d
}n t|d}|d}t|d} t|| |||}
|
||fV tdd S )Nc S s t d| S )Nzenv:// rendezvous: )r4 )r3 r r r r5 s z'_env_rendezvous_handler.._errorc s d| S )Nz-environment variable %s expected, but not setr )var)r5 r r
_env_error s z+_env_rendezvous_handler.._env_error)env_varrE c s$ t jj| d }|s | n|S d S )N)r> rF rG )rR Zenv_val)rQ r r _get_env_or_raise s
z2_env_rendezvous_handler.._get_env_or_raisec s s | ]}|j d V qdS )r N)r )r r r r r r s z*_env_rendezvous_handler..r r ZRANKr Z
WORLD_SIZEZ MASTER_ADDRZ MASTER_PORTz3Unable to perform re-rendezvous using env:// method) rH r r) r* r" r r@ rN r ) r rL r/ rS r0 r" r r Z master_addrZ master_portrC r )rQ r5 r _env_rendezvous_handler s rT Ztcpenvfiler# r# )r# r# )# urllib.parser r ImportErrorr' r> r8 datetimer typingr r r Z
torch._sixZ_sixr% Ztorch.distributedr r r
r constantsr
r r rH r@ r1 r4 rD boolrI rN rO rT r r r r s0
&" -