FM / library

/usr/share/doc/python2-docs/html/_sources/library/

/usr/share/doc/python2-docs/html/_sources/library

Name	Size	Mode	Actions
2to3.rst.txt	14647	0644	edit dl rm
abc.rst.txt	7219	0644	edit dl rm
aepack.rst.txt	4257	0644	edit dl rm
aetools.rst.txt	3532	0644	edit dl rm
aetypes.rst.txt	4257	0644	edit dl rm
aifc.rst.txt	7086	0644	edit dl rm
al.rst.txt	5309	0644	edit dl rm
allos.rst.txt	695	0644	edit dl rm
anydbm.rst.txt	4111	0644	edit dl rm
archiving.rst.txt	424	0644	edit dl rm
argparse.rst.txt	74685	0644	edit dl rm
array.rst.txt	10524	0644	edit dl rm
ast.rst.txt	10111	0644	edit dl rm
asynchat.rst.txt	9206	0644	edit dl rm
asyncore.rst.txt	12937	0644	edit dl rm
atexit.rst.txt	3910	0644	edit dl rm
audioop.rst.txt	10395	0644	edit dl rm
autogil.rst.txt	1015	0644	edit dl rm
base64.rst.txt	6252	0644	edit dl rm
basehttpserver.rst.txt	10404	0644	edit dl rm
bastion.rst.txt	2611	0644	edit dl rm
bdb.rst.txt	12456	0644	edit dl rm
binascii.rst.txt	6508	0644	edit dl rm
binhex.rst.txt	1910	0644	edit dl rm
bisect.rst.txt	5415	0644	edit dl rm
bsddb.rst.txt	7575	0644	edit dl rm
bz2.rst.txt	8064	0644	edit dl rm
calendar.rst.txt	11282	0644	edit dl rm
carbon.rst.txt	15956	0644	edit dl rm
cd.rst.txt	11974	0644	edit dl rm
cgi.rst.txt	22853	0644	edit dl rm
cgihttpserver.rst.txt	2788	0644	edit dl rm
cgitb.rst.txt	2874	0644	edit dl rm
chunk.rst.txt	4955	0644	edit dl rm
cmath.rst.txt	7647	0644	edit dl rm
cmd.rst.txt	8565	0644	edit dl rm
code.rst.txt	7115	0644	edit dl rm
codecs.rst.txt	66966	0644	edit dl rm
codeop.rst.txt	3774	0644	edit dl rm
collections.rst.txt	41448	0644	edit dl rm
colorpicker.rst.txt	913	0644	edit dl rm
colorsys.rst.txt	1819	0644	edit dl rm
commands.rst.txt	2595	0644	edit dl rm
compileall.rst.txt	4678	0644	edit dl rm
compiler.rst.txt	37464	0644	edit dl rm
configparser.rst.txt	19607	0644	edit dl rm
constants.rst.txt	2328	0644	edit dl rm
contextlib.rst.txt	6012	0644	edit dl rm
cookie.rst.txt	9543	0644	edit dl rm
cookielib.rst.txt	27875	0644	edit dl rm
copy.rst.txt	3351	0644	edit dl rm
copy_reg.rst.txt	2329	0644	edit dl rm
crypt.rst.txt	2292	0644	edit dl rm
crypto.rst.txt	355	0644	edit dl rm
csv.rst.txt	22757	0644	edit dl rm
ctypes.rst.txt	90513	0644	edit dl rm
curses.ascii.rst.txt	9046	0644	edit dl rm
curses.panel.rst.txt	2740	0644	edit dl rm
curses.rst.txt	74877	0644	edit dl rm
custominterp.rst.txt	570	0644	edit dl rm
datatypes.rst.txt	864	0644	edit dl rm
datetime.rst.txt	74679	0644	edit dl rm
dbhash.rst.txt	3865	0644	edit dl rm
dbm.rst.txt	3117	0644	edit dl rm
debug.rst.txt	446	0644	edit dl rm
decimal.rst.txt	70926	0644	edit dl rm
development.rst.txt	640	0644	edit dl rm
difflib.rst.txt	30716	0644	edit dl rm
dircache.rst.txt	1813	0644	edit dl rm
dis.rst.txt	23248	0644	edit dl rm
distribution.rst.txt	426	0644	edit dl rm
distutils.rst.txt	1958	0644	edit dl rm
dl.rst.txt	3392	0644	edit dl rm
doctest.rst.txt	73985	0644	edit dl rm
docxmlrpcserver.rst.txt	3802	0644	edit dl rm
dumbdbm.rst.txt	2841	0644	edit dl rm
dummy_thread.rst.txt	1058	0644	edit dl rm
dummy_threading.rst.txt	799	0644	edit dl rm
easydialogs.rst.txt	10346	0644	edit dl rm
email-examples.rst.txt	1271	0644	edit dl rm
email.charset.rst.txt	9654	0644	edit dl rm
email.encoders.rst.txt	2376	0644	edit dl rm
email.errors.rst.txt	4006	0644	edit dl rm
email.generator.rst.txt	6132	0644	edit dl rm
email.header.rst.txt	7531	0644	edit dl rm
email.iterators.rst.txt	2421	0644	edit dl rm
email.message.rst.txt	25217	0644	edit dl rm
email.mime.rst.txt	9917	0644	edit dl rm
email.parser.rst.txt	10327	0644	edit dl rm
email.rst.txt	16111	0644	edit dl rm
email.utils.rst.txt	6475	0644	edit dl rm
ensurepip.rst.txt	5022	0644	edit dl rm
errno.rst.txt	6708	0644	edit dl rm
exceptions.rst.txt	18343	0644	edit dl rm
fcntl.rst.txt	7361	0644	edit dl rm
filecmp.rst.txt	5348	0644	edit dl rm
fileformats.rst.txt	302	0644	edit dl rm
fileinput.rst.txt	7417	0644	edit dl rm
filesys.rst.txt	806	0644	edit dl rm
fl.rst.txt	17646	0644	edit dl rm
fm.rst.txt	2699	0644	edit dl rm
fnmatch.rst.txt	3104	0644	edit dl rm
formatter.rst.txt	13245	0644	edit dl rm
fpectl.rst.txt	4171	0644	edit dl rm
fpformat.rst.txt	1747	0644	edit dl rm
fractions.rst.txt	5297	0644	edit dl rm
framework.rst.txt	11444	0644	edit dl rm
frameworks.rst.txt	378	0644	edit dl rm
ftplib.rst.txt	15720	0644	edit dl rm
functions.rst.txt	75554	0644	edit dl rm
functools.rst.txt	7448	0644	edit dl rm
future_builtins.rst.txt	2010	0644	edit dl rm
gc.rst.txt	9014	0644	edit dl rm
gdbm.rst.txt	4882	0644	edit dl rm
gensuitemodule.rst.txt	3113	0644	edit dl rm
getopt.rst.txt	6670	0644	edit dl rm
getpass.rst.txt	1876	0644	edit dl rm
gettext.rst.txt	29040	0644	edit dl rm
gl.rst.txt	6009	0644	edit dl rm
glob.rst.txt	2414	0644	edit dl rm
grp.rst.txt	2256	0644	edit dl rm
gzip.rst.txt	4828	0644	edit dl rm
hashlib.rst.txt	7379	0644	edit dl rm
heapq.rst.txt	13185	0644	edit dl rm
hmac.rst.txt	3073	0644	edit dl rm
hotshot.rst.txt	4289	0644	edit dl rm
htmllib.rst.txt	7379	0644	edit dl rm
htmlparser.rst.txt	11640	0644	edit dl rm
httplib.rst.txt	37454	0644	edit dl rm
i18n.rst.txt	409	0644	edit dl rm
ic.rst.txt	5005	0644	edit dl rm
idle.rst.txt	22182	0644	edit dl rm
imageop.rst.txt	4001	0644	edit dl rm
imaplib.rst.txt	17221	0644	edit dl rm
imgfile.rst.txt	2765	0644	edit dl rm
imghdr.rst.txt	2635	0644	edit dl rm
imp.rst.txt	12593	0644	edit dl rm
importlib.rst.txt	1126	0644	edit dl rm
imputil.rst.txt	7023	0644	edit dl rm
index.rst.txt	2287	0644	edit dl rm
inspect.rst.txt	28151	0644	edit dl rm
internet.rst.txt	950	0644	edit dl rm
intro.rst.txt	2803	0644	edit dl rm
io.rst.txt	39015	0644	edit dl rm
ipc.rst.txt	630	0644	edit dl rm
itertools.rst.txt	36324	0644	edit dl rm
jpeg.rst.txt	3858	0644	edit dl rm
json.rst.txt	25545	0644	edit dl rm
keyword.rst.txt	617	0644	edit dl rm
language.rst.txt	523	0644	edit dl rm
linecache.rst.txt	1887	0644	edit dl rm
locale.rst.txt	24975	0644	edit dl rm
logging.config.rst.txt	31560	0644	edit dl rm
logging.handlers.rst.txt	28153	0644	edit dl rm
logging.rst.txt	46882	0644	edit dl rm
mac.rst.txt	791	0644	edit dl rm
macos.rst.txt	3824	0644	edit dl rm
macosa.rst.txt	3964	0644	edit dl rm
macostools.rst.txt	4017	0644	edit dl rm
macpath.rst.txt	650	0644	edit dl rm
mailbox.rst.txt	68129	0644	edit dl rm
mailcap.rst.txt	3675	0644	edit dl rm
markup.rst.txt	1249	0644	edit dl rm
marshal.rst.txt	5689	0644	edit dl rm
math.rst.txt	10920	0644	edit dl rm
md5.rst.txt	2815	0644	edit dl rm
mhlib.rst.txt	3966	0644	edit dl rm
mimetools.rst.txt	4504	0644	edit dl rm
mimetypes.rst.txt	9839	0644	edit dl rm
mimewriter.rst.txt	3360	0644	edit dl rm
mimify.rst.txt	3519	0644	edit dl rm
miniaeframe.rst.txt	2564	0644	edit dl rm
misc.rst.txt	248	0644	edit dl rm
mm.rst.txt	447	0644	edit dl rm
mmap.rst.txt	10481	0644	edit dl rm
modulefinder.rst.txt	3380	0644	edit dl rm
modules.rst.txt	382	0644	edit dl rm
msilib.rst.txt	19062	0644	edit dl rm
msvcrt.rst.txt	4346	0644	edit dl rm
multifile.rst.txt	6613	0644	edit dl rm
multiprocessing.rst.txt	92621	0644	edit dl rm
mutex.rst.txt	1939	0644	edit dl rm
netdata.rst.txt	432	0644	edit dl rm
netrc.rst.txt	3119	0644	edit dl rm
new.rst.txt	2653	0644	edit dl rm
nis.rst.txt	2108	0644	edit dl rm
nntplib.rst.txt	14531	0644	edit dl rm
numbers.rst.txt	8007	0644	edit dl rm
numeric.rst.txt	751	0644	edit dl rm
operator.rst.txt	22092	0644	edit dl rm
optparse.rst.txt	77101	0644	edit dl rm
os.path.rst.txt	13094	0644	edit dl rm
os.rst.txt	82580	0644	edit dl rm
ossaudiodev.rst.txt	17309	0644	edit dl rm
othergui.rst.txt	2721	0644	edit dl rm
parser.rst.txt	15385	0644	edit dl rm
pdb.rst.txt	16043	0644	edit dl rm
persistence.rst.txt	826	0644	edit dl rm
pickle.rst.txt	37262	0644	edit dl rm
pickletools.rst.txt	1997	0644	edit dl rm
pipes.rst.txt	3786	0644	edit dl rm
pkgutil.rst.txt	7714	0644	edit dl rm
platform.rst.txt	9554	0644	edit dl rm
plistlib.rst.txt	4134	0644	edit dl rm
popen2.rst.txt	7022	0644	edit dl rm
poplib.rst.txt	6220	0644	edit dl rm
posix.rst.txt	3630	0644	edit dl rm
posixfile.rst.txt	7200	0644	edit dl rm
pprint.rst.txt	9071	0644	edit dl rm
profile.rst.txt	28666	0644	edit dl rm
pty.rst.txt	1762	0644	edit dl rm
pwd.rst.txt	2725	0644	edit dl rm
pyclbr.rst.txt	3296	0644	edit dl rm
pydoc.rst.txt	4083	0644	edit dl rm
pyexpat.rst.txt	28891	0644	edit dl rm
python.rst.txt	514	0644	edit dl rm
py_compile.rst.txt	2479	0644	edit dl rm
queue.rst.txt	7034	0644	edit dl rm
quopri.rst.txt	2670	0644	edit dl rm
random.rst.txt	13319	0644	edit dl rm
re.rst.txt	55102	0644	edit dl rm
readline.rst.txt	10513	0644	edit dl rm
repr.rst.txt	4718	0644	edit dl rm
resource.rst.txt	9595	0644	edit dl rm
restricted.rst.txt	3327	0644	edit dl rm
rexec.rst.txt	11742	0644	edit dl rm
rfc822.rst.txt	14037	0644	edit dl rm
rlcompleter.rst.txt	2494	0644	edit dl rm
robotparser.rst.txt	2190	0644	edit dl rm
runpy.rst.txt	6938	0644	edit dl rm
sched.rst.txt	4644	0644	edit dl rm
scrolledtext.rst.txt	1379	0644	edit dl rm
select.rst.txt	20705	0644	edit dl rm
sets.rst.txt	14973	0644	edit dl rm
sgi.rst.txt	322	0644	edit dl rm
sgmllib.rst.txt	10664	0644	edit dl rm
sha.rst.txt	2807	0644	edit dl rm
shelve.rst.txt	8319	0644	edit dl rm
shlex.rst.txt	11313	0644	edit dl rm
shutil.rst.txt	13491	0644	edit dl rm
signal.rst.txt	10719	0644	edit dl rm
simplehttpserver.rst.txt	4559	0644	edit dl rm
simplexmlrpcserver.rst.txt	10881	0644	edit dl rm
site.rst.txt	7880	0644	edit dl rm
smtpd.rst.txt	2464	0644	edit dl rm
smtplib.rst.txt	14906	0644	edit dl rm
sndhdr.rst.txt	1759	0644	edit dl rm
socket.rst.txt	40765	0644	edit dl rm
socketserver.rst.txt	22512	0644	edit dl rm
someos.rst.txt	599	0644	edit dl rm
spwd.rst.txt	2825	0644	edit dl rm
sqlite3.rst.txt	35448	0644	edit dl rm
ssl.rst.txt	76554	0644	edit dl rm
stat.rst.txt	7778	0644	edit dl rm
statvfs.rst.txt	1300	0644	edit dl rm
stdtypes.rst.txt	122668	0644	edit dl rm
string.rst.txt	44085	0644	edit dl rm
stringio.rst.txt	4166	0644	edit dl rm
stringprep.rst.txt	4238	0644	edit dl rm
strings.rst.txt	746	0644	edit dl rm
struct.rst.txt	17100	0644	edit dl rm
subprocess.rst.txt	33648	0644	edit dl rm
sun.rst.txt	249	0644	edit dl rm
sunau.rst.txt	7124	0644	edit dl rm
sunaudio.rst.txt	5849	0644	edit dl rm
symbol.rst.txt	975	0644	edit dl rm
symtable.rst.txt	5064	0644	edit dl rm
sys.rst.txt	47520	0644	edit dl rm
sysconfig.rst.txt	7639	0644	edit dl rm
syslog.rst.txt	3931	0644	edit dl rm
tabnanny.rst.txt	1999	0644	edit dl rm
tarfile.rst.txt	28214	0644	edit dl rm
telnetlib.rst.txt	7482	0644	edit dl rm
tempfile.rst.txt	10504	0644	edit dl rm
termios.rst.txt	3733	0644	edit dl rm
test.rst.txt	17659	0644	edit dl rm
textwrap.rst.txt	8552	0644	edit dl rm
thread.rst.txt	6588	0644	edit dl rm
threading.rst.txt	32468	0644	edit dl rm
time.rst.txt	26020	0644	edit dl rm
timeit.rst.txt	11598	0644	edit dl rm
tix.rst.txt	22693	0644	edit dl rm
tk.rst.txt	1612	0644	edit dl rm
tkinter.rst.txt	33456	0644	edit dl rm
token.rst.txt	2451	0644	edit dl rm
tokenize.rst.txt	5570	0644	edit dl rm
trace.rst.txt	6724	0644	edit dl rm
traceback.rst.txt	10711	0644	edit dl rm
ttk.rst.txt	57411	0644	edit dl rm
tty.rst.txt	1011	0644	edit dl rm
turtle.rst.txt	64097	0644	edit dl rm
types.rst.txt	6200	0644	edit dl rm
undoc.rst.txt	6550	0644	edit dl rm
unicodedata.rst.txt	5736	0644	edit dl rm
unittest.rst.txt	82952	0644	edit dl rm
unix.rst.txt	490	0644	edit dl rm
urllib.rst.txt	24960	0644	edit dl rm
urllib2.rst.txt	35211	0644	edit dl rm
urlparse.rst.txt	16071	0644	edit dl rm
user.rst.txt	2748	0644	edit dl rm
userdict.rst.txt	9290	0644	edit dl rm
uu.rst.txt	2369	0644	edit dl rm
uuid.rst.txt	8378	0644	edit dl rm
warnings.rst.txt	19815	0644	edit dl rm
wave.rst.txt	5047	0644	edit dl rm
weakref.rst.txt	12926	0644	edit dl rm
webbrowser.rst.txt	9863	0644	edit dl rm
whichdb.rst.txt	931	0644	edit dl rm
windows.rst.txt	273	0644	edit dl rm
winsound.rst.txt	5066	0644	edit dl rm
wsgiref.rst.txt	30567	0644	edit dl rm
xdrlib.rst.txt	8077	0644	edit dl rm
xml.dom.minidom.rst.txt	11208	0644	edit dl rm
xml.dom.pulldom.rst.txt	1571	0644	edit dl rm
xml.dom.rst.txt	40182	0644	edit dl rm
xml.etree.elementtree.rst.txt	35707	0644	edit dl rm
xml.rst.txt	6090	0644	edit dl rm
xml.sax.handler.rst.txt	15371	0644	edit dl rm
xml.sax.reader.rst.txt	12257	0644	edit dl rm
xml.sax.rst.txt	6460	0644	edit dl rm
xml.sax.utils.rst.txt	3560	0644	edit dl rm
xmlrpclib.rst.txt	22559	0644	edit dl rm
zipfile.rst.txt	18871	0644	edit dl rm
zipimport.rst.txt	5957	0644	edit dl rm
zlib.rst.txt	12766	0644	edit dl rm
_winreg.rst.txt	23305	0644	edit dl rm
__builtin__.rst.txt	1494	0644	edit dl rm
__future__.rst.txt	4952	0644	edit dl rm
__main__.rst.txt	535	0644	edit dl rm

Edit: /usr/share/doc/python2-docs/html/_sources/library/urllib.rst.txt (24960B)

:mod:`urllib` --- Open arbitrary resources by URL
=================================================

.. module:: urllib
   :synopsis: Open an arbitrary network resource by URL (requires sockets).

.. note::
    The :mod:`urllib` module has been split into parts and renamed in
    Python 3 to :mod:`urllib.request`, :mod:`urllib.parse`,
    and :mod:`urllib.error`. The :term:`2to3` tool will automatically adapt
    imports when converting your sources to Python 3.
    Also note that the :func:`urllib.request.urlopen` function in Python 3 is
    equivalent to :func:`urllib2.urlopen` and that :func:`urllib.urlopen` has
    been removed.

.. index::
   single: WWW
   single: World Wide Web
   single: URL

This module provides a high-level interface for fetching data across the World
Wide Web.  In particular, the :func:`urlopen` function is similar to the
built-in function :func:`open`, but accepts Universal Resource Locators (URLs)
instead of filenames.  Some restrictions apply --- it can only open URLs for
reading, and no seek operations are available.

.. seealso::

The `Requests package <http://docs.python-requests.org/>`_
    is recommended for a higher-level HTTP client interface.

.. versionchanged:: 2.7.9

For HTTPS URIs, :mod:`urllib` performs all the neccessary certificate and hostname checks by default.

.. warning::

For Python versions earlier than 2.7.9, urllib does not attempt to validate the server certificates of HTTPS URIs. Use at your own risk!

High-level interface
--------------------

.. function:: urlopen(url[, data[, proxies[, context]]])

Open a network object denoted by a URL for reading.  If the URL does not
   have a scheme identifier, or if it has :file:`file:` as its scheme
   identifier, this opens a local file (without :term:`universal newlines`);
   otherwise it opens a socket to a server somewhere on the network.  If the
   connection cannot be made the :exc:`IOError` exception is raised.  If all
   went well, a file-like object is returned.  This supports the following
   methods: :meth:`read`, :meth:`readline`, :meth:`readlines`, :meth:`fileno`,
   :meth:`close`, :meth:`info`, :meth:`getcode` and :meth:`geturl`.  It also
   has proper support for the :term:`iterator` protocol. One caveat: the
   :meth:`read` method, if the size argument is omitted or negative, may not
   read until the end of the data stream; there is no good way to determine
   that the entire stream from a socket has been read in the general case.

Except for the :meth:`info`, :meth:`getcode` and :meth:`geturl` methods,
   these methods have the same interface as for file objects --- see section
   :ref:`bltin-file-objects` in this manual.  (It is not a built-in file object,
   however, so it can't be used at those few places where a true built-in file
   object is required.)

.. index:: module: mimetools

The :meth:`info` method returns an instance of the class
   :class:`mimetools.Message` containing meta-information associated with the
   URL.  When the method is HTTP, these headers are those returned by the server
   at the head of the retrieved HTML page (including Content-Length and
   Content-Type).  When the method is FTP, a Content-Length header will be
   present if (as is now usual) the server passed back a file length in response
   to the FTP retrieval request. A Content-Type header will be present if the
   MIME type can be guessed.  When the method is local-file, returned headers
   will include a Date representing the file's last-modified time, a
   Content-Length giving file size, and a Content-Type containing a guess at the
   file's type. See also the description of the :mod:`mimetools` module.

The :meth:`geturl` method returns the real URL of the page.  In some cases, the
   HTTP server redirects a client to another URL.  The :func:`urlopen` function
   handles this transparently, but in some cases the caller needs to know which URL
   the client was redirected to.  The :meth:`geturl` method can be used to get at
   this redirected URL.

The :meth:`getcode` method returns the HTTP status code that was sent with the
   response, or ``None`` if the URL is no HTTP URL.

If the *url* uses the :file:`http:` scheme identifier, the optional *data*
   argument may be given to specify a ``POST`` request (normally the request type
   is ``GET``).  The *data* argument must be in standard
   :mimetype:`application/x-www-form-urlencoded` format; see the :func:`urlencode`
   function below.

The :func:`urlopen` function works transparently with proxies which do not
   require authentication.  In a Unix or Windows environment, set the
   :envvar:`http_proxy`, or :envvar:`ftp_proxy` environment variables to a URL that
   identifies the proxy server before starting the Python interpreter.  For example
   (the ``'%'`` is the command prompt)::

% http_proxy="http://www.someproxy.com:3128"
      % export http_proxy
      % python
      ...

The :envvar:`no_proxy` environment variable can be used to specify hosts which
   shouldn't be reached via proxy; if set, it should be a comma-separated list
   of hostname suffixes, optionally with ``:port`` appended, for example
   ``cern.ch,ncsa.uiuc.edu,some.host:8080``.

In a Windows environment, if no proxy environment variables are set, proxy
   settings are obtained from the registry's Internet Settings section.

.. index:: single: Internet Config

In a Mac OS X  environment, :func:`urlopen` will retrieve proxy information
   from the OS X System Configuration Framework, which can be managed with
   Network System Preferences panel.

Alternatively, the optional *proxies* argument may be used to explicitly specify
   proxies.  It must be a dictionary mapping scheme names to proxy URLs, where an
   empty dictionary causes no proxies to be used, and ``None`` (the default value)
   causes environmental proxy settings to be used as discussed above.  For
   example::

# Use http://www.someproxy.com:3128 for HTTP proxying
      proxies = {'http': 'http://www.someproxy.com:3128'}
      filehandle = urllib.urlopen(some_url, proxies=proxies)
      # Don't use any proxies
      filehandle = urllib.urlopen(some_url, proxies={})
      # Use proxies from environment - both versions are equivalent
      filehandle = urllib.urlopen(some_url, proxies=None)
      filehandle = urllib.urlopen(some_url)

Proxies which require authentication for use are not currently supported;
   this is considered an implementation limitation.

The *context* parameter may be set to a :class:`ssl.SSLContext` instance to
   configure the SSL settings that are used if :func:`urlopen` makes a HTTPS
   connection.

.. versionchanged:: 2.3
      Added the *proxies* support.

.. versionchanged:: 2.6
      Added :meth:`getcode` to returned object and support for the
      :envvar:`no_proxy` environment variable.

.. versionchanged:: 2.7.9
      The *context* parameter was added.  All the neccessary certificate and hostname
      checks are done by default.

.. deprecated:: 2.6
      The :func:`urlopen` function has been removed in Python 3 in favor
      of :func:`urllib2.urlopen`.

.. function:: urlretrieve(url[, filename[, reporthook[, data[, context]]]])

Copy a network object denoted by a URL to a local file, if necessary. If the URL
   points to a local file, or a valid cached copy of the object exists, the object
   is not copied.  Return a tuple ``(filename, headers)`` where *filename* is the
   local file name under which the object can be found, and *headers* is whatever
   the :meth:`info` method of the object returned by :func:`urlopen` returned (for
   a remote object, possibly cached). Exceptions are the same as for
   :func:`urlopen`.

The second argument, if present, specifies the file location to copy to (if
   absent, the location will be a tempfile with a generated name). The third
   argument, if present, is a callable that will be called once on
   establishment of the network connection and once after each block read
   thereafter.  The callable will be passed three arguments; a count of blocks
   transferred so far, a block size in bytes, and the total size of the file.  The
   third argument may be ``-1`` on older FTP servers which do not return a file
   size in response to a retrieval request.

If the *url* uses the :file:`http:` scheme identifier, the optional *data*
   argument may be given to specify a ``POST`` request (normally the request type
   is ``GET``).  The *data* argument must in standard
   :mimetype:`application/x-www-form-urlencoded` format; see the :func:`urlencode`
   function below.

The *context* parameter may be set to a :class:`ssl.SSLContext` instance to
   configure the SSL settings that are used if :func:`urlretrieve` makes a HTTPS
   connection.

.. versionchanged:: 2.5
      :func:`urlretrieve` will raise :exc:`ContentTooShortError` when it detects that
      the amount of data available  was less than the expected amount (which is the
      size reported by a  *Content-Length* header). This can occur, for example, when
      the  download is interrupted.

The *Content-Length* is treated as a lower bound: if there's more data  to read,
      :func:`urlretrieve` reads more data, but if less data is available,  it raises
      the exception.

You can still retrieve the downloaded data in this case, it is stored  in the
      :attr:`content` attribute of the exception instance.

If no *Content-Length* header was supplied, :func:`urlretrieve` can not check
      the size of the data it has downloaded, and just returns it.  In this case you
      just have to assume that the download was successful.

.. versionchanged:: 2.7.9
      The *context* parameter was added.  All the neccessary certificate and hostname
      checks are done by default.

.. data:: _urlopener

The public functions :func:`urlopen` and :func:`urlretrieve` create an instance
   of the :class:`FancyURLopener` class and use it to perform their requested
   actions.  To override this functionality, programmers can create a subclass of
   :class:`URLopener` or :class:`FancyURLopener`, then assign an instance of that
   class to the ``urllib._urlopener`` variable before calling the desired function.
   For example, applications may want to specify a different
   :mailheader:`User-Agent` header than :class:`URLopener` defines.  This can be
   accomplished with the following code::

import urllib

class AppURLopener(urllib.FancyURLopener):
          version = "App/1.7"

urllib._urlopener = AppURLopener()

.. function:: urlcleanup()

Clear the cache that may have been built up by previous calls to
   :func:`urlretrieve`.

Utility functions
-----------------

.. function:: quote(string[, safe])

Replace special characters in *string* using the ``%xx`` escape. Letters,
   digits, and the characters ``'_.-'`` are never quoted. By default, this
   function is intended for quoting the path section of the URL. The optional
   *safe* parameter specifies additional characters that should not be quoted
   --- its default value is ``'/'``.

Example: ``quote('/~connolly/')`` yields ``'/%7econnolly/'``.

.. function:: quote_plus(string[, safe])

Like :func:`quote`, but also replaces spaces by plus signs, as required for
   quoting HTML form values when building up a query string to go into a URL.
   Plus signs in the original string are escaped unless they are included in
   *safe*.  It also does not have *safe* default to ``'/'``.

.. function:: unquote(string)

Replace ``%xx`` escapes by their single-character equivalent.

Example: ``unquote('/%7Econnolly/')`` yields ``'/~connolly/'``.

.. function:: unquote_plus(string)

Like :func:`unquote`, but also replaces plus signs by spaces, as required for
   unquoting HTML form values.

.. function:: urlencode(query[, doseq])

Convert a mapping object or a sequence of two-element tuples to a
   "percent-encoded" string, suitable to pass to :func:`urlopen` above as the
   optional *data* argument.  This is useful to pass a dictionary of form
   fields to a ``POST`` request.  The resulting string is a series of
   ``key=value`` pairs separated by ``'&'`` characters, where both *key* and
   *value* are quoted using :func:`quote_plus` above.  When a sequence of
   two-element tuples is used as the *query* argument, the first element of
   each tuple is a key and the second is a value. The value element in itself
   can be a sequence and in that case, if the optional parameter *doseq* is
   evaluates to ``True``, individual ``key=value`` pairs separated by ``'&'`` are
   generated for each element of the value sequence for the key.  The order of
   parameters in the encoded string will match the order of parameter tuples in
   the sequence. The :mod:`urlparse` module provides the functions
   :func:`parse_qs` and :func:`parse_qsl` which are used to parse query strings
   into Python data structures.

.. function:: pathname2url(path)

Convert the pathname *path* from the local syntax for a path to the form used in
   the path component of a URL.  This does not produce a complete URL.  The return
   value will already be quoted using the :func:`quote` function.

.. function:: url2pathname(path)

Convert the path component *path* from a percent-encoded URL to the local syntax for a
   path.  This does not accept a complete URL.  This function uses :func:`unquote`
   to decode *path*.

.. function:: getproxies()

This helper function returns a dictionary of scheme to proxy server URL
   mappings. It scans the environment for variables named ``<scheme>_proxy``,
   in case insensitive way, for all operating systems first, and when it cannot
   find it, looks for proxy information from Mac OSX System Configuration for
   Mac OS X and Windows Systems Registry for Windows.
   If both lowercase and uppercase environment variables exist (and disagree),
   lowercase is preferred.

.. note::

If the environment variable ``REQUEST_METHOD`` is set, which usually
      indicates your script is running in a CGI environment, the environment
      variable ``HTTP_PROXY`` (uppercase ``_PROXY``) will be ignored. This is
      because that variable can be injected by a client using the "Proxy:" HTTP
      header. If you need to use an HTTP proxy in a CGI environment, either use
      ``ProxyHandler`` explicitly, or make sure the variable name is in
      lowercase (or at least the ``_proxy`` suffix).

.. note::
    urllib also exposes certain utility functions like splittype, splithost and
    others parsing URL into various components. But it is recommended to use
    :mod:`urlparse` for parsing URLs rather than using these functions directly.
    Python 3 does not expose these helper functions from :mod:`urllib.parse`
    module.

URL Opener objects
------------------

.. class:: URLopener([proxies[, context[, **x509]]])

Base class for opening and reading URLs.  Unless you need to support opening
   objects using schemes other than :file:`http:`, :file:`ftp:`, or :file:`file:`,
   you probably want to use :class:`FancyURLopener`.

By default, the :class:`URLopener` class sends a :mailheader:`User-Agent` header
   of ``urllib/VVV``, where *VVV* is the :mod:`urllib` version number.
   Applications can define their own :mailheader:`User-Agent` header by subclassing
   :class:`URLopener` or :class:`FancyURLopener` and setting the class attribute
   :attr:`version` to an appropriate string value in the subclass definition.

The optional *proxies* parameter should be a dictionary mapping scheme names to
   proxy URLs, where an empty dictionary turns proxies off completely.  Its default
   value is ``None``, in which case environmental proxy settings will be used if
   present, as discussed in the definition of :func:`urlopen`, above.

The *context* parameter may be a :class:`ssl.SSLContext` instance.  If given,
   it defines the SSL settings the opener uses to make HTTPS connections.

Additional keyword parameters, collected in *x509*, may be used for
   authentication of the client when using the :file:`https:` scheme.  The keywords
   *key_file* and *cert_file* are supported to provide an  SSL key and certificate;
   both are needed to support client authentication.

:class:`URLopener` objects will raise an :exc:`IOError` exception if the server
   returns an error code.

.. versionchanged:: 2.7.9
      The *context* parameter was added.  All the neccessary certificate and hostname
      checks are done by default.

.. method:: open(fullurl[, data])

Open *fullurl* using the appropriate protocol.  This method sets up cache and
      proxy information, then calls the appropriate open method with its input
      arguments.  If the scheme is not recognized, :meth:`open_unknown` is called.
      The *data* argument has the same meaning as the *data* argument of
      :func:`urlopen`.

.. method:: open_unknown(fullurl[, data])

Overridable interface to open unknown URL types.

.. method:: retrieve(url[, filename[, reporthook[, data]]])

Retrieves the contents of *url* and places it in *filename*.  The return value
      is a tuple consisting of a local filename and either a
      :class:`mimetools.Message` object containing the response headers (for remote
      URLs) or ``None`` (for local URLs).  The caller must then open and read the
      contents of *filename*.  If *filename* is not given and the URL refers to a
      local file, the input filename is returned.  If the URL is non-local and
      *filename* is not given, the filename is the output of :func:`tempfile.mktemp`
      with a suffix that matches the suffix of the last path component of the input
      URL.  If *reporthook* is given, it must be a function accepting three numeric
      parameters.  It will be called after each chunk of data is read from the
      network.  *reporthook* is ignored for local URLs.

If the *url* uses the :file:`http:` scheme identifier, the optional *data*
      argument may be given to specify a ``POST`` request (normally the request type
      is ``GET``).  The *data* argument must in standard
      :mimetype:`application/x-www-form-urlencoded` format; see the :func:`urlencode`
      function below.

.. attribute:: version

Variable that specifies the user agent of the opener object.  To get
      :mod:`urllib` to tell servers that it is a particular user agent, set this in a
      subclass as a class variable or in the constructor before calling the base
      constructor.

.. class:: FancyURLopener(...)

:class:`FancyURLopener` subclasses :class:`URLopener` providing default handling
   for the following HTTP response codes: 301, 302, 303, 307 and 401.  For the 30x
   response codes listed above, the :mailheader:`Location` header is used to fetch
   the actual URL.  For 401 response codes (authentication required), basic HTTP
   authentication is performed.  For the 30x response codes, recursion is bounded
   by the value of the *maxtries* attribute, which defaults to 10.

For all other response codes, the method :meth:`http_error_default` is called
   which you can override in subclasses to handle the error appropriately.

.. note::

According to the letter of :rfc:`2616`, 301 and 302 responses to POST requests
      must not be automatically redirected without confirmation by the user.  In
      reality, browsers do allow automatic redirection of these responses, changing
      the POST to a GET, and :mod:`urllib` reproduces this behaviour.

The parameters to the constructor are the same as those for :class:`URLopener`.

.. note::

When performing basic authentication, a :class:`FancyURLopener` instance calls
      its :meth:`prompt_user_passwd` method.  The default implementation asks the
      users for the required information on the controlling terminal.  A subclass may
      override this method to support more appropriate behavior if needed.

The :class:`FancyURLopener` class offers one additional method that should be
   overloaded to provide the appropriate behavior:

.. method:: prompt_user_passwd(host, realm)

Return information needed to authenticate the user at the given host in the
      specified security realm.  The return value should be a tuple, ``(user,
      password)``, which can be used for basic authentication.

The implementation prompts for this information on the terminal; an application
      should override this method to use an appropriate interaction model in the local
      environment.

.. exception:: ContentTooShortError(msg[, content])

This exception is raised when the :func:`urlretrieve` function detects that the
   amount of the downloaded data is less than the  expected amount (given by the
   *Content-Length* header). The :attr:`content` attribute stores the downloaded
   (and supposedly truncated) data.

.. versionadded:: 2.5

:mod:`urllib` Restrictions
--------------------------

.. index::
     pair: HTTP; protocol
     pair: FTP; protocol

* Currently, only the following protocols are supported: HTTP, (versions 0.9 and
  1.0),  FTP, and local files.

* The caching feature of :func:`urlretrieve` has been disabled until I find the
  time to hack proper processing of Expiration time headers.

* There should be a function to query whether a particular URL is in the cache.

* For backward compatibility, if a URL appears to point to a local file but the
  file can't be opened, the URL is re-interpreted using the FTP protocol.  This
  can sometimes cause confusing error messages.

* The :func:`urlopen` and :func:`urlretrieve` functions can cause arbitrarily
  long delays while waiting for a network connection to be set up.  This means
  that it is difficult to build an interactive Web client using these functions
  without using threads.

.. index::
     single: HTML
     pair: HTTP; protocol
     module: htmllib

* The data returned by :func:`urlopen` or :func:`urlretrieve` is the raw data
  returned by the server.  This may be binary data (such as an image), plain text
  or (for example) HTML.  The HTTP protocol provides type information in the reply
  header, which can be inspected by looking at the :mailheader:`Content-Type`
  header.  If the returned data is HTML, you can use the module :mod:`htmllib` to
  parse it.

.. index:: single: FTP

* The code handling the FTP protocol cannot differentiate between a file and a
  directory.  This can lead to unexpected behavior when attempting to read a URL
  that points to a file that is not accessible.  If the URL ends in a ``/``, it is
  assumed to refer to a directory and will be handled accordingly.  But if an
  attempt to read a file leads to a 550 error (meaning the URL cannot be found or
  is not accessible, often for permission reasons), then the path is treated as a
  directory in order to handle the case when a directory is specified by a URL but
  the trailing ``/`` has been left off.  This can cause misleading results when
  you try to fetch a file whose read permissions make it inaccessible; the FTP
  code will try to read it, fail with a 550 error, and then perform a directory
  listing for the unreadable file. If fine-grained control is needed, consider
  using the :mod:`ftplib` module, subclassing :class:`FancyURLopener`, or changing
  *_urlopener* to meet your needs.

* This module does not support the use of proxies which require authentication.
  This may be implemented in the future.

.. index:: module: urlparse

* Although the :mod:`urllib` module contains (undocumented) routines to parse
  and unparse URL strings, the recommended interface for URL manipulation is in
  module :mod:`urlparse`.

.. _urllib-examples:

Examples
--------

Here is an example session that uses the ``GET`` method to retrieve a URL
containing parameters::

>>> import urllib
   >>> params = urllib.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})
   >>> f = urllib.urlopen("http://www.musi-cal.com/cgi-bin/query?%s" % params)
   >>> print f.read()

The following example uses the ``POST`` method instead::

>>> import urllib
   >>> params = urllib.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})
   >>> f = urllib.urlopen("http://www.musi-cal.com/cgi-bin/query", params)
   >>> print f.read()

The following example uses an explicitly specified HTTP proxy, overriding
environment settings::

>>> import urllib
   >>> proxies = {'http': 'http://proxy.example.com:8080/'}
   >>> opener = urllib.FancyURLopener(proxies)
   >>> f = opener.open("http://www.python.org")
   >>> f.read()

The following example uses no proxies at all, overriding environment settings::

>>> import urllib
   >>> opener = urllib.FancyURLopener({})
   >>> f = opener.open("http://www.python.org/")
   >>> f.read()

cmd: