Name	Size	Mode	Actions
api.cpython-36.pyc	1096	0644	editdlrm
clipboards.cpython-36.pyc	3392	0644	editdlrm
common.cpython-36.pyc	14064	0644	editdlrm
date_converters.cpython-36.pyc	1947	0644	editdlrm
feather_format.cpython-36.pyc	2936	0644	editdlrm
gbq.cpython-36.pyc	7459	0644	editdlrm
html.cpython-36.pyc	31977	0644	editdlrm
orc.cpython-36.pyc	1802	0644	editdlrm
parquet.cpython-36.pyc	9500	0644	editdlrm
parsers.cpython-36.pyc	88200	0644	editdlrm
pickle.cpython-36.pyc	5679	0644	editdlrm
pytables.cpython-36.pyc	130805	0644	editdlrm
spss.cpython-36.pyc	1328	0644	editdlrm
sql.cpython-36.pyc	52283	0644	editdlrm
stata.cpython-36.pyc	97563	0644	editdlrm
__init__.cpython-36.pyc	145	0644	editdlrm

3 :%Eg���@s�dZd
dl
mZ
d
dlmZmZmZ
d
dlmZ
d
dlZd
dl m Z mZmZm Z
d
dlZd
dlZd
dljjZd
dlmZmZmZ
d
dlmZ
d
d lmZ
d
d lmZmZ
d
dlm Z
d
dl!m"Z"
Gd d�de#�Z$Gdd�de%�Z&da'dd�Z(dd�Z)dd�Z*dQdd�
Z+dd�Z,dRdd�
Z-dSdd�
Z.e dTded �d!d"��
Z/e dUe0e ed �d$d"��
Z/dVee0eee efd �d%d"�Z/e dWded �d&d'��
Z1e dXe0e ed �d(d'��
Z1dYee0eee efd �d)d'�Z1e dZded �d*d+��
Z2e d[e0e ed �d,d+��
Z2d\ee0eee efd �d-d+�Z2d]dd/�
d0d1�Z3d^d2d3�
Z4e4Z5d4d5�Z6d_d7d8�
Z7Gd9d:�d:e �Z8Gd;d<�d�d>e9�Z:d?d@dAdBdCdDdAdE�Z;dFdG�ZGdKdL�dLe8�Z?GdMdN�dNe9�Z@d`dOdP�
ZAdS)az| Collection of query wrappers / abstractions to both facilitate data retrieval and to reduce dependency on DB-specific API. �)
�contextmanager)�date�datetime�time)
�partialN)�Iterator�Optional�Union�overload)�is_datetime64tz_dtype�is_dict_like�is_list_like)
�DatetimeTZDtype)
�isna)� DataFrame�Series)
�PandasObject)
�to_datetimec
@seZ
dZd
S)�SQLAlchemyRequiredN)�__name__� __module__�__qualname__�rr�9/usr/local/lib64/python3.6/site-packages/pandas/io/sql.pyrs
rc
@seZ
dZd
S)� DatabaseErrorN)rrrrrrrrs
rc
CsTtdkr2yd
dl
}
daWntk r0


daYnXtrLd
dl
}
t||
jj�SdSdS)Nr
TF)�_SQLALCHEMY_INSTALLED� sqlalchemy�ImportError� isinstance�engineZConnectable)�conrrrr�_is_sqlalchemy_connectable(s




r!cCs6|g
}|
d
k r2t|
d�r$||
g
7}n|t
|
�
g
7}|S)z9Convert SQL and params args to DBAPI2.0 compliant format.N�keys)�hasattr�list)�sql�params�argsrrr�_convert_params:s



r(c

Cs2|d
ks|dks|dkrg}nt|d�s.|g
}|S)z3Process parse_dates argument for read_sql functionsTNF�__iter__)
r#)
�parse_datesrrr�_process_parse_dates_argumentEs 


r+c Cs�t|t
�rt|f
d
di
|��
S|dkrJt|jjtj�sFt|jjtj�rJd}|dkrbt|d||
d�St |j�
rxt|d d�St|d||
d�SdS)N�errors�ignore�
s�
D�
d�
h�
m�ms�us�nsZcoerce)r,�unit�utcT)
r7)r,�formatr7)r/r0r1r2r.r3r4r5) r�dictr� issubclass�dtype�type�np�floating�integerr)�colr7r8rrr�_handle_date_columnPs






rAc Csjt|
�
}
x\|j
�D]P\}}t|j�
s,||
kry|
|}Wntk rP


d
}YnXt||d�||<qW|S)zz Force non-datetime columns to be read as such. Supports both string formatted and integer timestamp columns. N)
r8)r+�itemsrr;� TypeErrorrA)Z data_framer*�col_name�df_col�fmtrrr�_parse_date_columnses





rGTcCs4tj
||
|d
�}t||�}|dk r0|j|dd�
|S)z(Wrap result set of query in a DataFrame.)�columns�coerce_floatNT)
�inplace)r�from_recordsrG� set_index)�datarH� index_colrIr*�framerrr�_wrap_resultzs  
rPcCs2|d
krt|
�
}nt|dd�}t
||�}|j|�S)aY Execute the given SQL query using the provided connection object. Parameters ---------- sql : string SQL query to be executed. con : SQLAlchemy connectable(engine/connection) or sqlite3 connection Using SQLAlchemy makes it possible to use any DB supported by the library. If a DBAPI2 object, only sqlite3 is supported. cur : deprecated, cursor is obtained from connection, default: None params : list or tuple, optional, default: None List of parameters to pass to execute method. Returns ------- Results Iterable NT)
� is_cursor)�pandasSQL_builderr(�execute)r%r �curr&� pandas_sqlr'rrrrS�s 


rS)� chunksize�returnc
CsdS)
Nr)� table_namer �schemarNrIr*rHrVrrr�read_sql_table�srZ�
c
CsdS)
Nr)rXr rYrNrIr*rHrVrrrrZ�scCs�t|
�
}
t
|
�
std
�
�
ddl}ddlm}
| |
|d�} y| j|g
dd�
Wn8|jjk r�
}
zt d|�d ��
|�WYdd}~XnXt |
| d �}|j||||||d�} | dk r�| St d|�d �|
��
dS)a� Read SQL database table into a DataFrame. Given a table name and a SQLAlchemy connectable, returns a DataFrame. This function does not support DBAPI connections. Parameters ---------- table_name : str Name of SQL table in database. con : SQLAlchemy connectable or str A database URI could be provided as as str. SQLite DBAPI connection mode not supported. schema : str, default None Name of SQL schema in database to query (if database flavor supports this). Uses default schema if None (default). index_col : str or list of str, optional, default: None Column(s) to set as index(MultiIndex). coerce_float : bool, default True Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point. Can result in loss of Precision. parse_dates : list or dict, default None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg dict}``, where the arg dict corresponds to the keyword arguments of :func:`pandas.to_datetime` Especially useful with databases without native Datetime support, such as SQLite. columns : list, default None List of column names to select from SQL table. chunksize : int, default None If specified, returns an iterator where `chunksize` is the number of rows to include in each chunk. Returns ------- DataFrame or Iterator[DataFrame] A SQL table is returned as two-dimensional data structure with labeled axes. See Also -------- read_sql_query : Read SQL query into a DataFrame. read_sql : Read SQL query or database table into a DataFrame. Notes ----- Any datetime values with time zone information will be converted to UTC. Examples -------- >>> pd.read_sql_table('table_name', 'postgres:///db_name') # doctest:+SKIP z9read_sql_table only supported for SQLAlchemy connectable.r
N)
�MetaData)
rYT)�onlyZviewszTable z not found)
�meta)rNrIr*rHrV)�_engine_builderr!�NotImplementedErrorr�sqlalchemy.schemar\�reflect�excZInvalidRequestError� ValueError�SQLDatabase� read_table)rXr rYrNrIr*rHrVrr\r^�errrU�tablerrrrZ�s,A







$







c
CsdS)
Nr)r%r rNrIr&r*rVrrr�read_sql_query!
s ric
CsdS)
Nr)r%r rNrIr&r*rVrrrri.
s cCst|
�
}|j
||||||d
�S)a7 Read SQL query into a DataFrame. Returns a DataFrame corresponding to the result set of the query string. Optionally provide an `index_col` parameter to use one of the columns as the index, otherwise default integer index will be used. Parameters ---------- sql : str SQL query or SQLAlchemy Selectable (select or text object) SQL query to be executed. con : SQLAlchemy connectable, str, or sqlite3 connection Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported. index_col : str or list of str, optional, default: None Column(s) to set as index(MultiIndex). coerce_float : bool, default True Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point. Useful for SQL result sets. params : list, tuple or dict, optional, default: None List of parameters to pass to execute method. The syntax used to pass parameters is database driver dependent. Check your database driver documentation for which of the five syntax styles, described in PEP 249's paramstyle, is supported. Eg. for psycopg2, uses %(name)s so use params={'name' : 'value'}. parse_dates : list or dict, default: None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg dict}``, where the arg dict corresponds to the keyword arguments of :func:`pandas.to_datetime` Especially useful with databases without native Datetime support, such as SQLite. chunksize : int, default None If specified, return an iterator where `chunksize` is the number of rows to include in each chunk. Returns ------- DataFrame or Iterator[DataFrame] See Also -------- read_sql_table : Read SQL database table into a DataFrame. read_sql : Read SQL query or database table into a DataFrame. Notes ----- Any datetime values with time zone information parsed via the `parse_dates` parameter will be converted to UTC. )rNr&rIr*rV)rR� read_query)r%r rNrIr&r*rVrUrrrri;
s=






c
CsdS)
Nr)r%r rNrIr&r*rHrVrrr�read_sql�
srkc
CsdS)
Nr)r%r rNrIr&r*rHrVrrrrk�
sc Cs�t|
�
}t
|t�r(|j||||||d
�Sy|j|�
} Wntk rN


d} YnX| rz|jj|g
d�

|j||||||d�S|j||||||d
�SdS)a� Read SQL query or database table into a DataFrame. This function is a convenience wrapper around ``read_sql_table`` and ``read_sql_query`` (for backward compatibility). It will delegate to the specific function depending on the provided input. A SQL query will be routed to ``read_sql_query``, while a database table name will be routed to ``read_sql_table``. Note that the delegated function might have more specific notes about their functionality not listed here. Parameters ---------- sql : str or SQLAlchemy Selectable (select or text object) SQL query to be executed or a table name. con : SQLAlchemy connectable, str, or sqlite3 connection Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported. The user is responsible for engine disposal and connection closure for the SQLAlchemy connectable. See `here `_. index_col : str or list of str, optional, default: None Column(s) to set as index(MultiIndex). coerce_float : bool, default True Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point, useful for SQL result sets. params : list, tuple or dict, optional, default: None List of parameters to pass to execute method. The syntax used to pass parameters is database driver dependent. Check your database driver documentation for which of the five syntax styles, described in PEP 249's paramstyle, is supported. Eg. for psycopg2, uses %(name)s so use params={'name' : 'value'}. parse_dates : list or dict, default: None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg dict}``, where the arg dict corresponds to the keyword arguments of :func:`pandas.to_datetime` Especially useful with databases without native Datetime support, such as SQLite. columns : list, default: None List of column names to select from SQL table (only used when reading a table). chunksize : int, default None If specified, return an iterator where `chunksize` is the number of rows to include in each chunk. Returns ------- DataFrame or Iterator[DataFrame] See Also -------- read_sql_table : Read SQL database table into a DataFrame. read_sql_query : Read SQL query into a DataFrame. )rNr&rIr*rVF)
r])rNrIr*rHrVN) rRr�SQLiteDatabaserj� has_table� Exceptionr^rbrf) r%r rNrIr&r*rHrVrUZ_is_table_namerrrrk�
s:A








 













�fail)
rWc Csj|d krtd|�d��
�
t
||d�} t|t�r8|j�}nt|t�sJtd�
�
| j||
||||||| d�
d S)a; Write records stored in a DataFrame to a SQL database. Parameters ---------- frame : DataFrame, Series name : str Name of SQL table. con : SQLAlchemy connectable(engine/connection) or database string URI or sqlite3 DBAPI2 connection Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported. schema : str, optional Name of SQL schema in database to write to (if database flavor supports this). If None, use default schema (default). if_exists : {'fail', 'replace', 'append'}, default 'fail' - fail: If table exists, do nothing. - replace: If table exists, drop it, recreate it, and insert data. - append: If table exists, insert data. Create if does not exist. index : boolean, default True Write DataFrame index as a column. index_label : str or sequence, optional Column label for index column(s). If None is given (default) and `index` is True, then the index names are used. A sequence should be given if the DataFrame uses MultiIndex. chunksize : int, optional Specify the number of rows in each batch to be written at a time. By default, all rows will be written at once. dtype : dict or scalar, optional Specifying the datatype for columns. If a dictionary is used, the keys should be the column names and the values should be the SQLAlchemy types or strings for the sqlite3 fallback mode. If a scalar is provided, it will be applied to all columns. method : {None, 'multi', callable}, optional Controls the SQL insertion clause used: - None : Uses standard SQL ``INSERT`` clause (one per row). - 'multi': Pass multiple values in a single ``INSERT`` clause. - callable with signature ``(pd_table, conn, keys, data_iter)``. Details and a sample callable implementation can be found in the section :ref:`insert method `. .. versionadded:: 0.24.0 ro�replace�append�
'z' is not valid for if_exists)
rYz9'frame' argument should be either a Series or a DataFrame)� if_exists�index�index_labelrYrVr;�methodN)rorprq)rdrRrrZto_framerr`�to_sql)rO�namer rYrsrtrurVr;rvrUrrrrws$:













rwcCst|
|d
�}|j
|�
S)a- Check if DataBase has named table. Parameters ---------- table_name: string Name of SQL table. con: SQLAlchemy connectable(engine/connection) or sqlite3 DBAPI2 connection Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported. schema : string, default None Name of SQL schema in database to write to (if database flavor supports this). If None, use default schema (default). Returns ------- boolean )
rY)rRrm)rXr rYrUrrrrmZs
rmc
CsBt|t
�r>yd
dl}
Wntk r.


daYnX|
j|�
}|S|S)zw Returns a SQLAlchemy engine from a URI (if con is a string) else it just return con without modifying it. r
NF)r�strrrrZ create_engine)r rrrrr_us




r_FcCsBt|�
}t
|�
rt||
|d
�St|t�r2td�
�
nt||d�SdS)zm Convenience function to return the correct PandasSQL subclass based on the provided parameters. )rYr^z.Using URI string without sqlalchemy installed.)
rQN)r_r!rerryrrl)r rYr^rQrrrrR�s



rRc@s�eZ
dZd
Zd(dd�
Zdd �Zd d�Zdd �Zdd�Zdd�Z dd�Z dd�Zd)dd�
Zd*dd�
Z d+dd�
Zdd�Zdd�Zd d!�Zd,d"d#�
Zd$d%�Zd&d'�ZdS)-�SQLTablez� For mapping Pandas tables to SQL tables. Uses fact that table is reflected by SQLAlchemy to do better type conversions. Also holds various flags needed to avoid having to pass them between functions all the time. NTro�pandascCs�|
|_||_
||_||_|j||�|_||_||_| |_| |_ |dk rR|j �|_n|j
j|j|j�|_|jdkr�t d
|
�d��
�
dS)NzCould not init table 'rr)rx�pd_sql�prefixrO�_index_namertrYrsr"r;�_create_table_setuprh� get_tablerd)�selfrxZpandas_sql_enginerOrtrsr}rurYr"r;rrr�__init__�s 








zSQLTable.__init__c

Cs|jj
|j|j�S)
N)r|rmrxrY)
r�rrr�exists�s
zSQLTable.existsc
Cs$d
dlm
}

t|
|j�
j|jj�
�
S)Nr
)
�CreateTable)rar�ryrh�compiler|�connectable)r�r�rrr� sql_schema�s
zSQLTable.sql_schemac

Cs |jj
|jj�
|_|jj�
dS)
N)rhZ tometadatar|r^�create)
r�rrr�_execute_create�s
zSQLTable._execute_createc

Csx|j�rl|j
d
kr&td|j�d��
�
qt|j
dkrL|jj|j|j�
|j�
qt|j
dkrXqttd|j
�d��
�
n|j�
dS)NrozTable 'z' already exists.rprqrrz' is not valid for if_exists)r�rsrdrxr|� drop_tablerYr�)
r�rrrr��s







zSQLTable.createcs(�f
d
d�|D�
}|
j|j
j�|�
dS)a<
Execute SQL statement inserting data Parameters ---------- conn : sqlalchemy.engine.Engine or sqlalchemy.engine.Connection keys : list of str Column names data_iter : generator of list Each item contains a list of values to be inserted c
sg|]}
tt
�|
��
�qSr)r9�zip)�.0�row)
r"rr� �sz,SQLTable._execute_insert..N)rSrh�insert)r��connr"� data_iterrMr)
r"r�_execute_insert�s
zSQLTable._execute_insertcs(�f
d
d�|D�
}|
j|j
j|�
�

dS)a
Alternative to _execute_insert for DBs support multivalue INSERT. Note: multi-value insert is usually faster for analytics DBs and tables containing a few columns but performance degrades quickly with increase of columns. c
sg|]}
tt
�|
��
�qSr)r9r�)r�r�)
r"rrr��sz2SQLTable._execute_insert_multi..N)rSrhr�)r�r�r"r�rMr)
r"r�_execute_insert_multi�s
zSQLTable._execute_insert_multic
Cs0
|jdk rd|j
j�}
|j|
j_y|
jd
d�

Wqjtk r`
}
ztd|���
|�WYdd}~XqjXn|j
}
ttt|
j ��
}t |�
}dg
|}x�t|
j��
D]�\}\}}|j } | jjdkr�| j�} n(| jjdkr�| jd�
jt�
} n | jt�
} t| tj��
stt| �
�
�
|j�
rt| �
}d| |<| ||<q�W||fS)NT)
rJz!duplicate name in index/columns: �
Mr2�i8)rtrO�copy�namesZreset_indexrdr$�mapryrH�len� enumeraterBZ_valuesr;�kindZ to_pydatetime�view�astype�objectrr=Zndarray�AssertionErrorr<Z_can_hold_nar)r��temprg�column_namesZncols� data_list�
i�
_Zser�valsr0�maskrrr�insert_data�s0






$





 
zSQLTable.insert_datacs
|dkr|j}n2|d
kr |j
}n"t|�
r4t||�}ntd|���
�
|j�\}}t|j�
}|dkrddS|
dkrr|}
n|
dkr�td�
�
t||
�
d}|j j ��`}xXt|�
D]L} | |
�
t| d|
|���
�kr�Pt ��
fdd�|D�
�} |||| �
q�WWdQRXdS)NZmultizInvalid parameter `method`: r
z%chunksize argument should be non-zeror[c
sg|]}
|
�
���qSrr)r�Zarr)�end_i�start_irrr�=sz#SQLTable.insert..)r�r��callablerrdr�r�rO�intr|�run_transaction�range�minr�)r�rVrvZexec_insertr"r�Znrows�chunksr�r�Z chunk_iterr)r�r�rr�s0




 









zSQLTable.insertccs^xX|
j|�
}|sPqt
j|||d
�|_|j|d�

|jdk rN|jj|jdd�
|jV
qWdS)z,Return generator through chunked result set.)rHrI)
r*NT)
rJ)� fetchmanyrrKrO�_harmonize_columnsrtrL)r��resultrVrHrIr*rMrrr�_query_iterator@s




zSQLTable._query_iteratorcs�|dk rtt|�
d
krtd
dl
m}
�f
dd�|D�
}�jdk rjx,�jddd�D]}|jd
�jj|�
qNW||�
}n �jj�}�jj|�
} | j �} |dk r��j | || |
|d�S| j�}tj || |
d��_�j|d�

�jdk r��jj�jd d �
�jSdS)Nr
)
�selectc
sg|]}
�jj
|
�qSr)rh�
c)r��
n)
r�rrr�Ysz!SQLTable.read..r[)rIr*)rHrI)
r*T)
rJ�����)r�rr�rtr�rhr�r|rSr"r��fetchallrrKrOr�rL)r�rIr*rHrVr��cols�idxZ sql_selectr�r�rMr)
r�r�readTs0




 









z SQLTable.readcCs�|
d
kr�|jj
j}|dk rJt|t�s*|g
}t|�
|krFtd|���
�
n|S|dkrrd|jjkrr|jj
jdkrrdg
Sdd�t |jj
j �
D�
Sn"t|
t�r�|
g
St|
t�r�|
SdSdS)NTz@Length of 'index_label' should match number of levels, which is r[rtc
Ss&g|]\}
}|dk r|nd
|
���qS)NZlevel_r)r�r��
lrrrr��s
z(SQLTable._index_name..)rOrt�nlevelsrr$r�rdrHrxr�r�ry)r�rtrur�rrrr~ys(











zSQLTable._index_namecstg}�
jdk rLx.) rtr�rOZ_get_level_valuesrqryr�r�rH)r�r��column_names_and_typesr�Z idx_labelZidx_typer)r�r�r�_get_column_names_and_types�s






z$SQLTable._get_column_names_and_typesc
s�d
dlm
�m}
m}
|j|j�
}�f
dd�|D�
}|jdk rtt|j�
sP|jg
}n|j}|
|d|jdi
�
}|j |�

|j p�|jjj }d
dl m}
||j|d�} ||j| f|�d |i
�
S) Nr
)�Column�PrimaryKeyConstraint�Tablec
s g|]\}
}}�|
||d��qS)
)
rtr)r�rx�typ�is_index)
r�rrr��s
z0SQLTable._create_table_setup..rxZ_pk)
r\)
rYrY)rr�r�r�r��_sqlalchemy_typer"r rxrqrYr|r^rar\) r�r�r�r�rHr"ZpkcrYr\r^r)
r�rr�s





zSQLTable._create_table_setupcCs,
t|
�
}
�
x|j
jD�
]}|j}y�|j|}||
krny|
|}Wntk rX


d
}YnXt||d�|j|<w|j|j�
}|t ks�|t ks�|tkr�|tk}t||d�|j|<n\|tkr�|j |dd�|j|<n>t|�
|j�k�
r |tjd�
ks�|tk�
r |j |dd�|j|<Wqtk �
r"


YqXqWd
S)a Make the DataFrame's column types align with the SQL table column types. Need to work around limited NA value support. Floats are always fine, ints must always be floats if there are Null values. Booleans are hard because converting bool column with None replaces all Nones with false. Therefore only convert bool if there are no NA values. Datetimes should already be converted to np.datetime64 if supported, but here we also force conversion if required. N)
r8)
r7F)
r��int64)r+rhrHrxrOrCrA� _get_dtyper<rrr�floatr�r��countr=r;�bool�KeyError)r�r*Zsql_colrDrErF�col_typer7rrrr��s2















zSQLTable._harmonize_columnsc Cs^
|jpi}|
j
|kr |j|
j
Stj|
d
d�}ddlm}m}m}m}m }m } m} m}m }
|dksj|dkr�y|
jjdk r�|d
d�
SWn.tk r�


t|
dd�dk r�|d
d�
SYnX|S|d kr�tjd tdd�
|S|d k�r�|
jdkr�| dd�
S| dd�
Sn\|dk�
r|
jdk�
r| S|Sn<|dk�
r,|S|dk�
r:|S|dk�
rH|S|dk�
rZtd�
�
|S)NT)
�skipnar
) � TIMESTAMP� BigInteger�Boolean�Date�DateTime�Float�Integer�Text�Time� datetime64r)
�timezone�tz�timedelta64zlthe 'timedelta' type is not supported, and will be written as integer values (ns frequency) to the database.�)
� stacklevelr>Zfloat32�)
Z precision�5r?Zint32�booleanrr�complexzComplex datatypes not supported)r;rx�lib�infer_dtype�sqlalchemy.typesr�r�r�r�r�r�r�r�r��dtr��AttributeError�getattr�warnings�warn�UserWarningrd) r�r@r;r�r�r�r�r�r�r�r�r�r�rrrr��sH

,






















zSQLTable._sqlalchemy_typecCs�d
dlm
}m}m}m}m}m}
t|
|�r.tSt|
|�rBt j d�
St|
|�rZ|
jsVtSt St|
|�rhtSt|
|�rvtSt|
|�r�tStS)Nr
)r�r�r�r�r�r�r�)r�r�r�r�r�r�r�rr�r=r;r�rrrr�r�)r�Zsqltyper�r�r�r�r�r�rrrr�9s











zSQLTable._get_dtype)NTror{NNNN)NN)TN)TNNN)
N)rrr�__doc__r�r�r�r�r�r�r�r�r�r�r�r~r�rr�r�r�rrrrrz�s2






$ %  %$ 3?rzc@s eZ
dZd
Zdd�Zdd�ZdS)� PandasSQLz7 Subclasses Should define read_sql and to_sql. c
Ostd
�
�
dS)NzMPandasSQL must be created with an SQLAlchemy connectable or sqlite connection)
rd)r�r'�kwargsrrrrkUs

zPandasSQL.read_sqlc
Ostd
�
�
dS)NzMPandasSQL must be created with an SQLAlchemy connectable or sqlite connection)
rd)r�r'r�rrrrw[s

zPandasSQL.to_sqlN)rrrr�rkrwrrrrr�Psr�c@s�eZ
dZd
Zddd�
Zedd��
Zdd�Zdd d�
Ze ddd �
�
Z d dd�
ZeZd!dd�
Z edd��
Zd"dd�
Zd#dd�
Zd$dd�
Zd%dd�
ZdS)&rea� This class enables conversion between DataFrame and SQL databases using SQLAlchemy to handle DataBase abstraction. Parameters ---------- engine : SQLAlchemy connectable Connectable to connect with the database. Using SQLAlchemy makes it possible to use any DB supported by that library. schema : string, default None Name of SQL schema in database to write to (if database flavor supports this). If None, use default schema (default). meta : SQLAlchemy MetaData object, default None If provided, this MetaData object is used instead of a newly created. This allows to specify database flavor specific arguments in the MetaData object. NcCs.|
|_|s$d
dl
m}
||j|d�}||_dS)Nr
)
r\)
rY)r�rar\r^)r�rrYr^r\rrrr�vs


zSQLDatabase.__init__c
 cs4|jj
�� }
t|
d
�r|
V
n|jV
WdQRXdS)NrS)r��beginr#)r�Ztxrrrr�s

zSQLDatabase.run_transactionc
Os|jj
d
d�
j|
|�
S)z,Simple passthrough to SQLAlchemy connectableT)
Z no_parameters)r�Zexecution_optionsrS)r�r'r�rrrrS�s
zSQLDatabase.executeTc Cs"t|
|||d
�}|j
||||d�S)a� Read SQL database table into a DataFrame. Parameters ---------- table_name : string Name of SQL table in database. index_col : string, optional, default: None Column to set as index. coerce_float : boolean, default True Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point. This can result in loss of precision. parse_dates : list or dict, default: None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg}``, where the arg corresponds to the keyword arguments of :func:`pandas.to_datetime`. Especially useful with databases without native Datetime support, such as SQLite. columns : list, default: None List of column names to select from SQL table. schema : string, default None Name of SQL schema in database to query (if database flavor supports this). If specified, this overwrites the default schema of the SQL database object. chunksize : int, default None If specified, return an iterator where `chunksize` is the number of rows to include in each chunk. Returns ------- DataFrame See Also -------- pandas.read_sql_table SQLDatabase.read_query )rtrY)rIr*rHrV)rzr�) r�rXrNrIr*rHrYrVrhrrrrf�s4




zSQLDatabase.read_tableccs0x*|j|
�
}|sPqt
|||||d
�V
qWdS)z+Return generator through chunked result set)rNrIr*N)r�rP)r�rVrHrNrIr*rMrrrr��s







zSQLDatabase._query_iteratorcCs\t|
|�}|j
|�}|j�} |d
k r:|j||| |||d�S|j�} t| | |||d�}|Sd
S)a$ Read SQL query into a DataFrame. Parameters ---------- sql : string SQL query to be executed. index_col : string, optional, default: None Column name to use as index for the returned DataFrame object. coerce_float : boolean, default True Attempt to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point, useful for SQL result sets. params : list, tuple or dict, optional, default: None List of parameters to pass to execute method. The syntax used to pass parameters is database driver dependent. Check your database driver documentation for which of the five syntax styles, described in PEP 249's paramstyle, is supported. Eg. for psycopg2, uses %(name)s so use params={'name' : 'value'} parse_dates : list or dict, default: None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg dict}``, where the arg dict corresponds to the keyword arguments of :func:`pandas.to_datetime` Especially useful with databases without native Datetime support, such as SQLite. chunksize : int, default None If specified, return an iterator where `chunksize` is the number of rows to include in each chunk. Returns ------- DataFrame See Also -------- read_sql_table : Read SQL database table into a DataFrame. read_sql N)rNrIr*)r(rSr"r�r�rP)r�r%rNrIr*r&rVr'r�rHrMrOrrrrj�s&2 













zSQLDatabase.read_queryroc sn
�r t��
r �f
d
d�|
D�
��dk rlddl
m} m}
x2�j�D]&\}} t|| �
| �sBtd|�d��
�
qBWt|||
||||�d�}|j�
dd l m }
y|j|| d �
WnN|jk r�
}
z0d}t |j�
}tj||�r�td�
|�n|�
WYdd}~XnX|j��
rj|j��
rj|jj}|jj��}|j|�
p6|jj|d �}WdQRX||k�
rjd|�d�}tj|t�
dS)a� Write records stored in a DataFrame to a SQL database. Parameters ---------- frame : DataFrame name : string Name of SQL table. if_exists : {'fail', 'replace', 'append'}, default 'fail' - fail: If table exists, do nothing. - replace: If table exists, drop it, recreate it, and insert data. - append: If table exists, insert data. Create if does not exist. index : boolean, default True Write DataFrame index as a column. index_label : string or sequence, default None Column label for index column(s). If None is given (default) and `index` is True, then the index names are used. A sequence should be given if the DataFrame uses MultiIndex. schema : string, default None Name of SQL schema in database to write to (if database flavor supports this). If specified, this overwrites the default schema of the SQLDatabase object. chunksize : int, default None If not None, then rows will be written in batches of this size at a time. If None, all rows will be written at once. dtype : single type or dict of column name to SQL type, default None Optional specifying the datatype for columns. The SQL type should be a SQLAlchemy type. If all columns are of the same type, one single value can be used. method : {None', 'multi', callable}, default None Controls the SQL insertion clause used: * None : Uses standard SQL ``INSERT`` clause (one per row). * 'multi': Pass multiple values in a single ``INSERT`` clause. * callable with signature ``(pd_table, conn, keys, data_iter)``. Details and a sample callable implementation can be found in the section :ref:`insert method `. .. versionadded:: 0.24.0 c
si|] }
�|
�qSrr)r�rD)
r;rr� ^sz&SQLDatabase.to_sql..Nr
)� TypeEngine�to_instancezThe type of z is not a SQLAlchemy type)rOrtrsrurYr;)
rc)
rvz.(1054, "Unknown column 'inf' in 'field list'")zinf cannot be used with MySQL)rY� connectionzThe provided table name 'z�' is not found exactly as such in the database after writing the table, possibly due to case sensitivity issues. Consider using lower case table names.)rr�r�r�rBrrdrzr�rrcr�ZSQLAlchemyErrorry�orig�re�search�isdigit�islowerr�r�connect�table_namesr^rYr�r�r�)r�rOrxrsrtrurYrVr;rvr�r�r@�my_typerhrcrg�msgZerr_textrr�r�r)
r;rrw(sD5



















 zSQLDatabase.to_sqlc


Cs|jj
S)
N)r^�tables)
r�rrrr��szSQLDatabase.tablescCs|jj
|jjj|
|p|jj�S)
N)r�Zrun_callable�dialectrmr^rY)r�rxrYrrrrm�s

zSQLDatabase.has_tablecCsl|p |jj
}|r*|jjjd
j||
g�
�
}n|jjj|
�
}ddlm}
x"|jD]}t|j |�rLd|j _ qLW|S)N�
.r
)
�NumericF)r^rYr��get�joinrr�rHrr<Z asdecimal)r�rXrYZtblr��columnrrrr��s




zSQLDatabase.get_tablecCsH|p |jj
}|j|
|�rD|jj|
g
|d
�
|j|
|�j�
|jj�
dS)N)r]rY)r^rYrmrbr�Zdrop�clear)r�rXrYrrrr��s




zSQLDatabase.drop_tablecCs t|||
d
||d�}t
|j��
S)NF)rOrtr"r;)rzryr�)r�rOrXr"r;rhrrr�_create_sql_schema�s

zSQLDatabase._create_sql_schema)NN)NTNNNN)NTN)NTNNN)roTNNNNN)
N)
N)
N)NN)rrrr�r�rr�rSrf�staticmethodr�rjrkrw�propertyr�rmr�r�r
rrrrrebs< 




4



D





_   reZTEXTZREALZINTEGERr�ZDATEZTIME)�stringr>r?rrrr�c
CsTyt|�
j
d
d�jd
�
}
Wn4tk rN
}
ztd|�d��
|�WYdd}~XnX|
S)Nzutf-8�strictz%Cannot convert identifier to UTF-8: 'rr)ry�encode�decode�UnicodeErrorrd)rx�unamergrrr�_get_unicode_name�s



$
r
c
CsFt|�
}
t
|
�
std
�
�
|
jd�
}|dkr2td�
�
d|
jdd�dS)Nz$Empty table or column name specified�
r
z%SQLite identifier cannot contain NULs�
"z"")r
r�rd�findrp)rxr
Z nul_indexrrr�_get_valid_sqlite_name�s




r
zvThe spaces in these column names will not be changed. In pandas versions < 0.14, spaces were converted to underscores.csXeZ
dZd
Z�f
dd�Zdd�Zdd�Zdd �Zd d�Zdd �Z dd�Z dd�Z�ZS)�SQLiteTablezw Patch the SQLTable for fallback support. Instead of a table variable just use the Create Table statement. c
s*d
dl}|j
tdd��
t�j|
|�

dS)Nr
c

Ss |jd
�
S)Nz%H:%M:%S.%f)
�strftime)
r�rrr��sz&SQLiteTable.__init__..)�sqlite3Zregister_adapterr�superr�)r�r'r�r
)
� __class__rrr��s
zSQLiteTable.__init__c

Cstd
j
|j�
�
S)Nz; )ryr

rh)
r�rrrr��s
zSQLiteTable.sql_schemac
 Cs4|jj
�� }
x|jD]}|
j|�

qWWdQRXdS)
N)r|r�rhrS)r�r��stmtrrrr��s


zSQLiteTable._execute_createc

s�tt
t|jj��
}d
}t�|jdk rJx$|jddd�D]}|jd|�
q6W�f
dd�|D�
}dj|�
}dj|g
t |�
�
�
dj�
f
dd�t |
�
D�
�
}d �|j�
�d |�d|��}|S) N�
?r[r
c
sg|]}
�|
�
�qSrr)r�r
)
�escaperrr�sz0SQLiteTable.insert_statement..�
,c
3s|]}
d��d
�V
qdS)�
(�
)Nr)r�r�)
� row_wildcardsrr� sz/SQLiteTable.insert_statement..zINSERT INTO z (z ) VALUES r�)r$r�ryrOrHr
rtr�r

r�r�rx) r��num_rowsr��wldr�Zbracketed_namesZ col_names� wildcards�insert_statementr)r
r
rr#
�s






zSQLiteTable.insert_statementcCs t|�
}|
j
|jd
d�
|�
dS)Nr[)
r
)r$Zexecutemanyr#
)r�r�r"r�r�rrrr�s

zSQLiteTable._execute_insertcCs2t|�
}d
d�|D�
}|
j
|jt|�
d�
|�
dS)Nc
Ssg|]}
|
D]}|�qqSrr)r�r��
xrrrr�sz5SQLiteTable._execute_insert_multi..)
r
)r$rSr#
r�)r�r�r"r�r�Zflattened_datarrrr�s


z!SQLiteTable._execute_insert_multic
sR
|j|j
�
}
tjd
�
}dd�|
D�
}tt|j|��
rBtjt dd�
t ��f
dd�|
D�
}|jdk r�t|j�
r�t |j�
s�|jg
}n|j}dj�f
d d �|D�
�
}|jd|j�d|�d ��

d�|j�
ddj|�
dg
}dd�|
D�
}t|�
�
rNdj|�
} dj�f
dd �|D�
�
}|jd�d|jd| �
d�|j�
d|d �

|S)z� Return a list of SQL statements that creates a table reflecting the structure of a DataFrame. The first entry will be a CREATE TABLE statement while the rest will be CREATE INDEX statements. z\s+c
Ssg|]\}
}}|
�qSrr)r�rDr�rrrr�#sz3SQLiteTable._create_table_setup..�)
r�c
s"g|]\}
}}�|
�
d|�qS)
�
r)r��cname�ctyper�)
r
rrr�*sNz, c
3s|]}
�|
�
V
qdS)
Nr)r�r�)
r
rrr
2sz2SQLiteTable._create_table_setup..zCONSTRAINT z_pk PRIMARY KEY (r
z CREATE TABLE z ( z, z )c
Ssg|]\}
}}|r|
�qSrr)r�r'
r�r�rrrr�?sr�r
c
3s|]}
�|
�
V
qdS)
Nr)r�r�)
r
rrr
Bsz CREATE INDEX Zix_zON z ()r��_sql_type_namer�r��anyr�r�r�r��_SAFE_NAMES_WARNINGr
r"r�r r

rqrx) r�r��patr�Zcreate_tbl_stmtsr"Z cnames_brZcreate_stmtsZix_colsZcnamesr)
r
rrs,







"



6zSQLiteTable._create_table_setupcCs�|jpi}|
j
|kr||
j
Stj|
d
d�}|dkrJtjdtdd�
d}n,|dkrXd }n|d krfd}n|dkrvtd �
�
|tkr�d}t|S)NT)
r�r�zlthe 'timedelta' type is not supported, and will be written as integer values (ns frequency) to the database.r�)
r�r?r�r�emptyr
r�zComplex datatypes not supported) r;rxr�r�r�r�r�rd� _SQL_TYPES)r�r@r;r�rrrr)
Os&









zSQLiteTable._sql_type_name) rrrr�r�r�r�r#
r�r�rr)
� __classcell__rr)
r
rr
�s 5r
c@s�eZ
dZd
Zddd�
Zedd��
Zdd�Zeddd�
�
Z dd d�
Z dd�Zddd�
Zd dd�
Z d!dd�
Zd"dd�
Zd#dd�
Zd S)$rlz� Version of SQLDatabase to support SQLite connections (fallback without SQLAlchemy). This should only be used internally. Parameters ---------- con : sqlite connection object FcCs||_|
|_
dS)
N)rQr )r�r rQrrrr�{s

zSQLiteDatabase.__init__c
csT|jj
�}
z:y|
V
|jj�
Wn tk r@


|jj�
�YnXWd|
j�
XdS)
N)r �cursor�commitrn�rollback�close)r�rTrrrr�s





zSQLiteDatabase.run_transactionc
Os�|jr|j
}n |j
j�}y|j|
|�

|Stk r�
}
zvy|j
j�
WnBtk r�
}
z&td
|
d�d|�d��
}||�WYdd}~XnXtd|
d�d|���
}||�WYdd}~XnXdS)NzExecution failed on sql: r
�
z unable to rollbackzExecution failed on sql 'z': )rQr r0
rSrnr2
r)r�r'r�rTrcZ inner_exc�exrrrrS�s











zSQLiteDatabase.executeNTccsLxF|j|
�
}t
|�
tkr t|�
}|s0|j�
Pqt|||||d
�V
qWdS)z+Return generator through chunked result set)rNrIr*N)r�r<�tupler$r3
rP)r0
rVrHrNrIr*rMrrrr��s










zSQLiteDatabase._query_iteratorcCsnt|
|�}|j
|�}d
d�|jD�
} |dk rB|j||| |||d�S|j|�
} |j�
t| | |||d�}|SdS)Nc
Ssg|]}
|
d�qS)
r
r)r�Zcol_descrrrr��sz-SQLiteDatabase.read_query..)rNrIr*)r(rS�descriptionr��_fetchall_as_listr3
rP)r�r%rNrIr&r*rVr'r0
rHrMrOrrrrj�s(














zSQLiteDatabase.read_querycCs|
j�}t
|t�st|�
}|S)
N)r�rr$)r�rTr�rrrr8
�s



z SQLiteDatabase._fetchall_as_listroc s��r t��
r �f
d
d�|
D�
��dk r\x2�j
�D]&\} }t|t�s2t| �d|�d��
�
q2Wt|||
|||�d�}|j�
|j|| �
dS)ai Write records stored in a DataFrame to a SQL database. Parameters ---------- frame: DataFrame name: string Name of SQL table. if_exists: {'fail', 'replace', 'append'}, default 'fail' fail: If table exists, do nothing. replace: If table exists, drop it, recreate it, and insert data. append: If table exists, insert data. Create if it does not exist. index : boolean, default True Write DataFrame index as a column index_label : string or sequence, default None Column label for index column(s). If None is given (default) and `index` is True, then the index names are used. A sequence should be given if the DataFrame uses MultiIndex. schema : string, default None Ignored parameter included for compatibility with SQLAlchemy version of ``to_sql``. chunksize : int, default None If not None, then rows will be written in batches of this size at a time. If None, all rows will be written at once. dtype : single type or dict of column name to SQL type, default None Optional specifying the datatype for columns. The SQL type should be a string. If all columns are of the same type, one single value can be used. method : {None, 'multi', callable}, default None Controls the SQL insertion clause used: * None : Uses standard SQL ``INSERT`` clause (one per row). * 'multi': Pass multiple values in a single ``INSERT`` clause. * callable with signature ``(pd_table, conn, keys, data_iter)``. Details and a sample callable implementation can be found in the section :ref:`insert method `. .. versionadded:: 0.24.0 c
si|] }
�|
�qSrr)r�rD)
r;rrr�sz)SQLiteDatabase.to_sql..Nz (z) not a string)rOrtrsrur;)rrBrryrdr
r�r�) r�rOrxrsrtrurYrVr;rvr@r�rhr)
r;rrw�s 4











zSQLiteDatabase.to_sqlcCs*d
}d|�d�}t|j
||
g
�j��
dkS)Nr
z;SELECT name FROM sqlite_master WHERE type='table' AND name=�
;r
)r�rSr�)r�rxrYr!
�queryrrrrm&s
zSQLiteDatabase.has_tablec
CsdS)
Nr)r�rXrYrrrr�0s
zSQLiteDatabase.get_tablecCsd
t|
�
��}|j
|�

dS)NzDROP TABLE )r
rS)r�rxrYZdrop_sqlrrrr�3s

zSQLiteDatabase.drop_tablecCs t|||
d
||d�}t
|j��
S)NF)rOrtr"r;)r
ryr�)r�rOrXr"r;rhrrrr
7s

z!SQLiteDatabase._create_sql_schema)
F)NTN)NTNNN)roTNNNNN)
N)
N)
N)NN)rrrr�r�rr�rSr
r�rjr8
rwrmr�r�r
rrrrrlps.  









>  rlcCst|d
�
}|j
||
||d�S)a� Get the SQL db table schema for the given frame. Parameters ---------- frame : DataFrame name : string name of SQL table keys : string or sequence, default: None columns to use a primary key con: an open SQL database connection object or a SQLAlchemy connectable Using SQLAlchemy makes it possible to use any DB supported by that library, default: None If a DBAPI2 object, only sqlite3 is supported. dtype : dict of column name to SQL type, default None Optional specifying the datatype for columns. The SQL type should be a SQLAlchemy type, or a string for sqlite3 fallback connection. )
r )r"r;)rRr
)rOrxr"r r;rUrrr� get_schema>s
r;
)NN)NTN)NN)NNTNNN)NNTNNr[)NNTNNN)NTNNN)NTNNr[)NTNNN)NTNNNN)NTNNNr[)NTNNNN)NroTNNNN)
N)NNF)NNN)Br�� contextlibrrrr� functoolsrr��typingrrr r r�Znumpyr=Zpandas._libs.libZ_libsr�Zpandas.core.dtypes.commonrrr Zpandas.core.dtypes.dtypesrZpandas.core.dtypes.missingrZpandas.core.apirrZpandas.core.baserZpandas.core.tools.datetimesrrr�IOErrorrrr!r(r+rArGrPrSrZr�rirkrwrmZtable_existsr_rRrzr�rer.
r
r
r+
r
rlr;
rrrr�s�








   

















V














@

















c






H  <\





O