/usr/local/lib/python3.6/site-packages/datasets/filesystems/__pycache__
Edit: /usr/local/lib/python3.6/site-packages/datasets/filesystems/__pycache__/compression.cpython-36.pyc (6142B)
3
<%EgV @ s d dl Z d dlmZ d dlZd dlmZ d dlmZ G dd deZG dd deZ G d d
d
eZ
G d d d eZ G d
d deZ G dd deZ
dS ) N)Optional)AbstractArchiveFileSystem)DEFAULT_BLOCK_SIZEc s e Zd ZU dZdZdZedZedZedee e e e
d fdd
Z e fddZ
d d
Zed d d
ZdeedddZ ZS ) BaseCompressedFileFileSystemzFRead contents of compressed file as a filesystem with one file inside. N)fotarget_protocoltarget_optionsc s t j| f| tj|fd|| jd|p*i | _tjj| jjj dd | _
d| j
krn| j
d| j
j d n| j
| _ d| _
dS )a
The compressed file system can be instantiated from any compressed file.
It reads the contents of compressed file as a filesystem with one file inside, as if it was an archive.
The single file inside the filesystem is named after the compresssed file,
without the compression extension at the end of the filename.
Args:
fo (:obj:``str``): Path to compressed file. Will fetch file using ``fsspec.open()``
mode (:obj:``str``): Currently, only 'rb' accepted
target_protocol(:obj:``str``, optional): To override the FS protocol inferred from a URL.
target_options (:obj:``dict``, optional): Kwargs passed when instantiating the target FS.
rb)modeprotocol compressionz::r .N)super__init__fsspecopenr
fileospathbasenamesplitcompressed_namerindexuncompressed_name dir_cache)selfr r r kwargs) __class__ J/usr/local/lib/python3.6/site-packages/datasets/filesystems/compression.pyr s z%BaseCompressedFileFileSystem.__init__c s t j|jdS )N/)r _strip_protocollstrip)clsr )r r r r" 0 s z,BaseCompressedFileFileSystem._strip_protocolc C s8 | j d kr4| jjj| jjd| ji}|d |i| _ d S )Nname)r r fsinfor r )r fr r r _get_dirs5 s
z&BaseCompressedFileFileSystem._get_dirs)r c C s | j j j S )N)r r read)r r r r r cat: s z BaseCompressedFileFileSystem.catr
T)r r c K s6 | j |}|dkr,td| d| jj d| jj S )Nr
zTried to read with mode z on file z opened with mode 'rb')r"
ValueErrorr r r )r r r
block_sizeZ
autocommitZ
cache_optionsr r r r _open= s
z"BaseCompressedFileFileSystem._open)r NN)r
NTN)__name__
__module__ __qualname____doc__Z root_markerr strr
extensionr dictr classmethodr" r) r+ r.
__classcell__r r )r r r s
r c @ s e Zd ZdZdZdZdZdS )
Bz2FileSystemz?Read contents of BZ2 file as a filesystem with one file inside.bz2z.bz2N)r/ r0 r1 r2 r r
r4 r r r r r8 L s r8 c @ s e Zd ZdZdZdZdZdS )GzipFileSystemz@Read contents of GZIP file as a filesystem with one file inside.gzipz.gzN)r/ r0 r1 r2 r r
r4 r r r r r: T s r: c @ s e Zd ZdZdZdZdZdS )
Lz4FileSystemz?Read contents of LZ4 file as a filesystem with one file inside.Zlz4z.lz4N)r/ r0 r1 r2 r r
r4 r r r r r< \ s r< c @ s e Zd ZdZdZdZdZdS ) XzFileSystemzFRead contents of .xz (LZMA) file as a filesystem with one file inside.xzz.xzN)r/ r0 r1 r2 r r
r4 r r r r r= d s r= c sL e Zd ZdZdZdZdZdddefeee e e e
e d fdd
Z Z
S ) ZstdFileSystemz
Read contents of zstd file as a filesystem with one file inside.
Note that reading in binary mode with fsspec isn't supported yet:
https://github.com/indygreg/python-zstandard/issues/136
Zzstdz.zstr
N)r r r r r- c sN t jf |||||d| | jjG dd d fdd}|| j_d S )N)r r r r r- c @ s< e Zd Zdd Zdd Zdd Zdd Zd d
Zd d Zd
S )z,ZstdFileSystem.__init__.
.WrappedFilec S s
|| _ d S )N)_file)r file_r r r r s z5ZstdFileSystem.__init__..WrappedFile.__init__c S s | j j | S )N)r@ __enter__)r r r r rB s
z6ZstdFileSystem.__init__..WrappedFile.__enter__c _ s | j j|| d S )N)r@ __exit__)r argsr r r r rC s z5ZstdFileSystem.__init__..WrappedFile.__exit__c S s
t | jS )N)iterr@ )r r r r __iter__ s z5ZstdFileSystem.__init__..WrappedFile.__iter__c S s
t | jS )N)nextr@ )r r r r __next__ s z5ZstdFileSystem.__init__..WrappedFile.__next__c S s t | j|S )N)getattrr@ )r attrr r r __getattr__ s z8ZstdFileSystem.__init__..WrappedFile.__getattr__N) r/ r0 r1 r rB rC rF rH rK r r r r WrappedFile s rL c s | |S )Nr )rD r )rL _enterr r fixed_enter s z,ZstdFileSystem.__init__..fixed_enter)r r r rB )r r r r r r- r rN )r )rL rM r r x s zZstdFileSystem.__init__)r/ r0 r1 r2 r r
r4 r r3 r r5 intr r7 r r )r r r? l s r? )r typingr r Zfsspec.archiver Z fsspec.utilsr r r8 r: r< r= r? r r r r s C