/usr/local/lib/python3.6/site-packages/datasets/features/__pycache__
Edit: /usr/local/lib/python3.6/site-packages/datasets/features/__pycache__/image.cpython-36.pyc (10600B)
3
<%Eg/ @ s@ U d dl Z d dlmZmZ d dlmZ d dlmZmZm Z m
Z
m Z m Z m
Z
d dlZd dlZddlmZ ddlmZ ddlmZ dd lmZ dd
lmZm Z m Z erd dl Z d d l m!Z! da"e e e# "eG d
d dZ$e e# dddZ%de&dddZ'de(dddZ)e
e e# e e( e ej* e d f e e( dddZ+dS ) N) dataclassfield)BytesIO)
TYPE_CHECKINGAnyClassVarDictListOptionalUnion )config)xopen)
array_cast)
is_local_path)first_non_null_valueno_op_if_value_is_nullstring_to_dict ) FeatureTypec @ s e Zd ZU dZdZedZee dZ e
e e j e j
e j dZe
e ed dddZedd Zeeeejdf ed
d d Zdedd
d
dZedeedf f dddZee j e j e j f e j dddZ de j ee j dddZ dS )Imagea Image feature to read image data from an image file.
Input: The Image feature accepts as input:
- A :obj:`str`: Absolute path to the image file (i.e. random access is allowed).
- A :obj:`dict` with the keys:
- path: String with relative path of the image file to the archive file.
- bytes: Bytes of the image file.
This is useful for archived files with sequential access.
- An :obj:`np.ndarray`: NumPy array representing an image.
- A :obj:`PIL.Image.Image`: PIL image object.
Args:
decode (:obj:`bool`, default ``True``): Whether to decode the image data. If `False`,
returns the underlying dictionary in the format {"path": image_path, "bytes": image_bytes}.
Examples:
```py
>>> from datasets import load_dataset, Image
>>> ds = load_dataset("beans", split="train")
>>> ds.features["image"]
Image(decode=True, id=None)
>>> ds[0]["image"]
>>> ds = ds.cast_column('image', Image(decode=False))
{'bytes': None,
'path': '/root/.cache/huggingface/datasets/downloads/extracted/b0a21163f78769a2cf11f58dfc767fb458fc7cea5c05dccc0144a2c0f0bc1292/train/healthy/healthy_train.85.jpg'}
```
TNzPIL.Image.Image)bytespathF)defaultinitreprc C s | j S )N)pa_type)self r A/usr/local/lib/python3.6/site-packages/datasets/features/image.py__call__C s zImage.__call__)valuereturnc C s t jrddl}ntdt|tr,|ddS t|tjrZ|jj |j
tj }dt |dS t||jjrpt
|S |jddk rtjj|d rd|jddS |jddk s|jddk r|jd|jddS td| d dS )
a Encode example into a format for Arrow.
Args:
value (:obj:`str`, :obj:`np.ndarray`, :obj:`PIL.Image.Image` or :obj:`dict`): Data passed as input to Image feature.
Returns:
:obj:`dict` with "path" and "bytes" fields
r Nz4To support encoding images, please install 'Pillow'.)r r r )r r r zUAn image sample should have one of 'path' or 'bytes' but they are missing or None in .)r
PIL_AVAILABLE PIL.Image ImportError
isinstancestrnpndarrayr fromarrayastypeuint8image_to_bytesencode_pil_imagegetosr isfile
ValueError)r r! PILimager r r encode_exampleF s
zImage.encode_examplec C s | j stdtjr ddl}ntd|dkr2i }|d |d }}|dkr|dkrftd| dnt|r||jj |}nx|j
d d }y t |tj d }|j
|} W n tk
r d} Y nX t|d | d
}
t|
j }W dQ R X |jj |}n|jj t|}|j |S )aW Decode example image file into image data.
Args:
value (obj:`str` or :obj:`dict`): a string with the absolute image file path, a dictionary with
keys:
- path: String with absolute or relative image file path.
- bytes: The bytes of the image file.
token_per_repo_id (:obj:`dict`, optional): To access and decode
image files from private repositories on the Hub, you can pass
a dictionary repo_id (str) -> token (bool or str)
Returns:
:obj:`PIL.Image.Image`
zMDecoding is disabled for this feature. Please use Image(decode=True) instead.r Nz4To support decoding images, please install 'Pillow'.r r zCAn image should have one of 'path' or 'bytes' but both are None in r# z::r repo_idrb)use_auth_token)decode RuntimeErrorr
r$ r% r& r3 r r opensplitr ZHUB_DATASETS_URLr0 r r readload) r r! Ztoken_per_repo_idr4 r bytes_r5
source_urlr7 r9 fr r r decode_exampleh s2
zImage.decode_exampler )r" c C s( ddl m} | jr| S |d|ddS )zfIf in the decodable state, return the feature itself, otherwise flatten the feature into a dictionary.r )Valuebinarystring)r r )featuresrE r; )r rE r r r flatten s
z
Image.flatten)storager" c s~ t jrddl ntdtjj|jrdtjdgt | tj
d}tj j ||gddg|j
d}ntjj|jr|jjddkr|jd}n tjdgt | tj
d}|jjddkr|jd}n tjdgt | tj d}tj j ||gddg|j
d}nntjj|jrrtj fdd |j D tj
d}tjdgt | tj d}tj j ||gddg|j
d}t|| jS )
a Cast an Arrow array to the Image arrow storage type.
The Arrow types that can be converted to the Image pyarrow storage type are:
- pa.string() - it must contain the "path" data
- pa.struct({"bytes": pa.binary()})
- pa.struct({"path": pa.string()})
- pa.struct({"bytes": pa.binary(), "path": pa.string()}) - order doesn't matter
- pa.list(*) - it must contain the image array data
Args:
storage (Union[pa.StringArray, pa.StructArray, pa.ListArray]): PyArrow array to cast.
Returns:
pa.StructArray: Array in the Image arrow storage type, that is
pa.struct({"bytes": pa.binary(), "path": pa.string()})
r Nz4To support encoding images, please install 'Pillow'.)typer r )maskc s2 g | ]*}|d k r*t jjtj|tjnd qS )N)r. r r+ r) arrayr- ).0Zarr)r4 r r
s z&Image.cast_storage..)r
r$ r% r& patypes is_stringrK rM lenrF StructArray from_arraysis_nullZ is_structZget_field_indexr rG Zis_list to_pylistr r )r rJ bytes_array
path_arrayr )r4 r cast_storage s.
"
zImage.cast_storage)rJ
drop_pathsr" c s t dd tj fdd|j D tj d}|rPtjdgt| tj dn|jd}tjj ||gddg|j
d }t || j S )
av Embed image files into the Arrow array.
Args:
storage (pa.StructArray): PyArrow array to embed.
drop_paths (bool, default ``True``): If True, the paths are set to None.
Returns:
pa.StructArray: Array in the Image arrow storage type, that is
pa.struct({"bytes": pa.binary(), "path": pa.string()})
c S s" t | d}|j }W d Q R X |S )Nr8 )r r? )r rC rA r r r
path_to_bytes s z*Image.embed_storage..path_to_bytesc s8 g | ]0}|d k r0|d d kr( |d q2|d nd qS )Nr r r )rN x)r\ r r rO s z'Image.embed_storage..)rK Nr r )rL )
r rP rM rW rF rS rG r rT rU rV r r )r rJ r[ rX rY r )r\ r
embed_storage s
* zImage.embed_storage)N)T)!__name__
__module__ __qualname____doc__r; boolidr
r( Zdtyper rP structrF rG r r r _typer r dictr) r* r6 rD r rI Z StringArrayrT Z ListArrayrZ r^ r r r r r s
! "1
"1r )r" c C sR t jrdd l} ntdtd krN| jj tt| jj j
t| jj j
@ atS )Nr z4To support encoding images, please install 'Pillow'.) r
r$ r% r& _IMAGE_COMPRESSION_FORMATSr r listsetZOPENkeysZSAVE)r4 r r r list_image_compression_formats s
$rl zPIL.Image.Image)r5 r" c C s2 t }| jt kr| jnd}| j||d |j S )zhConvert a PIL Image object to bytes using native compression if possible, otherwise use PNG compression.ZPNG)format)r rm rl savegetvalue)r5 bufferrm r r r r. s r. c C s2 t | dr | jdkr | jd dS d t| dS d S )Nfilename )r r )hasattrrq r. )r5 r r r r/
s r/ )objsr" c s t jrddl ntd| rt| \}}t|tr@dd | D S t|tjr^ fdd| D S t| j j rt
t fdd| D S | S n| S dS )zrEncode a list of objects into a format suitable for creating an extension array of type :obj:`ImageExtensionType`.r Nz4To support encoding images, please install 'Pillow'.c S s" g | ]}|d k r|d dnd qS )N)r r r )rN objr r r rO s z2objects_to_list_of_image_dicts..c s6 g | ].}|d k r.d t jj|jtjdnd qS )N)r r )r. r r+ r, r) r- )rN ru )r4 r r rO s c s g | ] } |qS r r )rN ru )obj_to_image_dict_funcr r rO ' s ) r
r$ r% r& r r' r( r) r* r r r/ )rt _ru r )r4 rv r objects_to_list_of_image_dicts s
rx ),r1 Z dataclassesr r ior typingr r r r r r
r Znumpyr) ZpyarrowrP rr r
Z#download.streaming_download_managerr tabler Zutils.file_utilsr Zutils.py_utilsr r r r% r4 rH r rh r( r rl r r. rg r/ r* rx r r r r s, $ ]