tensorbay.dataset.data¶

Data.

Data is the most basic data unit of a Dataset. It contains path information of a data sample and its corresponding labels.

class tensorbay.dataset.data.DataBase(path: str, *, timestamp: Optional[float] = None)[source]¶

Bases: tensorbay.utility.attr.AttrsMixin, tensorbay.utility.repr.ReprMixin

DataBase is a base class for the file and label combination.

Parameters

path – The file path.
timestamp – The timestamp for the file.

path¶: The file path.

timestamp¶

The timestamp for the file.

Type: float

label¶

The Label instance that contains all the label information of the file.

Type: tensorbay.label.label.Label

static loads(contents: Dict[str, Any]) → _Type[source]¶

Loads data subclass from a dict containing data information.

Parameters

contents –

A dict containing the information of the data, which looks like:

{
    "localPath", "remotePath" or "authPath": <str>,
    "timestamp": <float>,
    "label": {
        "CLASSIFICATION": {...},
        "BOX2D": {...},
        "BOX3D": {...},
        "POLYGON2D": {...},
        "POLYLINE2D": {...},
        "KEYPOINTS2D": {...},
        "SENTENCE": {...}
    }
}

Returns

A Data, RemoteData or class`AuthData` instance containing information from the given dict.

Raises

KeyError – When the “localPath”, “remotePath” or “authPath” not in contents.

class tensorbay.dataset.data.Data(local_path: str, *, target_remote_path: Optional[str] = None, timestamp: Optional[float] = None)[source]¶

Bases: tensorbay.dataset.data.DataBase

Data is a combination of a specific local file and its label.

It contains the file local path, label information of the file and the file metadata, such as timestamp.

A Data instance contains one or several types of labels.

Parameters

local_path – The file local path.
target_remote_path – The file remote path after uploading to tensorbay.
timestamp – The timestamp for the file.

path¶

The file local path.

Type: str

timestamp¶

The timestamp for the file.

Type: float

label¶

The Label instance that contains all the label information of the file.

Type: tensorbay.label.label.Label

target_remote_path¶: The target remote path of the data.

classmethod loads(contents: Dict[str, Any]) → tensorbay.dataset.data._T[source]¶

Loads Data from a dict containing local data information.

Parameters

contents –

A dict containing the information of the data, which looks like:

{
    "localPath": <str>,
    "timestamp": <float>,
    "label": {
        "CLASSIFICATION": {...},
        "BOX2D": {...},
        "BOX3D": {...},
        "POLYGON2D": {...},
        "POLYLINE2D": {...},
        "KEYPOINTS2D": {...},
        "SENTENCE": {...}
    }
}

Returns

A Data instance containing information from the given dict.

open() → _io.BufferedReader[source]¶

Return the binary file pointer of this file.

The local file pointer will be obtained by build-in open().

Returns: The local file pointer for this data.

dumps() → Dict[str, Any][source]¶

Dumps the local data into a dict.

Returns

Dumped data dict, which looks like:

{
    "localPath": <str>,
    "timestamp": <float>,
    "label": {
        "CLASSIFICATION": {...},
        "BOX2D": {...},
        "BOX3D": {...},
        "POLYGON2D": {...},
        "POLYLINE2D": {...},
        "KEYPOINTS2D": {...},
        "SENTENCE": {...}
    }
}

class tensorbay.dataset.data.RemoteData(remote_path: str, *, timestamp: Optional[float] = None, url_getter: Optional[Callable[[str], str]] = None)[source]¶

Bases: tensorbay.dataset.data.DataBase

RemoteData is a combination of a specific tensorbay dataset file and its label.

It contains the file remote path, label information of the file and the file metadata, such as timestamp.

A RemoteData instance contains one or several types of labels.

Parameters

remote_path – The file remote path.
timestamp – The timestamp for the file.
url_getter – The url getter of the remote file.

path¶

The file remote path.

Type: str

timestamp¶

The timestamp for the file.

Type: float

label¶

The Label instance that contains all the label information of the file.

Type: tensorbay.label.label.Label

classmethod loads(contents: Dict[str, Any]) → tensorbay.dataset.data._T[source]¶

Loads RemoteData from a dict containing remote data information.

Parameters

contents –

A dict containing the information of the data, which looks like:

{
    "remotePath": <str>,
    "timestamp": <float>,
    "label": {
        "CLASSIFICATION": {...},
        "BOX2D": {...},
        "BOX3D": {...},
        "POLYGON2D": {...},
        "POLYLINE2D": {...},
        "KEYPOINTS2D": {...},
        "SENTENCE": {...}
    }
}

Returns

A Data instance containing information from the given dict.

get_url() → str[source]¶

Return the url of the data hosted by tensorbay.

Returns: The url of the data.
Raises: ValueError – When the url_getter is missing.

open() → http.client.HTTPResponse[source]¶

Return the binary file pointer of this file.

The remote file pointer will be obtained by urllib.request.urlopen().

Returns: The remote file pointer for this data.

dumps() → Dict[str, Any][source]¶

Dumps the remote data into a dict.

Returns

Dumped data dict, which looks like:

{
    "remotePath": <str>,
    "timestamp": <float>,
    "label": {
        "CLASSIFICATION": {...},
        "BOX2D": {...},
        "BOX3D": {...},
        "POLYGON2D": {...},
        "POLYLINE2D": {...},
        "KEYPOINTS2D": {...},
        "SENTENCE": {...}
    }
}

class tensorbay.dataset.data.AuthData(cloud_path: str, *, target_remote_path: Optional[str] = None, timestamp: Optional[float] = None, url_getter: Optional[Callable[[str], str]] = None)[source]¶

Bases: tensorbay.dataset.data.DataBase

AuthData is a combination of a specific cloud storaged file and its label.

It contains the cloud storage file path, label information of the file and the file metadata, such as timestamp.

An AuthData instance contains one or several types of labels.

Parameters

cloud_path – The cloud file path.
target_remote_path – The file remote path after uploading to tensorbay.
timestamp – The timestamp for the file.
url_getter – The url getter of the remote file.

path¶

The cloud file path.

Type: str

timestamp¶

The timestamp for the file.

Type: float

label¶

The Label instance that contains all the label information of the file.

Type: tensorbay.label.label.Label

classmethod loads(contents: Dict[str, Any]) → tensorbay.dataset.data._T[source]¶

Loads AuthData from a dict containing remote data information.

Parameters

contents –

A dict containing the information of the data, which looks like:

{
    "cloudPath": <str>,
    "timestamp": <float>,
    "label": {
        "CLASSIFICATION": {...},
        "BOX2D": {...},
        "BOX3D": {...},
        "POLYGON2D": {...},
        "POLYLINE2D": {...},
        "KEYPOINTS2D": {...},
        "SENTENCE": {...}
    }
}

Returns

An AuthData instance containing information from the given dict.

dumps() → Dict[str, Any][source]¶

Dumps the auth data into a dict.

Returns

Dumped data dict, which looks like:

{
    "cloudPath": <str>,
    "timestamp": <float>,
    "label": {
        "CLASSIFICATION": {...},
        "BOX2D": {...},
        "BOX3D": {...},
        "POLYGON2D": {...},
        "POLYLINE2D": {...},
        "KEYPOINTS2D": {...},
        "SENTENCE": {...}
    }
}

get_url() → str[source]¶

Return the url of the auth data.

Returns: The url of the auth data.
Raises: ValueError – When the url_getter is missing.

open() → http.client.HTTPResponse[source]¶

Return the binary file pointer of this file.

Returns: The cloud file pointer of this file path.