tensorbay.dataset.data¶
Data.
Data
is the most basic data unit of a Dataset
.
It contains path information of a data sample and its corresponding labels.
- class tensorbay.dataset.data.DataBase(path: str, *, timestamp: Optional[float] = None)[source]¶
Bases:
tensorbay.utility.attr.AttrsMixin
,tensorbay.utility.repr.ReprMixin
DataBase is a base class for the file and label combination.
- Parameters
path – The file path.
timestamp – The timestamp for the file.
- path¶
The file path.
- timestamp¶
The timestamp for the file.
- Type
float
- static loads(contents: Dict[str, Any]) _Type [source]¶
Loads data subclass from a dict containing data information.
- Parameters
contents –
A dict containing the information of the data, which looks like:
{ "localPath", "remotePath" or "authPath": <str>, "timestamp": <float>, "label": { "CLASSIFICATION": {...}, "BOX2D": {...}, "BOX3D": {...}, "POLYGON2D": {...}, "POLYLINE2D": {...}, "KEYPOINTS2D": {...}, "SENTENCE": {...} } }
- Returns
A
Data
,RemoteData
or class`AuthData` instance containing information from the given dict.- Raises
KeyError – When the “localPath”, “remotePath” or “authPath” not in contents.
- class tensorbay.dataset.data.Data(local_path: str, *, target_remote_path: Optional[str] = None, timestamp: Optional[float] = None)[source]¶
Bases:
tensorbay.dataset.data.DataBase
Data is a combination of a specific local file and its label.
It contains the file local path, label information of the file and the file metadata, such as timestamp.
A Data instance contains one or several types of labels.
- Parameters
local_path – The file local path.
target_remote_path – The file remote path after uploading to tensorbay.
timestamp – The timestamp for the file.
- path¶
The file local path.
- Type
str
- timestamp¶
The timestamp for the file.
- Type
float
- target_remote_path¶
The target remote path of the data.
- classmethod loads(contents: Dict[str, Any]) tensorbay.dataset.data._T [source]¶
Loads
Data
from a dict containing local data information.- Parameters
contents –
A dict containing the information of the data, which looks like:
{ "localPath": <str>, "timestamp": <float>, "label": { "CLASSIFICATION": {...}, "BOX2D": {...}, "BOX3D": {...}, "POLYGON2D": {...}, "POLYLINE2D": {...}, "KEYPOINTS2D": {...}, "SENTENCE": {...} } }
- Returns
A
Data
instance containing information from the given dict.
- open() _io.BufferedReader [source]¶
Return the binary file pointer of this file.
The local file pointer will be obtained by build-in
open()
.- Returns
The local file pointer for this data.
- dumps() Dict[str, Any] [source]¶
Dumps the local data into a dict.
- Returns
Dumped data dict, which looks like:
{ "localPath": <str>, "timestamp": <float>, "label": { "CLASSIFICATION": {...}, "BOX2D": {...}, "BOX3D": {...}, "POLYGON2D": {...}, "POLYLINE2D": {...}, "KEYPOINTS2D": {...}, "SENTENCE": {...} } }
- class tensorbay.dataset.data.RemoteData(remote_path: str, *, timestamp: Optional[float] = None, url_getter: Optional[Callable[[str], str]] = None)[source]¶
Bases:
tensorbay.dataset.data.DataBase
RemoteData is a combination of a specific tensorbay dataset file and its label.
It contains the file remote path, label information of the file and the file metadata, such as timestamp.
A RemoteData instance contains one or several types of labels.
- Parameters
remote_path – The file remote path.
timestamp – The timestamp for the file.
url_getter – The url getter of the remote file.
- path¶
The file remote path.
- Type
str
- timestamp¶
The timestamp for the file.
- Type
float
- classmethod loads(contents: Dict[str, Any]) tensorbay.dataset.data._T [source]¶
Loads
RemoteData
from a dict containing remote data information.- Parameters
contents –
A dict containing the information of the data, which looks like:
{ "remotePath": <str>, "timestamp": <float>, "label": { "CLASSIFICATION": {...}, "BOX2D": {...}, "BOX3D": {...}, "POLYGON2D": {...}, "POLYLINE2D": {...}, "KEYPOINTS2D": {...}, "SENTENCE": {...} } }
- Returns
A
Data
instance containing information from the given dict.
- get_url() str [source]¶
Return the url of the data hosted by tensorbay.
- Returns
The url of the data.
- Raises
ValueError – When the url_getter is missing.
- open() http.client.HTTPResponse [source]¶
Return the binary file pointer of this file.
The remote file pointer will be obtained by
urllib.request.urlopen()
.- Returns
The remote file pointer for this data.
- dumps() Dict[str, Any] [source]¶
Dumps the remote data into a dict.
- Returns
Dumped data dict, which looks like:
{ "remotePath": <str>, "timestamp": <float>, "label": { "CLASSIFICATION": {...}, "BOX2D": {...}, "BOX3D": {...}, "POLYGON2D": {...}, "POLYLINE2D": {...}, "KEYPOINTS2D": {...}, "SENTENCE": {...} } }
- class tensorbay.dataset.data.AuthData(cloud_path: str, *, target_remote_path: Optional[str] = None, timestamp: Optional[float] = None, url_getter: Optional[Callable[[str], str]] = None)[source]¶
Bases:
tensorbay.dataset.data.DataBase
AuthData is a combination of a specific cloud storaged file and its label.
It contains the cloud storage file path, label information of the file and the file metadata, such as timestamp.
An AuthData instance contains one or several types of labels.
- Parameters
cloud_path – The cloud file path.
target_remote_path – The file remote path after uploading to tensorbay.
timestamp – The timestamp for the file.
url_getter – The url getter of the remote file.
- path¶
The cloud file path.
- Type
str
- timestamp¶
The timestamp for the file.
- Type
float
- classmethod loads(contents: Dict[str, Any]) tensorbay.dataset.data._T [source]¶
Loads
AuthData
from a dict containing remote data information.- Parameters
contents –
A dict containing the information of the data, which looks like:
{ "cloudPath": <str>, "timestamp": <float>, "label": { "CLASSIFICATION": {...}, "BOX2D": {...}, "BOX3D": {...}, "POLYGON2D": {...}, "POLYLINE2D": {...}, "KEYPOINTS2D": {...}, "SENTENCE": {...} } }
- Returns
An
AuthData
instance containing information from the given dict.
- dumps() Dict[str, Any] [source]¶
Dumps the auth data into a dict.
- Returns
Dumped data dict, which looks like:
{ "cloudPath": <str>, "timestamp": <float>, "label": { "CLASSIFICATION": {...}, "BOX2D": {...}, "BOX3D": {...}, "POLYGON2D": {...}, "POLYLINE2D": {...}, "KEYPOINTS2D": {...}, "SENTENCE": {...} } }