tensorbay.client.dataset#

The remote dataset on TensorBay.

class tensorbay.client.dataset.DatasetClientBase(name, dataset_id, gas, *, status, alias, is_public)[source]#

Bases: tensorbay.client.version.VersionControlMixin

This class defines the basic concept of the dataset client.

A DatasetClientBase contains the information needed for determining a unique dataset on TensorBay, and provides a series of method within dataset scope, such as DatasetClientBase.list_segment_names() and DatasetClientBase.upload_catalog().

Parameters
  • name (str) – Dataset name.

  • dataset_id (str) – Dataset ID.

  • gas (GAS) – The initial client to interact between local and TensorBay.

  • status (tensorbay.client.status.Status) – The version control status of the dataset.

  • alias (str) – Dataset alias.

  • is_public (bool) – Whether the dataset is public.

Return type

None

name#

Dataset name.

dataset_id#

Dataset ID.

status#

The version control status of the dataset.

property is_public: bool#

Return whether the dataset is public.

Returns

Whether the dataset is public.

property cache_enabled: bool#

Whether the cache is enabled.

Returns

Whether the cache is enabled.

property squash_and_merge: tensorbay.client.version.SquashAndMerge#

Get class SquashAndMerge.

Returns

Required SquashAndMerge.

Get class BasicSearch.

Returns

Required BasicSearch.

enable_cache(cache_path='')[source]#

Enable cache when open the remote data of the dataset.

Parameters

cache_path (str) – The path to store the cache.

Raises

StatusError – When enable cache under draft status.

Return type

None

update_notes(*, is_continuous=None, bin_point_cloud_fields=Ellipsis)[source]#

Update the notes.

Parameters
  • is_continuous (Optional[bool]) – Whether the data is continuous.

  • bin_point_cloud_fields (Optional[Iterable[str]]) – The field names of the bin point cloud files in the dataset.

Return type

None

get_notes()[source]#

Get the notes.

Returns

The Notes.

Return type

tensorbay.dataset.dataset.Notes

list_segment_names()[source]#

List all segment names in a certain commit.

Returns

The PagingList of segment names.

Return type

tensorbay.client.lazy.PagingList[str]

get_catalog()[source]#

Get the catalog of the certain commit.

Returns

Required Catalog.

Return type

tensorbay.label.catalog.Catalog

upload_catalog(catalog)[source]#

Upload a catalog to the draft.

Parameters

catalog (tensorbay.label.catalog.Catalog) – Catalog to upload.

Return type

None

delete_segment(name)[source]#

Delete a segment of the draft.

Parameters

name (str) – Segment name.

Return type

None

get_label_statistics()[source]#

Get label statistics of the dataset.

Returns

Required Statistics.

Return type

tensorbay.client.statistics.Statistics

get_total_size()[source]#

Get total data size of the dataset and the unit is byte.

Returns

The total data size of the dataset.

Return type

int

class tensorbay.client.dataset.DatasetClient(name, dataset_id, gas, *, status, alias, is_public)[source]#

Bases: tensorbay.client.dataset.DatasetClientBase

This class defines DatasetClient.

DatasetClient inherits from DataClientBase and provides more methods within a dataset scope, such as DatasetClient.get_segment(), DatasetClient.commit and DatasetClient.upload_segment(). In contrast to FusionDatasetClient, a DatasetClient has only one sensor.

Parameters
Return type

None

get_or_create_segment(name='default')[source]#

Get or create a segment with the given name.

Parameters

name (str) – The name of the fusion segment.

Returns

The created SegmentClient with given name.

Return type

tensorbay.client.segment.SegmentClient

create_segment(name='default')[source]#

Create a segment with the given name.

Parameters

name (str) – The name of the fusion segment.

Returns

The created SegmentClient with given name.

Raises

NameConflictError – When the segment exists.

Return type

tensorbay.client.segment.SegmentClient

copy_segment(source_name, target_name=None, *, source_client=None, strategy='abort')[source]#

Copy segment to this dataset.

Parameters
  • source_name (str) – The source name of the copied segment.

  • target_name (Optional[str]) – The target name of the copied segment. This argument is used to specify a new name of the copied segment. If None, the name of the copied segment will not be changed after copy.

  • source_client (Optional[tensorbay.client.dataset.DatasetClient]) – The source dataset client of the copied segment. This argument is used to specify where the copied segment comes from when the copied segment is from another commit, draft or even another dataset. If None, the copied segment comes from this dataset.

  • strategy (str) –

    The strategy of handling the name conflict. There are three options:

    1. ”abort”: stop copying and raise exception;

    2. ”override”: the source segment will override the origin segment;

    3. ”skip”: keep the origin segment.

Returns

The client of the copied target segment.

Return type

tensorbay.client.segment.SegmentClient

move_segment(source_name, target_name, *, strategy='abort')[source]#

Move/Rename segment in this dataset.

Parameters
  • source_name (str) – The source name of the moved segment.

  • target_name (str) – The target name of the moved segment.

  • strategy (str) –

    The strategy of handling the name conflict. There are three options:

    1. ”abort”: stop moving and raise exception;

    2. ”override”: the source segment will override the origin segment;

    3. ”skip”: keep the origin segment.

Returns

The client of the moved target segment.

Return type

tensorbay.client.segment.SegmentClient

get_segment(name='default')[source]#

Get a segment in a certain commit according to given name.

Parameters

name (str) – The name of the required segment.

Returns

The required SegmentClient.

Raises

ResourceNotExistError – When the required segment does not exist.

Return type

tensorbay.client.segment.SegmentClient

upload_segment(segment, *, jobs=1, skip_uploaded_files=False, quiet=False, _is_cli=False)[source]#

Upload a Segment to the dataset.

This function will upload all info contains in the input Segment, which includes:

  • Create a segment using the name of input Segment.

  • Upload all Data in the Segment to the dataset.

Parameters
  • segment (tensorbay.dataset.segment.Segment) – The Segment contains the information needs to be upload.

  • jobs (int) – The number of the max workers in multi-thread uploading method.

  • skip_uploaded_files (bool) – True for skipping the uploaded files.

  • quiet (bool) – Set to True to stop showing the upload process bar.

  • _is_cli (bool) – Whether the method is called by CLI.

Raises

Exception – When the upload got interrupted by Exception.

Returns

The SegmentClient used for uploading the data in the segment.

Return type

tensorbay.client.segment.SegmentClient

get_diff(*, head=None)[source]#

Get a brief diff between head and its parent commit.

Parameters

head (Optional[Union[str, int]]) – Target version identification. Type int for draft number, type str for revision. If not given, use the current commit id.

Return type

tensorbay.client.diff.DatasetDiff

Examples

>>> self.get_diff(head="b382450220a64ca9b514dcef27c82d9a")
Returns

The brief diff between head and its parent commit.

Parameters

head (Optional[Union[str, int]]) –

Return type

tensorbay.client.diff.DatasetDiff

class tensorbay.client.dataset.FusionDatasetClient(name, dataset_id, gas, *, status, alias, is_public)[source]#

Bases: tensorbay.client.dataset.DatasetClientBase

This class defines FusionDatasetClient.

FusionDatasetClient inherits from DatasetClientBase and provides more methods within a fusion dataset scope, such as FusionDatasetClient.get_segment(), FusionDatasetClient.commit and FusionDatasetClient.upload_segment(). In contrast to DatasetClient, a FusionDatasetClient has multiple sensors.

Parameters
Return type

None

get_or_create_segment(name='default')[source]#

Get or create a fusion segment with the given name.

Parameters

name (str) – The name of the fusion segment.

Returns

The created FusionSegmentClient with given name.

Return type

tensorbay.client.segment.FusionSegmentClient

create_segment(name='default')[source]#

Create a fusion segment with the given name.

Parameters

name (str) – The name of the fusion segment.

Returns

The created FusionSegmentClient with given name.

Raises

NameConflictError – When the segment exists.

Return type

tensorbay.client.segment.FusionSegmentClient

copy_segment(source_name, target_name=None, *, source_client=None, strategy='abort')[source]#

Copy segment to this dataset.

Parameters
  • source_name (str) – The source name of the copied segment.

  • target_name (Optional[str]) – The target name of the copied segment. This argument is used to specify a new name of the copied segment. If None, the name of the copied segment will not be changed after copy.

  • source_client (Optional[tensorbay.client.dataset.FusionDatasetClient]) – The source dataset client of the copied segment. This argument is used to specify where the copied segment comes from when the copied segment is from another commit, draft or even another dataset. If None, the copied segment comes from this dataset.

  • strategy (str) –

    The strategy of handling the name conflict. There are three options:

    1. ”abort”: stop copying and raise exception;

    2. ”override”: the source segment will override the origin segment;

    3. ”skip”: keep the origin segment.

Returns

The client of the copied target segment.

Return type

tensorbay.client.segment.FusionSegmentClient

move_segment(source_name, target_name, *, strategy='abort')[source]#

Move/Rename segment in this dataset.

Parameters
  • source_name (str) – The source name of the moved segment.

  • target_name (str) – The target name of the moved segment.

  • strategy (str) –

    The strategy of handling the name conflict. There are three options:

    1. ”abort”: stop moving and raise exception;

    2. ”override”: the source segment will override the origin segment;

    3. ”skip”: keep the origin segment.

Returns

The client of the moved target segment.

Return type

tensorbay.client.segment.FusionSegmentClient

get_segment(name='default')[source]#

Get a fusion segment in a certain commit according to given name.

Parameters

name (str) – The name of the required fusion segment.

Returns

The required FusionSegmentClient.

Raises

ResourceNotExistError – When the required fusion segment does not exist.

Return type

tensorbay.client.segment.FusionSegmentClient

upload_segment(segment, *, jobs=1, skip_uploaded_files=False, quiet=False)[source]#

Upload a fusion segment object to the draft.

This function will upload all info contains in the input FusionSegment, which includes:

  • Create a segment using the name of input fusion segment object.

  • Upload all sensors in the segment to the dataset.

  • Upload all frames in the segment to the dataset.

Parameters
  • segment (tensorbay.dataset.segment.FusionSegment) – The FusionSegment.

  • jobs (int) – The number of the max workers in multi-thread upload.

  • skip_uploaded_files (bool) – Set it to True to skip the uploaded files.

  • quiet (bool) – Set to True to stop showing the upload process bar.

Raises

Exception – When the upload got interrupted by Exception.

Returns

The FusionSegmentClient

used for uploading the data in the segment.

Return type

tensorbay.client.segment.FusionSegmentClient