tensorbay.apps.sextant#

Interact with sextant app at graviti marketplace.

class tensorbay.apps.sextant.Evaluation(evaluation_id, created_at, benchmark)[source]#

Bases: object

This class defines Evaluation.

Parameters
  • evaluation_id (str) – Evaluation ID.

  • created_at (int) – Created time of the evaluation.

  • benchmark (Benchmark) – The Benchmark.

Return type

None

get_result()[source]#

Get the result of the evaluation.

Returns

The result dict of the evaluation.

Return type

Dict[str, Any]

class tensorbay.apps.sextant.Benchmark(name, benchmark_id, sextant, *, dataset_id=None, commit_id=None, categories=None, iou_threshold=None, customized_metrics=None)[source]#

Bases: object

This class defines Benchmark.

Parameters
  • name (str) – Name of the Benchmark.

  • dataset_id (Optional[str]) – ID of the dataset on which this benchmark based.

  • commit_id (Optional[str]) – ID of the commit which is used as the evaluation benchmark.

  • benchmark_id (str) – Benchmark ID.

  • sextant (Sextant) – The SextantClient.

  • categories (Optional[List[str]]) – The needed evaluation categories, if not given, all categories will be used.

  • iou_threshold (Optional[float]) – The IoU threshold.

  • customized_metrics (Optional[str]) – Https url of the github repository.

Return type

None

create_evaluation(dataset_id, commit_id)[source]#

Create an evaluation task.

Parameters
  • dataset_id (str) – Id of the needed evaluation dataset.

  • commit_id (str) – Id of the needed commit.

Returns

The created evaluation instance.

Return type

tensorbay.apps.sextant.Evaluation

list_evaluations()[source]#

List all evaluations.

Returns

A list of evaluations.

Return type

tensorbay.client.lazy.PagingList[tensorbay.apps.sextant.Evaluation]

class tensorbay.apps.sextant.Sextant(access_key, url='')[source]#

Bases: tensorbay.client.requests.Client

This class defines Sextant.

Parameters
  • access_key (str) – User’s access key.

  • url (str) – The URL of the graviti gas website.

Return type

None

list_benchmarks()[source]#

List all benchmarks.

Returns

The list of Benchmark instances.

Return type

tensorbay.client.lazy.PagingList[tensorbay.apps.sextant.Benchmark]

get_benchmark(name)[source]#

Get a benchmark instance by name.

Parameters

name (str) – Name of the Benchmark.

Returns

The Benchmark instance with the given name.

Raises

ResourceNotExistError – When the required benchmark does not exist.

Return type

tensorbay.apps.sextant.Benchmark