tensorbay.client.requests

Class Client and method multithread_upload.

Client can send POST, PUT, and GET requests to the TensorBay Dataset Open API.

multithread_upload() creates a multi-thread framework for uploading.

class tensorbay.client.requests.Config[source]

Bases: object

This is a base class defining the concept of Request Config.

max_retries

Maximum retry times of the request.

allowed_retry_methods

The allowed methods for retrying request.

allowed_retry_status

The allowed status for retrying request. If both methods and status are fitted, the retrying strategy will work.

timeout

Timeout value of the request in seconds.

is_internal

Whether the request is from internal.

class tensorbay.client.requests.TimeoutHTTPAdapter(*args: Any, timeout: Optional[int] = None, **kwargs: Any)[source]

Bases: requests.adapters.HTTPAdapter

This class defines the http adapter for setting the timeout value.

Parameters

  • *args – Extra arguments to initialize TimeoutHTTPAdapter.

  • timeout – Timeout value of the post request in seconds.

  • **kwargs – Extra keyword arguments to initialize TimeoutHTTPAdapter.

send(request: requests.models.PreparedRequest, stream: Any = False, timeout: Optional[Any] = None, verify: Any = True, cert: Optional[Any] = None, proxies: Optional[Any] = None) → Any[source]

Send the request.

Parameters

  • request – The PreparedRequest being sent.

  • stream – Whether to stream the request content.

  • timeout – Timeout value of the post request in seconds.

  • verify – A path string to a CA bundle to use or a boolean which controls whether to verify the server’s TLS certificate.

  • cert – User-provided SSL certificate.

  • proxies – Proxies dict applying to the request.

Returns

Response object.

class tensorbay.client.requests.UserSession[source]

Bases: requests.sessions.Session

This class defines UserSession.

request(method: str, url: str, *args: Any, **kwargs: Any) → requests.models.Response[source]

Make the request.

Parameters

  • method – Method for the request.

  • url – URL for the request.

  • *args – Extra arguments to make the request.

  • **kwargs – Extra keyword arguments to make the request.

Returns

Response of the request.

Raises

ResponseError – If post response error.

class tensorbay.client.requests.Client(access_key: str, url: str = '')[source]

Bases: object

This class defines Client.

Client defines the client that saves the user and URL information and supplies basic call methods that will be used by derived clients, such as sending GET, PUT and POST requests to TensorBay Open API.

Parameters

  • access_key – User’s access key.

  • url – The URL of the graviti gas website.

property session: tensorbay.client.requests.UserSession

Create and return a session per PID so each sub-processes will use their own session.

Returns

The session corresponding to the process.

open_api_do(method: str, section: str, dataset_id: str = '', **kwargs: Any) → requests.models.Response[source]

Send a request to the TensorBay Open API.

Parameters

  • method – The method of the request.

  • section – The section of the request.

  • dataset_id – Dataset ID.

  • **kwargs – Extra keyword arguments to send in the POST request.

Raises

ResponseError – When the status code OpenAPI returns is unexpected.

Returns

Response of the request.

do(method: str, url: str, **kwargs: Any) → requests.models.Response[source]

Send a request.

Parameters

  • method – The method of the request.

  • url – The URL of the request.

  • **kwargs – Extra keyword arguments to send in the GET request.

Returns

Response of the request.

class tensorbay.client.requests.Tqdm(*_, **__)[source]

Bases: tqdm.std.tqdm

A wrapper class of tqdm for showing the process bar.

Parameters

  • total – The number of excepted iterations.

  • disable – Whether to disable the entire progress bar.

update_callback(_: Any) → None[source]

Callback function for updating process bar when multithread task is done.

update_for_skip(condition: bool) → bool[source]

Update process bar for the items which are skipped in builtin filter function.

Parameters

condition – The filter condition, the process bar will be updated if condition is False.

Returns

The input condition.

tensorbay.client.requests.multithread_upload(function: Callable[[tensorbay.client.requests._T], Optional[tensorbay.client.requests._R]], arguments: Iterable[tensorbay.client.requests._T], *, callback: Optional[Callable[[Tuple[tensorbay.client.requests._R, ...]], None]] = None, jobs: int = 1, pbar: tensorbay.client.requests.Tqdm) → None[source]

Multi-thread upload framework.

Parameters

  • function – The upload function.

  • arguments – The arguments of the upload function.

  • callback – The callback function.

  • jobs – The number of the max workers in multi-thread uploading procession.

  • pbar – The Tqdm instance for showing the upload process bar.

class tensorbay.client.requests.MultiCallbackTask(*, function: Callable[[tensorbay.client.requests._T], Optional[tensorbay.client.requests._R]], callback: Callable[[Tuple[tensorbay.client.requests._R, ...]], None], size: int = 50)[source]

Bases: Generic[tensorbay.client.requests._T, tensorbay.client.requests._R]

A class for callbacking in multi-thread work.

Parameters

  • function – The function of a single thread.

  • callback – The callback function.

  • size – The size of the task queue to send a callback.

work(argument: tensorbay.client.requests._T) → None[source]

Do the work of a single thread.

Parameters

argument – The argument of the function.

last_callback() → None[source]

Send the last callback when all works have been done.