stelar.client.proxy package

Submodules

stelar.client.proxy.decl module

class stelar.client.proxy.decl.ProxyState(value)

Bases: Enum

CLEAN = 2

DIRTY = 3

EMPTY = 1

ERROR = 0

stelar.client.proxy.decl.tag_join(vocname: str | None, tagname: str) → str

Return the tagspec of a tag and a vocabulary.

Parameters:

• vocname (str|None) – the vocabulary name

• tagname – the tag name

Returns:

(str) the tagspec

stelar.client.proxy.decl.tag_split(tagspec: str) → tuple[str | None, str]

Split a tagspec into a pair or (<vocabulary-name> , <tag-name>).

Properly, a tagspec is either <tag-name> or <vocabulary-name>:<tag-name>, where

<tagname>

is a string made only of lower-case alphanumerics, hyphen (-) and underscore (_), and of length in [2,100]

<vocabulary-name>

is any string (which may contain spaces and other ascii characters) of length [2,100].

Parameters:

(str) (tagspec)

Returns:

(str|None, str)

Return type:

a pair of (<vocabulary-name> , <tag-name>)

Raises:

ValueError – if the tagspec is not valid:

stelar.client.proxy.decl.validate_tagname(tagname: str) → bool

Check if a string is formatted correctly as a tagname

stelar.client.proxy.decl.validate_tagspec(tagspec: str) → bool

Check if a string is formatted correctly as a tagspec

stelar.client.proxy.derived module

class stelar.client.proxy.derived.derived_property(method=None, **kwargs)

Bases: Property

Derived properties are used to cache some ‘expensive’ value into the proxy.

The value is recomputed at each proxy_sync(), during a call to ‘proxy_from_entity()’. This value is then returned by the getter. Additional ‘vanilla python properties’ or methods can be declared in the usual way, to implement richer functionality.

convert_entity_to_proxy(proxy: Proxy, entity: Any, **kwargs)

Update proxy dict to represent this property from the entity

convert_proxy_to_entity(proxy: Proxy, entity: dict, **kwargs)

Update entity dict to represent this property from the proxy. The changes flag is true when the proxy_attr is actually the Proxy.proxy_changes dict.

convert_to_create(proxy_type, create_props: dict, entity_props: dict, **kwargs)

Convert a value to be used for entity creation.

Parameters:

• client (Client) – The client used for creation.

• proxy_type (ProxyClass) – The entity type being created.

• create_props – The object passed to the create client call

• entity_props – The entity object given to the create API call.

set(obj, value)

Low-level setter

stelar.client.proxy.exceptions module

exception stelar.client.proxy.exceptions.ConflictError(conflicted_proxy, new_entity, *args)

Bases: ProxyError

Update for an entity with unsynchronized changes

exception stelar.client.proxy.exceptions.ConversionError(property, conv_type)

Bases: EntityError

A problem with conversion to and from entity

exception stelar.client.proxy.exceptions.EntityError

Bases: ValueError

Raised when the system encounters a malformed entity

exception stelar.client.proxy.exceptions.EntityNotFound(proxy_type, eid, operation, purged=False)

Bases: ProxyOperationError

Indicate that an entity is not found

exception stelar.client.proxy.exceptions.ErrorState

Bases: ProxyError

Operation on proxy in ERROR state

exception stelar.client.proxy.exceptions.InvalidationError

Bases: ProxyError

Invalidation attempt on dirty proxy

exception stelar.client.proxy.exceptions.ProxyError

Bases: RuntimeError

The proxying state is not consistent or an illegal proxying command was issued

exception stelar.client.proxy.exceptions.ProxyOperationError(proxy_type, eid, operation, *args, **kwargs)

Bases: ProxyError

An error occurred during an API operation

stelar.client.proxy.extras module

class stelar.client.proxy.extras.ExtrasProperty(**kwargs)

Bases: Property

autodoc(doc, repr_type, repr_constraints)

convert_entity_to_proxy(proxy: Proxy, entity, **kwargs)

Update proxy dict to represent this property from the entity

convert_proxy_to_entity(proxy: Proxy, entity: dict, **kwargs)

Update entity dict to represent this property from the proxy. The changes flag is true when the proxy_attr is actually the Proxy.proxy_changes dict.

convert_to_create(proxy_type, create_props, entity_props, **kwargs)

Convert a value to be used for entity creation.

Parameters:

• client (Client) – The client used for creation.

• proxy_type (ProxyClass) – The entity type being created.

• create_props – The object passed to the create client call

• entity_props – The entity object given to the create API call.

get(obj: Proxy)

Low-level getter

touch(obj: Proxy) → bool

Mark the extras as changed.

This overloads the default touch, because after the touch, the extras dict needs to be copied (else, changing the ‘proxy_attr’ instance will change the ‘proxy_changed’ instance, which is not what we want).

class stelar.client.proxy.extras.ExtrasProxy(registry: Registry, eid: str | UUID | None = None, entity=None)

Bases: Proxy

The class implements a Proxy with an ‘extras’ field.

There are two API variants for extra arguments in the CKAN data catalog: * Resource entities allow additional fields in their objects.:

"foo": "bar"

• Packages, Groups and Organizations maintain a separate field called extras, whose format is a list of dicts.:

  "extras": [{"key": "foo", "value": "bar"}]
  

Our own implementation follows the resource approach, by utilizing dynamic attributes.

stelar.client.proxy.fieldvalidation module

Proxy field validation

Proxied entity fields (attributes) have two representations: one that appears in the entity (JSON object stored in a dict) and one that is used by the proxy objects.

client proxy value <–> json entity value

For example, dates are represented as datetime objects in the proxy and as strings in the JSON entity. Conversion between the two is done via the two conversion methods of the classes herein.

class stelar.client.proxy.fieldvalidation.AnyField(repr_type='Any', *args, **kwargs)

Bases: FieldValidator

A very promiscuous basic validator.

convert_to_entity(value, **kwargs)

convert_to_proxy(value, **kwargs)

default_value()

repr_type()

class stelar.client.proxy.fieldvalidation.BasicField(ftype, **kwargs)

Bases: AnyField

Given ftype T, accept value if it is an instance of T or if T(value) succeeds. Conversion to T is performed.

Subclasses include basic types: str, int, bool

repr_type()

to_ftype(value, **kwargs)

Validator stage for BasicField

class stelar.client.proxy.fieldvalidation.BoolField(**kwargs)

Bases: BasicField

A bool field validator

class stelar.client.proxy.fieldvalidation.DateField(**kwargs)

Bases: FieldValidator

convert_to_entity(value: datetime, **kwargs) → str

convert_to_proxy(value: str, **kwargs) → datetime

repr_type()

to_date(value: Any, **kwargs) → tuple[datetime, bool]

Validation stage for dates.

class stelar.client.proxy.fieldvalidation.DateFmtField(datefmt, **kwargs)

Bases: FieldValidator

convert_to_entity(value: datetime, **kwargs) → str

convert_to_proxy(value: str, **kwargs) → datetime

repr_type()

to_date(value: Any, **kwargs) → tuple[datetime, bool]

Validation stage for dates.

class stelar.client.proxy.fieldvalidation.DictField(key_type, value_type, **kwargs)

Bases: AnyField

check_dict(value, **kwargs)

class stelar.client.proxy.fieldvalidation.EnumeratedField(*args, **kwargs)

Bases: AnyField

Fields with a fixed set of legal values.

Subclasses can redefine the VALUES class attribute.

VALUES = []

oneof(value)

repr_type()

class stelar.client.proxy.fieldvalidation.ExecStateField

Bases: EnumeratedField

VALUES = ['running', 'succeeded', 'failed']

class stelar.client.proxy.fieldvalidation.FieldValidator(*, strict: bool = False, nullable: bool = True, default: Any = Ellipsis, minimum_value: Any = None, maximum_value: Any = None, maximum_len: int | None = None, minimum_len: int | None = None)

Bases: object

Provide simple validation and conversion for entity fields.

A validation is a sequence of checks. Each check can: - Raise an exception of ValueError - Apply a value conversion and continue - Apply a value conversion and terminate

Any function that takes as input a value, and returns a pair (newvalue, done) where done is a boolean, can be used as a check.

At the end of all conversions, if no conversion signaled done, the strict attribute is checked. If True, an error is raised, else (the default) conversion succeeds.

add_check(check_func, pri: int)

check_length(value, **kwargs)

check_maximum(value, **kwargs)

check_minimum(value, **kwargs)

check_null(value, **kwargs)

convert_to_entity(value, **kwargs)

convert_to_proxy(value, **kwargs)

default_value()

repr_constraints()

validate(value, **kwargs)

class stelar.client.proxy.fieldvalidation.FrozenField(**kwargs)

Bases: AnyField

A field that accepts any JSON-serializable value.

check_freeze(value, **kwargs)

Validator stage for JSONField

convert_to_proxy(value, **kwargs)

repr_type()

class stelar.client.proxy.fieldvalidation.IntField(**kwargs)

Bases: BasicField

An int field validator

class stelar.client.proxy.fieldvalidation.JSONField(**kwargs)

Bases: AnyField

A field that accepts any JSON-serializable value.

check_json(value, **kwargs)

Validator stage for JSONField

convert_to_entity(value, **kwargs)

convert_to_proxy(value, **kwargs)

repr_type()

class stelar.client.proxy.fieldvalidation.LicenseNameField(**kwargs)

Bases: NameField

NAME_PATTERN = re.compile('[a-z0-9_-]+')

class stelar.client.proxy.fieldvalidation.ListField(element_type, **kwargs)

Bases: AnyField

check_list(value, **kwargs)

repr_type()

class stelar.client.proxy.fieldvalidation.NameField(**kwargs)

Bases: StrField

Name fields are non-nullable string fields whose value must follow a pattern.

NAME_PATTERN = re.compile('[a-z0-9_-]+')

check_name(value: str, **kwargs)

class stelar.client.proxy.fieldvalidation.StateField

Bases: EnumeratedField

VALUES = ['active', 'deleted']

class stelar.client.proxy.fieldvalidation.StrField(**kwargs)

Bases: BasicField

A string field validator

class stelar.client.proxy.fieldvalidation.TagNameField(**kwargs)

Bases: NameField

NAME_PATTERN = re.compile('[A-Za-z0-9 ._-]+')

class stelar.client.proxy.fieldvalidation.UUIDField(**kwargs)

Bases: BasicField

convert_to_entity(value: UUID, **kwargs) → str

convert_to_proxy(value: str, **kwargs) → UUID

repr_type()

class stelar.client.proxy.fieldvalidation.VocabNameField(**kwargs)

Bases: NameField

NAME_PATTERN = re.compile('.+')

stelar.client.proxy.property module

class stelar.client.proxy.property.DictProperty(key_type, value_type, *, nullable=False, **kwargs)

Bases: ProxyProperty

class stelar.client.proxy.property.DictProxy(proxy, property)

Bases: PropertyProxy, MutableMapping

class stelar.client.proxy.property.Id(entity_name=None, doc=None)

Bases: Property

A Python descriptor for implementing entity ID access.

class stelar.client.proxy.property.ListProperty(value_type, *, nullable=False, **kwargs)

Bases: ProxyProperty

class stelar.client.proxy.property.ListProxy(proxy, property)

Bases: PropertyProxy, MutableSequence

insert(index, value)

S.insert(index, value) – insert value before index

class stelar.client.proxy.property.NameId(entity_name=None, doc=None, validator=<class 'stelar.client.proxy.fieldvalidation.NameField'>)

Bases: Property

Many entities have a name field with a unique constaint.

class stelar.client.proxy.property.Property(*, validator=None, updatable=False, optional=False, entity_name=None, doc=None, create_default=None, short=None)

Bases: object

A Python descriptor for implementing access and updating of fields of proxy objects.

A property object performs the following roles:

• Is a python ‘descriptor’ (implements getter, setter and deleter) for proxy classes

• Holds a number of metadata that determine the behaviour of the proxy (validation, conversion to entity, nullabilty, updatability, optionality, view, default values at creation, etc)

• Performs internalization/externalization for the data.

Properties are the mechanism for most of the functionality of the STELAR client.

autodoc(doc, repr_type, repr_constraints)

convert_entity_to_proxy(proxy: Proxy, entity: Any, **kwargs)

Update proxy dict to represent this property from the entity

convert_proxy_to_entity(proxy: Proxy, entity: dict, **kwargs)

Update entity dict to represent this property from the proxy. The changes flag is true when the proxy_attr is actually the Proxy.proxy_changes dict.

convert_to_create(proxy_type: type, create_props: dict[str, Any], entity_props: dict[str, Any], **kwargs)

Convert a value to be used for entity creation.

Parameters:

• client (Client) – The client used for creation.

• proxy_type (ProxyClass) – The entity type being created.

• create_props – The object passed to the create client call

• entity_props – The entity object given to the create API call.

get(obj)

Low-level getter

missing(*, proxy=None, registry=None, **kwargs)

Provides missing values during entity creation.

Args: proxy (a ‘create proxy’ is ok) or registry, one muste be provided.

property qualname

set(obj, value)

Low-level setter

touch(obj: Proxy) → bool

Transition the initial value of a clean proxy to the ‘proxy_changed’ dictionary.

This is done only on the first update to an attribute, in order to allow for the proxy_reset() functionality.

Parameters:

obj (Proxy) – The proxy object whose property is touched.

Returns:

True if the touch actually happened, False if the property was already touched.

validate(obj, value)

class stelar.client.proxy.property.PropertyProxy(proxy, property)

Bases: object

class stelar.client.proxy.property.ProxyProperty(property_proxy_class, **kwargs)

Bases: Property

A proxy property allows for properties whose values are objects.

As an example, a dict-valued property can be accessed via a proxy object, which detects changes to the dictionary and automatically updates the entity proxy.

For this to work, the proxy property is parametrized with a PropertyProxy class, whose instances are to be returned by the __get__ method.

touch(obj: Proxy) → bool

Mark the property as changed.

stelar.client.proxy.proxy module

class stelar.client.proxy.proxy.Proxy(registry: Registry, eid: str | UUID | None = None, entity=None)

Bases: object

Base class for all proxy objects of the STELAR entities.

Proxy objects are managed by Registry. The primary implementation of Registry is Client.

A proxy can be in one of four states:

• EMPTY: there is no entity data in the proxy

• CLEAN: the data loaded by the last proxy_sync() operation is not changed

• DIRTY: the data loaded by the last proxy_sync() operation has been changed

• ERROR: the proxy is illegal! This state can be the result of deleting entity

Attributes are used to hold property values:

proxy_registry:

The Registry instance that this proxy belongs to

proxy_id:

The UUID of the proxies entity

proxy_attr:

A dict of all loaded attributes. When None, the entity has not yet been loaded. The state is EMPTY

proxy_changed:

A dict of all changed attributes (loaded values of updated attributes) since last upload. When None, the entity is CLEAN (or EMPTY), else the entity is DIRTY.

To initialize a proxy object, one must supply either an entity id or an entity JSON body. The proxy_id is never changed. When an entity is deleted, set the proxy_id to None, to mark the proxy state as ERROR.

After initialization, the state of a proxy is EMPTY.

Proxies are handlers for entities of the Stelar Service API. Entities are manipulated by a CRUD-like API. Besides creation and deletion, entities are manipulated by two additional I/O API operations: - fetch, which returns an entity data from the API - update, which accepts a spec of the updates to apply to an entity. This operation often returns the updated object after updates are applied.

The following operations operate on proxies:

proxy_sync(entity=None):

Save any pending updates to make the state CLEAN. Load the proxy data from the Stelar Service API, to make sure the proxy has the latest. When entity is not None, use it to avoid a fetch.

proxy_invalidate(force=False):

Make the object EMPTY. If the proxy is DIRTY, an IvalidationError is raised, unless force is specified as True.

proxy_reset():

Make a DIRTY object to CLEAN, by restoring the property values of the last sync().

proxy_sync(entity=None):

Make an entity CLEAN.

delete(purge: bool = False)

Delete the entity and mark the proxy as invalid. Entity classes can overload this method, to perform the actual API delete. When successful, they can then invoke Proxy.delete() to mark this proxy as invalid.

classmethod new(regspec: Registry | RegistryCatalog, *, autosync: bool = True, **fields)

Return a non-affiliated proxy instance, i.e. a ‘creation proxy’.

This proxy instance’s id is UUID(int=0), which is indicative of a non-valid id. The purpose of such an object is to create a new entity, by calling ‘proxy_sync()’.

While the UUID is 0, this entity is not registered in the registry. Therefore, multiple such entries can exist. However, once the entity is proxy_sync’d, its id changes to a proper one and the object is registered in the registry.

Parameters:

• cls (Type[ProxyClass]) – the proxy type for the new proxy.

• regspec (Registry|RegistryCatalog) – Registry spec for the new object. If regspec is a Registry, its catalog is used to locate a suitable registry for the ‘cls’ proxy type, via regspec.catalog.registry_for(cls). If regspec is a catalog, it provides the registry via regspec.registry_for(cls).

• (bool (autosync) –

  True): When True, the new entity is created (by calling proxy_sync() on the proxy) before returning. When False, the new entity is not yet created, a call to proxy_sync() must be done later.

  Note that the proxy_aytosync field of the returned proxy is set to true, regardless of the value of this parameter.

• default –

  True): When True, the new entity is created (by calling proxy_sync() on the proxy) before returning. When False, the new entity is not yet created, a call to proxy_sync() must be done later.

  Note that the proxy_aytosync field of the returned proxy is set to true, regardless of the value of this parameter.

• fields (dict[str][Any]) – values to initialize the new entity from

Returns:

A proxy to a new entity. If autosync if False, the new proxy does not yet correspond to an entity.

classmethod new_entity(catalog: RegistryCatalog = None, /, **fields) → Entity

Return a set of fields for creating a new entity.

Parameters:

fields (dict[str,Any])

proxy_attr: dict[str, Any] | None

proxy_autocommit()

Try a call to proxy_sync() if the proxy is in auto sync.

This call is invoked whenever the proxy is changed. If the proxy is in autosync mode and DIRTY, the method will try to sync the proxy with the API. If this fails, the proxy is reset to CLEAN.

If the proxy is not DIRTY, or not in autosync mode, the method does nothing.

proxy_autosync: bool

proxy_changed: dict[str, Any] | None

proxy_from_entity(entity: Any)

Update the proxy_attr dictionary from a given entity.

proxy_id: UUID | None

proxy_invalidate(*, force=False)

Make this proxy object EMPTY, discarding any entity data.

If the proxy object is DIRTY, a InvalidationError exception is raised, unless force is true.

Parameters:

force (bool) – Invalidate even if DIRTY, without raising exception. Deafults to False.

Raises:

InvalidationError – If called on a DIRTY proxy with force being False

proxy_is_purged()

Called to designate that this proxy is referring to a non-existent entity and should be marked as such.

This type of marking happens when an entity is purged.

proxy_registry: Registry

proxy_reset()

If proxy is EMPTY, do nothing. If the proxy is DIRTY, make it CLEAN by restoring the values changed since the last sync.

property proxy_state: ProxyState

Return the proxy state

proxy_sync(entity=None)

Sync the data between the proxy and the API entity.

After a sync, the proxy is CLEAN and consistent with the underlying entity in the Data Catalog. This method must be overloaded in subclasses, to cater to the details of different types of entities.

In order to sync, the method works as follows:

1. If the proxy is DIRTY:

   • updates are sent to the API.

   • The API optionally returns a new entity object. If so, override the entity parameter.

   • Make object CLEAN (by setting proxy_changed to None)

2. If entity is None: load entity from API

3. Update the proxy data from entity. This may involve updating additional proxies with data contained in the given entity.

For typical operations, the implementation can use the following mathods: ` self.proxy_from_entity(entity) self.proxy_to_entity(attrset) -> entity `

proxy_to_Series(*, sync_empty: bool = True, include_null: bool = False, include_extras: bool = False, simplify: bool = True) → Series

Return a pandas Series for this entity.

The pandas Series index will contain entity fields. The values will be simplified, to reflect

Parameters:

• sync_empty (bool, default=True) – call proxy_sync() if state is EMPTY

• include_null (bool, default=False) – include fields that evaluate to False

• include_extras (bool, default=False) – also include any extras fields

• simplify (bool, default=True) – return a more printable, simpler representation

proxy_to_entity(attrset: set[str] | dict[str, Any] | None = None) → dict[str, Any]

Return an entity from the proxy values.

Note that the entity returned will not contain the id attribute.

Parameters:

attrset (set of property names, optional) –

If not None, determines the set of names to add to the entity.

Any type of object, where the expression name in attrset is valid, can be used.

Use this to only add names of changed properties to an entity: self.proxy_to_entity(self.proxy_changed)

Returns:

An entity dict containing all values

specified.

Return type:

entity (dict)

property s: Series

property sl: Series

property sraw: Series

property ss: Series

property sx: Series

property sxl: Series

update(**updates: Any)

Update a bunch of attributes in a single operation.

stelar.client.proxy.proxy.deferred_sync(*proxies)

stelar.client.proxy.proxycursor module

class stelar.client.proxy.proxycursor.ProxyCursor(client: Client, proxy_type: Type[ProxyClass])

Bases: Registry[ProxyClass]

Proxy cursors provide access functions to the collection of all entities of a given type.

The cursor is used to fetch entities in a lazy way, and to provide a way to iterate over the entities, check for the existence of an entity by name, etc.

MAX_FETCH = 1000

create(**updates) → ProxyClass

Create a new entity of this type.

fetch(*, limit, offset) → Iterator[ProxyClass]

Fetch entities of this type.

fetch_list(*, limit, offset) → list[str]

Lists entities of this type.

get(name_or_id, default=None) → ProxyClass

Get an entity by name or ID.

stelar.client.proxy.proxylist module

class stelar.client.proxy.proxylist.ProxyList(client: Client, proxy_type: Type[ProxyClass])

Bases: Generic[ProxyClass]

Base class for “dynamic lists” of proxies.

The instances maintain lists of IDs and translate list operations as if the list actually contained proxy objects.

property DF

Return a dataframe over all fields.

property coll: list[UUID]

property df

Return a dataframe over the default fields.

get_slice(slc: slice) → list[UUID]

Return a list of UUIDs for the given slice.

This method is used to implement slicing operations on the ProxyList. It should be overriden in subclasses depending on the underlying implementation the the collection.

property ids

resolve_proxy(item)

This is the main routine that transforms elements of the list to proxies.

to_df(*additional_fields, fields=None, simplify=True)

Generate a pandas dataframe for the list of proxy entities.

The dataframe is generated by fetching and tabulating a subset of fields, for each entity in the list.

class stelar.client.proxy.proxylist.ProxySublist(property: RefList, owner: Proxy)

Bases: ProxyList

A proxy class that translates collection operations to operations on an entity sub-collection.

property coll

get_slice(slc: slice) → list[UUID]

Return a list of UUIDs for the given slice.

This method is used to implement slicing operations on the ProxyList. It should be overriden in subclasses depending on the underlying implementation the the collection.

class stelar.client.proxy.proxylist.ProxyVec(client: Client, proxy_type: Type[ProxyClass], members: list[UUID])

Bases: ProxyList

A list of IDs appearing as proxies.

The underlying data is a list of UUIDs. At each element access, the correpsonding element is fetched from the registry.

property coll

get_slice(slc: slice) → list[UUID]

Return a list of UUIDs for the given slice.

This method is used to implement slicing operations on the ProxyList. It should be overriden in subclasses depending on the underlying implementation the the collection.

class stelar.client.proxy.proxylist.ShortenedUUID(hex=None, bytes=None, bytes_le=None, fields=None, int=None, version=None, *, is_safe=SafeUUID.unknown)

Bases: UUID

int

is_safe

stelar.client.proxy.proxylist.simplified(value, property)

stelar.client.proxy.proxysync module

class stelar.client.proxy.proxysync.ProxySynclist(l: list[Proxy] = [])

Bases: object

A container for the proxies that need to be sync’d after an operation.

add(ref: Proxy | ProxyList)

Add a proxy or the elements of a proxy list to the synclist.

Any other types are ignored.

on_create(proxy_type: Type[ProxyClass], properties: dict[str, Any])

on_create_proxy(proxy: ProxyClass)

on_delete(proxy: Proxy)

on_update(proxy: Proxy, prop: Property, newref: Proxy)

on_update_all(proxy: ProxyClass)

sync()

trigger_properties(schema: Schema)

stelar.client.proxy.refs module

class stelar.client.proxy.refs.RefField(ref_property: Reference, ref_typename: str, **kwargs)

Bases: AnyField

This field validator is specialized for reference properties

convert_to_entity(value: UUID, **kwargs) → str

convert_to_proxy(value: str, **kwargs) → UUID

property proxy_type

repr_type()

to_uuid(value, **kwargs)

class stelar.client.proxy.refs.RefList(*args, element_validator=<class 'stelar.client.proxy.fieldvalidation.UUIDField'>, **kwargs)

Bases: Reference

A proxy property that manages a sub-schema (a sub-collection of other properties)

VALIDATOR_CLASS

alias of RefListField

convert_entity_to_proxy(proxy, entity, **kwargs)

Update proxy dict to represent this property from the entity

convert_proxy_to_entity(proxy, entity, **kwargs)

Update entity dict to represent this property from the proxy. The changes flag is true when the proxy_attr is actually the Proxy.proxy_changes dict.

get(obj)

Low-level getter

classmethod property_type_name(proxy_type) → str

set(obj, value)

Low-level setter

class stelar.client.proxy.refs.RefListField(ref_property: RefList, ref_typename: str, *, element_validator=None, **kwargs)

Bases: AnyField

This field validator is specialized for reference properties

convert_to_entity(value: list, **kwargs) → str

convert_to_proxy(value: str, **kwargs) → UUID

property proxy_type

repr_type()

to_list(value, **kwargs)

class stelar.client.proxy.refs.Reference(proxy_type, nullable=False, trigger_sync=False, **kwargs)

Bases: Property

A proxy property which is a reference to an entity.

VALIDATOR_CLASS

alias of RefField

get(obj)

Low-level getter

classmethod property_type_name(proxy_type) → str

property proxy_type

The class of the proxy object pointed to by this

registry_for(obj) → Registry

Return the registry for the referrent, given owner

validator_for(proxy_type, **kwargs)

stelar.client.proxy.registry module

class stelar.client.proxy.registry.Registry(catalog: RegistryCatalog, proxy_type)

Bases: Generic[ProxyClass]

A registry is a factory and collection of proxies.

The registry maintains a weak reference to its elements, so that a proxy which is not referenced any more, is deleted from the registry.

Registries are crucial as they are used to maintain the following important invariant:

‘For each entity, there exists no more than one proxy (under the same client) at any time’.

This invariant guarantees that, any updates to an entity (via a client) can only happen on a uniquely defined object. This solution saves on memory and avoids complex state synchronization problems.

Registries are typed: only proxies of the same type belong to a registry.

A collection of registries of different types form a ‘catalog’ (of type RegistryCatalog).

fetch_proxy(eid: UUID) → ProxyClass

Return a proxy for the provided entity id.

If a proxy needs to be created, it will be initialized in the EMPTY state by the provided entity object.

Parameters:

eid – The provided entity ID.

Returns:

a proxy initialized with the provided entity.

fetch_proxy_for_entity(entity) → ProxyClass

Return a proxy for the entity object provided.

The returned proxy will be created if needed, and it will be initialized in the CLEAN state by the provided entity object.

Parameters:

entity – The provided entity object.

Returns:

a proxy initialized with the provided entity.

invalidate()

Invalidate clean entries in the registry.

This method invalidates all CLEAN proxies in the registry, setting them to the EMPTY state. This will cause all subsequent accesses of these proxies to retrieve fresh data from the KLMS.

The method refrains from touching dirty proxies, in order to avoid losing any unsaved changes.

purge_proxy(proxy)

register_proxy_for_entity(proxy: Proxy, entity: dict)

Register an existing proxy ebject for the given entity.

For success, 1. the proxy id must be UUID(int=0) 2. the entity must contain an id 3. the id must not exist in the registry already.

On success, the entity id is copied into the proxy and the proxy is registered.

On failure, a conflict error is raised.

Parameters:

• proxy – The proxy object to be registered.

• entity – The entity object to be registered.

Returns:

None

Raises:

ConflictError – If the proxy cannot be registered (is deleted or has an ID)

class stelar.client.proxy.registry.RegistryCatalog(*args, **kwargs)

Bases: object

Class that implements a catalog of registries, for different entity types.

The purpose of the catalog is to provide all context information needed for data manipulation and transformation between proxy space and the STELAR API.

Indeed, the RegistryCatalog class is intended to become one of the base classes of Client. However, it is a separate class, for better design and testability.

add_registry_for(cls: Type[ProxyClass], registry: Registry[ProxyClass])

This method adds a registry to the catalog. It is often called from inside the constructor of a registry.

fetch_active_vocabularies()

This method should return a list of dicts, {“name”: “…”, “id”: “…”}.

More items are also allowed. It so happens that this is exactly the format returned by CKAN’s “vocabulary_list” API call. :-)

invalidate()

Invalidate all registries in the catalog.

This method invalidates all CLEAN proxies in all registries, setting them to the EMPTY state. This will cause all subsequent accesses of these proxies to retrieve fresh data from the KLMS.

registry_for(cls: Type[ProxyClass] | str) → Registry[ProxyClass]

Return the registry for the given type.

registry_for_type(type_name: str) → Registry[ProxyClass]

Return the registry for the given type name.

Entities with type name include packages, groups/organizations, resource parents.

Parameters:

type_name – The name of the type for which to return the registry.

Returns:

The registry for the given type name.

registry_stats()

Return a Series with the number of proxies held for each entity type.

stelar.client.proxy.schema module

class stelar.client.proxy.schema.Schema(cls)

Bases: object

A class that holds all information related to an entity proxy class. This includes the list of proxied properties, and other information

SHORT_NAMES = {'author', 'id', 'maintainer', 'name', 'resource_type', 'title', 'url'}

static check_non_entity(cls)

property class_name

Return the class name for the

entity_schema: dict[str, Schema] = {'Dataset': <stelar.client.proxy.schema.Schema object>, 'Group': <stelar.client.proxy.schema.Schema object>, 'ImageRegistryToken': <stelar.client.proxy.schema.Schema object>, 'License': <stelar.client.proxy.schema.Schema object>, 'Organization': <stelar.client.proxy.schema.Schema object>, 'Policy': <stelar.client.proxy.schema.Schema object>, 'Process': <stelar.client.proxy.schema.Schema object>, 'Resource': <stelar.client.proxy.schema.Schema object>, 'Tag': <stelar.client.proxy.schema.Schema object>, 'Task': <stelar.client.proxy.schema.Schema object>, 'Tool': <stelar.client.proxy.schema.Schema object>, 'User': <stelar.client.proxy.schema.Schema object>, 'Vocabulary': <stelar.client.proxy.schema.Schema object>, 'Workflow': <stelar.client.proxy.schema.Schema object>}

classmethod for_entity(name: str) → Schema

Return the schema for the given named entity

Parameters:

name (str) – the name of an entity (e.g. ‘Dataset’, ‘Group’, etc)

Returns:

the schema for the give entity, or None.

get_id(entity) → str

Return the entity ID from the entity object

properties: dict[str, Property]

short_list(additional={})

Return a ‘short list’ of important field names.

The main purpose for this is to provide a reduced set of names to use in visual representations of proxy instances.

stelar.client.proxy.schema.inheritance_for_entity_class(cls: type)

stelar.client.proxy.tag module

class stelar.client.proxy.tag.TagList(doc='The tag list', short=True, **kwargs)

Bases: Property

convert_entity_to_proxy(proxy, entity, **kwargs)

Update proxy dict to represent this property from the entity

convert_proxy_to_entity(proxy, entity, **kwargs)

Update entity dict to represent this property from the proxy. The changes flag is true when the proxy_attr is actually the Proxy.proxy_changes dict.

convert_to_create(proxy_type, create_props, entity_props, *, catalog, **kwargs)

Convert a value to be used for entity creation.

Parameters:

• client (Client) – The client used for creation.

• proxy_type (ProxyClass) – The entity type being created.

• create_props – The object passed to the create client call

• entity_props – The entity object given to the create API call.

validate(proxy, value)

class stelar.client.proxy.tag.TagListField

Bases: AnyField

convert_to_entity(value: TagSpecList, *, vocindex: VocabularyIndex, **kwargs) → TagDictList

convert_to_proxy(value: TagDictList, *, vocindex: VocabularyIndex, **kwargs) → TagSpecList

to_taglist(tagl, *, vocindex, **kwargs)

class stelar.client.proxy.tag.TaggableProxy(registry: Registry, eid: str | UUID | None = None, entity=None)

Bases: Proxy

A virtual base class for all proxies to entities that are taggable.

add_tags(*tags: str)

Add multiple tags to this entity.

Parameters:

*tags (str) – The tags to add, in tagspec format.

remove_tags(*tags: str)

Remove multiple tags from this entity.

Parameters:

*tags (str) – The tags to remove, in tagspec format.

tags

The tag list not null

Type:

<stelar.client.proxy.tag.TagListField object at 0x7fda2595b090>

class stelar.client.proxy.tag.VocabularyIndex(catalog: RegistryCatalog, *args, **kwargs)

Bases: object

This is a component to be included as part of a registry catalog. It is responsible for indexing the tag vocabularies of a STELAR API, translating vocabulary names to IDs and back. As vocabularies do not change often at all, this index should only be included once in the lifetime of a client.

property id_to_name

property id_to_tags

property name_to_id

property name_to_tags

refresh_tag_vocabularies()

The registry catalog

tagspec_to_id(tagspec: str) → str

Convert a tagspec to a tag ID

validate_tagspec(tagspec: str) → bool

Check if a string is formatted correctly as a tagspec

stelar.client.proxy.typing module

Module contents

The STELAR client’s proxying package.

The definitions here implement entity proxying by the client. Entities include Data Catalog entities, authorization entities, etc.