gluonts.itertools module

class gluonts.itertools.Cached(iterable: gluonts.itertools.SizedIterable)

Bases: object

An iterable wrapper, which caches values in a list while iterated.

The primary use-case for this is to avoid re-computing the elements of the sequence, in case the inner iterable does it on demand.

This should be used to wrap deterministic iterables, i.e. iterables where the data generation process is not random, and that yield the same elements when iterated multiple times.

consumed: list

iterable: gluonts.itertools.SizedIterable

provider: Iterable

class gluonts.itertools.Chain(iterables: Collection[gluonts.itertools.SizedIterable])

Bases: object

Chain multiple iterables into a single one.

This is a thin wrapper around itertools.chain.

iterables: Collection[gluonts.itertools.SizedIterable]

class gluonts.itertools.Cyclic(iterable: gluonts.itertools.SizedIterable)

Bases: object

Like itertools.cycle, but does not store the data.

iterable: gluonts.itertools.SizedIterable

stream()

Return a continuous stream of self that has no fixed start.

When re-iterating Cyclic it will yield elements from the start of the passed iterable. However, this is not always desired; e.g. in training we want to treat training data as an infinite stream of values and not start at the beginning of the dataset for each epoch.

>>> from toolz import take
>>> c = Cyclic([1, 2, 3, 4])
>>> assert list(take(5, c)) == [1, 2, 3, 4, 1]
>>> assert list(take(5, c)) == [1, 2, 3, 4, 1]

>>> s = Cyclic([1, 2, 3, 4]).stream()
>>> assert list(take(5, s)) == [1, 2, 3, 4, 1]
>>> assert list(take(5, s)) == [2, 3, 4, 1, 2]

class gluonts.itertools.Filter(fn: 'Callable', iterable: 'SizedIterable')

Bases: object

fn: Callable

iterable: gluonts.itertools.SizedIterable

class gluonts.itertools.Fuse(collections: typing.List[typing.Sequence], _lengths: typing.List[int] = <factory>)

Bases: object

Fuse collections together to act as single collections.

>>> a = [0, 1, 2]
>>> b = [3, 4, 5]
>>> fused = Fuse([a, b])
>>> assert len(a) + len(b) == len(fused)
>>> list(fused[2:5])
[2, 3, 4]
>>> list(fused[3:])
[3, 4, 5]

This is similar to Chain, but also allows to select directly into the data. While Chain creates a single Iterable out of a set of Iterable``s, ``Fuse creates a single Collection out of a set of ``Collection``s.

collections: List[Sequence]

class gluonts.itertools.IterableSlice(iterable, length)

Bases: object

An iterable version of itertools.islice, i.e. one that can be iterated over multiple times:

>>> isl = IterableSlice(iter([1, 2, 3, 4, 5]), 3)
>>> list(isl)
[1, 2, 3]
>>> list(isl)
[4, 5]
>>> list(isl)
[]

This needs to be a class to support re-entry iteration.

class gluonts.itertools.Map(fn: 'Callable', iterable: 'SizedIterable')

Bases: object

fn: Callable

iterable: gluonts.itertools.SizedIterable

class gluonts.itertools.PickleCached(iterable: gluonts.itertools.SizedIterable, cached: bool = False, _path: pathlib.Path = <factory>)

Bases: object

A caching wrapper for iterable using pickle to store cached values on disk.

See Cached for more information.

cached: bool = False

iterable: gluonts.itertools.SizedIterable

class gluonts.itertools.PseudoShuffled(iterable: gluonts.itertools.SizedIterable, shuffle_buffer_length: int)

Bases: object

Yield items from a given iterable in a pseudo-shuffled order.

iterable: gluonts.itertools.SizedIterable

shuffle_buffer_length: int

class gluonts.itertools.RandomYield(iterables: typing.List[typing.Iterable], probabilities: typing.Optional[typing.List[float]] = None, random_state: numpy.random.mtrand.RandomState = <factory>)

Bases: object

Given a list of Iterables iterables, generate samples from them.

When probabilities is given, sample iteratbles according to it. When probabilities is not given, sample iterables uniformly.

When one iterable exhausts, the sampling probabilities for it will be set to 0.

>>> from toolz import take
>>> a = [1, 2, 3]
>>> b = [4, 5, 6]
>>> it = iter(RandomYield([a, b], probabilities=[1, 0]))
>>> list(take(5, it))
[1, 2, 3]

>>> a = [1, 2, 3]
>>> b = [4, 5, 6]
>>> it = iter(RandomYield([Cyclic(a), b], probabilities=[1, 0]))
>>> list(take(5, it))
[1, 2, 3, 1, 2]

iterables: List[Iterable]

probabilities: Optional[List[float]] = None

random_state: numpy.random.mtrand.RandomState

class gluonts.itertools.SizedIterable(*args, **kwargs)

Bases: typing_extensions.Protocol

class gluonts.itertools.SizedIterableSlice(iterable, length)

Bases: gluonts.itertools.IterableSlice

Same as IterableSlice but also supports len():

>>> isl = SizedIterableSlice([1, 2, 3, 4, 5], 3)
>>> len(isl)
3

class gluonts.itertools.StarMap(fn: 'Callable', iterable: 'SizedIterable')

Bases: object

fn: Callable

iterable: gluonts.itertools.SizedIterable

gluonts.itertools.batcher(iterable: Iterable[gluonts.itertools.T], batch_size: int) → Iterator[List[gluonts.itertools.T]]

Groups elements from iterable into batches of size batch_size.

>>> list(batcher("ABCDEFG", 3))
[['A', 'B', 'C'], ['D', 'E', 'F'], ['G']]

Unlike the grouper proposed in the documentation of itertools, batcher doesn’t fill up missing values.

gluonts.itertools.chop(at: int, take: int) → slice

Create slice using an index at and amount take.

>>> x = [0, 1, 2, 3, 4]
>>> x[chop(at=1, take=2)]
[1, 2]
>>>
>>> x[chop(at=-2, take=2)]
[3, 4]
>>> x[chop(at=3, take=-2)]
[1, 2]

gluonts.itertools.columns_to_rows(columns: Dict[gluonts.itertools.K, Sequence[gluonts.itertools.V]]) → List[Dict[gluonts.itertools.K, gluonts.itertools.V]]

Transpose column-orientation to row-orientation.

>>> columns_to_rows({'a': [1, 3], 'b': [2, 4]})
[{'a': 1, 'b': 2}, {'a': 3, 'b': 4}]

gluonts.itertools.inverse(dct: Dict[gluonts.itertools.K, gluonts.itertools.V]) → Dict[gluonts.itertools.V, gluonts.itertools.K]

Inverse a dictionary; keys become values and values become keys.

gluonts.itertools.join_items(left, right, how='outer', default=None)

Iterate over joined dictionary items.

Yields triples of key, left_value, right_value.

Similar to SQL join statements the join behaviour is controlled by how:

• outer (default): use keys from left and right

• inner: only use keys which appear in both left and right

• strict: like inner, but throws error if keys mismatch

• left: use only keys from left

• right: use only keys from right

If a key is not present in either input, default is chosen instead.

gluonts.itertools.maybe_len(obj) → Optional[int]

gluonts.itertools.partition(it: Iterable[gluonts.itertools.T], fn: Callable[[gluonts.itertools.T], bool]) → Tuple[List[gluonts.itertools.T], List[gluonts.itertools.T]]

Partition it into two lists given predicate fn.

This is similar to the recipe defined in Python’s itertools docs, however this method consumes the iterator directly and returns lists instead of iterators.

gluonts.itertools.pluck_attr(seq='__no__default__', name='__no__default__', default=<object object>)

Get attribute name from elements in seq.

gluonts.itertools.power_set(iterable)

Generate all possible subsets of the given iterable, as tuples.

>>> list(power_set(["a", "b"]))
[(), ('a',), ('b',), ('a', 'b')]

Adapted from https://docs.python.org/3/library/itertools.html#itertools-recipes

gluonts.itertools.prod(xs)

Compute the product of the elements of an iterable object.

gluonts.itertools.replace(values: Sequence[gluonts.itertools.T], idx: int, value: gluonts.itertools.T) → Sequence[gluonts.itertools.T]

Replace value at index idx with value.

Like setitem, but for tuples.

>>> replace((1, 2, 3, 4), -1, 99)
(1, 2, 3, 99)

gluonts.itertools.roundrobin(*iterables)

roundrobin(‘ABC’, ‘D’, ‘EF’) –> A D E B F C

Taken from: https://docs.python.org/3/library/itertools.html#recipes.

gluonts.itertools.rows_to_columns(rows: typing.Sequence[typing.Dict[gluonts.itertools.K, gluonts.itertools.V]], wrap: typing.Callable[[typing.Sequence[gluonts.itertools.V]], typing.Sequence[gluonts.itertools.V]] = <function <lambda>>) → Dict[gluonts.itertools.K, Sequence[gluonts.itertools.V]]

Transpose rows of dicts, to one dict containing columns.

>>> rows_to_columns([{'a': 1, 'b': 2}, {'a': 3, 'b': 4}])
{'a': [1, 3], 'b': [2, 4]}

This can also be understood as stacking the values of each dict onto each other.

gluonts.itertools.select(keys, source: dict, ignore_missing: bool = False) → dict

Select subset of source dictionaries.

>>> d = {"a": 1, "b": 2, "c": 3}
>>> select(["a", "b"], d)
{'a': 1, 'b': 2}

gluonts.itertools.split(xs: Sequence, indices: List[int]) → List[Sequence]

Split xs into subsets given indices.

>>> split("abcdef", [1, 3])
['a', 'bc', 'def']

This is similar to numpy.split when passing a list of indices, but this version does not convert the underlying data into arrays.

gluonts.itertools.split_into(xs: Sequence, n: int) → Sequence

Split xs into n parts of similar size.

>>> split_into("abcd", 2)
['ab', 'cd']
>>> split_into("abcd", 3)
['ab', 'c', 'd']

gluonts.itertools.trim_nans(xs, trim='fb')

Trim the leading and/or trailing NaNs from a 1-D array or sequence.

Like np.trim_zeros but for NaNs.