NumPy: Stream Arrays

Skip this section on a first reading; read it when you want to use NumPy arrays.

You can create a stream whose elements are NumPy objects. Such streams belong to the type StreamArray. An element of a StreamArray can be a type that you define (called a NumPy dtype) or it can be a default array of floats. We start by describing StreamArrays whose elements are default arrays of floats and later describe user-defined types.

Creating a stream array

We introduce the idea with an example. The statement:

s = StreamArray(name='s', dimension=3)

creates a stream s where s[i] is a NumPy array consisting of 3 (i.e. dimension) elements. You can think of the stream array s as an array with an unbounded number of rows and 3 columns. When items are appended to a stream array, each row of the stream array is filled in sequence.

The parameters of StreamArray are: name, dimension, dtype, initial_value, and num_in_memory

  • name is optional and is a string. The default is ‘no_name’.

  • dimension is optional and is the dimension of elements of the stream array. The default is 0.

  • dtype is optional and the type of the rows of the stream array. The default is float.

  • initial_value is optional and is the initial value of the stream array. The default (which is almost always used) is None.

  • num_in_memory is optional and can be ignored for the time being. It is used for memory management.

When dimension is 0

The dimension parameter can be a non-negative integer or a tuple or list. If dimension is 0 then each element of the stream array belongs to type dtype. In this case, think of the stream array as a 1-D array of unbounded length with elements of type dtype. For example:

t = StreamArray()

makes t a stream array where t is effectively an array of unbounded size where each element of t is a float.

When dimension is a positive integer

If dimension is a positive integer then each element of the stream is a 1-D array whose length is dimension. Stream array s, the first example, is an instance of such a dimension. Another example is,

u = StreamArray(name='u', dimension=2, dtype=int)

makes u a stream array called u. Think of u as an array with an unbounded number of rows where each row of u is an array consisting of 2 integers.

When dimension is a tuple or list

Each element of the tuple must be a positive integer. Think of the stream array as having an unbounded number of rows where each row is an N-dimensional array where N is the length of the tuple. The lengths of the N-dimensional array are given by the tuple. For example,

v = StreamArray(dimension=(3,4), dtype=int)

makes v a stream array where each row of v is a 3 x 4 array of integers. For example, an element of v could be the array a:

a = np.array([
        [0, 1, 2, 3],
        [4, 5, 6, 7],
        [8, 9, 10, 11]])

appending a numpy array to a stream array

You can append an array to a stream provided that they have the same dimensions. Here are examples of appending to stream arrays created in the previous paragraph.

Append a singleton array of float to stream array t

t.append(np.array(1.0))

Append a 1-D array of size 3 of floats to stream array s

a = np.zeros(3)
s.append(a)

makes a the NumPy array: np.array([ 0.,  0.,  0.]) and appends the array a to stream array s. At this point, the zeroth row of s is np.array([ 0.,  0.,  0.]).

Append a 1-D array of size 2 of ints to stream array u

u.append(np.array([0, 1])

Append a 3 x 4 array of integers to stream array v

v.append(np.array([
        [0, 1, 2, 3],
        [4, 5, 6, 7],
        [8, 9, 10, 11]]))


extending a stream array

You can extend a stream array by an array consisting of multiple rows provided that the dimensions of rows of the array and the stream are identical.

Extend stream array t by an array of floats

t.extend(np.array([2.0, 3.0]))

The statement is equivalent to:

t.append(np.array(2.0))
t.append(np.array(3.0))

Since t was np.array([1.0]) before the above statements are executed, its value after the statement is np.array([1.0, 2.0, 3.0]).

Extend stream array s by an array of arrays

An example of an array consisting of 2 rows where each row has 3 elements is b:

b = np.array([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
s.extend(b)

The above statement is equivalent to:

s.append(np.array([1.0, 2.0, 3.0])
s.append(np.array([4.0, 5.0, 6.0])

Since s is np.array([[0.0, 0.0, 0.0]]) before the above statements are executed, then at this point, its content is:

np.array([[0.0, 0.0, 0.0],
          [1.0, 2.0, 3.0],
          [4.0, 5.0, 6.0]])

Extend stream array u by an array of arrays

u.extend(np.array([[2,3], [4,5], [6,7]]))

Extend stream array v

v.extend(np.array([
   [[12, 13, 14, 15],[16, 17, 18, 19], [20, 21, 22, 23]],
   [[24, 25, 26, 27], [28, 29, 30, 31], [32, 33, 34, 35]]]))

user-defined types

An example of a user-defined type is:

txyz_dtype = np.dtype([('time','int'), ('data', '3float')])

An example of an object, c, of this type is created by:

c = np.array((1, [0.0, 1.0, 2.0]), dtype=txyz_dtype)

Then, c[‘time’] is np.array(1), and c['data'] is np.array([ 0., 1., 2.]

Creating a stream array with user-defined types

y = StreamArray(dimension=0, dtype=txyz_dtype)

creates a stream array, y, whose elements are of type txyz_dtype.

Appending to a stream array with user-defined types

We can append c to the stream array,

y.append(c)

At this point the value of y is a singleton array containing c, i.e. y’s value is

np.array([(1, [0., 1., 2.])], dtype=txyz_dtype))

EXTending a stream array with user-defined types

d = np.array([
   (2, [3., 4., 5.]), (3, [6., 7., 8.])], dtype=txyz_dtype)

creates an array d with two elements of type txyz_dtype. We can extend y with d:

y.extend(d)

and at this point y’s value is:

np.array([
   (1, [0., 1., 2.]), 
   (2, [3., 4., 5.]), 
   (3, [6., 7., 8.])], 
   dtype=txyz_dtype))

You can also create stream arrays with user defined types and arbitrary dimensions. A dimension of 0 is used most often with user-defined types.

Next: Agent Types

Previous: agents and streams