Map: 1-in, 1-out

In this section we describe functions that operate on streams. You can create such functions using decorators or wrappers. We next describe the use of decorators. A description of wrappers is found in Wrappers.

Notational Conventions

The suffix e to refers to functions that operate on an element of a stream, and w for functions on sliding windows of streams. For example, map_e refers to the map function element by element whereas map_w refers to functions that operate on sliding windows of the stream.

We convert a terminating function f that operates on conventional Python bounded-sized to a non-terminating function that operates on endless streams by decorating f. IoTPy has many decorators. We start with decorators map_e and map_w for procedures, and their functional variants fmap_e and fmap_w. The decorators, map_e and map_w, are used to get procedures that operate on streams whereas fmap_e and fmap_w are used to get functions that return streams.

The j-th element of a stream s is s[j].

MAP

Suppose you are given a conventional Python function, f, that terminates and operates on a conventional Python bounded-size object such as a list or an array and returns a conventional Python object. An example of such a function is double.

double(v): return 2*v

You want to decorate, f, so that the decorated function has two parameters:

  1. in_stream an input stream

  2. out_stream an output stream

When a value v is appended to in_stream, the decorated function appends f(v) to out_stream. The decoration for this purpose is @map_e. For example, consider the decorated double function 

@map_e
def double(v): return 2*v

If we call the decorated function with the following call, where x and y are streams (or stream arrays):

double(in_stream=x, out_stream=y)

then: 

y[n] = 2*x[n]

Example: polynomial

@map_e
def f(v): return 3*v**3 +2*v**2 + v + 1

The call f(x, y), where x and y are streams, creates an agent that has x as an input stream and y as an output stream, and which makes

y[n] = 3*x[n]**3 +2*x[n]**2 + x[n] + 1

STATE

You can pass a state to a function by including the parameter called state in the call and providing its initial value, as in:

@map_e
def f(v, state): return g(v,state), h(v,state)
f(in_stream=x, out_stream=y, state=initial_state)

Let state[n] be the n-th value of state, for n = 0, 1, 2,… Then:

state[0] = initial_state
y[n] = g(x[n], state[n])
state[n+1] = h(x[n], state[n])

where y[n] is the n-th value appended by the procedure to stream y and state[n] is the n-th value of state for n = 0, 1, 2, …

EXAMPLE: difference between successive elements

We want to create an agent with an input stream, x, and an output stream, y, where:

y[0] = 0,  and,   y[n+1] = x[n+1] - x[n]

We write a function, deltas, which has a state. Each call to this function is passed the value of state that resulted from the previous call. The initial value of state is specified in the call to the (decorated) function deltas by passing its value in a keyword argument named state.

@map_e
def deltas(input_value, state):
  difference = input_value - state
  next_state = input_value
  return difference, next_state
deltas(in_stream=x, out_stream=y, state=0)

The above code ensures state[0] = 0 because the call to deltas has the term, state=0. Comparing this example to the generic version given in the previous paragraph, the g() function in the previous paragraph corresponds to difference, and the h() function corresponds to next_state. So:

state[0] = 0
state[n+1] = x[n]
y[0] = x[0]
y[n+1] = x[n+1] - x[n]

A less verbose style of writing the above function is:

@map_e
def deltas(u, v): return u - v, u
deltas(x, y, state=0)

Note that you must explicitly use the keyword state in the call if the procedure has state. This is because state is a keyword parameter.

KEYWORD PARAMETERS OTHER THAN STATE

You can pass an arbitrary number of keyword parameters to a function as shown in the following example which has a keyword parameter called exponent. The call ensures y[n] = x[n]**3.

@map_e
def exponentiate(v, exponent): return v**exponent
exponentiate(in_stream=x, out_stream=y, exponent=3)

Note that the call to exponentiate must have the keyword exponent, though in_stream and out_stream can be positional arguments as in:

exponentiate(x, y, exponent=3)

Example: evaluate polynomial

@map_e
def evaluate_polynomial(number, polynomial):
    return np.polyval(polynomial, number)

np.polyval(polynomial, number) returns the evaluation of polynomial at number.

The n-th element of the output stream of the decorated function consists of the evaluation of a polynomial at the n-th element of the input stream. In the following test, the polynomial is given by its coefficients [1, 0, 1] which defines the polynomial

f(v) = 1*v*v + 0*v + 1

Then if the input stream contains the elements 1.0, 4.0, 3.0, 0.0, …. then the output stream contains 2.0, 17.0, 10.0, 1.0, …. The parameter polynomial is passed as a keyword argument in the call to evaluate_polynomial().

def test_evaluate_polynomial():
    # Declare streams
    x = StreamArray('x')
    y = StreamArray('y')
    # Create agent
    evaluate_polynomial(x, y, polynomial=[1, 0, 1])
    # Put data in streams and run
    x.extend(np.array([1.0, 4.0, 3.0, 0.0]))
    run()
    assert np.array_equal(
        recent_values(y), np.array([2.0, 17.0, 10.0, 1.0]))

KEYWORD PARAMETERS WITH STATE

You can have both keyword parameters and state as in the following example.

@map_e
def anomalous_change(v, state, max_change):
   delta = v - state
   next_state = v
   next_output = (delta > max_change)
return next_output, next_state

Given streams x and y, if you call:

anomalous_change(in_stream=x, out_stream=y, state=0, max_change=10)

Then at the n-th call to the undecorated function anomalous_change for n = 0, 1, 2, …, the values passed to the function are:

v = x[n]
state[0] = 0, and, state[n+1] = x[n]
max_change = 10

and therefore:

delta[0] = x[0], and, delta[n+1] = x[n+1] - x[n]
y[0] = (x[0] > 10)
y[n+1] = (x[n+1] - x[n] > 10)

Download examples of @map_e and @fmap_e from GitHub at IoTPy/examples/op/examples_map_e.py.

WINDOWS

IoTPy has multiple windowing operations. We begin with windows specified by a fixed size and fixed step size. A window is a list. (Later, we will describe windows that are NumPy arrays.) A window is specified by a window_size and step_size. The length of a window is its window_size. A sliding window is moved forward by its step_size. The j-th window specified of a stream x is:

x[j*step_size : j*step_size + window_size] for j >= 0

For example, if window_size is 3 and step_size is 2 then the sequence of windows is the following sequence of lists:

x[0, 1, 2],  x[2, 3, 4],  x[4, 5, 6], …

Use the decorator @map_w for mapping operations on windows.

@map_w
def f(window): return g(window)
f(in_stream=x, out_stream=y, window_size=w, step_size=s)

This call sets:

y[n] = g(x[n*s : n*s+w])

For example,

@map_w
def sum_window(window): return sum(window)

sum_window(in_stream=x, out_stream=y,
           window_size=2, step_size=2)

sets

y[n] = x[2*n] + x[2*n+1]

You can use keywords and state for @map_w in exactly the same way as for @map_e. For example

@map_w
def sum_window_plus_addend(window, addend):
   return sum(window) + addend

sum_window_plus_addend(
          in_stream=x, out_stream=y,
           window_size=2, step_size=2,
           addend=100)

sets:

y[n] = x[2n] + x[2*n+1] + 100

Functional Form

The functional form obtained with the decorator @fmap_e can be more convenient in some cases than the procedural form obtained by @map_e, as illustrated in the following example.

@fmap_e
def f(v): return g(v)
y=f(x)

and

@map_e
def h(v): return g(v)
h(in_stream=x, out_stream=z)

The relationship between @fmap_e and @map_e is similar to the relationship between @f_add and @r_add.

  1. In the functional form with @fmap_e the output stream is created by the call to f whereas in the procedural form with @map_e , the output stream must be declared before the call to f and is passed as a parameter to f.

  2. The n-th value Z[n] appended to the output stream z by the procedure is the n-th value returned by the function f.

The functional form gives you the convenience of using functional notation such as:

z = f(g(x,y) * h(w,x,y))

where w, x, y, z are streams.

You can use keyword parameters and state with the functional form in the exactly the same way as for the procedural form.

Functional Form with Windows

The decorator @fmap_w is the functional form of @map_w in exactly the same way that @fmap_e is the functional form of @map_e.

@fmap_w
def f(window): return g(window)
y = f(x, window_size=w, step_size=s)

is the functional version of the procedure:

@map_w
def f(window): return g(window)
f(in_stream=x, out_stream=y, window_size=w, step_size=s)

Download examples of @map_w and @fmap_w from GitHub at IoTPy/examples/op/examples_map_e.py and IoTPy/examples/op/examples_map_w.py.

Another simple example is in IoTPy/examples/signal_processing_examples/window_dot_product.py.

Examples that use wrappers (rather than decorators) are found in IoTPy/examples/map_element_examples.py, IoTPy/examples/map_window_examples.py, and IoTPy/examples/filter_element_examples.py. In some cases, explicit wrappers are more syntactically convenient that decorators.

Let’s look at an example from acoustics: generating an echo: Example. Acoustics Echo.