Overview

The Two Goals of IoTPy

  1. Get non-terminating functions that operate on endless streams by reusing terminating functions that operate on bounded-size objects.

  2. Build multithreaded, multicore, distributed applications that connect sensors and social media to the cloud by simply connecting streams.

Sensors, social media, news feeds, webcams and other sources generate streams of data. Many applications ingest and analyze data streams to control actuators, generate alerts, and feed continuous displays. IoTPy helps you to build such applications.

For a partial list of applications see "Event Processing: Designing IT Systems for Agile Companies" and  "Event Processing in Action".

Goal 1: Building streaming applications by REUSING libraries

Powerful libraries, such as Scikit, exist in a variety of domains including data analytics, artificial intelligence, signal processing and science. Programs in these libraries apply a function to input data to obtain output and then terminate. By contrast, streaming applications are perpetual processes that analyze endless data streams without terminating. IoTPy helps to make the wealth of non-streaming Python software available for streaming applications. It does so by providing a collection of methods for encapsulating non-streaming programs to get applications that operate on endless streams. 

Goal 2: Processing at the Edge and throughout the Network

Efficient data-streaming applications process data at the edge, right where it is generated, and then send on summarized information to other computers for further processing. For example an accelerometer may generate 250 samples per second, and only a few seconds a day may be relevant to crisis monitoring application. Initial processing on single-board computers, such as the Raspberry Pi, discards irrelevant information.

Concurrent computation in streaming applications occurs at three levels. (1) Sensors that acquire data and actuators that control devices run in separate threads of computation. If a sensor or an actuator stops, the corresponding thread waits while other threads in the computer continue to execute. (2) Multiple processes in a multicore computer execute in parallel. Some Raspberry Pi models, for example, have multiple cores. (3) Applications often have multiple virtual machines, distributed across a network and in the cloud.

IoTPy uses a single construct - connect input and output streams together - to deal with all three types of concurrency.

Disadvantages of IoTPy

IoTPy has not been optimized for performance.  IoTPy does not have its own security protocols for execution on distributed computing systems; it uses the protocols of the platforms on which it runs.

IoTPy is a work in progress. We are adding example applications and functions that we find to be helpful in application development. The core functionality and interfaces are stable; however, the code and documentation and are being modified to better fit standards. An application that runs on the current code base will run, without change, in future releases.

Please extend and modify the package if you wish.

Next: Streams