By   November 7, 2018

Flink – Interactive Scala Shell

Interactive scala shell is available in Flink download package. It allows us to run flink streams and batch processing statements in scala by building and executing the jobs directly created from the shell. We can simply call execute() method, which should request to run the job.

The shell supports including additional jars added to the class path using -a, –addclasspath argument. This can allow us to use types from the jars.

scala shell & Flink Cluster

scala shell & Flink Cluster

Here we are starting up a local flink cluster and connecting to it. It provides prebuilt execution environments to implement batch (benv) or streaming programs (senv).

In the following we are using senv to create a DataStream from integer elements. We can then calculating the twice of each elements of the DataStream. In order to check output we can simply print() the elements to the console. In order to build the execution graph, we can simply call execute() method on senv.

Executing on a Remote Flink Cluster

If we are connected to a remote Flink cluster, we can verify the execution from the web dashboard. Since our Stream just has a few elements and the closes, the job finishes very fast. Afterwards, we can see it in the Completed Jobs section.

Here is the simple execution plan of our job: