How To: Install Apache Flink and run example

flinkI had to give a brief intro presentation on Apache Flink in my Big Data & Technologies class, including installing Apache Flink on my local machine and running an example. I thought the steps I took to do so might be useful to others, so sharing here:

First, what is Apache Flink?

Flink (German for quick/nimble) is a big data processing engine that can run in both streaming & batch modes. It can be deployed on your local machine, on a cluster (can run on YARN), or in the cloud.

Installing Apache Flink

I am running Ubuntu 14.04.4 LTS on my machine, just fyi, and running everything from the terminal.

Run Apache Flink

The following will run Flink on your local machine:

  • cd into the unpacked directory (mine is flink-1.0.1)
  • in directory, start local session: bin/start-local.sh or ./bin/start-local.sh
  • make sure it’s running by going here (link will open in browser)
    • if successful, should see task manager
  • try this easy example using batch word count:
    ./bin/flink run ./examples/batch/WordCount.jar
  • now try this input/output example:
    • download this text file into your home directory (for easy coding):
      wget -O hamlet.txt http://www.gutenberg.org/cache/epub/1787/pg1787.txt
    • while in the flink directory, enter the following in the terminal:
      ./bin/flink run ./examples/streaming/WordCount.jar
      –input file:///home/user/hamlet.txt –output file:///home/user/output.txt

For more command line examples, see this resource.

Stop Apache Flink

To exit Flink from the terminal, type ./bin/stop-local.sh

Resources

You can read the paper I wrote giving a quick overview of Apache Flink here, and the presentation I gave in class from that paper here.

Advertisements

2 thoughts on “How To: Install Apache Flink and run example

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s