An implementation of Beam's {@link TimerInternals}, that also provides serialization functionality. The latter is used when snapshots of the current s

A DoFn that keys each candidate by all its prefixes.

To run the example, first open a socket on a terminal by executing the command: <li> <li> <code>nc -lk 9999</code> </li> </li> and then la

A translator of a {@link PTransform}.

A PTransform that writes a bounded PCollection to standard output.

This is the interface state has to implement in order for it to be fault tolerant when executed by the FlinkPipelineRunner.

Special composite transform translator. Only called if the CoGroup is two dimensional. @param <K>

Flink {@link org.apache.flink.api.common.typeutils.TypeComparator} for {@link com.google.cloud.dataflow.sdk.coders.Coder}.

Flink {@link org.apache.flink.api.common.typeinfo.TypeInformation} for Dataflow {@link com.google.cloud.dataflow.sdk.coders.Coder}s.

Flink {@link org.apache.flink.api.common.typeutils.TypeSerializer} for Dataflow {@link com.google.cloud.dataflow.sdk.coders.Coder}s

Wrapper that wraps a {@link com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn} in a Flink {@link org.apache.flink.api.common.accumulators.Acc

Class used to store tag-count pairs.

A transform containing a basic TF-IDF pipeline. The input consists of KV objects where the key is the document's URI and the value is a piece of the d

A PTransform that takes as input a list of tokens and returns the most common tokens per prefix.

Lower latency, but more expensive.

Cheaper but higher latency. <p> Returns two PCollections, the first is top prefixes of size greater than minPrefix, and the second is top prefixes of

Transform for printing the contents of a {@link com.google.cloud.dataflow.sdk.values.PCollection}. to standard output. This is Flink-specific and wil

Wrapper for {@link DataInputView}. We need this because Flink reads data using a {@link org.apache.flink.core.memory.DataInputView} while Dataflow {@l

Wrapper for {@link org.apache.flink.core.memory.DataOutputView}. We need this because Flink writes data using a {@link org.apache.flink.core.memory.Da

Examines each row (country info) in the input table. Output a KV with the key the country code, and the value the country name.

Examines each row (event) in the input table. Output a KV with the key the country code of the event, and the value a string encoding event informatio

An abstract class that encapsulates the common code of the the {@link com.google.cloud.dataflow.sdk.transforms.ParDo.Bound} and {@link com.google.clou

FlinkBatchPipelineTranslator knows how to translate Pipeline objects into Flink Jobs. This is based on {@link com.google.cloud.dataflow.sdk.runners.Da

Translators for transforming Dataflow {@link com.google.cloud.dataflow.sdk.transforms.PTransform}s to Flink {@link org.apache.flink.api.java.DataSet}s

This is a hack for transforming a {@link com.google.cloud.dataflow.sdk.transforms.Create} operation. Flink does not allow {@code null} in it's equival

Encapsulates a {@link com.google.cloud.dataflow.sdk.transforms.DoFn} inside a Flink {@link org.apache.flink.api.common.functions.RichMapPartitionFunct

This class is the key class implementing all the windowing/triggering logic of Apache Beam. To provide full compatibility and support for all the wind

This class groups the elements by key. It assumes that already the incoming stream is composed of <code>[Key,Value]</code> pairs.

Flink {@link org.apache.flink.api.common.functions.GroupReduceFunction} for executing a {@link com.google.cloud.dataflow.sdk.transforms.GroupByKey} op

Encapsulates a {@link com.google.cloud.dataflow.sdk.transforms.DoFn} that uses side outputs inside a Flink {@link org.apache.flink.api.common.function

A FlatMap function that filters out those elements that don't belong in this output. We need this to implement MultiOutput ParDo functions.

A wrapper for the {@link com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti} Beam transformation.

A wrapper for the {@link com.google.cloud.dataflow.sdk.transforms.ParDo.Bound} Beam transformation.

Flink {@link org.apache.flink.api.common.functions.GroupCombineFunction} for executing a {@link com.google.cloud.dataflow.sdk.transforms.Combine.PerKe

The class that instantiates and manages the execution of a given job. Depending on if the job is a Streaming or Batch processing one, it creates the a

Options which can be used to configure a Flink PipelineRunner.

A {@link PipelineRunner} that executes the operations in the pipeline by first translating them to a Flink Plan and then executing them either locally

The role of this class is to translate the Beam operators to their Flink counterparts. If we have a streaming job, this is instantiated as a {@link Fl

Flink {@link org.apache.flink.api.common.functions.GroupReduceFunction} for executing a {@link com.google.cloud.dataflow.sdk.transforms.Combine.PerKey

AuteService registrar - will register FlinkRunner and FlinkOptions as possible pipeline runner services. It ends up in META-INF/services and gets pic

Result of executing a {@link com.google.cloud.dataflow.sdk.Pipeline} with Flink. This has methods to query to job runtime and the final values of {@li

An implementation of the Beam {@link StateInternals}. This implementation simply keeps elements in memory. This state is periodically checkpointed by

This flat map function bootstraps from collection elements and turns them into WindowedValues (as required by the Flink runner).

This is a {@link FlinkPipelineTranslator} for streaming jobs. Its role is to translate the user-provided {@link com.google.cloud.dataflow.sdk.values.P

This class contains all the mappings between Beam and Flink <b>streaming</b> transformations. The {@link FlinkStreamingPipelineTranslator} traverses t

{@link com.google.cloud.dataflow.sdk.Pipeline} for testing Dataflow programs on the {@link com.dataartisans.flink.dataflow.FlinkPipelineRunner}.

A SimpleFunction that converts a Word and Count into a printable string.

Takes as input a the top candidates per prefix, and emits an entity suitable for writing to Datastore.

Translates a GroupByKey while ignoring window assignments. This is identical to the {@link GroupByKeyOnlyTranslatorBatch}

Version of {@link java.io.ByteArrayOutputStream} that allows to retrieve the internal byte[] buffer without incurring an array copy.

Copied from {@link com.google.cloud.dataflow.examples.JoinExamples} because the code is private there.

To run the example, first open two sockets on two terminals by executing the commands: <li> <li> <code>nc -lk 9999</code>, and </li>

Unfortunately we need to copy the code from the Dataflow SDK because it is not public there.

Just an auxiliary interface to bypass the fact that java anonymous classes cannot implement multiple interfaces.

Flink {@link org.apache.flink.api.common.typeutils.TypeComparator} for {@link com.google.cloud.dataflow.sdk.coders.KvCoder}. We have a special compara