Package org.apache.storm.kafka.spout

Class ByTopicRecordTranslator<K,V>

java.lang.Object
org.apache.storm.kafka.spout.ByTopicRecordTranslator<K,V>
Type Parameters:
K - the key of the incoming Records
V - the value of the incoming Records
All Implemented Interfaces:
Serializable, Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>>, RecordTranslator<K,V>
public class ByTopicRecordTranslator<K,V> extends Object implements RecordTranslator<K,V>
Based off of a given Kafka topic a ConsumerRecord came from it will be translated to a Storm tuple and emitted to a given stream.
See Also:

  • Constructor Details

    • ByTopicRecordTranslator

      public ByTopicRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields, String stream)
      Create a simple record translator that will use func to extract the fields of the tuple, named by fields, and emit them to stream. This will handle all topics not explicitly set elsewhere.
      Parameters:
      func - extracts and turns them into a list of objects to be emitted
      fields - the names of the fields extracted
      stream - the stream to emit these fields on.

    • ByTopicRecordTranslator

      public ByTopicRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields)
      Create a simple record translator that will use func to extract the fields of the tuple, named by fields, and emit them to the default stream. This will handle all topics not explicitly set elsewhere.
      Parameters:
      func - extracts and turns them into a list of objects to be emitted
      fields - the names of the fields extracted

    • ByTopicRecordTranslator

      public ByTopicRecordTranslator(RecordTranslator<K,V> defaultTranslator)
      Create a record translator with the given default translator.
      Parameters:
      defaultTranslator - a translator that will be used for all topics not explicitly set with one of the variants of forTopic(java.lang.String, org.apache.storm.kafka.spout.RecordTranslator).

  • Method Details

    • forTopic

      public ByTopicRecordTranslator<K,V> forTopic(String topic, Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields)
      Configure a translator for a given topic with tuples to be emitted to the default stream.
      Parameters:
      topic - the topic this should be used for
      func - extracts and turns them into a list of objects to be emitted
      fields - the names of the fields extracted
      Returns:
      this to be able to chain configuration
      Throws:
      IllegalStateException - if the topic is already registered to another translator
      IllegalArgumentException - if the Fields for the stream this emits to do not match any already configured Fields for the same stream

    • forTopic

      public ByTopicRecordTranslator<K,V> forTopic(String topic, Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields, String stream)
      Configure a translator for a given topic.
      Parameters:
      topic - the topic this should be used for
      func - extracts and turns them into a list of objects to be emitted
      fields - the names of the fields extracted
      stream - the stream to emit the tuples to.
      Returns:
      this to be able to chain configuration
      Throws:
      IllegalStateException - if the topic is already registered to another translator
      IllegalArgumentException - if the Fields for the stream this emits to do not match any already configured Fields for the same stream

    • forTopic

      public ByTopicRecordTranslator<K,V> forTopic(String topic, RecordTranslator<K,V> translator)
      Configure a translator for a given kafka topic.
      Parameters:
      topic - the topic this translator should handle
      translator - the translator itself
      Returns:
      this to be able to chain configuration
      Throws:
      IllegalStateException - if the topic is already registered to another translator
      IllegalArgumentException - if the Fields for the stream this emits to do not match any already configured Fields for the same stream

    • apply

      public List<Object> apply(org.apache.kafka.clients.consumer.ConsumerRecord<K,V> record)
      Description copied from interface: RecordTranslator
      Translate the ConsumerRecord into a list of objects that can be emitted.
      Specified by:
      apply in interface Func<K,V>
      Specified by:
      apply in interface RecordTranslator<K,V>
      Parameters:
      record - the record to translate
      Returns:
      the objects in the tuple. Return a KafkaTuple if you want to route the tuple to a non-default stream. Return null to discard an invalid ConsumerRecord if KafkaSpoutConfig.Builder.setEmitNullTuples(boolean) is set to false.

    • getFieldsFor

      public Fields getFieldsFor(String stream)
      Description copied from interface: RecordTranslator
      Get the fields associated with a stream. The streams passed in are returned by the RecordTranslator.streams() method.
      Specified by:
      getFieldsFor in interface RecordTranslator<K,V>
      Parameters:
      stream - the stream the fields are for
      Returns:
      the fields for that stream.

    • streams

      public List<String> streams()
      Description copied from interface: RecordTranslator
      Get the list of streams this translator will handle.
      Specified by:
      streams in interface RecordTranslator<K,V>
      Returns:
      the list of streams that this will handle.