org.apache.storm.kafka.spout

Class ByTopicRecordTranslator<K,V>

java.lang.Object

org.apache.storm.kafka.spout.ByTopicRecordTranslator<K,V>

Type Parameters:

K - the key of the incoming Records

V - the value of the incoming Records

All Implemented Interfaces:

Serializable, Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>>, RecordTranslator<K,V>

public class ByTopicRecordTranslator<K,V>
extends Object
implements RecordTranslator<K,V>

Based off of a given Kafka topic a ConsumerRecord came from it will be translated to a Storm tuple and emitted to a given stream.

See Also:

Serialized Form

Field Summary

Fields inherited from interface org.apache.storm.kafka.spout.RecordTranslator

DEFAULT_STREAM

Constructor Summary

Constructors

Constructor and Description
ByTopicRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields) Create a simple record translator that will use func to extract the fields of the tuple, named by fields, and emit them to the default stream.
ByTopicRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields, String stream) Create a simple record translator that will use func to extract the fields of the tuple, named by fields, and emit them to stream.
ByTopicRecordTranslator(RecordTranslator<K,V> defaultTranslator) Create a record translator with the given default translator.

Method Summary

Modifier and Type	Method and Description
List<Object>	apply(org.apache.kafka.clients.consumer.ConsumerRecord<K,V> record) Translate the ConsumerRecord into a list of objects that can be emitted.
ByTopicRecordTranslator<K,V>	forTopic(String topic, Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields) Configure a translator for a given topic with tuples to be emitted to the default stream.
ByTopicRecordTranslator<K,V>	forTopic(String topic, Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields, String stream) Configure a translator for a given topic.
ByTopicRecordTranslator<K,V>	forTopic(String topic, RecordTranslator<K,V> translator) Configure a translator for a given kafka topic.
Fields	getFieldsFor(String stream) Get the fields associated with a stream.
List<String>	streams() Get the list of streams this translator will handle.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ByTopicRecordTranslator

public ByTopicRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func,
                               Fields fields,
                               String stream)

Create a simple record translator that will use func to extract the fields of the tuple, named by fields, and emit them to stream. This will handle all topics not explicitly set elsewhere.

Parameters:

func - extracts and turns them into a list of objects to be emitted

fields - the names of the fields extracted

stream - the stream to emit these fields on.

ByTopicRecordTranslator

public ByTopicRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func,
                               Fields fields)

Create a simple record translator that will use func to extract the fields of the tuple, named by fields, and emit them to the default stream. This will handle all topics not explicitly set elsewhere.

Parameters:

func - extracts and turns them into a list of objects to be emitted

fields - the names of the fields extracted

ByTopicRecordTranslator

public ByTopicRecordTranslator(RecordTranslator<K,V> defaultTranslator)

Create a record translator with the given default translator.

Parameters:

defaultTranslator - a translator that will be used for all topics not explicitly set with one of the variants of forTopic(java.lang.String, org.apache.storm.kafka.spout.RecordTranslator).

Method Detail

forTopic

public ByTopicRecordTranslator<K,V> forTopic(String topic,
                                             Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func,
                                             Fields fields)

Configure a translator for a given topic with tuples to be emitted to the default stream.

Parameters:

topic - the topic this should be used for

func - extracts and turns them into a list of objects to be emitted

fields - the names of the fields extracted

Returns:

this to be able to chain configuration

Throws:

IllegalStateException - if the topic is already registered to another translator

IllegalArgumentException - if the Fields for the stream this emits to do not match any already configured Fields for the same stream

forTopic

public ByTopicRecordTranslator<K,V> forTopic(String topic,
                                             Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func,
                                             Fields fields,
                                             String stream)

Configure a translator for a given topic.

Parameters:

topic - the topic this should be used for

func - extracts and turns them into a list of objects to be emitted

fields - the names of the fields extracted

stream - the stream to emit the tuples to.

Returns:

this to be able to chain configuration

Throws:

IllegalStateException - if the topic is already registered to another translator

IllegalArgumentException - if the Fields for the stream this emits to do not match any already configured Fields for the same stream

forTopic

public ByTopicRecordTranslator<K,V> forTopic(String topic,
                                             RecordTranslator<K,V> translator)

Configure a translator for a given kafka topic.

Parameters:

topic - the topic this translator should handle

translator - the translator itself

Returns:

this to be able to chain configuration

Throws:

IllegalStateException - if the topic is already registered to another translator

IllegalArgumentException - if the Fields for the stream this emits to do not match any already configured Fields for the same stream

apply

public List<Object> apply(org.apache.kafka.clients.consumer.ConsumerRecord<K,V> record)

Description copied from interface: RecordTranslator

Translate the ConsumerRecord into a list of objects that can be emitted.

Specified by:

apply in interface Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>>

Specified by:

apply in interface RecordTranslator<K,V>

Parameters:

record - the record to translate

Returns:

the objects in the tuple. Return a KafkaTuple if you want to route the tuple to a non-default stream. Return null to discard an invalid ConsumerRecord if KafkaSpoutConfig.Builder.setEmitNullTuples(boolean) is set to false.

getFieldsFor

public Fields getFieldsFor(String stream)

Description copied from interface: RecordTranslator

Get the fields associated with a stream. The streams passed in are returned by the RecordTranslator.streams() method.

Specified by:

getFieldsFor in interface RecordTranslator<K,V>

Parameters:

stream - the stream the fields are for

Returns:

the fields for that stream.

streams

public List<String> streams()

Description copied from interface: RecordTranslator

Get the list of streams this translator will handle.

Specified by:

streams in interface RecordTranslator<K,V>

Returns:

the list of streams that this will handle.