org.apache.storm.hdfs.bolt

Class AvroGenericRecordBolt

java.lang.Object

org.apache.storm.topology.base.BaseComponent

org.apache.storm.topology.base.BaseRichBolt

org.apache.storm.hdfs.bolt.AbstractHdfsBolt

org.apache.storm.hdfs.bolt.AvroGenericRecordBolt

All Implemented Interfaces:

Serializable, IBolt, IComponent, IRichBolt

public class AvroGenericRecordBolt
extends AbstractHdfsBolt

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.storm.hdfs.bolt.AbstractHdfsBolt

collector, configKey, fileNameFormat, fileRetryCount, fs, fsUrl, hdfsConfig, maxOpenFiles, offset, partitioner, rotationActions, rotationCounterMap, rotationPolicy, rotationTimer, syncPolicy, tickTupleInterval, writeLock, writers

Constructor Summary

Constructors

Constructor and Description
AvroGenericRecordBolt()

Method Summary

Modifier and Type	Method and Description
AvroGenericRecordBolt	addRotationAction(RotationAction action)
protected void	doPrepare(Map conf, TopologyContext topologyContext, OutputCollector collector)
protected String	getWriterKey(Tuple tuple) AvroGenericRecordBolt must override this method because messages with different schemas cannot be written to the same file.
protected AbstractHDFSWriter	makeNewWriter(org.apache.hadoop.fs.Path path, Tuple tuple)
AvroGenericRecordBolt	withConfigKey(String configKey)
AvroGenericRecordBolt	withFileNameFormat(FileNameFormat fileNameFormat)
AvroGenericRecordBolt	withFsUrl(String fsUrl)
AvroGenericRecordBolt	withMaxOpenFiles(int maxOpenFiles)
AvroGenericRecordBolt	withPartitioner(Partitioner partitioner)
AvroGenericRecordBolt	withRotationPolicy(FileRotationPolicy rotationPolicy)
AvroGenericRecordBolt	withSyncPolicy(SyncPolicy syncPolicy)
AvroGenericRecordBolt	withTickTupleIntervalSeconds(int interval)

Methods inherited from class org.apache.storm.hdfs.bolt.AbstractHdfsBolt

cleanup, declareOutputFields, execute, getBasePathForNextFile, getComponentConfiguration, prepare, rotateOutputFile

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AvroGenericRecordBolt

public AvroGenericRecordBolt()

Method Detail

withFsUrl

public AvroGenericRecordBolt withFsUrl(String fsUrl)

withConfigKey

public AvroGenericRecordBolt withConfigKey(String configKey)

withFileNameFormat

public AvroGenericRecordBolt withFileNameFormat(FileNameFormat fileNameFormat)

withSyncPolicy

public AvroGenericRecordBolt withSyncPolicy(SyncPolicy syncPolicy)

withRotationPolicy

public AvroGenericRecordBolt withRotationPolicy(FileRotationPolicy rotationPolicy)

addRotationAction

public AvroGenericRecordBolt addRotationAction(RotationAction action)

withTickTupleIntervalSeconds

public AvroGenericRecordBolt withTickTupleIntervalSeconds(int interval)

withMaxOpenFiles

public AvroGenericRecordBolt withMaxOpenFiles(int maxOpenFiles)

withPartitioner

public AvroGenericRecordBolt withPartitioner(Partitioner partitioner)

doPrepare

protected void doPrepare(Map conf,
                         TopologyContext topologyContext,
                         OutputCollector collector)
                  throws IOException

Specified by:

doPrepare in class AbstractHdfsBolt

Throws:

IOException

getWriterKey

protected String getWriterKey(Tuple tuple)

AvroGenericRecordBolt must override this method because messages with different schemas cannot be written to the same file. By treating the complete schema as the “key” AbstractHdfsBolt will associate a different writer for every distinct schema.

Specified by:

getWriterKey in class AbstractHdfsBolt

makeNewWriter

protected AbstractHDFSWriter makeNewWriter(org.apache.hadoop.fs.Path path,
                                           Tuple tuple)
                                    throws IOException

Specified by:

makeNewWriter in class AbstractHdfsBolt

Throws:

IOException