org.apache.storm.topology.base

Class BaseStatefulWindowedBolt<T extends State>

java.lang.Object

org.apache.storm.topology.base.BaseWindowedBolt

org.apache.storm.topology.base.BaseStatefulWindowedBolt<T>

All Implemented Interfaces:

Serializable, IComponent, IStatefulComponent<T>, IStatefulWindowedBolt<T>, IWindowedBolt

public abstract class BaseStatefulWindowedBolt<T extends State>
extends BaseWindowedBolt
implements IStatefulWindowedBolt<T>

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.storm.topology.base.BaseWindowedBolt

BaseWindowedBolt.Count, BaseWindowedBolt.Duration

Field Summary

Fields inherited from class org.apache.storm.topology.base.BaseWindowedBolt

timestampExtractor, windowConfiguration

Constructor Summary

Constructors

Constructor and Description
BaseStatefulWindowedBolt()

Method Summary

Modifier and Type	Method and Description
boolean	isPersistent() If the stateful windowed bolt should have its windows persisted in state and maintain a subset of events in memory.
long	maxEventsInMemory() The maximum number of window events to keep in memory.
void	preCommit(long txid) This is a hook for the component to perform some actions just before the framework commits its state.
void	prePrepare(long txid) This is a hook for the component to perform some actions just before the framework prepares its state.
void	preRollback() This is a hook for the component to perform some actions just before the framework rolls back the prepared state.
BaseStatefulWindowedBolt<T>	withLag(BaseWindowedBolt.Duration duration) Specify the maximum time lag of the tuple timestamp in milliseconds.
BaseStatefulWindowedBolt<T>	withLateTupleStream(String streamName) Specify a stream id on which late tuples are going to be emitted.
BaseStatefulWindowedBolt<T>	withMaxEventsInMemory(long maxEventsInMemory) The maximum number of window events to keep in memory.
BaseStatefulWindowedBolt<T>	withMessageIdField(String fieldName) Specify the name of the field in the tuple that holds the message id.
BaseStatefulWindowedBolt<T>	withPersistence() If set, the stateful windowed bolt would use the backend state for window persistence and only keep a sub-set of events in memory as specified by withMaxEventsInMemory(long).
BaseStatefulWindowedBolt<T>	withTimestampExtractor(TimestampExtractor timestampExtractor) Specify the timestamp extractor implementation.
BaseStatefulWindowedBolt<T>	withTimestampField(String fieldName) Specify a field in the tuple that represents the timestamp as a long value.
BaseStatefulWindowedBolt<T>	withTumblingWindow(BaseWindowedBolt.Count count) A count based tumbling window.
BaseStatefulWindowedBolt<T>	withTumblingWindow(BaseWindowedBolt.Duration duration) A time duration based tumbling window.
BaseStatefulWindowedBolt<T>	withWatermarkInterval(BaseWindowedBolt.Duration interval) Specify the watermark event generation interval.
BaseStatefulWindowedBolt<T>	withWindow(BaseWindowedBolt.Count windowLength) A tuple count based window that slides with every incoming tuple.
BaseStatefulWindowedBolt<T>	withWindow(BaseWindowedBolt.Count windowLength, BaseWindowedBolt.Count slidingInterval) Tuple count based sliding window configuration.
BaseStatefulWindowedBolt<T>	withWindow(BaseWindowedBolt.Count windowLength, BaseWindowedBolt.Duration slidingInterval) Tuple count and time duration based sliding window configuration.
BaseStatefulWindowedBolt<T>	withWindow(BaseWindowedBolt.Duration windowLength) A time duration based window that slides with every incoming tuple.
BaseStatefulWindowedBolt<T>	withWindow(BaseWindowedBolt.Duration windowLength, BaseWindowedBolt.Count slidingInterval) Time duration and count based sliding window configuration.
BaseStatefulWindowedBolt<T>	withWindow(BaseWindowedBolt.Duration windowLength, BaseWindowedBolt.Duration slidingInterval) Time duration based sliding window configuration.

Methods inherited from class org.apache.storm.topology.base.BaseWindowedBolt

cleanup, declareOutputFields, getComponentConfiguration, getTimestampExtractor, prepare

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.storm.topology.IStatefulComponent

initState

Methods inherited from interface org.apache.storm.topology.IWindowedBolt

cleanup, execute, getTimestampExtractor, prepare

Methods inherited from interface org.apache.storm.topology.IComponent

declareOutputFields, getComponentConfiguration

Constructor Detail

BaseStatefulWindowedBolt

public BaseStatefulWindowedBolt()

Method Detail

withWindow

public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Count windowLength,
                                              BaseWindowedBolt.Count slidingInterval)

Tuple count based sliding window configuration.

Overrides:

withWindow in class BaseWindowedBolt

Parameters:

windowLength - the number of tuples in the window

slidingInterval - the number of tuples after which the window slides

withWindow

public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Count windowLength,
                                              BaseWindowedBolt.Duration slidingInterval)

Tuple count and time duration based sliding window configuration.

Overrides:

withWindow in class BaseWindowedBolt

Parameters:

windowLength - the number of tuples in the window

slidingInterval - the time duration after which the window slides

withWindow

public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Duration windowLength,
                                              BaseWindowedBolt.Count slidingInterval)

Time duration and count based sliding window configuration.

Overrides:

withWindow in class BaseWindowedBolt

Parameters:

windowLength - the time duration of the window

slidingInterval - the number of tuples after which the window slides

withWindow

public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Duration windowLength,
                                              BaseWindowedBolt.Duration slidingInterval)

Time duration based sliding window configuration.

Overrides:

withWindow in class BaseWindowedBolt

Parameters:

windowLength - the time duration of the window

slidingInterval - the time duration after which the window slides

withWindow

public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Count windowLength)

A tuple count based window that slides with every incoming tuple.

Overrides:

withWindow in class BaseWindowedBolt

Parameters:

windowLength - the number of tuples in the window

withWindow

public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Duration windowLength)

A time duration based window that slides with every incoming tuple.

Overrides:

withWindow in class BaseWindowedBolt

Parameters:

windowLength - the time duration of the window

withTumblingWindow

public BaseStatefulWindowedBolt<T> withTumblingWindow(BaseWindowedBolt.Count count)

A count based tumbling window.

Overrides:

withTumblingWindow in class BaseWindowedBolt

Parameters:

count - the number of tuples after which the window tumbles

withTumblingWindow

public BaseStatefulWindowedBolt<T> withTumblingWindow(BaseWindowedBolt.Duration duration)

A time duration based tumbling window.

Overrides:

withTumblingWindow in class BaseWindowedBolt

Parameters:

duration - the time duration after which the window tumbles

withTimestampField

public BaseStatefulWindowedBolt<T> withTimestampField(String fieldName)

Specify a field in the tuple that represents the timestamp as a long value. If this field is not present in the incoming tuple, an IllegalArgumentException will be thrown. The field MUST contain a timestamp in milliseconds

Overrides:

withTimestampField in class BaseWindowedBolt

Parameters:

fieldName - the name of the field that contains the timestamp

withTimestampExtractor

public BaseStatefulWindowedBolt<T> withTimestampExtractor(TimestampExtractor timestampExtractor)

Specify the timestamp extractor implementation.

Overrides:

withTimestampExtractor in class BaseWindowedBolt

Parameters:

timestampExtractor - the TimestampExtractor implementation

withLateTupleStream

public BaseStatefulWindowedBolt<T> withLateTupleStream(String streamName)

Specify a stream id on which late tuples are going to be emitted. They are going to be accessible via the WindowedBoltExecutor.LATE_TUPLE_FIELD field. It must be defined on a per-component basis, and in conjunction with the BaseWindowedBolt.withTimestampField(java.lang.String), otherwise IllegalArgumentException will be thrown.

Overrides:

withLateTupleStream in class BaseWindowedBolt

Parameters:

streamName - the name of the stream used to emit late tuples on

withLag

public BaseStatefulWindowedBolt<T> withLag(BaseWindowedBolt.Duration duration)

Specify the maximum time lag of the tuple timestamp in milliseconds. It means that the tuple timestamps cannot be out of order by more than this amount.

Overrides:

withLag in class BaseWindowedBolt

Parameters:

duration - the max lag duration

withWatermarkInterval

public BaseStatefulWindowedBolt<T> withWatermarkInterval(BaseWindowedBolt.Duration interval)

Specify the watermark event generation interval. For tuple based timestamps, watermark events are used to track the progress of time

Overrides:

withWatermarkInterval in class BaseWindowedBolt

Parameters:

interval - the interval at which watermark events are generated

withMessageIdField

public BaseStatefulWindowedBolt<T> withMessageIdField(String fieldName)

Specify the name of the field in the tuple that holds the message id. This is used to track the windowing boundaries and re-evaluating the windowing operation during recovery of IStatefulWindowedBolt

Parameters:

fieldName - the name of the field that contains the message id

withPersistence

public BaseStatefulWindowedBolt<T> withPersistence()

If set, the stateful windowed bolt would use the backend state for window persistence and only keep a sub-set of events in memory as specified by withMaxEventsInMemory(long).

withMaxEventsInMemory

public BaseStatefulWindowedBolt<T> withMaxEventsInMemory(long maxEventsInMemory)

The maximum number of window events to keep in memory. This is meaningful only if withPersistence() is also set. As the number of events in memory grows close to the maximum, the events that are less likely to be used again are evicted and persisted. The default value for this is 1,000,000.

Parameters:

maxEventsInMemory - the maximum number of window events to keep in memory

isPersistent

public boolean isPersistent()

Description copied from interface: IStatefulWindowedBolt

If the stateful windowed bolt should have its windows persisted in state and maintain a subset of events in memory.

The default is to keep all the window events in memory.

Specified by:

isPersistent in interface IStatefulWindowedBolt<T extends State>

Returns:

true if the windows should be persisted

maxEventsInMemory

public long maxEventsInMemory()

Description copied from interface: IStatefulWindowedBolt

The maximum number of window events to keep in memory.

Specified by:

maxEventsInMemory in interface IStatefulWindowedBolt<T extends State>

preCommit

public void preCommit(long txid)

Description copied from interface: IStatefulComponent

This is a hook for the component to perform some actions just before the framework commits its state.

Specified by:

preCommit in interface IStatefulComponent<T extends State>

prePrepare

public void prePrepare(long txid)

Description copied from interface: IStatefulComponent

This is a hook for the component to perform some actions just before the framework prepares its state.

Specified by:

prePrepare in interface IStatefulComponent<T extends State>

preRollback

public void preRollback()

Description copied from interface: IStatefulComponent

This is a hook for the component to perform some actions just before the framework rolls back the prepared state.

Specified by:

preRollback in interface IStatefulComponent<T extends State>