Package org.apache.storm.topology.base
Class BaseStatefulWindowedBolt<T extends State>
java.lang.Object
org.apache.storm.topology.base.BaseWindowedBolt
org.apache.storm.topology.base.BaseStatefulWindowedBolt<T>
- All Implemented Interfaces:
Serializable
,IComponent
,IStatefulComponent<T>
,IStatefulWindowedBolt<T>
,IWindowedBolt
public abstract class BaseStatefulWindowedBolt<T extends State>
extends BaseWindowedBolt
implements IStatefulWindowedBolt<T>
- See Also:
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.storm.topology.base.BaseWindowedBolt
BaseWindowedBolt.Count, BaseWindowedBolt.Duration
-
Field Summary
Fields inherited from class org.apache.storm.topology.base.BaseWindowedBolt
timestampExtractor, windowConfiguration
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionboolean
If the stateful windowed bolt should have its windows persisted in state and maintain a subset of events in memory.long
The maximum number of window events to keep in memory.void
preCommit
(long txid) This is a hook for the component to perform some actions just before the framework commits its state.void
prePrepare
(long txid) This is a hook for the component to perform some actions just before the framework prepares its state.void
This is a hook for the component to perform some actions just before the framework rolls back the prepared state.withLag
(BaseWindowedBolt.Duration duration) Specify the maximum time lag of the tuple timestamp in milliseconds.withLateTupleStream
(String streamName) Specify a stream id on which late tuples are going to be emitted.withMaxEventsInMemory
(long maxEventsInMemory) The maximum number of window events to keep in memory.withMessageIdField
(String fieldName) Specify the name of the field in the tuple that holds the message id.If set, the stateful windowed bolt would use the backend state for window persistence and only keep a sub-set of events in memory as specified bywithMaxEventsInMemory(long)
.withTimestampExtractor
(TimestampExtractor timestampExtractor) Specify the timestamp extractor implementation.withTimestampField
(String fieldName) Specify a field in the tuple that represents the timestamp as a long value.A count based tumbling window.withTumblingWindow
(BaseWindowedBolt.Duration duration) A time duration based tumbling window.Specify the watermark event generation interval.withWindow
(BaseWindowedBolt.Count windowLength) A tuple count based window that slides with every incoming tuple.withWindow
(BaseWindowedBolt.Count windowLength, BaseWindowedBolt.Count slidingInterval) Tuple count based sliding window configuration.withWindow
(BaseWindowedBolt.Count windowLength, BaseWindowedBolt.Duration slidingInterval) Tuple count and time duration based sliding window configuration.withWindow
(BaseWindowedBolt.Duration windowLength) A time duration based window that slides with every incoming tuple.withWindow
(BaseWindowedBolt.Duration windowLength, BaseWindowedBolt.Count slidingInterval) Time duration and count based sliding window configuration.withWindow
(BaseWindowedBolt.Duration windowLength, BaseWindowedBolt.Duration slidingInterval) Time duration based sliding window configuration.Methods inherited from class org.apache.storm.topology.base.BaseWindowedBolt
cleanup, declareOutputFields, getComponentConfiguration, getTimestampExtractor, prepare
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.storm.topology.IComponent
declareOutputFields, getComponentConfiguration
Methods inherited from interface org.apache.storm.topology.IStatefulComponent
initState
Methods inherited from interface org.apache.storm.topology.IWindowedBolt
cleanup, execute, getTimestampExtractor, prepare
-
Constructor Details
-
BaseStatefulWindowedBolt
public BaseStatefulWindowedBolt()
-
-
Method Details
-
withWindow
public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Count windowLength, BaseWindowedBolt.Count slidingInterval) Tuple count based sliding window configuration.- Overrides:
withWindow
in classBaseWindowedBolt
- Parameters:
windowLength
- the number of tuples in the windowslidingInterval
- the number of tuples after which the window slides
-
withWindow
public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Count windowLength, BaseWindowedBolt.Duration slidingInterval) Tuple count and time duration based sliding window configuration.- Overrides:
withWindow
in classBaseWindowedBolt
- Parameters:
windowLength
- the number of tuples in the windowslidingInterval
- the time duration after which the window slides
-
withWindow
public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Duration windowLength, BaseWindowedBolt.Count slidingInterval) Time duration and count based sliding window configuration.- Overrides:
withWindow
in classBaseWindowedBolt
- Parameters:
windowLength
- the time duration of the windowslidingInterval
- the number of tuples after which the window slides
-
withWindow
public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Duration windowLength, BaseWindowedBolt.Duration slidingInterval) Time duration based sliding window configuration.- Overrides:
withWindow
in classBaseWindowedBolt
- Parameters:
windowLength
- the time duration of the windowslidingInterval
- the time duration after which the window slides
-
withWindow
A tuple count based window that slides with every incoming tuple.- Overrides:
withWindow
in classBaseWindowedBolt
- Parameters:
windowLength
- the number of tuples in the window
-
withWindow
A time duration based window that slides with every incoming tuple.- Overrides:
withWindow
in classBaseWindowedBolt
- Parameters:
windowLength
- the time duration of the window
-
withTumblingWindow
A count based tumbling window.- Overrides:
withTumblingWindow
in classBaseWindowedBolt
- Parameters:
count
- the number of tuples after which the window tumbles
-
withTumblingWindow
A time duration based tumbling window.- Overrides:
withTumblingWindow
in classBaseWindowedBolt
- Parameters:
duration
- the time duration after which the window tumbles
-
withTimestampField
Specify a field in the tuple that represents the timestamp as a long value. If this field is not present in the incoming tuple, anIllegalArgumentException
will be thrown. The field MUST contain a timestamp in milliseconds- Overrides:
withTimestampField
in classBaseWindowedBolt
- Parameters:
fieldName
- the name of the field that contains the timestamp
-
withTimestampExtractor
Specify the timestamp extractor implementation.- Overrides:
withTimestampExtractor
in classBaseWindowedBolt
- Parameters:
timestampExtractor
- theTimestampExtractor
implementation
-
withLateTupleStream
Specify a stream id on which late tuples are going to be emitted. They are going to be accessible via theWindowedBoltExecutor.LATE_TUPLE_FIELD
field. It must be defined on a per-component basis, and in conjunction with theBaseWindowedBolt.withTimestampField(java.lang.String)
, otherwiseIllegalArgumentException
will be thrown.- Overrides:
withLateTupleStream
in classBaseWindowedBolt
- Parameters:
streamName
- the name of the stream used to emit late tuples on
-
withLag
Specify the maximum time lag of the tuple timestamp in milliseconds. It means that the tuple timestamps cannot be out of order by more than this amount.- Overrides:
withLag
in classBaseWindowedBolt
- Parameters:
duration
- the max lag duration
-
withWatermarkInterval
Specify the watermark event generation interval. For tuple based timestamps, watermark events are used to track the progress of time- Overrides:
withWatermarkInterval
in classBaseWindowedBolt
- Parameters:
interval
- the interval at which watermark events are generated
-
withMessageIdField
Specify the name of the field in the tuple that holds the message id. This is used to track the windowing boundaries and re-evaluating the windowing operation during recovery of IStatefulWindowedBolt- Parameters:
fieldName
- the name of the field that contains the message id
-
withPersistence
If set, the stateful windowed bolt would use the backend state for window persistence and only keep a sub-set of events in memory as specified bywithMaxEventsInMemory(long)
. -
withMaxEventsInMemory
The maximum number of window events to keep in memory. This is meaningful only ifwithPersistence()
is also set. As the number of events in memory grows close to the maximum, the events that are less likely to be used again are evicted and persisted. The default value for this is1,000,000
.- Parameters:
maxEventsInMemory
- the maximum number of window events to keep in memory
-
isPersistent
public boolean isPersistent()Description copied from interface:IStatefulWindowedBolt
If the stateful windowed bolt should have its windows persisted in state and maintain a subset of events in memory.The default is to keep all the window events in memory.
- Specified by:
isPersistent
in interfaceIStatefulWindowedBolt<T extends State>
- Returns:
- true if the windows should be persisted
-
maxEventsInMemory
public long maxEventsInMemory()Description copied from interface:IStatefulWindowedBolt
The maximum number of window events to keep in memory.- Specified by:
maxEventsInMemory
in interfaceIStatefulWindowedBolt<T extends State>
-
preCommit
public void preCommit(long txid) Description copied from interface:IStatefulComponent
This is a hook for the component to perform some actions just before the framework commits its state.- Specified by:
preCommit
in interfaceIStatefulComponent<T extends State>
-
prePrepare
public void prePrepare(long txid) Description copied from interface:IStatefulComponent
This is a hook for the component to perform some actions just before the framework prepares its state.- Specified by:
prePrepare
in interfaceIStatefulComponent<T extends State>
-
preRollback
public void preRollback()Description copied from interface:IStatefulComponent
This is a hook for the component to perform some actions just before the framework rolls back the prepared state.- Specified by:
preRollback
in interfaceIStatefulComponent<T extends State>
-