Fork me on GitHub

Latest News

The Apache Storm community is pleased to announce that version 3.0.0 has been released and is available from the downloads page.

πŸš€ Apache Storm 3.0.0 β€” The New Major Release

Storm 3.0.0 is a major milestone for the project, packed with new features: complete removal of the Clojure dependency, Zstd compression for inter-worker communication and cluster state, AIMD-based dynamic batch sizing, a new jitter-aware stream grouping, significant scheduler improvements, a leaner binary distribution, and much more.

⚠️ Java 25 is required to run Apache Storm 3.0.0. Please ensure your environment is upgraded before migrating.

The Java API is fully backwards-compatible with Storm 2.x. Existing Java topologies and client code should migrate seamlessly β€” no API changes are required. The one exception is the Clojure DSL, which has been removed; see the section below.

Simultaneously with this release, Apache Storm 2.8.9 has been published as the final release of the 2.x line. The Storm 2.x branch will no longer be maintained. All community efforts will be focused on the Storm 3.x release stream.

See the Storm 2.8.9 release announcement for details on the end-of-life of the 2.x series.

Clojure support removed

⚠️ Breaking change for Clojure topology users

Storm 3.0.0 removes Clojure support. The Clojure DSL and the storm-clojure module are no longer available.

If your topology is written using the Clojure API (backtype.storm.clojure or org.apache.storm.clojure), you must rewrite it using the Java API before upgrading to Storm 3.0.0. The Java API covers all the same primitives and the migration is straightforward for most topologies.

Clojure is no longer a transitive dependency of any Storm module. Clusters running only Java topologies are unaffected.

Two Distribution Flavours

Starting with Storm 3.0.0, the binary distribution is available in two flavours (#8819):

apache-storm-3.0.0-lite

A leaner package with optional integrations removed from the bundle. Add only what your topology needs via Maven or by dropping jars into the lib-worker directory.

Removed:

  • Optional Hadoop integration jars
  • Optional Kafka integration jars
  • Duplicated shared jars

Gains:

  • Significantly smaller distribution size
  • Faster downloads and deploys

apache-storm-3.0.0

The full distribution, bundling all optional integrations as in previous releases. Drop-in replacement for users upgrading from Storm 2.x who want to keep their existing setup unchanged.

Both flavours are available on the downloads page.

More highlights

πŸ—œοΈ Zstd compression opt-in #8707 #8653

Inter-worker tuple traffic and ZooKeeper cluster state can now be compressed with Zstd, reducing network bandwidth and coordination-layer load. Enabled per-topology or cluster-wide via configuration.

〰️ JitterAwareStreamGrouping opt-in #8593

A new stream grouping that routes tuples away from high-jitter tasks using an RFC 1889 EWMA jitter metric. Opt in per stream by replacing shuffleGrouping with customGrouping("stream", new JitterAwareStreamGrouping()).

πŸ“ˆ AIMD batch sizing opt-in #8796

Producer batch sizes in JCQueue can now adapt dynamically under backpressure using additive-increase / multiplicative-decrease, improving throughput without manual tuning. Enable with topology.producer.batch.dynamic: true.

πŸ“… Scheduler improvements #8709 #8778

IsolationScheduler now sorts candidate hosts by free slots as a secondary criterion (after total capacity), reducing unnecessary worker evictions β€” this applies automatically. EvenScheduler gains an opt-in pass that redistributes workers onto supervisors returning from maintenance (nimbus.even.rebalance.idle.supervisor.enabled: true).

🐳 Local dev cluster #8706

A Docker Compose cluster under docker/ with Nimbus, ZooKeeper, two Supervisors, Prometheus, and Grafana. Includes netsim.sh for injecting network latency and packet loss between nodes without modifying the host network.

πŸ”§ Y2038 fix #8788

Worker heartbeat timestamps migrated from 32-bit to 64-bit integers, eliminating the overflow that would have caused heartbeat failures on clusters running past January 2038.

Thanks

Special thanks are due to all those who have contributed to Apache Storm -- whether through direct code contributions, documentation, bug reports, or helping other users on the mailing lists. Your efforts are much appreciated.

Changes in this Release - Storm 3.0.0

Issues addressed in the 3.0.0 release of Storm. Documentation for this release is available at the Apache Storm project site.

Bug fixes

Workers & Heartbeats

  • [#8905] - Fix worker launch: add lib-common to the supervisor's worker classpath and restore the lib-worker drop-in directory
  • [#8896] - Fix NPE in WorkerState.transferLocalBatch causing worker crashes
  • [#8800] - Filter worker heartbeats by per-topology timeout
  • [#8788] - Fix Y2038 worker heartbeat overflow by migrating timestamps to i64
  • [#7897] - [STORM-4116] Heartbeats mechanism is affected by Y2038 bug

Scheduler

  • [#8798] - Fix double max.free.per.topology cap in EvenScheduler idle rebalance
  • [#8705] - Remove shuffle post sorting assignable hosts in Isolation Scheduler
  • [#8708] - Isolation Scheduler Can Have Free Slots Check as a Secondary Sorting Check
  • [#8704] - Isolation Scheduler Does Shuffle After Sort so sorting does not makes sense

Web UI

  • [#8889] - fix(webapp): guard Flux viewer against js-yaml 5.x empty-input throw

Windowing

  • [#8790] - Fix late tuple stream serialization in windowed bolts

Kafka

  • [#8589] - fix(topology_lag): Kafka Topology Lag breaking when no offsets are commited for topic/partition

Enhancements

Clojure Removal

  • [#8537] - Removed all uses of Clojure, version 3.0.0-SNAPSHOT
  • [#8455] - Replace Thread.sleep and Testing.whileTimeout with Awaitility in tests
  • [#8446] - Port Clojure tests to Java: cluster_test (Phase 3)
  • [#8445] - Port Clojure tests to Java and remove Clojure dependency
  • [#8444] - Port Clojure tests to Java: scheduler_test and multitenant_scheduler_…
  • [#8443] - Port Clojure tests to Java: versioned_store_test and tuple_test

Messaging & Performance

  • [#8892] - Decouple the control plane from the data plane on receive queues
  • [#8796] - Add AIMD based dynamic producer batch sizing to `JCQueue`
  • [#8707] - Add tuple compression for inter-worker communication
  • [#8701] - Introduce component-level compression configuration for remote worker traffic
  • [#8593] - Add jitter control loop and `JitterAwareStreamGrouping`
  • [#8586] - Introduce jitter metric based on RFC 1889 Appendix A

Compression

  • [#8653] - Zstd compression for thrift serialization (storm cluster state)
  • [#8652] - Add zstd compression support as an alternative to GZIP in serialization

Scheduler

  • [#8778] - EvenScheduler: opt-in round-robin rebalance onto returning idle supervisors
  • [#8789] - Skip stale worker heartbeats from orphaned worker directories
  • [#8714] - Optimize ZooKeeper read requests and remove redundant checks
  • [#8709] - Isolation Scheduler Hosts Sorting | Add secondary sort (No of free slots) and tertiary sort (Hostname)
  • [#8811] - scheduler: replace reflection with @VisibleForTesting in IsolationSchedulerTest

Distribution & Build

  • [#8819] - Reduce binary distribution size: unbundle optional Hadoop/Kafka deps + de-duplicate shared jars
  • [#8541] - build: introduce aggregator modules and unify pom display names
  • [#8582] - storm-submit-tools: modernize Java code, no API surface changes
  • [#8813] - Improve RELEASING.md accuracy and completeness

Java & Platform

  • [#8893] - Bump Java baseline to 25 on master
  • [#8594] - Bump Java to 21, update docs for Storm 3.x and javadoc.io migration
  • [#8699] - Make storm more windows friendly

CI

  • [#8887] - build: run ASF allowlist check on every PR so the required check reports
  • [#8817] - ci: add ASF allowlist check for GitHub Actions (Dependabot guard)

Other

  • [#8706] - Docker compose local (dev) distributed Storm cluster with full observability and network simulation
  • [#8702] - Fix checkstyle violations (#8585 - Phase 3 - Part 1)

Documentation

  • [#8700] - Update checkstyle and Modernize config (#8585 - Phase 1, Phase 2)
  • [#8587] - [STORM-3962] Update integration page for Kafka links
  • [#8584] - Fix outdated code references in message passing documentation
  • [#8539] - docs: drop stale Clojure references after Clojure removal
  • [#7744] - [STORM-3962] Update integration page for Kafka links

Dependency upgrades

  • [#8901] - build(deps): bump actions/setup-java from 5.5.0 to 5.6.0
  • [#8858] - build(deps): bump cytoscape-dagre from 3.0.0 to 4.0.0 in /storm-webapp
  • [#8857] - build(deps-dev): bump start-server-and-test from 3.0.5 to 3.0.11 in /storm-webapp
  • [#8856] - build(deps): bump cytoscape from 3.33.4 to 3.34.0 in /storm-webapp
  • [#8855] - build(deps): bump js-yaml from 4.2.0 to 5.2.0 in /storm-webapp
  • [#8854] - build(deps-dev): bump webpack from 5.107.2 to 5.108.3 in /storm-webapp
  • [#8853] - build(deps-dev): bump cypress from 15.16.0 to 15.18.0 in /storm-webapp
  • [#8852] - build(deps-dev): bump webpack-cli from 7.0.3 to 7.1.0 in /storm-webapp
  • [#8851] - build(deps): bump com.puppycrawl.tools:checkstyle from 13.5.0 to 13.7.0 in /examples
  • [#8850] - build(deps): bump com.github.luben:zstd-jni from 1.5.7-9 to 1.5.7-11 in /examples
  • [#8849] - build(deps): bump org.apache:apache from 38 to 39 in /examples
  • [#8848] - build(deps): bump org.apache.kafka:kafka-clients from 4.3.0 to 4.3.1 in /examples
  • [#8847] - build(deps): bump org.junit:junit-bom from 6.1.0 to 6.1.1 in /examples
  • [#8846] - build(deps): bump com.fasterxml.woodstox:woodstox-core from 7.2.0 to 7.2.1 in /examples
  • [#8845] - build(deps): bump jetty.version from 12.1.9 to 12.1.10 in /examples
  • [#8844] - build(deps): bump com.google.errorprone:error_prone_annotations from 2.49.0 to 2.50.0 in /examples
  • [#8843] - build(deps): bump activemq.version from 6.2.6 to 6.2.7 in /examples
  • [#8842] - build(deps): bump byte-buddy.version from 1.18.8 to 1.18.10 in /examples
  • [#8841] - build(deps-dev): bump org.jacoco:jacoco-maven-plugin from 0.8.14 to 0.8.15
  • [#8840] - build(deps): bump io.netty:netty-bom from 4.2.14.Final to 4.2.15.Final in /examples
  • [#8839] - build(deps): bump io.netty:netty-bom from 4.2.14.Final to 4.2.15.Final
  • [#8838] - build(deps): bump com.fasterxml.woodstox:woodstox-core from 7.2.0 to 7.2.1
  • [#8837] - build(deps): bump spring.version from 7.0.7 to 7.0.8 in /examples
  • [#8836] - build(deps): bump com.zaxxer:HikariCP from 7.0.2 to 7.1.0 in /examples
  • [#8835] - build(deps): bump com.github.luben:zstd-jni from 1.5.7-9 to 1.5.7-11
  • [#8834] - build(deps): bump commons-logging:commons-logging from 1.3.6 to 1.4.0 in /examples
  • [#8833] - build(deps): bump storm.kafka.client.version from 4.3.0 to 4.3.1
  • [#8832] - build(deps): bump com.zaxxer:HikariCP from 7.0.2 to 7.1.0
  • [#8831] - build(deps): bump com.google.errorprone:error_prone_annotations from 2.49.0 to 2.50.0
  • [#8830] - build(deps-dev): bump com.github.eirslett:frontend-maven-plugin from 2.0.0 to 2.0.1
  • [#8829] - build(deps): bump byte-buddy.version from 1.18.8 to 1.18.10
  • [#8828] - build(deps): bump org.apache.hbase:hbase-client from 2.6.5-hadoop3 to 2.6.6-hadoop3
  • [#8827] - build(deps): bump activemq.version from 6.2.6 to 6.2.7
  • [#8826] - build(deps): bump org.junit:junit-bom from 6.1.0 to 6.1.1
  • [#8825] - build(deps): bump commons-logging:commons-logging from 1.3.6 to 1.4.0
  • [#8824] - build(deps): bump spring.version from 7.0.7 to 7.0.8
  • [#8823] - build(deps): bump org.apache:apache from 38 to 39
  • [#8822] - build(deps): bump jetty.version from 12.1.9 to 12.1.10
  • [#8821] - build(deps): bump netty-tcnative.version from 2.0.77.Final to 2.0.80.Final
  • [#8820] - build(deps): bump prometheus.client.version from 1.6.1 to 1.8.0
  • [#8751] - Bump webpack-cli from 7.0.2 to 7.0.3 in /storm-webapp
  • [#8750] - Bump terser-webpack-plugin from 5.6.0 to 5.6.1 in /storm-webapp
  • [#8749] - Bump cypress from 15.15.0 to 15.16.0 in /storm-webapp
  • [#8748] - Bump js-yaml from 4.1.1 to 4.2.0 in /storm-webapp
  • [#8747] - Bump js-cookie from 3.0.7 to 3.0.8 in /storm-webapp
  • [#8746] - Bump cytoscape from 3.33.3 to 3.33.4 in /storm-webapp
  • [#8745] - Bump webpack from 5.106.2 to 5.107.2 in /storm-webapp
  • [#8743] - Bump com.fasterxml.jackson:jackson-bom from 2.21.3 to 2.22.0 in /examples
  • [#8742] - Bump io.netty:netty-bom from 4.2.13.Final to 4.2.14.Final in /examples
  • [#8741] - Bump com.puppycrawl.tools:checkstyle from 13.4.2 to 13.5.0 in /examples
  • [#8740] - Bump org.apache.kafka:kafka-clients from 4.2.0 to 4.3.0 in /examples
  • [#8739] - Bump com.github.luben:zstd-jni from 1.5.7-8 to 1.5.7-9 in /examples
  • [#8738] - Bump org.apache.commons:commons-configuration2 from 2.15.0 to 2.15.1 in /examples
  • [#8737] - Bump org.junit:junit-bom from 6.0.3 to 6.1.0 in /examples
  • [#8736] - Bump org.ow2.asm:asm from 9.10 to 9.10.1 in /examples
  • [#8735] - Bump surefire.version from 3.5.5 to 3.5.6 in /examples
  • [#8734] - Bump org.apache.maven.plugins:maven-surefire-report-plugin from 3.5.5 to 3.5.6 in /examples
  • [#8733] - Bump activemq.version from 6.2.5 to 6.2.6 in /examples
  • [#8732] - Bump metrics.version from 4.2.38 to 4.2.39 in /examples
  • [#8731] - Bump com.fasterxml.woodstox:woodstox-core from 7.1.1 to 7.2.0 in /examples
  • [#8730] - Bump com.fasterxml.woodstox:woodstox-core from 7.1.1 to 7.2.0
  • [#8729] - Bump org.ow2.asm:asm from 9.10 to 9.10.1
  • [#8728] - Bump io.netty:netty-bom from 4.2.13.Final to 4.2.14.Final
  • [#8727] - Bump com.fasterxml.jackson:jackson-bom from 2.21.3 to 2.22.0
  • [#8726] - Bump storm.kafka.client.version from 4.2.0 to 4.3.0
  • [#8725] - Bump org.apache.commons:commons-configuration2 from 2.15.0 to 2.15.1
  • [#8724] - Bump org.apache.maven.plugins:maven-dependency-plugin from 3.10.0 to 3.11.0
  • [#8723] - Bump com.fasterxml.jackson.core:jackson-databind from 2.21.3 to 2.22.0
  • [#8722] - Bump surefire.version from 3.5.5 to 3.5.6
  • [#8721] - Bump org.junit:junit-bom from 6.0.3 to 6.1.0
  • [#8720] - Bump com.github.luben:zstd-jni from 1.5.7-8 to 1.5.7-9
  • [#8719] - Bump redis.clients:jedis from 7.5.0 to 7.5.2
  • [#8718] - Bump metrics.version from 4.2.38 to 4.2.39
  • [#8717] - Bump dropwizard.version from 5.0.1 to 5.0.2
  • [#8716] - Bump org.apache.maven.plugins:maven-surefire-report-plugin from 3.5.5 to 3.5.6
  • [#8715] - Bump activemq.version from 6.2.5 to 6.2.6
  • [#8713] - Bump qs and @cypress/request in /storm-webapp
  • [#8676] - Bump org.ow2.asm:asm from 9.9.1 to 9.10
  • [#8674] - Bump org.ow2.asm:asm from 9.9.1 to 9.10 in /examples
  • [#8672] - Bump com.puppycrawl.tools:checkstyle from 8.2 to 13.4.2 in /examples
  • [#8671] - Bump org.apache.commons:commons-configuration2 from 2.14.0 to 2.15.0
  • [#8669] - Bump slf4j.version from 2.0.17 to 2.0.18 in /examples
  • [#8668] - Bump org.apache.commons:commons-configuration2 from 2.14.0 to 2.15.0 in /examples
  • [#8667] - Bump slf4j.version from 2.0.17 to 2.0.18
  • [#8664] - Bump org.apache.maven:maven-resolver-provider from 3.9.15 to 3.9.16
  • [#8661] - Bump js-cookie from 3.0.5 to 3.0.7 in /storm-webapp
  • [#8659] - Bump cypress from 15.14.2 to 15.15.0 in /storm-webapp
  • [#8658] - Bump start-server-and-test from 3.0.4 to 3.0.5 in /storm-webapp
  • [#8657] - Bump cytoscape-dagre from 2.5.0 to 3.0.0 in /storm-webapp
  • [#8656] - Bump vis-network from 10.0.3 to 10.1.0 in /storm-webapp
  • [#8638] - Bump org.apache.thrift:libthrift from 0.22.0 to 0.23.0
  • [#8634] - Bump org.apache.logging.log4j:log4j-bom from 2.25.4 to 2.26.0
  • [#8633] - Bump io.netty:netty-bom from 4.2.12.Final to 4.2.13.Final in /examples
  • [#8629] - Bump com.github.ben-manes.caffeine:caffeine from 3.2.3 to 3.2.4
  • [#8625] - Bump org.apache:apache from 37 to 38
  • [#8623] - Bump com.github.ben-manes.caffeine:caffeine from 3.2.3 to 3.2.4 in /examples
  • [#8618] - Bump org.apache:apache from 37 to 38 in /examples
  • [#8616] - Bump io.netty:netty-bom from 4.2.12.Final to 4.2.13.Final
  • [#8615] - Bump com.puppycrawl.tools:checkstyle from 8.2 to 13.4.2 in /examples
  • [#8612] - Bump terser-webpack-plugin from 5.5.0 to 5.6.0 in /storm-webapp
  • [#8611] - Bump jetty.version from 12.1.8 to 12.1.9 in /examples
  • [#8608] - Bump vis-data from 8.0.3 to 8.0.4 in /storm-webapp
  • [#8605] - Bump start-server-and-test from 3.0.2 to 3.0.4 in /storm-webapp
  • [#8603] - Bump jetty.version from 12.1.8 to 12.1.9
  • [#8600] - Bump vis-network from 10.0.2 to 10.0.3 in /storm-webapp
  • [#8598] - Bump org.apache.logging.log4j:log4j-bom from 2.25.4 to 2.26.0 in /examples
  • [#8597] - Bump org.apache.thrift:libthrift from 0.22.0 to 0.23.0 in /examples
  • [#8580] - Remove dead dependencyManagement entries and unused version properties
  • [#8578] - Bump cytoscape from 3.33.2 to 3.33.3 in /storm-webapp
  • [#8577] - Bump terser-webpack-plugin from 5.4.0 to 5.5.0 in /storm-webapp
  • [#8576] - Bump cypress from 15.13.0 to 15.14.2 in /storm-webapp
  • [#8575] - Bump datatables.net from 2.3.7 to 2.3.8 in /storm-webapp
  • [#8574] - Bump datatables.net-bs5 from 2.3.7 to 2.3.8 in /storm-webapp
  • [#8573] - Bump webpack from 5.105.4 to 5.106.2 in /storm-webapp
  • [#8572] - Bump datatables.net-dt from 2.3.7 to 2.3.8 in /storm-webapp
  • [#8571] - Bump start-server-and-test from 3.0.0 to 3.0.2 in /storm-webapp
  • [#8569] - Bump mockito.version from 5.14.2 to 5.23.0 in /examples
  • [#8567] - Bump io.github.git-commit-id:git-commit-id-maven-plugin from 9.0.1 to 10.0.0 in /examples
  • [#8566] - Bump com.google.code.gson:gson from 2.13.2 to 2.14.0 in /examples
  • [#8564] - Bump commons-io:commons-io from 2.21.0 to 2.22.0 in /examples
  • [#8560] - Bump org.yaml:snakeyaml from 2.2 to 2.6 in /examples
  • [#8559] - Bump com.fasterxml.jackson:jackson-bom from 2.21.2 to 2.21.3 in /examples
  • [#8558] - Bump spring.version from 7.0.6 to 7.0.7 in /examples
  • [#8557] - Bump activemq.version from 6.2.4 to 6.2.5 in /examples
  • [#8556] - Bump commons-codec:commons-codec from 1.21.0 to 1.22.0 in /examples
  • [#8555] - Bump io.github.git-commit-id:git-commit-id-maven-plugin from 9.0.1 to 10.0.0
  • [#8554] - Bump netty-tcnative.version from 2.0.75.Final to 2.0.77.Final
  • [#8553] - Bump redis.clients:jedis from 7.4.1 to 7.5.0
  • [#8552] - Bump com.fasterxml.jackson.core:jackson-databind from 2.21.2 to 2.21.3
  • [#8551] - Bump org.apache.maven:maven-resolver-provider from 3.9.14 to 3.9.15
  • [#8550] - Bump spring.version from 7.0.6 to 7.0.7
  • [#8549] - Bump commons-codec:commons-codec from 1.21.0 to 1.22.0
  • [#8548] - Bump com.fasterxml.jackson:jackson-bom from 2.21.2 to 2.21.3
  • [#8547] - Bump com.google.code.gson:gson from 2.13.2 to 2.14.0
  • [#8546] - Bump joda-time:joda-time from 2.14.1 to 2.14.2
  • [#8545] - Bump commons-io:commons-io from 2.21.0 to 2.22.0
  • [#8544] - Bump prometheus.client.version from 1.5.1 to 1.6.1
  • [#8543] - Bump activemq.version from 6.2.4 to 6.2.5
  • [#8540] - build: replace storm-maven-plugins with git-commit-id-maven-plugin