Package org.apache.storm.daemon.nimbus
Class Nimbus
- java.lang.Object
-
- org.apache.storm.daemon.nimbus.Nimbus
-
- All Implemented Interfaces:
DaemonCommon
,Shutdownable
,Nimbus.Iface
public class Nimbus extends Object implements Nimbus.Iface, Shutdownable, DaemonCommon
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
Nimbus.StandaloneINimbus
-
Field Summary
Fields Modifier and Type Field Description static SimpleVersion
MIN_VERSION_SUPPORT_RPC_HEARTBEAT
static Subject
NIMBUS_SUBJECT
static List<org.apache.storm.shade.org.apache.zookeeper.data.ACL>
ZK_ACLS
-
Constructor Summary
Constructors Constructor Description Nimbus(Map<String,Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, TopoCache topoCache, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry)
Nimbus(Map<String,Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry)
Nimbus(Map<String,Object> conf, INimbus inimbus, StormMetricsRegistry metricsRegistry)
-
Method Summary
-
-
-
Field Detail
-
ZK_ACLS
public static final List<org.apache.storm.shade.org.apache.zookeeper.data.ACL> ZK_ACLS
-
MIN_VERSION_SUPPORT_RPC_HEARTBEAT
public static final SimpleVersion MIN_VERSION_SUPPORT_RPC_HEARTBEAT
-
NIMBUS_SUBJECT
public static final Subject NIMBUS_SUBJECT
-
-
Constructor Detail
-
Nimbus
public Nimbus(Map<String,Object> conf, INimbus inimbus, StormMetricsRegistry metricsRegistry) throws Exception
- Throws:
Exception
-
Nimbus
public Nimbus(Map<String,Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry) throws Exception
- Throws:
Exception
-
Nimbus
public Nimbus(Map<String,Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, TopoCache topoCache, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry) throws Exception
- Throws:
Exception
-
-
Method Detail
-
getNimbusAcls
public static List<org.apache.storm.shade.org.apache.zookeeper.data.ACL> getNimbusAcls(Map<String,Object> conf)
-
getVersionForKey
public static int getVersionForKey(String key, NimbusInfo nimbusInfo, org.apache.storm.shade.org.apache.curator.framework.CuratorFramework zkClient) throws KeyNotFoundException
- Throws:
KeyNotFoundException
-
topoIdsToClean
public static Set<String> topoIdsToClean(IStormClusterState state, BlobStore store, Map<String,Object> conf)
-
cleanInbox
public static void cleanInbox(String dirLoc, int seconds)
Deletes jar files in dirLoc older than seconds.- Parameters:
dirLoc
- the location to look in for fileseconds
- how old is too old and should be deleted
-
topologiesOnSupervisor
public static List<String> topologiesOnSupervisor(Map<String,Assignment> assignments, String supervisorId)
-
setAuthorizationHandler
public void setAuthorizationHandler(IAuthorizer authorizationHandler)
-
getHeartbeatsCache
public HeartbeatCache getHeartbeatsCache()
-
addSupervisor
public void addSupervisor(Supervisor supervisor)
Used for local cluster.- Parameters:
supervisor
-Supervisor
-
checkAuthorization
public void checkAuthorization(String topoName, Map<String,Object> topoConf, String operation) throws AuthorizationException
- Throws:
AuthorizationException
-
checkAuthorization
public void checkAuthorization(String topoName, Map<String,Object> topoConf, String operation, ReqContext context) throws AuthorizationException
- Throws:
AuthorizationException
-
filterAuthorized
public Set<String> filterAuthorized(String operation, Collection<String> topoIds) throws NotAliveException, AuthorizationException, IOException
-
rmDependencyJarsInTopology
public void rmDependencyJarsInTopology(String topoId)
-
rmTopologyKeys
public void rmTopologyKeys(String topoId)
-
forceDeleteTopoDistDir
public void forceDeleteTopoDistDir(String topoId) throws IOException
- Throws:
IOException
-
doCleanup
public void doCleanup()
Cleanup topologies and Jars.
-
awaitLeadership
public boolean awaitLeadership(long timeout, TimeUnit timeUnit) throws InterruptedException
- Throws:
InterruptedException
-
submitTopology
public void submitTopology(String name, String uploadedJarLocation, String jsonConf, StormTopology topology) throws AlreadyAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
submitTopology
in interfaceNimbus.Iface
- Throws:
AlreadyAliveException
InvalidTopologyException
AuthorizationException
org.apache.storm.thrift.TException
-
submitTopologyWithOpts
public void submitTopologyWithOpts(String topoName, String uploadedJarLocation, String jsonConf, StormTopology topology, SubmitOptions options) throws AlreadyAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
submitTopologyWithOpts
in interfaceNimbus.Iface
- Throws:
AlreadyAliveException
InvalidTopologyException
AuthorizationException
org.apache.storm.thrift.TException
-
setUpAckerExecutorConfigs
public static void setUpAckerExecutorConfigs(String topoName, Map<String,Object> totalConfToSave, Map<String,Object> totalConf, int estimatedNumWorker)
-
killTopology
public void killTopology(String name) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
killTopology
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
killTopologyWithOpts
public void killTopologyWithOpts(String topoName, KillOptions options) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
killTopologyWithOpts
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
activate
public void activate(String topoName) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
activate
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
deactivate
public void deactivate(String topoName) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
deactivate
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
rebalance
public void rebalance(String topoName, RebalanceOptions options) throws NotAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
rebalance
in interfaceNimbus.Iface
- Throws:
NotAliveException
InvalidTopologyException
AuthorizationException
org.apache.storm.thrift.TException
-
setLogConfig
public void setLogConfig(String topoId, LogConfig config) throws org.apache.storm.thrift.TException
- Specified by:
setLogConfig
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
getLogConfig
public LogConfig getLogConfig(String topoId) throws org.apache.storm.thrift.TException
- Specified by:
getLogConfig
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
debug
public void debug(String topoName, String componentId, boolean enable, double samplingPercentage) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
Description copied from interface:Nimbus.Iface
Enable/disable logging the tuples generated in topology via an internal EventLogger bolt. The component name is optional and if null or empty, the debug flag will apply to the entire topology. The 'samplingPercentage' will limit loggging to a percentage of generated tuples.- Specified by:
debug
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
setWorkerProfiler
public void setWorkerProfiler(String topoId, ProfileRequest profileRequest) throws org.apache.storm.thrift.TException
- Specified by:
setWorkerProfiler
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
getComponentPendingProfileActions
public List<ProfileRequest> getComponentPendingProfileActions(String id, String componentId, ProfileAction action) throws org.apache.storm.thrift.TException
- Specified by:
getComponentPendingProfileActions
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
uploadNewCredentials
public void uploadNewCredentials(String topoName, Credentials credentials) throws NotAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
uploadNewCredentials
in interfaceNimbus.Iface
- Throws:
NotAliveException
InvalidTopologyException
AuthorizationException
org.apache.storm.thrift.TException
-
beginCreateBlob
public String beginCreateBlob(String key, SettableBlobMeta meta) throws AuthorizationException, KeyAlreadyExistsException, org.apache.storm.thrift.TException
- Specified by:
beginCreateBlob
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyAlreadyExistsException
org.apache.storm.thrift.TException
-
beginUpdateBlob
public String beginUpdateBlob(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException
- Specified by:
beginUpdateBlob
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyNotFoundException
org.apache.storm.thrift.TException
-
uploadBlobChunk
public void uploadBlobChunk(String session, ByteBuffer chunk) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
uploadBlobChunk
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
finishBlobUpload
public void finishBlobUpload(String session) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
finishBlobUpload
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
cancelBlobUpload
public void cancelBlobUpload(String session) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
cancelBlobUpload
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getBlobMeta
public ReadableBlobMeta getBlobMeta(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException
- Specified by:
getBlobMeta
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyNotFoundException
org.apache.storm.thrift.TException
-
setBlobMeta
public void setBlobMeta(String key, SettableBlobMeta meta) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException
- Specified by:
setBlobMeta
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyNotFoundException
org.apache.storm.thrift.TException
-
beginBlobDownload
public BeginDownloadResult beginBlobDownload(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException
- Specified by:
beginBlobDownload
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyNotFoundException
org.apache.storm.thrift.TException
-
downloadBlobChunk
public ByteBuffer downloadBlobChunk(String session) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
downloadBlobChunk
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
deleteBlob
public void deleteBlob(String key) throws AuthorizationException, KeyNotFoundException, IllegalStateException, org.apache.storm.thrift.TException
- Specified by:
deleteBlob
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyNotFoundException
IllegalStateException
org.apache.storm.thrift.TException
-
listBlobs
public ListBlobsResult listBlobs(String session) throws org.apache.storm.thrift.TException
- Specified by:
listBlobs
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
getBlobReplication
public int getBlobReplication(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException
- Specified by:
getBlobReplication
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyNotFoundException
org.apache.storm.thrift.TException
-
updateBlobReplication
public int updateBlobReplication(String key, int replication) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException
- Specified by:
updateBlobReplication
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyNotFoundException
org.apache.storm.thrift.TException
-
createStateInZookeeper
public void createStateInZookeeper(String key) throws org.apache.storm.thrift.TException
- Specified by:
createStateInZookeeper
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
beginFileUpload
public String beginFileUpload() throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
beginFileUpload
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
uploadChunk
public void uploadChunk(String location, ByteBuffer chunk) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
uploadChunk
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
finishFileUpload
public void finishFileUpload(String location) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
finishFileUpload
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
downloadChunk
public ByteBuffer downloadChunk(String id) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
downloadChunk
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getNimbusConf
public String getNimbusConf() throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getNimbusConf
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologyInfo
public TopologyInfo getTopologyInfo(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologyInfo
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologyInfoByName
public TopologyInfo getTopologyInfoByName(String name) throws org.apache.storm.thrift.TException
- Specified by:
getTopologyInfoByName
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
getTopologyInfoByNameWithOpts
public TopologyInfo getTopologyInfoByNameWithOpts(String name, GetInfoOptions options) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologyInfoByNameWithOpts
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologyInfoWithOpts
public TopologyInfo getTopologyInfoWithOpts(String topoId, GetInfoOptions options) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologyInfoWithOpts
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologyPageInfo
public TopologyPageInfo getTopologyPageInfo(String topoId, String window, boolean includeSys) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologyPageInfo
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getSupervisorPageInfo
public SupervisorPageInfo getSupervisorPageInfo(String superId, String host, boolean includeSys) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getSupervisorPageInfo
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getComponentPageInfo
public ComponentPageInfo getComponentPageInfo(String topoId, String componentId, String window, boolean includeSys) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getComponentPageInfo
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologyConf
public String getTopologyConf(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologyConf
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getTopology
public StormTopology getTopology(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
Description copied from interface:Nimbus.Iface
Returns the compiled topology that contains ackers and metrics consumsers. CompareNimbus.Iface.getUserTopology(String id)
.- Specified by:
getTopology
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getUserTopology
public StormTopology getUserTopology(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
Description copied from interface:Nimbus.Iface
Returns the user specified topology as submitted originally. CompareNimbus.Iface.getTopology(String id)
.- Specified by:
getUserTopology
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologyHistory
public TopologyHistoryInfo getTopologyHistory(String user) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologyHistory
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getClusterInfo
public ClusterSummary getClusterInfo() throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getClusterInfo
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologySummaries
public List<TopologySummary> getTopologySummaries() throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologySummaries
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologySummaryByName
public TopologySummary getTopologySummaryByName(String name) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologySummaryByName
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologySummary
public TopologySummary getTopologySummary(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getTopologySummary
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getLeader
public NimbusSummary getLeader() throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getLeader
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
isTopologyNameAllowed
public boolean isTopologyNameAllowed(String name) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
isTopologyNameAllowed
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getOwnerResourceSummaries
public List<OwnerResourceSummary> getOwnerResourceSummaries(String owner) throws AuthorizationException, org.apache.storm.thrift.TException
- Specified by:
getOwnerResourceSummaries
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getSupervisorAssignments
public SupervisorAssignments getSupervisorAssignments(String nodeId) throws AuthorizationException, org.apache.storm.thrift.TException
Description copied from interface:Nimbus.Iface
Get assigned assignments for a specific supervisor- Specified by:
getSupervisorAssignments
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
sendSupervisorWorkerHeartbeats
public void sendSupervisorWorkerHeartbeats(SupervisorWorkerHeartbeats heartbeats) throws AuthorizationException, org.apache.storm.thrift.TException
Description copied from interface:Nimbus.Iface
Send supervisor worker heartbeats for a specific supervisor- Specified by:
sendSupervisorWorkerHeartbeats
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
sendSupervisorWorkerHeartbeat
public void sendSupervisorWorkerHeartbeat(SupervisorWorkerHeartbeat hb) throws AuthorizationException, org.apache.storm.thrift.TException
Description copied from interface:Nimbus.Iface
Send supervisor local worker heartbeat when a supervisor is unreachable- Specified by:
sendSupervisorWorkerHeartbeat
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
shutdown
public void shutdown()
- Specified by:
shutdown
in interfaceShutdownable
-
isWaiting
public boolean isWaiting()
- Specified by:
isWaiting
in interfaceDaemonCommon
-
processWorkerMetrics
public void processWorkerMetrics(WorkerMetrics metrics) throws org.apache.storm.thrift.TException
- Specified by:
processWorkerMetrics
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
isRemoteBlobExists
public boolean isRemoteBlobExists(String blobKey) throws AuthorizationException, org.apache.storm.thrift.TException
Description copied from interface:Nimbus.Iface
Decide if the blob is removed from cluster.- Specified by:
isRemoteBlobExists
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
-