Package org.apache.storm.scheduler

Interface ISchedulingState

  • All Known Implementing Classes:
    Cluster, SingleTopologyCluster
    public interface ISchedulingState
    An interface that provides access to the current scheduling state. The scheduling state is not guaranteed to be thread safe.

    • Method Detail

      • getTopologies

        Topologies getTopologies()
        Get all of the topologies.
        Returns:
        all of the topologies that are a part of the cluster.

      • needsSchedulingTopologies

        List<TopologyDetails> needsSchedulingTopologies()
        Get all of the topologies that need scheduling.
        Returns:
        all of the topologies that are not fully scheduled.

      • needsScheduling

        boolean needsScheduling​(TopologyDetails topology)
        Does the topology need scheduling.

        A topology needs scheduling if one of the following conditions holds:

        • Although the topology is assigned slots, but is squeezed. i.e. the topology is assigned less slots than desired.
        • There are unassigned executors in this topology

      • needsSchedulingRas

        boolean needsSchedulingRas​(TopologyDetails topology)
        Like needsScheduling(TopologyDetails) but does not take into account the number of workers requested. This is because the number of workers is ignored in RAS
        Parameters:
        topology - the topology to check
        Returns:
        true if the topology needs scheduling else false.

      • getBlacklistedHosts

        Set<String> getBlacklistedHosts()
        Get all of the hosts that are blacklisted.
        Returns:
        all of the hosts that are blacklisted

      • isBlackListed

        boolean isBlackListed​(String supervisorId)
        Check is a given supervisor is on a blacklisted host.
        Parameters:
        supervisorId - the id of the supervisor
        Returns:
        true if it is else false

      • isBlacklistedHost

        boolean isBlacklistedHost​(String host)
        Check if a given host is blacklisted.
        Parameters:
        host - the name of the host
        Returns:
        true if it is else false.

      • getHost

        String getHost​(String supervisorId)
        Map a supervisor to a given host.
        Parameters:
        supervisorId - the id of the supervisor
        Returns:
        the actual host name the supervisor is on

      • getUnassignedExecutors

        Collection<ExecutorDetails> getUnassignedExecutors​(TopologyDetails topology)
        get the unassigned executors of the topology.
        Parameters:
        topology - the topology to check
        Returns:
        the unassigned executors of the topology.

      • getNeedsSchedulingExecutorToComponents

        Map<ExecutorDetails,​String> getNeedsSchedulingExecutorToComponents​(TopologyDetails topology)
        Get the executor to component name map for executors that need to be scheduled.
        Parameters:
        topology - the topology this is for
        Returns:
        a executor -> component-id map which needs scheduling in this topology.

      • getNeedsSchedulingComponentToExecutors

        Map<String,​List<ExecutorDetails>> getNeedsSchedulingComponentToExecutors​(TopologyDetails topology)
        Get the component name to executor list for executors that need to be scheduled.
        Parameters:
        topology - the topology this is for
        Returns:
        a component-id -> executors map which needs scheduling in this topology.

      • getAssignablePorts

        Set<Integer> getAssignablePorts​(SupervisorDetails supervisor)
        Get the ports that are not blacklisted.
        Parameters:
        supervisor - the supervisor
        Returns:
        the ports that are not blacklisted

      • getAvailableSlots

        List<WorkerSlot> getAvailableSlots()
        Get all the available worker slots in the cluster.

      • getNonBlacklistedAvailableSlots

        List<WorkerSlot> getNonBlacklistedAvailableSlots​(List<String> blacklistedSupervisorIds)
        Get all the available worker slots in the cluster, that are not blacklisted.
        Parameters:
        blacklistedSupervisorIds - list of supervisor ids that should also be considered blacklisted.

      • getAssignableSlots

        List<WorkerSlot> getAssignableSlots​(SupervisorDetails supervisor)
        Return all non-blacklisted slots on this supervisor.
        Parameters:
        supervisor - the supervisor
        Returns:
        the non-blacklisted slots

      • getAssignableSlots

        List<WorkerSlot> getAssignableSlots()
        Get all non-blacklisted slots in the cluster.

      • isSlotOccupied

        boolean isSlotOccupied​(WorkerSlot slot)
        Check if a slot is occupied or not.
        Parameters:
        slot - the slot be to checked.
        Returns:
        true if the specified slot is occupied.

      • getAssignedNumWorkers

        int getAssignedNumWorkers​(TopologyDetails topology)
        Get the number of workers assigned to a topology.
        Parameters:
        topology - the topology this is for
        Returns:
        the number of workers assigned to this topology.

      • getAvailableResources

        NormalizedResourceOffer getAvailableResources​(SupervisorDetails sd)
        Get the resources on the supervisor that are available to be scheduled.
        Parameters:
        sd - the supervisor.
        Returns:
        the resources available to be scheduled.

      • wouldFit

        boolean wouldFit​(WorkerSlot ws,
                 ExecutorDetails exec,
                 TopologyDetails td,
                 NormalizedResourceOffer resourcesAvailable,
                 double maxHeap)
        Would scheduling exec on ws fit? With a heap <= maxHeap total memory added <= memoryAvailable and cpu added <= cpuAvailable.
        Parameters:
        ws - the slot to put it in
        exec - the executor to investigate
        td - the topology detains for this executor
        resourcesAvailable - all the available resources
        maxHeap - the maximum heap size for ws
        Returns:
        true it fits else false

      • getAssignmentById

        SchedulerAssignment getAssignmentById​(String topologyId)
        get the current assignment for the topology.

      • getSupervisorById

        SupervisorDetails getSupervisorById​(String nodeId)
        Get a specific supervisor with the nodeId.

      • getSupervisorsByHost

        List<SupervisorDetails> getSupervisorsByHost​(String host)
        Get all the supervisors on the specified host.
        Parameters:
        host - hostname of the supervisor
        Returns:
        the SupervisorDetails object.

      • getNonBlacklistedClusterAvailableResources

        NormalizedResourceOffer getNonBlacklistedClusterAvailableResources​(Collection<String> blacklistedSupervisorIds)
        Get the resources in the cluster that are available for scheduling.
        Parameters:
        blacklistedSupervisorIds - other ids that are tentatively blacklisted.

      • getClusterTotalCpuResource

        double getClusterTotalCpuResource()
        Get the total amount of CPU resources in cluster.

      • getClusterTotalMemoryResource

        double getClusterTotalMemoryResource()
        Get the total amount of memory resources in cluster.

      • getClusterTotalGenericResources

        Map<String,​Double> getClusterTotalGenericResources()
        Get the total amount of generic resources (excluding CPU and memory) in cluster.

      • getNetworkTopography

        Map<String,​List<String>> getNetworkTopography()
        Get the network topography (rackId -> nodes in the rack).

      • getHostToRack

        default Map<String,​String> getHostToRack()
        Get host -> rack map - the inverse of networkTopography.

      • getStatusMap

        Map<String,​String> getStatusMap()
        Get all topology scheduler statuses.

      • getTopologyResourcesMap

        Map<String,​TopologyResources> getTopologyResourcesMap()
        Get the amount of resources used by topologies. Used for displaying resource information on the UI.
        Returns:
        a map that contains multiple topologies and the resources the topology requested and assigned. Key: topology id Value: an array that describes the resources the topology requested and assigned in the following format: {requestedMemOnHeap, requestedMemOffHeap, requestedCpu, assignedMemOnHeap, assignedMemOffHeap, assignedCpu}

      • getSupervisorsResourcesMap

        Map<String,​SupervisorResources> getSupervisorsResourcesMap()
        Get the amount of used and free resources on a supervisor. Used for displaying resource information on the UI
        Returns:
        a map where the key is the supervisor id and the value is a map that represents resource usage for a supervisor in the following format: {totalMem, totalCpu, usedMem, usedCpu}

      • getWorkerResourcesMap

        Map<String,​Map<WorkerSlot,​WorkerResources>> getWorkerResourcesMap()
        Gets the reference to the full topology->worker resource map.
        Returns:
        map of topology -> map of worker slot ->resources for that worker

      • getWorkerResources

        WorkerResources getWorkerResources​(WorkerSlot ws)
        Get the resources for a given slot.
        Parameters:
        ws - the slot
        Returns:
        the resources currently assigned

      • getScheduledMemoryForNode

        double getScheduledMemoryForNode​(String nodeId)
        Get the total memory currently scheduled on a node.
        Parameters:
        nodeId - the id of the node
        Returns:
        the total memory currently scheduled on the node

      • getScheduledCpuForNode

        double getScheduledCpuForNode​(String nodeId)
        Get the total cpu currently scheduled on a node.
        Parameters:
        nodeId - the id of the node
        Returns:
        the total cpu currently scheduled on the node

      • getAssignedRacks

        default Set<String> getAssignedRacks​(String... topologyIds)
        Determine the list of racks on which topologyIds have been assigned. Note that the returned set may contain DNSToSwitchMapping.DEFAULT_RACK if getHostToRack() is null or does not contain the assigned host.
        Parameters:
        topologyIds - for which assignments are examined.
        Returns:
        set of racks on which assignments have been made.