All Implemented Interfaces:

public class NodeSorter extends Object implements INodeSorter
  • Field Details

  • Constructor Details

  • Method Details

    • prepare

      public void prepare(ExecutorDetails exec)
      Description copied from interface: INodeSorter
      Prepare for node sorting. This method must be called before INodeSorter.getSortedRacks() and INodeSorter.sortAllNodes().
      Specified by:
      prepare in interface INodeSorter
      exec - optional, may be null.
    • sortObjectResources

      protected List<ObjectResourcesItem> sortObjectResources(ObjectResourcesSummary resourcesSummary, ExecutorDetails exec, NodeSorter.ExistingScheduleFunc existingScheduleFunc)
      Scheduling uses sortAllNodes() which eventually calls this method whose behavior can be altered by setting nodeSortType.
      resourcesSummary - contains all individual ObjectResourcesItem as well as cumulative stats
      exec - executor for which the sorting is done
      existingScheduleFunc - a function to get existing executors already scheduled on this object
      a sorted list of ObjectResourcesItem
    • makeHostToNodeIds

      protected List<String> makeHostToNodeIds(List<String> hosts)
    • sortAllNodes

      public Iterable<String> sortAllNodes()
      Specified by:
      sortAllNodes in interface INodeSorter
    • getSortedRacks

      public List<ObjectResourcesItem> getSortedRacks()
      Racks are sorted by two criteria.

      1) the number executors of the topology that needs to be scheduled is already on the rack in descending order. The reasoning to sort based on criterion 1 is so we schedule the rest of a topology on the same rack as the existing executors of the topology.

      2) the subordinate/subservient resource availability percentage of a rack in descending order We calculate the resource availability percentage by dividing the resource availability on the rack by the resource availability of the entire cluster By doing this calculation, racks that have exhausted or little of one of the resources mentioned above will be ranked after racks that have more balanced resource availability. So we will be less likely to pick a rack that have a lot of one resource but a low amount of another.

      Specified by:
      getSortedRacks in interface INodeSorter
      a sorted list of racks
    • hostnameToNodes

      public List<RasNode> hostnameToNodes(String hostname)
      hostname to Ids.
      hostname - the hostname.
      the ids n that node.