org.apache.storm.container.cgroup

Class CgroupManager

java.lang.Object

org.apache.storm.container.DefaultResourceIsolationManager

org.apache.storm.container.cgroup.CgroupManager

All Implemented Interfaces:

ResourceIsolationInterface

public class CgroupManager
extends DefaultResourceIsolationManager

Class that implements ResourceIsolationInterface that manages cgroups.

Field Summary

Fields inherited from class org.apache.storm.container.DefaultResourceIsolationManager

conf, runAsUser

Constructor Summary

Constructors

Constructor and Description
CgroupManager()

Method Summary

Modifier and Type	Method and Description
void	cleanup(String user, String workerId, int port) This function will be called when the worker needs to shutdown.
protected Set<Long>	getAllPids(String workerId) Get all of the pids that are a part of this container.
List<String>	getLaunchCommand(String workerId, List<String> existingCommand) To compose launch command based on workerId and existing command.
long	getMemoryUsage(String user, String workerId, int port) Get the current memory usage of the a given worker.
long	getSystemFreeMemoryMb() Get the amount of free memory in MB.
protected static boolean	isOnLinux() Extracting out to mock it for tests.
boolean	isResourceManaged() This class doesn’t really manage resources.
void	launchWorkerProcess(String user, String topologyId, Map<String,Object> topoConf, int port, String workerId, List<String> command, Map<String,String> env, String logPrefix, ExitCodeCallback processExitCallback, File targetDir) After reserving resources for the worker (i.e.
void	prepare(Map<String,Object> conf) initialize data structures.
void	reserveResourcesForWorker(String workerId, Integer totalMem, Integer cpuNum, String numaId) This function should be used prior to starting the worker to reserve resources for the worker.

Methods inherited from class org.apache.storm.container.DefaultResourceIsolationManager

areAllProcessesDead, forceKill, kill, runProfilingCommand

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CgroupManager

public CgroupManager()

Method Detail

prepare

public void prepare(Map<String,Object> conf)
             throws IOException

initialize data structures.

Specified by:

prepare in interface ResourceIsolationInterface

Overrides:

prepare in class DefaultResourceIsolationManager

Parameters:

conf - storm confs

Throws:

IOException - on any error.

reserveResourcesForWorker

public void reserveResourcesForWorker(String workerId,
                                      Integer totalMem,
                                      Integer cpuNum,
                                      String numaId)
                               throws SecurityException

Description copied from interface: ResourceIsolationInterface

This function should be used prior to starting the worker to reserve resources for the worker.

Specified by:

reserveResourcesForWorker in interface ResourceIsolationInterface

Overrides:

reserveResourcesForWorker in class DefaultResourceIsolationManager

Parameters:

workerId - worker id of the worker to start

totalMem - the amount of memory for the worker or null if not enforced

cpuNum - the amount of cpu for the worker or null if not enforced

numaId - NUMA zone if applicable the worker should be bound to

Throws:

SecurityException

cleanup

public void cleanup(String user,
                    String workerId,
                    int port)
             throws IOException

Description copied from interface: ResourceIsolationInterface

This function will be called when the worker needs to shutdown. This function should include logic to clean up after a worker is shutdown.

Specified by:

cleanup in interface ResourceIsolationInterface

Overrides:

cleanup in class DefaultResourceIsolationManager

Parameters:

user - the user of the worker

workerId - worker id to shutdown and clean up after

port - the port of the worker

Throws:

IOException

isOnLinux

protected static boolean isOnLinux()

Extracting out to mock it for tests.

Returns:

true if on Linux.

launchWorkerProcess

public void launchWorkerProcess(String user,
                                String topologyId,
                                Map<String,Object> topoConf,
                                int port,
                                String workerId,
                                List<String> command,
                                Map<String,String> env,
                                String logPrefix,
                                ExitCodeCallback processExitCallback,
                                File targetDir)
                         throws IOException

Description copied from interface: ResourceIsolationInterface

After reserving resources for the worker (i.e. calling reserveResourcesForWorker), this function can be used to launch worker process.

Specified by:

launchWorkerProcess in interface ResourceIsolationInterface

Overrides:

launchWorkerProcess in class DefaultResourceIsolationManager

Parameters:

user - the user who runs the command as

topologyId - the Id of the topology

topoConf - the topology configuration

port - the port where the worker is on

workerId - the Id of the worker

command - the command to run

env - the environment to run the command

logPrefix - the prefix to include in the logs

processExitCallback - a callback for when the process exits

targetDir - the working directory to run the command in

Throws:

IOException - on I/O exception

getLaunchCommand

public List<String> getLaunchCommand(String workerId,
                                     List<String> existingCommand)

To compose launch command based on workerId and existing command.

Parameters:

workerId - the worker id

existingCommand - the current command to run that may need to be modified

Returns:

new commandline with necessary additions to launch worker

getAllPids

protected Set<Long> getAllPids(String workerId)
                        throws IOException

Get all of the pids that are a part of this container.

Overrides:

getAllPids in class DefaultResourceIsolationManager

Parameters:

workerId - the worker id

Returns:

all of the pids that are a part of this container

Throws:

IOException

getMemoryUsage

public long getMemoryUsage(String user,
                           String workerId,
                           int port)
                    throws IOException

Description copied from interface: ResourceIsolationInterface

Get the current memory usage of the a given worker.

Specified by:

getMemoryUsage in interface ResourceIsolationInterface

Overrides:

getMemoryUsage in class DefaultResourceIsolationManager

Parameters:

user - the user that the worker is running as

workerId - the id of the worker

port - the port of the worker

Returns:

the amount of memory the worker is using in bytes or -1 if not supported

Throws:

IOException - on I/O exception

getSystemFreeMemoryMb

public long getSystemFreeMemoryMb()
                           throws IOException

Description copied from interface: ResourceIsolationInterface

Get the amount of free memory in MB. This might not be the entire box, it might be within a parent resource group.

Specified by:

getSystemFreeMemoryMb in interface ResourceIsolationInterface

Overrides:

getSystemFreeMemoryMb in class DefaultResourceIsolationManager

Returns:

The amount of memory in MB that are free on the system.

Throws:

IOException - on I/O exception

isResourceManaged

public boolean isResourceManaged()

Description copied from class: DefaultResourceIsolationManager

This class doesn’t really manage resources.

Specified by:

isResourceManaged in interface ResourceIsolationInterface

Overrides:

isResourceManaged in class DefaultResourceIsolationManager

Returns:

false