org.apache.storm.container

Interface ResourceIsolationInterface

All Known Implementing Classes:

CgroupManager

public interface ResourceIsolationInterface

A plugin to support resource isolation and limitation within Storm.

Method Summary

Modifier and Type	Method and Description
List<String>	getLaunchCommand(String workerId, List<String> existingCommand) After reserving resources for the worker (i.e.
List<String>	getLaunchCommandPrefix(String workerId) After reserving resources for the worker (i.e.
long	getMemoryUsage(String workerId) Get the current memory usage of the a given worker.
Set<Long>	getRunningPids(String workerId) Get the list of PIDs currently in an isolated container.
long	getSystemFreeMemoryMb() Get the system free memory in MB.
void	prepare(Map<String,Object> conf) Called when starting up.
void	releaseResourcesForWorker(String workerId) This function will be called when the worker needs to shutdown.
void	reserveResourcesForWorker(String workerId, Integer workerMemory, Integer workerCpu, String numaId) This function should be used prior to starting the worker to reserve resources for the worker.

Method Detail

prepare

void prepare(Map<String,Object> conf)
      throws IOException

Called when starting up.

Parameters:

conf - the cluster config

Throws:

IOException - on any error.

reserveResourcesForWorker

void reserveResourcesForWorker(String workerId,
                               Integer workerMemory,
                               Integer workerCpu,
                               String numaId)

This function should be used prior to starting the worker to reserve resources for the worker.

Parameters:

workerId - worker id of the worker to start

workerMemory - the amount of memory for the worker or null if not enforced

workerCpu - the amount of cpu for the worker or null if not enforced

numaId - NUMA zone if applicable the worker should be bound to

releaseResourcesForWorker

void releaseResourcesForWorker(String workerId)

This function will be called when the worker needs to shutdown. This function should include logic to clean up after a worker is shutdown.

Parameters:

workerId - worker id to shutdown and clean up after

getLaunchCommand

List<String> getLaunchCommand(String workerId,
                              List<String> existingCommand)

After reserving resources for the worker (i.e. calling reserveResourcesForWorker). This function can be used to get the modified command line to launch the worker with resource isolation

Parameters:

existingCommand - the current command to run that may need to be modified.

Returns:

new commandline with necessary additions to launch worker with resource isolation

getLaunchCommandPrefix

List<String> getLaunchCommandPrefix(String workerId)

After reserving resources for the worker (i.e. calling reserveResourcesForWorker). this function can be used to get the launch command prefix

Parameters:

workerId - the of the worker

Returns:

the command line prefix for launching a worker with resource isolation

getRunningPids

Set<Long> getRunningPids(String workerId)
                  throws IOException

Get the list of PIDs currently in an isolated container.

Parameters:

workerId - the id of the worker to get these for

Returns:

the set of PIDs, this will be combined with other ways of getting PIDs. An Empty set if no PIDs are found.

Throws:

IOException - on any error

getMemoryUsage

long getMemoryUsage(String workerId)
             throws IOException

Get the current memory usage of the a given worker.

Parameters:

workerId - the id of the worker

Returns:

the amount of memory the worker is using in bytes or -1 if not supported

Throws:

IOException - on any error.

getSystemFreeMemoryMb

long getSystemFreeMemoryMb()
                    throws IOException

Get the system free memory in MB.

Returns:

The amount of memory in bytes that are free on the system. This might not be the entire box, it might be within a parent resource group.

Throws:

IOException - on any error.