Package org.apache.storm.container.docker

Class DockerManager

java.lang.Object

org.apache.storm.container.oci.OciContainerManager

org.apache.storm.container.docker.DockerManager

All Implemented Interfaces:

ResourceIsolationInterface

public class DockerManager
extends OciContainerManager

For security, we can launch worker processes inside the docker container. This class manages the interaction with docker containers including launching, stopping, profiling and etc.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.storm.container.oci.OciContainerManager

OciContainerManager.CmdType

Field Summary

Fields inherited from class org.apache.storm.container.oci.OciContainerManager

cgroupParent, cgroupRootPath, conf, memoryCgroupRootPath, memoryCoreAtRoot, nscdPath, readonlyBindmounts, readwriteBindmounts, seccompJsonFile, stormHome, TMP_DIR, validatedNumaMap, workerToCores, workerToCpu, workerToMemoryMb, workerToMemoryZone

Constructor Summary

Constructors

Constructor	Description
DockerManager()

Method Summary

Modifier and Type	Method	Description
boolean	areAllProcessesDead(String user, String workerId)	Currently it only checks if the container is alive.
void	cleanup(String user, String workerId, int port)	This function will be called when the worker needs to shutdown.
void	forceKill(String user, String workerId)	Kill the given worker forcefully.
long	getMemoryUsage(String user, String workerId, int port)	Get the current memory usage of the a given worker.
boolean	isResourceManaged()	Return true if resources are being managed.
void	kill(String user, String workerId)	Kill the given worker.
void	launchWorkerProcess(String user, String topologyId, Map<String,Object> topoConf, int port, String workerId, List<String> command, Map<String,String> env, String logPrefix, ExitCodeCallback processExitCallback, File targetDir)	After reserving resources for the worker (i.e.
void	prepare(Map<String,Object> conf)	Called when starting up.
boolean	runProfilingCommand(String user, String workerId, List<String> command, Map<String,String> env, String logPrefix, File targetDir)	Run profiling command in the container.

Methods inherited from class org.apache.storm.container.oci.OciContainerManager

commandFilePath, getImageName, getSystemFreeMemoryMb, reserveResourcesForWorker, writeToCommandFile

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DockerManager

public DockerManager()

Method Detail

prepare

public void prepare(Map<String,Object> conf)
             throws IOException

Description copied from interface: ResourceIsolationInterface

Called when starting up.

Specified by:

prepare in interface ResourceIsolationInterface

Overrides:

prepare in class OciContainerManager

Parameters:

conf - the cluster config

Throws:

IOException - on any error.

launchWorkerProcess

public void launchWorkerProcess(String user,
                                String topologyId,
                                Map<String,Object> topoConf,
                                int port,
                                String workerId,
                                List<String> command,
                                Map<String,String> env,
                                String logPrefix,
                                ExitCodeCallback processExitCallback,
                                File targetDir)
                         throws IOException

Description copied from interface: ResourceIsolationInterface

After reserving resources for the worker (i.e. calling reserveResourcesForWorker), this function can be used to launch worker process.

Parameters:

user - the user who runs the command as

topologyId - the Id of the topology

topoConf - the topology configuration

port - the port where the worker is on

workerId - the Id of the worker

command - the command to run

env - the environment to run the command

logPrefix - the prefix to include in the logs

processExitCallback - a callback for when the process exits

targetDir - the working directory to run the command in

Throws:

IOException - on I/O exception

getMemoryUsage

public long getMemoryUsage(String user,
                           String workerId,
                           int port)
                    throws IOException

Description copied from interface: ResourceIsolationInterface

Get the current memory usage of the a given worker.

Parameters:

user - the user that the worker is running as

workerId - the id of the worker

port - the port of the worker

Returns:

the amount of memory the worker is using in bytes or -1 if not supported

Throws:

IOException - on I/O exception

kill

public void kill(String user,
                 String workerId)
          throws IOException

Description copied from interface: ResourceIsolationInterface

Kill the given worker.

Parameters:

user - the user that the worker is running as

workerId - the id of the worker to kill

Throws:

IOException - on I/O exception

forceKill

public void forceKill(String user,
                      String workerId)
               throws IOException

Description copied from interface: ResourceIsolationInterface

Kill the given worker forcefully.

Parameters:

user - the user that the worker is running as

workerId - the id of the worker to kill

Throws:

IOException - on I/O exception

areAllProcessesDead

public boolean areAllProcessesDead(String user,
                                   String workerId)
                            throws IOException

Currently it only checks if the container is alive. If the worker process inside the container dies, the container will exit. So we only need to check if the container is running to know if the worker process is still alive.

Parameters:

user - the user of the processes

workerId - the id of the worker to kill

Returns:

true if all processes are dead

Throws:

IOException - on I/O exception

runProfilingCommand

public boolean runProfilingCommand(String user,
                                   String workerId,
                                   List<String> command,
                                   Map<String,String> env,
                                   String logPrefix,
                                   File targetDir)
                            throws IOException,
                                   InterruptedException

Run profiling command in the container.

Parameters:

user - the user that the worker is running as

workerId - the id of the worker

command - the command to run. The profiler to be used is configured in worker-launcher.cfg.

env - the environment to run the command

logPrefix - the prefix to include in the logs

targetDir - the working directory to run the command in

Returns:

true if the command succeeds, false otherwise.

Throws:

IOException - on I/O exception

InterruptedException - if interrupted

cleanup

public void cleanup(String user,
                    String workerId,
                    int port)
             throws IOException

Description copied from interface: ResourceIsolationInterface

This function will be called when the worker needs to shutdown. This function should include logic to clean up after a worker is shutdown.

Specified by:

cleanup in interface ResourceIsolationInterface

Overrides:

cleanup in class OciContainerManager

Parameters:

user - the user of the worker

workerId - worker id to shutdown and clean up after

port - the port of the worker

Throws:

IOException

isResourceManaged

public boolean isResourceManaged()

Description copied from interface: ResourceIsolationInterface

Return true if resources are being managed. The DefaultResourceIsolationManager will have it return false since it doesn't really manage resources.

Returns:

true if resources are being managed.