org.apache.storm.hdfs.blobstore

Class HdfsClientBlobStore

java.lang.Object

org.apache.storm.blobstore.ClientBlobStore

org.apache.storm.hdfs.blobstore.HdfsClientBlobStore

All Implemented Interfaces:

Shutdownable

public class HdfsClientBlobStore
extends ClientBlobStore

Client to access the HDFS blobStore. At this point, this is meant to only be used by the supervisor. Don’t trust who the client says they are so pass null for all Subjects.

The HdfsBlobStore implementation takes care of the null Subjects. It assigns Subjects based on what hadoop says who the users are. These users must be configured accordingly in the SUPERVISOR_ADMINS for ACL validation and for the supervisors to download the blobs. This API is only used by the supervisor in order to talk directly to HDFS.

Field Summary

Fields inherited from class org.apache.storm.blobstore.ClientBlobStore

conf

Constructor Summary

Constructors

Constructor and Description
HdfsClientBlobStore()

Method Summary

Modifier and Type	Method and Description
AtomicOutputStream	createBlobToExtend(String key, SettableBlobMeta meta) Client facing API to create a blob.
void	createStateInZookeeper(String key) Creates state inside a zookeeper.
void	deleteBlob(String key) Client facing API to delete a blob.
InputStreamWithMeta	getBlob(String key) Client facing API to read a blob.
ReadableBlobMeta	getBlobMeta(String key) Client facing API to read the metadata information.
int	getBlobReplication(String key) Client facing API to read the replication of a blob.
Iterator<String>	listKeys()
void	prepare(Map conf) Sets up the client API by parsing the configs.
void	setBlobMetaToExtend(String key, SettableBlobMeta meta) Client facing API to set the metadata for a blob.
boolean	setClient(Map conf, NimbusClient client) Client facing API to set a nimbus client.
void	shutdown()
AtomicOutputStream	updateBlob(String key) Client facing API to update a blob.
int	updateBlobReplication(String key, int replication) Client facing API to update the replication of a blob.

Methods inherited from class org.apache.storm.blobstore.ClientBlobStore

createBlob, setBlobMeta

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HdfsClientBlobStore

public HdfsClientBlobStore()

Method Detail

prepare

public void prepare(Map conf)

Description copied from class: ClientBlobStore

Sets up the client API by parsing the configs.

Specified by:

prepare in class ClientBlobStore

Parameters:

conf - The storm conf containing the config details.

createBlobToExtend

public AtomicOutputStream createBlobToExtend(String key,
                                             SettableBlobMeta meta)
                                      throws AuthorizationException,
                                             KeyAlreadyExistsException

Description copied from class: ClientBlobStore

Client facing API to create a blob.

Specified by:

createBlobToExtend in class ClientBlobStore

Parameters:

key - blob key name.

meta - contains ACL information.

Returns:

AtomicOutputStream returns an output stream into which data can be written.

Throws:

AuthorizationException

KeyAlreadyExistsException

updateBlob

public AtomicOutputStream updateBlob(String key)
                              throws AuthorizationException,
                                     KeyNotFoundException

Description copied from class: ClientBlobStore

Client facing API to update a blob.

Specified by:

updateBlob in class ClientBlobStore

Parameters:

key - blob key name.

Returns:

AtomicOutputStream returns an output stream into which data can be written.

Throws:

AuthorizationException

KeyNotFoundException

getBlobMeta

public ReadableBlobMeta getBlobMeta(String key)
                             throws AuthorizationException,
                                    KeyNotFoundException

Description copied from class: ClientBlobStore

Client facing API to read the metadata information.

Specified by:

getBlobMeta in class ClientBlobStore

Parameters:

key - blob key name.

Returns:

AtomicOutputStream returns an output stream into which data can be written.

Throws:

AuthorizationException

KeyNotFoundException

setBlobMetaToExtend

public void setBlobMetaToExtend(String key,
                                SettableBlobMeta meta)
                         throws AuthorizationException,
                                KeyNotFoundException

Description copied from class: ClientBlobStore

Client facing API to set the metadata for a blob.

Specified by:

setBlobMetaToExtend in class ClientBlobStore

Parameters:

key - blob key name.

meta - contains ACL information.

Throws:

AuthorizationException

KeyNotFoundException

deleteBlob

public void deleteBlob(String key)
                throws AuthorizationException,
                       KeyNotFoundException

Description copied from class: ClientBlobStore

Client facing API to delete a blob.

Specified by:

deleteBlob in class ClientBlobStore

Parameters:

key - blob key name.

Throws:

AuthorizationException

KeyNotFoundException

getBlob

public InputStreamWithMeta getBlob(String key)
                            throws AuthorizationException,
                                   KeyNotFoundException

Description copied from class: ClientBlobStore

Client facing API to read a blob.

Specified by:

getBlob in class ClientBlobStore

Parameters:

key - blob key name.

Returns:

an InputStream to read the metadata for a blob.

Throws:

AuthorizationException

KeyNotFoundException

listKeys

public Iterator<String> listKeys()

Specified by:

listKeys in class ClientBlobStore

Returns:

Iterator for a list of keys currently present in the blob store.

getBlobReplication

public int getBlobReplication(String key)
                       throws AuthorizationException,
                              KeyNotFoundException

Description copied from class: ClientBlobStore

Client facing API to read the replication of a blob.

Specified by:

getBlobReplication in class ClientBlobStore

Parameters:

key - blob key name.

Returns:

int indicates the replication factor of a blob.

Throws:

AuthorizationException

KeyNotFoundException

updateBlobReplication

public int updateBlobReplication(String key,
                                 int replication)
                          throws AuthorizationException,
                                 KeyNotFoundException

Description copied from class: ClientBlobStore

Client facing API to update the replication of a blob.

Specified by:

updateBlobReplication in class ClientBlobStore

Parameters:

key - blob key name.

replication - int indicates the replication factor a blob has to be set.

Returns:

int indicates the replication factor of a blob.

Throws:

AuthorizationException

KeyNotFoundException

setClient

public boolean setClient(Map conf,
                         NimbusClient client)

Description copied from class: ClientBlobStore

Client facing API to set a nimbus client.

Specified by:

setClient in class ClientBlobStore

Parameters:

conf - storm conf

client - NimbusClient

Returns:

indicates where the client connection has been setup.

createStateInZookeeper

public void createStateInZookeeper(String key)

Description copied from class: ClientBlobStore

Creates state inside a zookeeper. Required for blobstore to write to zookeeper when Nimbus HA is turned on in order to maintain state consistency

Specified by:

createStateInZookeeper in class ClientBlobStore

shutdown

public void shutdown()