org.apache.storm.blobstore

Class LocalFsBlobStore

java.lang.Object

org.apache.storm.blobstore.BlobStore

org.apache.storm.blobstore.LocalFsBlobStore

All Implemented Interfaces:

Shutdownable

public class LocalFsBlobStore
extends BlobStore

Provides a local file system backed blob store implementation for Nimbus.

For a local blob store the user and the supervisor use NimbusBlobStore Client API in order to talk to nimbus through thrift. The authentication and authorization here is based on the subject. We currently have NIMBUS_ADMINS and SUPERVISOR_ADMINS configuration. NIMBUS_ADMINS are given READ, WRITE and ADMIN access whereas the SUPERVISOR_ADMINS are given READ access in order to read and download the blobs form the nimbus.

The ACLs for the blob store are validated against whether the subject is a NIMBUS_ADMIN, SUPERVISOR_ADMIN or USER who has read, write or admin privileges in order to perform respective operations on the blob.

For local blob store 1. The USER interacts with nimbus to upload and access blobs through NimbusBlobStore Client API. 2. The USER sets the ACLs, and the blob access is validated against these ACLs. 3. The SUPERVISOR interacts with nimbus through the NimbusBlobStore Client API to download the blobs. The supervisors principal should match the set of users configured into SUPERVISOR_ADMINS. Here, the PrincipalToLocalPlugin takes care of mapping the principal to user name before the ACL validation.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.storm.blobstore.BlobStore

BlobStore.BlobStoreFileInputStream, BlobStore.BlobStoreFileOutputStream, BlobStore.KeyTranslationIterator

Field Summary

Fields

Modifier and Type	Field and Description
protected BlobStoreAclHandler	_aclHandler
static org.slf4j.Logger	LOG

Fields inherited from class org.apache.storm.blobstore.BlobStore

BASE_BLOBS_DIR_NAME

Constructor Summary

Constructors

Constructor and Description
LocalFsBlobStore()

Method Summary

Modifier and Type	Method and Description
boolean	checkForBlobOrDownload(String key)
void	checkForBlobUpdate(String key)
AtomicOutputStream	createBlob(String key, SettableBlobMeta meta, Subject who) Creates the blob.
void	deleteBlob(String key, Subject who) Deletes the blob data and metadata.
void	fullCleanup(long age)
InputStreamWithMeta	getBlob(String key, Subject who) Gets the InputStream to read the blob details
ReadableBlobMeta	getBlobMeta(String key, Subject who) Gets the current version of metadata for a blob to be viewed by the user or downloaded by the supervisor.
int	getBlobReplication(String key, Subject who) Gets the replication factor of the blob.
Iterator<String>	listKeys() Returns an iterator with all the list of keys currently available on the blob store.
void	prepare(Map conf, String overrideBase, NimbusInfo nimbusInfo) Allows us to initialize the blob store
void	setBlobMeta(String key, SettableBlobMeta meta, Subject who) Sets the metadata with renewed acls for the blob.
void	shutdown()
AtomicOutputStream	updateBlob(String key, Subject who) Updates the blob data.
int	updateBlobReplication(String key, int replication, Subject who) Modifies the replication factor of the blob.

Methods inherited from class org.apache.storm.blobstore.BlobStore

createBlob, createBlob, filterAndListKeys, readBlob, readBlobTo, validateKey

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

_aclHandler

protected BlobStoreAclHandler _aclHandler

Constructor Detail

LocalFsBlobStore

public LocalFsBlobStore()

Method Detail

prepare

public void prepare(Map conf,
                    String overrideBase,
                    NimbusInfo nimbusInfo)

Description copied from class: BlobStore

Allows us to initialize the blob store

Specified by:

prepare in class BlobStore

Parameters:

conf - The storm configuration

overrideBase - The directory path to store the blobs

nimbusInfo - Contains the nimbus host, port and leadership information.

createBlob

public AtomicOutputStream createBlob(String key,
                                     SettableBlobMeta meta,
                                     Subject who)
                              throws AuthorizationException,
                                     KeyAlreadyExistsException

Description copied from class: BlobStore

Creates the blob.

Specified by:

createBlob in class BlobStore

Parameters:

key - Key for the blob.

meta - Metadata which contains the acls information

who - Is the subject creating the blob.

Returns:

AtomicOutputStream returns a stream into which the data can be written.

Throws:

AuthorizationException

KeyAlreadyExistsException

updateBlob

public AtomicOutputStream updateBlob(String key,
                                     Subject who)
                              throws AuthorizationException,
                                     KeyNotFoundException

Description copied from class: BlobStore

Updates the blob data.

Specified by:

updateBlob in class BlobStore

Parameters:

key - Key for the blob.

who - Is the subject having the write privilege for the blob.

Returns:

AtomicOutputStream returns a stream into which the data can be written.

Throws:

AuthorizationException

KeyNotFoundException

getBlobMeta

public ReadableBlobMeta getBlobMeta(String key,
                                    Subject who)
                             throws AuthorizationException,
                                    KeyNotFoundException

Description copied from class: BlobStore

Gets the current version of metadata for a blob to be viewed by the user or downloaded by the supervisor.

Specified by:

getBlobMeta in class BlobStore

Parameters:

key - Key for the blob.

who - Is the subject having the read privilege for the blob.

Returns:

AtomicOutputStream returns a stream into which the data can be written.

Throws:

AuthorizationException

KeyNotFoundException

setBlobMeta

public void setBlobMeta(String key,
                        SettableBlobMeta meta,
                        Subject who)
                 throws AuthorizationException,
                        KeyNotFoundException

Description copied from class: BlobStore

Sets the metadata with renewed acls for the blob.

Specified by:

setBlobMeta in class BlobStore

Parameters:

key - Key for the blob.

meta - Metadata which contains the updated acls information.

who - Is the subject having the write privilege for the blob.

Throws:

AuthorizationException

KeyNotFoundException

deleteBlob

public void deleteBlob(String key,
                       Subject who)
                throws AuthorizationException,
                       KeyNotFoundException

Description copied from class: BlobStore

Deletes the blob data and metadata.

Specified by:

deleteBlob in class BlobStore

Parameters:

key - Key for the blob.

who - Is the subject having write privilege for the blob.

Throws:

AuthorizationException

KeyNotFoundException

getBlob

public InputStreamWithMeta getBlob(String key,
                                   Subject who)
                            throws AuthorizationException,
                                   KeyNotFoundException

Description copied from class: BlobStore

Gets the InputStream to read the blob details

Specified by:

getBlob in class BlobStore

Parameters:

key - Key for the blob.

who - Is the subject having the read privilege for the blob.

Returns:

InputStreamWithMeta has the additional file length and version information.

Throws:

AuthorizationException

KeyNotFoundException

listKeys

public Iterator<String> listKeys()

Description copied from class: BlobStore

Returns an iterator with all the list of keys currently available on the blob store.

Specified by:

listKeys in class BlobStore

Returns:

Iterator

shutdown

public void shutdown()

getBlobReplication

public int getBlobReplication(String key,
                              Subject who)
                       throws Exception

Description copied from class: BlobStore

Gets the replication factor of the blob.

Specified by:

getBlobReplication in class BlobStore

Parameters:

key - Key for the blob.

who - Is the subject having the read privilege for the blob.

Returns:

BlobReplication object containing the replication factor for the blob.

Throws:

Exception

updateBlobReplication

public int updateBlobReplication(String key,
                                 int replication,
                                 Subject who)
                          throws AuthorizationException,
                                 KeyNotFoundException

Description copied from class: BlobStore

Modifies the replication factor of the blob.

Specified by:

updateBlobReplication in class BlobStore

Parameters:

key - Key for the blob.

replication - The replication factor the blob has to be set.

who - Is the subject having the update privilege for the blob

Returns:

BlobReplication object containing the updated replication factor for the blob.

Throws:

AuthorizationException

KeyNotFoundException

checkForBlobOrDownload

public boolean checkForBlobOrDownload(String key)
                               throws KeyNotFoundException

Throws:

KeyNotFoundException

checkForBlobUpdate

public void checkForBlobUpdate(String key)

fullCleanup

public void fullCleanup(long age)
                 throws IOException

Throws:

IOException