public abstract class BlobStore extends Object implements Shutdownable, AutoCloseable
Provides a way to store blobs that can be downloaded. Blobs must be able to be uploaded and listed from Nimbus, and downloaded from the Supervisors. It is a key value based store. Key being a string and value being the blob data.
ACL checking must take place against the provided subject. If the blob store does not support Security it must validate that all ACLs set are always WORLD, everything.
The users can upload their blobs through the blob store command line. The command line also allows us to update and delete blobs.
Modifying the replication factor only works for HdfsBlobStore as for the LocalFsBlobStore the replication is dependent on the number of Nimbodes available.
Modifier and Type | Class and Description |
---|---|
protected class |
BlobStore.BlobStoreFileInputStream
Input stream implementation used for writing both the metadata containing the acl information and the blob data.
|
protected class |
BlobStore.BlobStoreFileOutputStream
Output stream implementation used for reading the metadata and data information.
|
static class |
BlobStore.KeyTranslationIterator
Blob store implements its own version of iterator to list the blobs.
|
Modifier and Type | Field and Description |
---|---|
protected static String |
BASE_BLOBS_DIR_NAME |
Constructor and Description |
---|
BlobStore() |
Modifier and Type | Method and Description |
---|---|
void |
close() |
void |
createBlob(String key,
byte[] data,
SettableBlobMeta meta,
Subject who)
Wrapper called to create the blob which contains the byte data.
|
void |
createBlob(String key,
InputStream in,
SettableBlobMeta meta,
Subject who)
Wrapper called to create the blob which contains the byte data.
|
abstract AtomicOutputStream |
createBlob(String key,
SettableBlobMeta meta,
Subject who)
Creates the blob.
|
abstract void |
deleteBlob(String key,
Subject who)
Deletes the blob data and metadata.
|
<R> Set<R> |
filterAndListKeys(KeyFilter<R> filter)
Filters keys based on the KeyFilter passed as the argument.
|
abstract InputStreamWithMeta |
getBlob(String key,
Subject who)
Gets the InputStream to read the blob details.
|
abstract ReadableBlobMeta |
getBlobMeta(String key,
Subject who)
Gets the current version of metadata for a blob to be viewed by the user or downloaded by the supervisor.
|
abstract int |
getBlobReplication(String key,
Subject who)
Gets the replication factor of the blob.
|
abstract Iterator<String> |
listKeys()
Returns an iterator with all the list of keys currently available on the blob store.
|
abstract void |
prepare(Map<String,Object> conf,
String baseDir,
NimbusInfo nimbusInfo,
ILeaderElector leaderElector)
Allows us to initialize the blob store.
|
byte[] |
readBlob(String key,
Subject who)
Wrapper around readBlobTo which returns a ByteArray output stream.
|
void |
readBlobTo(String key,
OutputStream out,
Subject who)
Reads the blob from the blob store and writes it into the output stream.
|
abstract void |
setBlobMeta(String key,
SettableBlobMeta meta,
Subject who)
Sets the metadata with renewed acls for the blob.
|
abstract void |
setLeaderElector(ILeaderElector leaderElector)
Sets leader elector (only used by LocalFsBlobStore to help sync blobs between Nimbi.
|
void |
startSyncBlobs()
Start the syncing blobs between the local running instance of the BlobStore and others.
|
Set<String> |
storedTopoIds()
Get IDs stored in blob store.
|
void |
updateBlob(String key,
byte[] data,
Subject who)
Wrapper called to create the blob which contains the byte data.
|
abstract AtomicOutputStream |
updateBlob(String key,
Subject who)
Updates the blob data.
|
abstract int |
updateBlobReplication(String key,
int replication,
Subject who)
Modifies the replication factor of the blob.
|
static void |
validateKey(String key)
Validates key checking for potentially harmful patterns.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
shutdown
protected static final String BASE_BLOBS_DIR_NAME
public static final void validateKey(String key) throws IllegalArgumentException
Validates key checking for potentially harmful patterns.
key
- Key for the blobIllegalArgumentException
public abstract void prepare(Map<String,Object> conf, String baseDir, NimbusInfo nimbusInfo, ILeaderElector leaderElector)
Allows us to initialize the blob store.
conf
- The storm configurationbaseDir
- The directory path to store the blobsnimbusInfo
- Contains the nimbus host, port and leadership informationpublic void startSyncBlobs() throws KeyNotFoundException, AuthorizationException
Start the syncing blobs between the local running instance of the BlobStore and others. A no-op for the HdfsBlobStore where HDFS itself does the syncing but for the LocalFsBlobStore ZK state updates are run periodically here.
public abstract AtomicOutputStream createBlob(String key, SettableBlobMeta meta, Subject who) throws AuthorizationException, KeyAlreadyExistsException
Creates the blob.
key
- Key for the blobmeta
- Metadata which contains the acls informationwho
- Is the subject creating the blobAuthorizationException
KeyAlreadyExistsException
public void createBlob(String key, byte[] data, SettableBlobMeta meta, Subject who) throws AuthorizationException, KeyAlreadyExistsException, IOException
Wrapper called to create the blob which contains the byte data.
key
- Key for the blobdata
- Byte data that needs to be uploadedmeta
- Metadata which contains the acls informationwho
- Is the subject creating the blobAuthorizationException
KeyAlreadyExistsException
IOException
public void createBlob(String key, InputStream in, SettableBlobMeta meta, Subject who) throws AuthorizationException, KeyAlreadyExistsException, IOException
Wrapper called to create the blob which contains the byte data.
key
- Key for the blobin
- InputStream from which the data is read to be written as a part of the blobmeta
- Metadata which contains the acls informationwho
- Is the subject creating the blobAuthorizationException
KeyAlreadyExistsException
IOException
public abstract AtomicOutputStream updateBlob(String key, Subject who) throws AuthorizationException, KeyNotFoundException
Updates the blob data.
key
- Key for the blobwho
- Is the subject having the write privilege for the blobAuthorizationException
KeyNotFoundException
public void updateBlob(String key, byte[] data, Subject who) throws AuthorizationException, IOException, KeyNotFoundException
Wrapper called to create the blob which contains the byte data.
key
- Key for the blobdata
- Byte data that needs to be uploadedwho
- Is the subject creating the blobAuthorizationException
IOException
KeyNotFoundException
public abstract ReadableBlobMeta getBlobMeta(String key, Subject who) throws AuthorizationException, KeyNotFoundException
Gets the current version of metadata for a blob to be viewed by the user or downloaded by the supervisor.
key
- Key for the blobwho
- Is the subject having the read privilege for the blobAuthorizationException
KeyNotFoundException
public abstract void setLeaderElector(ILeaderElector leaderElector)
Sets leader elector (only used by LocalFsBlobStore to help sync blobs between Nimbi.
public abstract void setBlobMeta(String key, SettableBlobMeta meta, Subject who) throws AuthorizationException, KeyNotFoundException
Sets the metadata with renewed acls for the blob.
key
- Key for the blobmeta
- Metadata which contains the updated acls informationwho
- Is the subject having the write privilege for the blobAuthorizationException
KeyNotFoundException
public abstract void deleteBlob(String key, Subject who) throws AuthorizationException, KeyNotFoundException
Deletes the blob data and metadata.
key
- Key for the blobwho
- Is the subject having write privilege for the blobAuthorizationException
KeyNotFoundException
public abstract InputStreamWithMeta getBlob(String key, Subject who) throws AuthorizationException, KeyNotFoundException
Gets the InputStream to read the blob details.
key
- Key for the blobwho
- Is the subject having the read privilege for the blobAuthorizationException
KeyNotFoundException
public abstract Iterator<String> listKeys()
Returns an iterator with all the list of keys currently available on the blob store.
Iterator<String>
public abstract int getBlobReplication(String key, Subject who) throws Exception
Gets the replication factor of the blob.
key
- Key for the blobwho
- Is the subject having the read privilege for the blobException
public abstract int updateBlobReplication(String key, int replication, Subject who) throws AuthorizationException, KeyNotFoundException, IOException
Modifies the replication factor of the blob.
key
- Key for the blobreplication
- The replication factor the blob has to be setwho
- Is the subject having the update privilege for the blobAuthorizationException
KeyNotFoundException
IOException
public void close()
close
in interface AutoCloseable
public <R> Set<R> filterAndListKeys(KeyFilter<R> filter)
Filters keys based on the KeyFilter passed as the argument.
filter
- KeyFilterR
- Typepublic void readBlobTo(String key, OutputStream out, Subject who) throws IOException, KeyNotFoundException, AuthorizationException
Reads the blob from the blob store and writes it into the output stream.
key
- Key for the blobout
- Output streamwho
- Is the subject having read privilege for the blobIOException
KeyNotFoundException
AuthorizationException
public byte[] readBlob(String key, Subject who) throws IOException, KeyNotFoundException, AuthorizationException
Wrapper around readBlobTo which returns a ByteArray output stream.
key
- Key for the blobwho
- Is the subject having the read privilege for the blobIOException
KeyNotFoundException
AuthorizationException
Copyright © 2022 The Apache Software Foundation. All rights reserved.