Class AsyncLocalizer

All Implemented Interfaces:

public class AsyncLocalizer extends Object implements AutoCloseable
Downloads and caches blobs locally.
  • Field Details

  • Constructor Details

  • Method Details

    • requestDownloadTopologyBlobs

      public CompletableFuture<Void> requestDownloadTopologyBlobs(LocalAssignment assignment, int port, BlobChangingCallback cb) throws IOException
      Request that all of the blobs necessary for this topology be downloaded. Note that this adds references to blobs asynchronously in background threads.
      assignment - the assignment that needs the blobs
      port - the port the assignment is a part of
      cb - a callback for when the blobs change. This is only for blobs that are tied to the lifetime of the worker.
      a Future that indicates when they are all downloaded.
      IOException - if there was an error while trying doing it.
    • start

      public void start()
      Start any background threads needed. This includes updating blobs and cleaning up unused blobs over the configured size limit.
    • close

      public void close() throws InterruptedException
      Specified by:
      close in interface AutoCloseable
    • recoverRunningTopology

      public void recoverRunningTopology(LocalAssignment currentAssignment, int port, BlobChangingCallback cb) throws IOException
      Do everything needed to recover the state in the AsyncLocalizer for a running topology.
      currentAssignment - the assignment for the topology.
      port - the port the assignment is on.
      cb - a callback for when the blobs are updated. This will only be for blobs that indicate that if they change the worker should be restarted.
      IOException - on any error trying to recover the state.
    • releaseSlotFor

      public void releaseSlotFor(LocalAssignment assignment, int port) throws IOException
      Remove this assignment/port as blocking resources from being cleaned up.
      assignment - the assignment the resources are for
      port - the port the topology is running on
      IOException - on any error
    • getClientBlobStore

      protected ClientBlobStore getClientBlobStore()