JLupin disk queue implementation - local storage

Overview

Queues with persistance to disk can be configured for only local communication. If so there is no need to configure shared storage for them. But there is a need to configure balancers for platform and queue protocol to communication only with scope of localhost.

Usage

You must change configuration for calling microserivce, main server, queue microservice and microservice with service defintion.

Calling microservice

You must setup queue delegator for sending task with localhost load balancer. To do so updated your Spring configuration with below beans configuration:

@Configuration
@ComponentScan({"com.jlupin.impl.microservice.samplenative"})
public class MicroserviceApplicationConfiguration {

    @Bean
    public JLupinLogger getJLupinLogger() {
         return JLupinSystemContainer.getInstance().getJLupinLogger();
    }

    @Bean
    public JLupinServiceRepository getJLupinServiceRepository() {
        return JLupinClientUtil.generateServiceRepository(5000, JLupinServiceRepositoryType.INNER_MICROSERVICE, null,
                                                          new JLupinServiceRepositoryConnectionToServerConfiguration(), getJLupinLogger());
    }

    @Bean("jlrmcDelegator")
    public JLupinDelegator getJLRMCDelegator() {
        JLupinLoadBalancer jLupinPriorityLoadBalancer = JLupinClientUtil.generateLoadBalancer(getJLupinServiceRepository(), 3, 5000,
                                                        JLupinClientUtil.JLupinLoadBalancerAlgorithmType.PRIORITY, getJLupinLogger());
        return JLupinClientUtil.generateLoadBalancerDelegator(PortType.JLRMC, jLupinPriorityLoadBalancer);
    }

    @Bean("queueDelegator")
    public JLupinDelegator getQueueDelegator() {
        JLupinLoadBalancer jLupinLocalhostLoadBalancer = JLupinClientUtil.generateLoadBalancer(getJLupinServiceRepository(), 3, 5000,
                                                         JLupinClientUtil.JLupinLoadBalancerAlgorithmType.LOCALHOST, getJLupinLogger());
        return JLupinClientUtil.generateLoadBalancerDelegator(PortType.QUEUE, jLupinLocalhostLoadBalancer);
    }

}

Important here is that the service repository instance is shared between all load balancers.

Main server

Update your main.yml file to setup load balancers for queue protocol:

[...]
PROCESS_MANAGER:
  [...]
  LOAD_BALANCER_SETTINGS:
    [...]
    QUEUE_PROTOCOL_SETTINGS:
      ALGORITHM:
        type: localhost
      PROPERTIES:
        changeServerIntervalInMillis: 5000
[...]

With this configuration load balancer will look for queue microservice only on localhost.

Queue microservice

Update your configuration.yml file to setup load balancers for queue protocol:

[...]
ENTRY_POINTS:
  [...]
  QUEUE:
    [...]
    LOAD_BALANCER_SETTINGS:
      ALGORITHM:
        type: localhost
      PROPERTIES:
        changeServerIntervalInMillis: 5000
        repeatsAmount: 3
[...]

Microservice with service called by queue

Update your configuration.yml file to setup load balancers for queue protocol:

[...]
ENTRY_POINTS:
  [...]
  QUEUE:
    [...]
    LOAD_BALANCER_SETTINGS:
      ALGORITHM:
        type: localhost
      PROPERTIES:
        changeServerIntervalInMillis: 5000
        repeatsAmount: 3
[...]

Storage type configuration

To use disk queue implementation configure your queues.yml configuration file with storage class name as shown below:

[...]
    storageClassName: 'com.jlupin.impl.microservice.partofjlupin.asynchronous.storage.queue.impl.disk.JLupinDiskQueueStorageImpl'
[...]

Storage configuration

Remember to set up your storage configuration in storage.yml file:

STORAGE:
  #absoluteRootPath: 'c:\\storage'
  maxTasksAmountInSingleDirectory: 100
  howOftenCheckingSubdirectoryInMillis: 5000
  howOftenGenerateHealthCheckFileInMillis: 10000
  takeControlOfAnotherQueueInMillis: 60000
  howLongWaitToTakeControlOfOwnStorage: 60000
  howOftenCheckAnotherHealthCheckFileInMillis: 10000
  maxTimeToHoldNonUseDirectoriesInMilliseconds: 86400000

Description:

Parameter	Description
`#absoluteRootPath`	Path to root location for storing queues. By default it is `platform/storage` directory.
`maxTasksAmountInSingleDirectory`	Maximum amount of tasks stored in a single directory.
`howOftenCheckingSubdirectoryInMillis`	Time period (expressed in milliseconds) at which subdirectories will be checked for new tasks and switched.
`howOftenGenerateHealthCheckFileInMillis`	Time period (expressed in milliseconds) at which queue will update it's healt check status.
`takeControlOfAnotherQueueInMillis`	Minimum time (expressed in milliseconds) required to pass before taking controler over another queue which is not alive.
`howLongWaitToTakeControlOfOwnStorage`	Time period (expressed in milliseconds) which queue will wait after setting it is alive before it will start processing it's own tasks.
`howOftenCheckAnotherHealthCheckFileInMillis`	Time period (expressed in milliseconds) at which queue will check if other queues are alive.
`maxTimeToHoldNonUseDirectoriesInMilliseconds`	Maximum alive time (expressed in milliseconds) for unused directories.

Documentation

CONTENTS