Run the worker shell

This page explains how to configure of the worker shell and how to run it.

Prepare the configuration files

Write a worker.yml file to declare your tasks, below an example with a Python task:

shell-tasks:
  - id: kcoloring-python
    command: python3 kcoloring.py
    workingDirectory: ./python-tasks
    description: This task computes a coloration of a partial Europe map with k colors.
    inputs:
    - name: number_of_color
      type: NUMERIC
      description: "Number of colors to be used by the coloration."
      required: true
    outputs:
    - name: coloration
      type: JSON
      description: "The coloration found: for each country, the associated color."

The value of command is the command to run (like in a command line shell) and workingDirectory is the directory where the command is executed. Make sure the command can be executed in workingDirectory (like with your command line shell in this directory).

Write an application.yml file with the following content, set the relevant property values:

spring:
application:
  name: MyCustomWorker
rabbitmq:
  username: decisionbrain
  password: decisionbrain
  host: localhost
  port: 5672
master:
url: http://localhost:8080/

Run the worker shell

Once your configuration files are ready, you can run the worker shell using the jar or using Gradle or using Docker.

Using the jar

Requirements:

Make sure to have the following files in the current directory (example with the Python task):

optimserver-worker-shell-3.5.0.jar
config/
 ├─ application.yml
 └─ worker.yml
python-tasks/
 └─ kcoloring.py

Then run the command:

java -jar optimserver-worker-shell-3.5.0.jar --worker.configurationFileName=config/worker.yml

Or using an environment variable:

export WORKER_CONFIGURATIONFILENAME=config/worker.yml
java -jar optimserver-worker-shell-3.5.0.jar

Using gradle

Requirements:

  • Java 11+
  • Gradle (or Gradle Wrapper)

Make sure to have the following files in the current directory (example with the Python task):

build.gradle
gradle.properties
config/
 ├─ application.yml
 └─ worker.yml
python-tasks/
 └─ kcoloring.py

Fill the build.gradle file with:

apply plugin: 'java'

repositories {
    mavenCentral()
    maven {
        url "https://nexus.decisionbrain.cloud/repository/maven-releases-dbos"
        credentials  {
            username = "$MAVEN_USER"
            password = "$MAVEN_PASSWORD"
        }
        authentication {                            
            basic(BasicAuthentication)
        }
    }
}

configurations { workerShellRuntime }
dependencies {
    workerShellRuntime("com.decisionbrain:optimserver-worker-shell:3.5.0")
}

task workerShell(type: JavaExec) {
    classpath = files(configurations.workerShellRuntime.first())
    workingDir = projectDir
    args('--worker.configurationFileName=config/worker.yml')
}

Fill the gradle.properties file with your Nexus credentials:

MAVEN_USER=...
MAVEN_PASSWORD=...

Then run the command:

gradle workerShell

Using Docker

Requirements:

  • Docker

Make sure to have the following files in the current directory (example with the Python task):

Dockerfile
config/
 ├─ application.yml
 └─ worker.yml
python-tasks/
 └─ kcoloring.py

Fill the Dockerfile file with:

FROM dbos-registry.decisionbrain.cloud/dbos-worker-shell:3.5.0

ENV DEBIAN_FRONTEND noninteractive
# Install binaries and libraries (example for the Python task):
RUN apt update \
    && apt install -y python3 python3-pip \
    && apt clean
# Get credentials from args
ARG MAVEN_USER
ARG MAVEN_PASSWORD
# Install the Python bridge
RUN pip3 install --extra-index-url "https://${MAVEN_USER}:${MAVEN_PASSWORD}@nexus.decisionbrain.cloud/repository/dbos-pypi/simple" optimserver-workerbridge==3.5.0 

COPY config/worker.yml $WORKER_CONFIGURATIONFILENAME
COPY python-tasks/kcoloring.py python-tasks/kcoloring.py

The base image dbos-registry.decisionbrain.cloud/dbos-worker-shell:3.5.0 is based on Ubuntu, so you can complete your custom image by installing the packages required to run your shell tasks.

The WORKER_CONFIGURATIONFILENAME build variable contains the path were the worker shell of the base image will look for the worker.yml file.

Build your image with:

docker build . -t custom-worker --build-arg MAVEN_USER=your-username --build-arg MAVEN_PASSWORD=your-password

If your worker uses the Cplex solver, Optimization Server provides a pre-packaged Docker image that includes:

  • Cplex 12.10
  • Python 3.7.10
  • The Python bridge (a small library that will help write your task)

It’s available at a Docker repository dedicated to Cplex images. The credentials you were supplied must have the required role to fetch these images.

Then, fill the Dockerfile file with:

FROM cplex-registry.decisionbrain.cloud/dbos/dbos-worker-shell-python-cplex:3.5.0

COPY config/worker.yml $WORKER_CONFIGURATIONFILENAME
COPY python-tasks/kcoloring.py python-tasks/kcoloring.py

And build your image with:

docker build . -t custom-worker

To run your image locally, run the following command (remove --network host if master runs inside Docker):

docker run -it --rm --network host custom-worker