Skip to content

Installation with Kubernetes

For cluster deployments we recommend installing Accurinds within a container orchestration system such as Kubernetes or a compatible platform. The orchestration system will handle tasks such as service discovery, storage provisioning, load balancing and scheduling for you, which would otherwise have to be configured manually.

This guide will outline the installation of Accurids in a cluster managed with Kubernetes.

Preparation

Accurids can be provisioned in any Kubernetes cluster with a sufficient amount of available resources for scheduling (as configured) and capable of fulfilling volume requests for the individual Accurids replicas. Furthermore, if you wish Accurids to be available from an external network (such as the public internet), an Ingress controller must be set up and the DNS for your externally resolvable Accurids instance name must be configured appropriately.

In case you would like to deploy Accurids to an existing cluster, the necessary configuration will depend heavily on your environment. The deployment of Accurids itself is described below, however you must ensure that the following dependencies are satisfied, either through services deployed within the cluster, or via external systems:

  • a PostgreSQL relational database (at least version 11),
  • an installation of Elasticsearch (at least version 8.5.1)

Bootstrapping of a cluster is out of the scope of this guide. Please refer to the corresponding section of the Kubernetes documentation if you would like to install Kubernetes on physical hosts or buy a turnkey solution from a cloud provider. From now on let us assume that you have configured a Kubernetes control plane, joined worker nodes to the cluster, configured a CSI driver with appropriate storage classes and an ingress controller, and have kubectl on your local machine set up to point to the cluster via the default context.

Prerequisites

This section describes the installation of Elasticsearch and PostgreSQL with the Helm package manager. The charts and configuration options used in this example are used for illustrative purposes and may not be appropriate for your specific use case. Please consider to consult a deployment specialist or to evaluate managed services for Elasticsearch or PostgreSQL before using Accurids for production workloads.

Before installing Elasticsearch, clarify resource requirements and the desired number of replicas. Then, install Elasticsearch from the Bitnami Elasticsearch Chart. The following example will deploy three Elasticsearch replicas with 50GiB of storage and 32GiB of total memory each into a new namespace elasticsearch.

helm repo add bitnami "https://charts.bitnami.com/bitnami"
helm install elasticsearch bitnami/elasticsearch -n elasticsearch --create-namespace \
  --version 21.3.6 \
  --set="master.masterOnly=false" \
  --set="master.replicaCount=3" \
  --set="data.replicaCount=0" \
  --set="coordinating.replicaCount=0" \
  --set="ingest.replicaCount=0" \
  --set="master.resources.requests.memory=32Gi" \
  --set="master.resources.requests.cpu=2" \
  --set="master.resources.limits.memory=40Gi" \
  --set="master.resources.limits.cpu=8" \
  --set="master.heapSize=32g" \
  --set="master.persistence.size=50Gi"

Similarly, proceed as follows to deploy PostgreSQL from the Bitnami PostgreSQL Chart to a new namespace postgres.

helm repo add bitnami "https://charts.bitnami.com/bitnami"
helm install postgresql bitnami/postgresql -n postgres --create-namespace \
  --set-string=postgresqlDatabase=postgres \
  --set-string=resources.requests.memory=8Gi \
  --set-string=resources.requests.cpu=2 \
  --set-string=resources.limits.memory=16Gi \
  --set-string=resources.limits.cpu=8

Installation

Accurids is distributed as a container image via Amazon ECR. The latest release is available at 858273179365.dkr.ecr.eu-central-1.amazonaws.com/accurids:stable. Access credentials to this repository are provided to you with your Accurids subscription.

For installing Accurids, we will set up the following resources in a newly created namespace accurids:

  • a StatefulSet to manage pods for Accurids nodes and PersistentVolumeClaims for their corresponding working directories,
  • a ConfigMap with essential configuration options (such as those regarding cluster configuration and database access),
  • a Secret with database credentials,
  • a Service for routing and service discovery,
  • an Ingress to make Accurids available outside of Kubernetes,
  • another Service to allow the Accurids nodes to discover each other,
  • permissions via a ClusterRole and ClusterRoleBindings to permit Accurids to discover other nodes.

Please note: Accurids requires a service account token to be available at the default location in order to perform service discovery within the namespace.

The Accurids image will write state to /working-dir. This location should be the mount point of a persistent volume to avoid data loss.

Creating a Database

Please create a new database and an associated role in PostgreSQL and note down the credentials. If you have deployed PostgreSQL as described above, you can get the superuser password with kubectl get secret --namespace postgres postgresql -o jsonpath="{.data.postgres-password}" | base64 -d. Furthermore, you can create a role and a database like this:

export SUPERUSER=postgres     # Your PostgreSQL superuser
export SUPERUSER_PASS=        # Fill in your superuser password
export ACCURIDS_USER=accurids # The desired application user
export TARGET_DB=accurids     # The desired target database
# Create the accurids user, you will be asked for the password
kubectl exec -it postgresql-0 -c postgresql -n postgres -- bash -c "PGPASSWORD='$SUPERUSER_PASS' createuser -U $SUPERUSER -P $ACCURIDS_USER"
kubectl exec postgresql-0 -c postgresql -n postgres -- bash -c "PGPASSWORD='$SUPERUSER_PASS' createdb -U $SUPERUSER -O $TARGET_DB $TARGET_DB"
kubectl exec postgresql-0 -c postgresql -n postgres -- bash -c "PGPASSWORD='$SUPERUSER_PASS' psql -U $SUPERUSER -c \"GRANT ALL PRIVILEGES ON DATABASE $TARGET_DB TO $ACCURIDS_USER;\""

Installing Accurids

You can use the Manifest provided below as a starting point for your configuration. The Ingress annotations assume that you use the Kubernetes NGINX Ingress Controller, but any other ingress mechanism should be compatible, as long as you ensure that the appropriate request and reponse headers are present and long-running requests or requests with large bodies for file uploads can be supported.

Please refer to the variables documentation page for an overview of Accurids configuration variables.

---
apiVersion: v1
kind: Namespace
metadata:
  name: accurids # Specify target namespace here
spec:
  finalizers:
  - kubernetes
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app: accurids
  name: accurids
  namespace: accurids # Specify target namespace here
spec:
  podManagementPolicy: Parallel
  replicas: 3 # Desired count of Accurids replicas
  selector:
    matchLabels:
      app: accurids
  serviceName: accurids
  template:
    metadata:
      annotations:
        prometheus.io/port: "8080"
      labels:
        app: accurids
    spec:
      automountServiceAccountToken: true
      containers:
      - env:
        - name: spring.datasource.username
          valueFrom:
            secretKeyRef:
              key: username
              name: accurids-db
              optional: false
        - name: spring.datasource.password
          valueFrom:
            secretKeyRef:
              key: password
              name: accurids-db
              optional: false
        envFrom:
        - configMapRef:
            name: accurids
            optional: false
        image: 858273179365.dkr.ecr.eu-central-1.amazonaws.com/accurids:stable
        imagePullPolicy: Always
        livenessProbe:
          failureThreshold: 10
          httpGet:
            path: /actuator/health/liveness
            port: 8080
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: accurids
        ports:
        - containerPort: 8080
          protocol: TCP
        readinessProbe:
          failureThreshold: 10
          httpGet:
            path: /actuator/health/readiness
            port: 8080
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources: # Configure Accurids resource requests/limits (per replica)
          limits:
            cpu: "16"
            memory: 32Gi
          requests:
            cpu: "2"
            memory: 8Gi
        volumeMounts:
        - mountPath: /working-dir
          mountPropagation: None
          name: working-dir
      dnsPolicy: ClusterFirst
      enableServiceLinks: true
      restartPolicy: Always
      shareProcessNamespace: false
      terminationGracePeriodSeconds: 30
  updateStrategy:
    type: RollingUpdate
  volumeClaimTemplates:
  - apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: working-dir
      namespace: default
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 1Ti # Desired size of Accurids volume (per replica)
      volumeMode: Filesystem
---
apiVersion: v1
kind: Service
metadata:
  name: accurids
  namespace: accurids # Specify target namespace here
spec:
  externalTrafficPolicy: Cluster
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: accurids
  sessionAffinity: None
  type: NodePort
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/cors-allow-methods: PUT, GET, POST, OPTIONS
    nginx.ingress.kubernetes.io/enable-cors: "true"
    # The following option limits the size of datasets that can be uploaded
    # via the UI. Adapt to your requirements, use 0 to have no limit at all.
    nginx.ingress.kubernetes.io/proxy-body-size: 5G
    nginx.ingress.kubernetes.io/proxy-buffer-size: "16k"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-max-temp-file-size: 8192m
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    nginx.ingress.kubernetes.io/server-snippet: |
      location /actuator {
        deny all;
      }
  name: accurids
  namespace: accurids # Specify target namespace here
spec:
  rules:
  - host: example.accurids.com # external domain name (if desired)
    http:
      paths:
      - backend:
          service:
            name: accurids
            port:
              number: 8080
        pathType: ImplementationSpecific
  tls: # In case you wish to terminate TLS at the ingress, configure certificates here
  - hosts:
    - example.accurids.com
    secretName: accurids-tls
---
apiVersion: v1
data:
  accurids.confirm-user-email: "false" # Change as desired
  accurids.instance: example.accurids.com # Specify instance name
  accurids.maxParallelImportsPerNode: "2"
  accurids.namespace: accurids # Specify target namespace here
  accurids.root-url: ""
  elasticsearch_cluster: elasticsearch
  elasticsearch_host: elasticsearch.elasticsearch
  JDK_JAVA_OPTIONS: -XX:MaxRAMPercentage=75.0
  management.endpoint.health.probes.enabled: "true"
  management.endpoints.web.exposure.include: prometheus,health
  spring.datasource.driverClassName: org.postgresql.Driver
  spring.datasource.hikari.leak-detection-threshold: "5000"
  spring.datasource.url: jdbc:postgresql://postgresql-ha-pgpool.postgres:5432/<DATABASE NAME> # Change database name
  spring.mail.password: # Specify mail server password
  spring.mail.username: # Specify mail server username
kind: ConfigMap
metadata:
  name: accurids
  namespace: accurids # Specify target namespace here
---
apiVersion: v1
stringData:
  database: # Specify database name here
  password: # Specify database password here
  username: # Specify database username here
kind: Secret
metadata:
  name: accurids-db
  namespace: accurids # Specify target namespace here
---
apiVersion: v1
kind: Service
metadata:
  name: hazelcast
spec:
  type: LoadBalancer
  selector:
    app: accurids
  ports:
  - name: hazelcast
    port: 5701
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: hazelcast-cluster-role
rules:
  - apiGroups:
      - ""
    resources:
      - endpoints
      - pods
      - nodes
      - services
    verbs:
      - get
      - list
  - apiGroups:
      - "discovery.k8s.io"
    resources:
      - endpointslices
    verbs:
      - get
      - list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: hazelcast-cluster-role-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: hazelcast-cluster-role
subjects:
  - kind: ServiceAccount
    name: default
    namespace: accurids # Specify target namespace here

Apply the manifest with kubectl apply. Accurids should now be deployed.

Configure Mail Services

When built-in user management is activated, Accurids can verify the registration by email.

This is not applicable, when OAuth2 Setup is used for authentication. In that case or if you want to disable email confirmation, set accurids.confirm-user-email=false. No further configurations are needed in this case and all mail-related variables mentioned below can be removed.

To setup the email service, set credentials for authentication with spring.mail.username and spring.mail.password. Specify the mail server to connect to with spring.mail.host and spring.mail.port. The pre-configured default port is 587. Please ensure that the mailserver is reachable from the server where Accurids runs.

Furthermore, the sender address of emails sent by Accurids can be customized with accurids.from-email. The default sender address is office@accurids.com.

Accessing the new Installation

In case you have configured an Ingress Accurids should now be available at the specified location. Otherwise, you can access Accurids via a port-forward to the service: kubectl port-forward svc/accurids -n accurids 8080.

If you are using local authentication, note that the first registered user will receive the "administrator" role.

Next Steps

You are now ready to configure your Accurids deployment further. We recommend to set up your license and configure backups right away.

Entering Your License

Visit the license documentation page for details on how to configure your license key.

Configuring Backups

Please refer to the Backup & Recovery documentation page to configure backups. Having a recent backup is indispensable in case of any failure event.

Preload Datasets

Please refer to the Preload Datasets documentation page.