Getting Started with Data Index

This guide walks you through deploying Data Index and verifying it works.

Prerequisites

Before you begin, ensure you have:

  • Kubernetes cluster running (or KIND for local development)

  • kubectl configured to access your cluster

  • curl and jq installed

Quick Install (KIND)

For local development with KIND:

# 1. Setup KIND cluster and PostgreSQL
cd data-index/scripts/kind
./setup-cluster.sh
MODE=postgresql ./install-dependencies.sh

# 2. Deploy Data Index service
./deploy-data-index.sh postgresql

# 3. Deploy FluentBit DaemonSet (PostgreSQL mode)
cd ../fluentbit
./generate-configmap.sh postgresql postgresql/kubernetes/configmap.yaml
kubectl apply -f postgresql/kubernetes/configmap.yaml
kubectl apply -f postgresql/kubernetes/daemonset.yaml

Verify Installation

Check Pods

All components should be running:

# Data Index service
kubectl get pods -n data-index

# FluentBit
kubectl get pods -n logging

# PostgreSQL
kubectl get pods -n postgresql

Expected output:

NAMESPACE     NAME                                  READY   STATUS
data-index    data-index-service-xxx                1/1     Running
logging       workflows-fluent-bit-mode1-xxx        1/1     Running
postgresql    postgresql-0                          1/1     Running

Test GraphQL API

Query the API to verify it’s responding:

# Port-forward to Data Index service
kubectl port-forward -n data-index svc/data-index-service 8080:8080 &

# Query GraphQL
curl -s http://localhost:8080/graphql \
  -H "Content-Type: application/json" \
  -d '{"query":"{ getWorkflowInstances { id name status } }"}' \
  | jq .

Expected response (may be empty if no workflows executed yet):

{
  "data": {
    "getWorkflowInstances": []
  }
}

Access GraphQL UI

Open the interactive GraphQL playground in your browser:

# Port-forward to Data Index service (if not already running)
kubectl port-forward -n data-index svc/data-index-service 8080:8080 &

# Open GraphQL UI in browser
open http://localhost:8080/q/graphql-ui/

The GraphQL UI provides:

  • Interactive query builder - Autocomplete and syntax highlighting

  • Schema explorer - Browse available queries and types

  • Query history - Saved queries for reuse

  • Documentation - Inline field descriptions

Use Ctrl+Space in the query editor for autocomplete suggestions.

Check Database

Verify database tables exist:

kubectl exec -n postgresql postgresql-0 -- \
  env PGPASSWORD=dataindex123 \
  psql -U dataindex -d dataindex \
  -c "\dt"

Expected tables:

 workflow_instances
 task_instances
 workflow_events_raw
 task_events_raw
 flyway_schema_history

Next Steps

Now that Data Index is running:

Troubleshooting

See the Troubleshooting Guide for common issues.