NoSQL DB Runner

The NoSQL DB Runner is a backend microservice that powers FlowX Database functionality. It provides MongoDB and Azure Cosmos DB operations support for FlowX Database collections and handles data persistence operations across FlowX processes. This guide provides detailed instructions for setting up the service and configuring its components through environment variables.

Configuration

The service comes with most of the needed configuration properties filled in, but there are several environment variables that need to be set up for your specific deployment.

Database configuration

The NoSQL DB Runner connects to MongoDB for data persistence. Configure the following variables:

Environment Variable	Description	Default Value
DB_USERNAME	MongoDB username for the service	nosql-db-runner
DB_PASSWORD	MongoDB password (should be stored as secret)	secret(mongodb-generic-devenv)
SPRING_DATA_MONGODB_URI	Complete MongoDB connection string	mongodb://${DB_USERNAME}:${DB_PASSWORD}@nosql-db-runner-mongodb-0.nosql-db-runner-mongodb-headless,nosql-db-runner-mongodb-1.nosql-db-runner-mongodb-headless,nosql-db-runner-mongodb-arbiter-0.nosql-db-runner-mongodb-arbiter-headless:27017/nosql-db-runner?retryWrites=false

The MongoDB URI supports replica set configuration for high availability. The connection string includes multiple MongoDB instances and an arbiter for proper replica set functionality.

Configuring authorization & access roles

Connect the NoSQL DB Runner to an OAuth 2.0 identity management platform:

Environment Variable	Description	Default Value
SECURITY_TYPE	Security type	oauth2
SECURITY_OAUTH2_BASE_SERVER_URL	Base URL for the OAuth 2.0 Authorization Server
SECURITY_OAUTH2_REALM	OAuth2 realm name
SECURITY_OAUTH2_CLIENT_CLIENT_ID	Client ID for token introspection
SECURITY_OAUTH2_CLIENT_CLIENT_SECRET	Client secret for token introspection
SECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_ID	Service account client ID	flowx-nosql-db-runner-sa
SECURITY_OAUTH2_SERVICE_ACCOUNT_ADMIN_CLIENT_SECRET	Service account client secret
SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_MAINAUTHPROVIDER_TOKEN_URI	Provider token URI	${SECURITY_OAUTH2_BASE_SERVER_URL}/realms/${SECURITY_OAUTH2_REALM}/protocol/openid-connect/token

FlowX operations document limit

This setting allows you to configure the maximum number of documents that can be retrieved from MongoDB in a single request.

Environment Variable	Description	Default Value
FLOWX_RUNOPERATIONS_DOCUMENTLIMIT	Maximum number of documents to be processed in a single operation	50

External MongoDB connection caching

Available starting with FlowX.AI 5.6.0

These settings apply when using Unmanaged MongoDB data sources. The NoSQL DB Runner caches connections to external MongoDB instances to avoid reconnecting on every operation.

Environment Variable	Description	Default Value
FLOWX_MONGO_CONNECTIONS_CACHE_MAX_ENTRIES	Maximum number of cached external MongoDB connections	100
FLOWX_MONGO_CONNECTIONS_CACHE_TTL	Time-to-live for cached connections (duration format, e.g. 1d, 12h)	1d

If you connect to many distinct external MongoDB instances, increase FLOWX_MONGO_CONNECTIONS_CACHE_MAX_ENTRIES to avoid frequent reconnections. Lower the TTL if your MongoDB credentials rotate frequently.

Configuring logging

Control log levels for different components of the service:

Environment Variable	Description	Default Value
LOGGING_CONFIG_FILE	Path to the logging configuration file	logback-spring.xml
LOGGING_LEVEL_ROOT	Root logger level for the Spring Boot service	INFO
LOGGING_LEVEL_APP	Application-specific log level	INFO

Using DEBUG log levels in production environments may impact performance and generate large log volumes. Consider using INFO or WARN levels for production deployments.

Configuring application management

Configure management endpoints and monitoring capabilities:

Environment Variable	Description	Default Value
MANAGEMENT_SERVER_PORT	Port for management endpoints	8081
MANAGEMENT_ENDPOINT_HEALTH_GROUP_LIVENESS_INCLUDE	Health checks for liveness probes	ping,buildInfo
MANAGEMENT_ENDPOINT_HEALTH_GROUP_READINESS_INCLUDE	Health checks for readiness probes	ping,buildInfo,mongo
MANAGEMENT_HEALTH_KUBERNETES_ENABLED	Enable Kubernetes health checks	false

Prometheus metrics configuration

Enable Prometheus metrics export for monitoring:

Environment Variable	Description	Default Value
MANAGEMENT_PROMETHEUS_METRICS_EXPORT_ENABLED	Enable Prometheus metrics export	false

The older form MANAGEMENT_METRICS_EXPORT_PROMETHEUS_ENABLED also works (both resolve to the same setting). The property path above is the preferred form.

---

Troubleshooting

Common issues

MongoDB connection failures

Symptoms: Service fails to start with database connection errors.

Solutions:

1. Verify MongoDB connection string format and credentials
2. Ensure MongoDB replica set is properly configured and accessible
3. Check network connectivity between service and MongoDB instances
4. Verify MongoDB user permissions and database access rights

Kafka connectivity issues

Symptoms: Service cannot consume or produce Kafka messages.

Solutions:

1. Verify SPRING_KAFKA_BOOTSTRAP_SERVERS points to the correct broker(s)
2. Check Kafka security protocol and authentication configuration
3. Ensure network policies allow traffic to the Kafka cluster
4. Verify topic names and consumer group assignments are correct

OAuth authentication errors

Symptoms: 401/403 errors when communicating with other FlowX services.

Solutions:

1. Verify OAuth client ID and client secret are correct
2. Check OAuth server URL and realm configuration
3. Ensure OAuth provider is accessible from the service
4. Validate token endpoint URI configuration

---

Related resources

Redis Configuration

Complete Redis setup including Sentinel and Cluster modes

Kafka Authentication

Configure Kafka security and authentication

IAM Configuration

Identity and access management setup