Audit setup

This guide will walk you through the process of setting up the Audit service and configuring it to meet your needs.

Infrastructure prerequisites

The Audit service requires the following components to be set up before it can be started:

• Docker engine
• Kafka
• Elasticsearch

Redis is not required. As of FlowX.AI 5.6.0, audit-core no longer depends on Redis. If you previously configured Redis for this service, those environment variables can be removed from your deployment.

---

Dependencies

The Audit service is built as a Docker image and runs on top of Kafka and Elasticsearch. Therefore, these services must be set up and running before starting the Audit service.

• Kafka configuration
• Authorization & access roles
• Elasticsearch
• Logging

---

Configuration

Configuring Kafka

To configure the Kafka server for the Audit service, set the following environment variables:

Connection settings

Variable	Description	Default Value
KAFKA_BOOTSTRAP_SERVERS	Kafka broker addresses (fallback: SPRING_KAFKA_BOOTSTRAP_SERVERS)	localhost:9092
KAFKA_SECURITY_PROTOCOL	Security protocol for Kafka connections (fallback: SPRING_KAFKA_SECURITY_PROTOCOL)	PLAINTEXT
KAFKA_MESSAGE_MAX_BYTES	Maximum message size (bytes)	52428800 (50 MB)

Consumer configuration

Variable	Description	Default Value
SPRING_KAFKA_CONSUMER_GROUPID	Consumer group ID for audit logs	audit-gid
KAFKA_CONSUMER_THREADS	Number of consumer threads	1
KAFKA_AUTHEXCEPTIONRETRYINTERVAL	Retry interval after auth failure (seconds)	10

OAuth authentication (when using SASL_PLAINTEXT)

Environment Variable	Description	Default Value
KAFKA_OAUTH_CLIENT_ID	OAuth client ID	kafka
KAFKA_OAUTH_CLIENT_SECRET	OAuth client secret	kafka-secret
KAFKA_OAUTH_TOKEN_ENDPOINT_URI	OAuth token endpoint	kafka.auth.localhost

When using the kafka-auth profile, the security protocol will automatically be set to SASL_PLAINTEXT and the SASL mechanism will be set to OAUTHBEARER.

Topic naming configuration

Variable	Description	Default Value
KAFKA_TOPIC_NAMING_PACKAGE	Package prefix for topic names	ai.flowx.
KAFKA_TOPIC_NAMING_ENVIRONMENT	Environment segment for topic names
KAFKA_TOPIC_NAMING_VERSION	Version suffix for topic names	.v1
KAFKA_TOPIC_NAMING_SEPARATOR	Primary separator for topic names	.
KAFKA_TOPIC_NAMING_SEPARATOR2	Secondary separator for topic names	-

Kafka topics

Variable	Description	Default Value
KAFKA_TOPIC_AUDIT_IN	Topic for receiving audit logs	ai.flowx.core.trigger.save.audit.v1

Configuring Elasticsearch

Configure Elasticsearch connection for audit data storage:

Variable	Description	Default Value
SPRING_ELASTICSEARCH_REST_PROTOCOL	Connection protocol	https
SPRING_ELASTICSEARCH_REST_URIS	URL(s) of Elasticsearch nodes (no protocol)	-
SPRING_ELASTICSEARCH_REST_DISABLESSL	Disable SSL verification	false
SPRING_ELASTICSEARCH_REST_USERNAME	Authentication username	-
SPRING_ELASTICSEARCH_REST_PASSWORD	Authentication password	-
FLOWX_ELASTICSEARCH_INDEXSETTINGS_NAME	Audit data stream name	audit-logs
FLOWX_ELASTICSEARCH_INDEXSETTINGS_SHARDS	Number of primary shards	2
FLOWX_ELASTICSEARCH_INDEXSETTINGS_REPLICAS	Number of replica shards	0

The Elasticsearch index settings determine how your audit data is distributed and replicated across the cluster. The number of shards affects search performance and indexing, while replicas provide redundancy.

CAS lib configuration

Environment Variable	Description	Default Value
FLOWX_SPICEDB_HOST	SpiceDB server hostname	spicedb
FLOWX_SPICEDB_PORT	SpiceDB server port	50051
FLOWX_SPICEDB_TOKEN	SpiceDB authentication token	-

Configuring logging

To control the log levels, set the following environment variables:

Variable	Description	Default Value
LOGGING_LEVEL_ROOT	Log level for root service	INFO
LOGGING_LEVEL_APP	Log level for application	INFO

---

Ingress configuration

The Audit Core service uses the standard FlowX.AI ingress pattern. For complete setup instructions including the full ingress template, CORS configuration, and troubleshooting, see the Ingress Configuration Guide.

Service-specific values for Audit Core:

• Ingress name: audit-core-admin
• Service path: /audit(/|$)(.*)(/|$)(.*)
• Service name: audit-core
• Rewrite target: /$2
• Fx-Workspace-Id: Required

Complete Ingress Configuration

View the centralized ingress guide for the complete configuration template, annotations reference, and best practices.

---

Troubleshooting

Common issues

Audit logs not appearing

Symptoms: Audit events are not being recorded or are missing from search results.

Solutions:

1. Verify the KAFKA_TOPIC_AUDIT_IN topic exists and matches the topic other services publish to (ai.flowx.core.trigger.save.audit.v1)
2. Check that the consumer group ID (SPRING_KAFKA_CONSUMER_GROUPID) is correctly configured and not conflicting with another consumer
3. Ensure Kafka connectivity by verifying KAFKA_BOOTSTRAP_SERVERS is reachable from the pod
4. Check consumer thread count (KAFKA_CONSUMER_THREADS) — increase if throughput is insufficient

Elasticsearch connection errors

Symptoms: Service fails to start or throws connection errors related to Elasticsearch.

Solutions:

1. Verify SPRING_ELASTICSEARCH_REST_URIS points to a reachable Elasticsearch node (do not include the protocol in this value)
2. Check that SPRING_ELASTICSEARCH_REST_PROTOCOL matches your Elasticsearch setup (https or http)
3. If using SSL, ensure SPRING_ELASTICSEARCH_REST_DISABLESSL is set appropriately and certificates are valid
4. Verify SPRING_ELASTICSEARCH_REST_USERNAME and SPRING_ELASTICSEARCH_REST_PASSWORD credentials are correct

High storage consumption

Symptoms: Elasticsearch disk usage grows rapidly or cluster health turns yellow/red.

Solutions:

1. Review your index lifecycle management (ILM) policies to ensure old audit data is rolled over and deleted on schedule
2. Adjust FLOWX_ELASTICSEARCH_INDEXSETTINGS_SHARDS and FLOWX_ELASTICSEARCH_INDEXSETTINGS_REPLICAS — fewer replicas reduce storage at the cost of redundancy
3. Check the FLOWX_ELASTICSEARCH_INDEXSETTINGS_NAME data stream for excessive index count
4. Consider reducing the retention period for audit logs in your ILM policy

Audit events delayed

Symptoms: Audit entries appear in Elasticsearch with significant delay after the triggering action.

Solutions:

1. Check Kafka consumer lag for the audit consumer group using your Kafka monitoring tools
2. Increase KAFKA_CONSUMER_THREADS to allow parallel consumption if lag is consistently high
3. Verify Elasticsearch cluster health — a degraded cluster slows down indexing
4. Check KAFKA_AUTHEXCEPTIONRETRYINTERVAL — frequent auth failures with long retry intervals can cause delays

---

Related resources

Audit Log

Learn about the audit logging system and how to view audit trails

Elasticsearch Indexing

Configure Elasticsearch indexing for process data