Table of Contents
- N_KAFKA
- Public Attributes
- Public Attributes Documentation
- variable tosend_dir
- variable topics
- variable topic
- variable subscription
- variable sended_dir
- variable schema_id
- variable rwlock
- variable received_events
- variable rd_kafka_topic
- variable rd_kafka_handle
- variable rd_kafka_conf
- variable polling_thread_status
- variable polling_thread
- variable poll_timeout
- variable poll_interval
- variable nb_waiting
- variable nb_queued
- variable nb_error
- variable monitored_directory_interval
- variable mode
- variable is_transactional
- variable groupid
- variable events_to_send
- variable event_production_enabled
- variable event_consumption_enabled
- variable event_cmd
- variable errstr
- variable error_timeout
- variable configuration
- variable bootstrap_servers
N_KAFKA
Module: KAFKA: generic event producer and consumer functions
structure of a KAFKA consumer or producer handle
#include "n_kafka.h"
Public Attributes
| Name | |
|---|---|
| char * | tosend_dir base directory the produced event files are read from ('to-send'). |
| char ** | topics list of topics to subscribe to |
| char * | topic kafka topic string |
| rd_kafka_topic_partition_list_t * | subscription subscribed topics |
| char * | sended_dir base directory produced event files are moved to once acknowledged ('sended'). |
| int | schema_id kafka schema id in network order |
| pthread_rwlock_t | rwlock access lock |
| LIST * | received_events list of received N_KAFKA_EVENT |
| rd_kafka_topic_t * | rd_kafka_topic kafka topic handle |
| rd_kafka_t * | rd_kafka_handle kafka handle (producer or consumer) |
| rd_kafka_conf_t * | rd_kafka_conf kafka structure handle |
| int | polling_thread_status polling thread status, 0 => off , 1 => on , 2 => wants to stop, will be turned out to 0 by exiting polling thread |
| pthread_t | polling_thread polling thread id |
| int32_t | poll_timeout poll timeout in msecs |
| int32_t | poll_interval poll interval in msecs |
| size_t | nb_waiting number of events waiting for an ack in the waiting list |
| size_t | nb_queued number of waiting events in the producer waiting list |
| size_t | nb_error number of errored events |
| int32_t | monitored_directory_interval monitored directory refresh interval in msecs |
| int | mode kafka handle mode: RD_KAFKA_CONSUMER or RD_KAFKA_PRODUCER |
| int | is_transactional TRUE when a 'transactional.id' is set in the producer config: produces are then wrapped in kafka transactions. |
| char * | groupid consumer group id |
| LIST * | events_to_send list of N_KAFKA_EVENT to send |
| int32_t | event_production_enabled bool flag to suspend or restart event production |
| int32_t | event_consumption_enabled bool flag to suspend or restart event consumption |
| char * | event_cmd eventual custom event_cmd |
| N_STR * | errstr kafka error string holder |
| int32_t | error_timeout retry interval in seconds for events stuck in N_KAFKA_EVENT_ERROR. |
| cJSON * | configuration kafka json configuration holder |
| char * | bootstrap_servers kafka bootstrap servers string |
Public Attributes Documentation
variable tosend_dir
char * tosend_dir;
base directory the produced event files are read from ('to-send').
Used together with sended_dir to compute the destination subpath when moving a file on ack. NULL keeps the legacy behaviour.
variable topics
char ** topics;
list of topics to subscribe to
variable topic
char * topic;
kafka topic string
variable subscription
rd_kafka_topic_partition_list_t * subscription;
subscribed topics
variable sended_dir
char * sended_dir;
base directory produced event files are moved to once acknowledged ('sended').
When NULL, acknowledged files are deleted (legacy behaviour) instead of being moved.
variable schema_id
int schema_id;
kafka schema id in network order
variable rwlock
pthread_rwlock_t rwlock;
access lock
variable received_events
LIST * received_events;
list of received N_KAFKA_EVENT
variable rd_kafka_topic
rd_kafka_topic_t * rd_kafka_topic;
kafka topic handle
variable rd_kafka_handle
rd_kafka_t * rd_kafka_handle;
kafka handle (producer or consumer)
variable rd_kafka_conf
rd_kafka_conf_t * rd_kafka_conf;
kafka structure handle
variable polling_thread_status
int polling_thread_status;
polling thread status, 0 => off , 1 => on , 2 => wants to stop, will be turned out to 0 by exiting polling thread
variable polling_thread
pthread_t polling_thread;
polling thread id
variable poll_timeout
int32_t poll_timeout;
poll timeout in msecs
variable poll_interval
int32_t poll_interval;
poll interval in msecs
variable nb_waiting
size_t nb_waiting;
number of events waiting for an ack in the waiting list
variable nb_queued
size_t nb_queued;
number of waiting events in the producer waiting list
variable nb_error
size_t nb_error;
number of errored events
variable monitored_directory_interval
int32_t monitored_directory_interval;
monitored directory refresh interval in msecs
variable mode
int mode;
kafka handle mode: RD_KAFKA_CONSUMER or RD_KAFKA_PRODUCER
variable is_transactional
int is_transactional;
TRUE when a 'transactional.id' is set in the producer config: produces are then wrapped in kafka transactions.
FALSE keeps the legacy fire-and-forget produce path.
variable groupid
char * groupid;
consumer group id
variable events_to_send
LIST * events_to_send;
list of N_KAFKA_EVENT to send
variable event_production_enabled
int32_t event_production_enabled;
bool flag to suspend or restart event production
variable event_consumption_enabled
int32_t event_consumption_enabled;
bool flag to suspend or restart event consumption
variable event_cmd
char * event_cmd;
eventual custom event_cmd
variable errstr
N_STR * errstr;
kafka error string holder
variable error_timeout
int32_t error_timeout;
retry interval in seconds for events stuck in N_KAFKA_EVENT_ERROR.
0 (default) disables retries, errored events stay errored until the caller dumps and reloads them.
variable configuration
cJSON * configuration;
kafka json configuration holder
variable bootstrap_servers
char * bootstrap_servers;
kafka bootstrap servers string
Modules
- AABB3D
- ACCEPT_POOL
- ALLEGRO4
- ALLEGRO5
- ANIM_LIB
- ASTAR
- ATLAS_LIB
- AVRO
- CLOCK_SYNC
- COMMONS
- CONFIGFILE
- CYPHER_BASE64
- CYPHER_VIGENERE
- DEAD_RECKONING
- ENUMS
- EXCEPTIONS
- GAME_ENV
- GIT
- GUI
- HASH_TABLE
- ISOMETRIC_ENGINE
- LIST
- LOG
- LOGNODUP
- LZ4
- NETWORKING
- NETWORK_MSG
- N_COOKIES
- N_DIFF
- N_DIGEST
- N_DISK_CACHE
- N_DNS
- N_ECC
- N_ENTROPY
- N_FILES
- N_FLUIDS
- N_GUI_COLOR
- N_GUI_COLORWHEEL
- N_GUI_CURVE
- N_GUI_GRID
- N_GUI_POPUP
- N_HEX
- N_HTTP3
- N_HTTP_COOKIE
- N_HTTP_FORM
- N_HTTP_MIME
- N_HTTP_REQ
- N_HTTP_RESP
- N_HTTP_ROUTER
- N_HTTP_SERVER
- N_ICC
- N_JSON
- N_KAFKA
- N_LINALG
- N_MODULE
- N_OPENSSL_COMPAT
- N_PRETTY
- N_PTH
- N_QUERY
- N_RANDOM
- N_SHA256
- N_STR
- N_TIFF
- N_TIME
- N_USER
- N_X509
- N_ZIP
- PARTICLES
- PCRE
- PHYSICS
- SIGNALS
- STACK
- THREADS
- TRAJECTORY
- TREE
- ZLIB