Lightweight monitoring

← HQbird server features

A new feature — lightweight monitoring — based on a trace plugin that updates monitoring counters in real time. Counters live in shared memory, so the feature works efficiently in Classic Server. Counter updates are not tied to locks, so they stay cheap. Shared memory defaults to 4 MB and does not grow after initialization. That keeps counter access mostly lock-free. You can change the size with the plugin MemorySize parameter (see below).

A small set of counters is supported today. Monitoring covers two kinds of objects: databases and attachments. At database level there is a separate counter set for finished attachments, so cumulative stats for a database are kept.

Service API

To read monitoring data, HQbird adds a new service:

/* Lightweight monitoring */
#define isc_action_svc_lwmonitoring  32

The service has two parameters. The first selects the operation. Today there is only “execute query”:

#define isc_spb_lwm_query  1

Four query kinds are implemented (below). Pass the query number (1–4) as the integer value of isc_spb_lwm_query.

The second parameter — database name — is optional and uses the usual isc_spb_dbname tag. If set, the query is limited to that database; otherwise data for all currently known databases is returned.

Response format

All queries return text, one line per element. An element may describe a database, an attachment, or an aggregate. Lines are JSON for easy parsing and future extension.

Every response starts with a header line:

json

{"idQuery" : 1, "tsQuery" : "2024-04-17T00:17:58" }

idQuery: query number (ID)
tsQuery: query start time (server local time, ISO-8601)

Query 1 — summary

Summary for known databases and attachments. Always one line:

json

{"cntDb" : 1, "activeAtt" : 1, "goneAtt" : 0}

cntDb: number of databases
activeAtt: number of active attachments
goneAtt: number of finished attachments

Query 2 — database list

One or more lines:

json

{"dbName" : "SECURITY2.FDB", "activeAtt" : 1, "goneAtt" : 2}

dbName: database file name
activeAtt: number of active attachments
goneAtt: number of finished attachments

Query 3 — per-database connection summary

One or more lines:

json

{"dbName" : "SECURITY2.FDB", "activeAtt" : 1, "goneAtt" : 0, "doneTran" : 1, "activeTran" : 0, "prepReq" : 0, "execReq" : 0, "activeReq" : 0}

dbName: database file name
activeAtt: active attachments
goneAtt: finished attachments
doneTran: completed transactions
activeTran: active transactions
prepReq: prepared statements
execReq: executed statements
activeReq: active statements

Additional counters:

rowsFetched: rows returned to the application
execTime: statement execution time, ms
errors: errors returned to the application
fetches: page fetches
reads: page reads
marks: marks (before a page is modified)
writes: page writes

Query 4 — attachments detail

Several lines of two kinds:

json

{"dbName" : "SECURITY2.FDB", "activeAtt" : 1, "goneAtt" : 0}

json

{"idAtt" : 0, "pidAtt" : 0, "doneTran" : 0, "activeTran" : 0, "prepReq" : 0, "execReq" : 0, "activeReq" : 0}

json

{"idAtt" : 13925, "pidAtt" : 11188, "doneTran" : 1, "activeTran" : 0, "prepReq" : 0, "execReq" : 0, "activeReq" : 0}

Type (a) lines describe databases; type (b) lines describe attachments to that database. After each (a) line there is at least one (b) line for that database.

idAtt: attachment id, or 0 for aggregate of finished attachments
pidAtt: server process id that serves this attachment

Errors

On failure the service either raises a status-vector exception or returns a JSON error line:

json

{"errorMsg" : "Error text"}

Example:

json

{"errorMsg" : "Unknown query: 5"}

Command examples

Local service manager:

Command

fbsvcmgr service_mgr action_lwmonitoring lwm_query 1

Remote host, query 4 for one database:

Command

fbsvcmgr host:service_mgr action_lwmonitoring lwm_query 4 dbname employee

Configuration

firebird.conf has a parameter for the lightweight monitoring plugin name:

ini

MonitoringPlugin =

By default the plugin is unset and monitoring is off. The HQbird plugin name is fblwmon:

ini

MonitoringPlugin = fblwmon

The plugin ships with HQbird in the plugins directory.

Plugin options are set in the usual way (see README.plugins.html), for example in plugins.conf:

ini

Plugin = fblwmon
{
  Config = lwmConfig
}

Config = lwmConfig
{
  # default 4MB
  MemorySize = 1M
}

MemorySize — size of the plugin shared memory in bytes (or with a suffix such as 1M).

Attachment limit

On top of lightweight monitoring, HQbird can limit concurrent attachments per database. The limit value is global for all databases (HQbird 2.5 has no per-database setting). Two integer parameters in firebird.conf:

ini

WarnCountAttachments = 0
MaxCountAttachments = 0

The check runs at connect time and does not count the attachment being created. It uses the active-attachment counter for that database:

if counter ≥ WarnCountAttachments and < MaxCountAttachments, the connection is allowed and firebird.log gets:

text

Database XXX: number of attachments is above warning level (N >= W)

if counter ≥ MaxCountAttachments, the connection is refused. The client sees:

text

Statement failed, SQLSTATE = HY000
Number of attachments exceeds allowed maximum

and firebird.log gets:

text

Database XXX: number of attachments is above maximum level (N >= M)

Not checked: connections to security.db, service connections, and connections with isc_dpb_no_db_triggers in the DPB.

WarnCountAttachments = 0 — no warnings
MaxCountAttachments = 0 — no hard limit

The limit requires the lightweight monitoring plugin to be installed and configured (see above).