HQbird: Advanced FirebirdSQL

HQbird is a distribution of the Firebird DBMS for enterprises, which includes additional functions that mainly improve performance for large and highly loaded databases, and a set of tools for organizing a full cycle of database maintenance without a DBA (including tools for performance optimization, monitoring, local and cloud backups, and recovery in case of failures).
In practice, HQbird accelerates the work of large databases (from 50 GB to 2 TB) and allows companies to organize work with large databases in such a way that they do not need to hire a dedicated database administrator.
The minimal hardware requirements for HQbird are 8Gb RAM and 4 cores. The first version of HQbird was released in 2015, the current version is HQbird 2024.

You can use HQbird with existing Firebird databases without any change in the database structure or metadata, without backup/restore, triggers, etc. You can switch between HQbird and vanilla (standard Firebird) at any time, without any changes or backup/restore, too.

Download trial version of HQbird now.

Download HQbird   Purchase HQbird

To put it simply, HQbird is an enterprise version of the open-source Firebird DBMS. In keeping with the tradition of open-source projects, we call Firebird the “vanilla” version: in the same way, there is a “vanilla” version of PostgreSQL and commercial versions of EnterpriseDB, PostgresPro, etc.

HQbird is not a “different” database in terms of compatibility with Firebird: there is no need to make a backup-restore when switching between HQbird and Firebird, no need to rewrite SQLs or change client applications.

Without any problems, you can install HQbird and the vanilla version of Firebird in parallel on the same server, work with the database file using HQbird, then switch to the vanilla Firebird, and vice versa.

HQbird's 100% compatibility with vanilla Firebird is the most important feature of HQbird!

Almost all functions that are developed for HQbird end up in Firebird within 1-2 versions: for example, replication appeared in HQbird for version 2.5, and it appeared in vanilla version 4.0, external connection pooling was developed in HQbird 3.0, and appeared in Firebird 4.0, multi-threaded backup, restore, sweep capabilities appeared starting with HQbird 2.5, and became available in Firebird 5.0, etc.

In addition to the new functionality, bugs fixed in HQbird are also fixed in the corresponding versions of vanilla Firebird.

Also, IBSurgeon Software provides public testing of Firebird and HQbird in terms of reliability and performance: the testing results are published on the website www.firebirdtest.com.

Here you can find the list of HQbird features

Please download the complete HQbird 2024 Description with detailed descriptions for more details.

# Features V5.0 V4.0 V3.0 V2.5 Level
1 High-performance native replication X X X X server
2 Replacing queries “in the air” X X X X server
3 Plugins to access ODBC/MySQL through Execute Statement On External X X X   server
4 Caching BLOBs in temp space X X X X server
5 Improvements in optimizer for JOINs and large sortings X X X   server
6 Cache of compiled statements X X X   server
7 Streaming/Change Data Capture (plugins for Kafka, JSON, etc) X X X   plugin
8 Full Text Search X X X   plugin
9 Multi-thread backup/restore/sweep, creation of indices X X X X server
10 Parallel Reading of consistent data X X X X server
11 Pool of External Connections X X X X server
12 Encryption X X X   plugin
13 Automatic correction of firebird.conf (DefaultDbCachePages) X X X X server
14 Advanced Monitoring of Performance (trace, MON, locks, CPU, RAM, frequency) X X X X tools
15 Monitoring of queries with large sortings X X X X server
16 Manage (and configure replication) many databases at once with command-line tools X X X X tools
17 Backups, Restore, and Automatic Backup/Restore X X X X tools
18 Transfer backups, segments, etc through FTP/sockets/Amazon S3 X X X X tools
19 Advanced database maintenance: proper garbage collection and more X X X X tools
20 Multi-instance support X X X X tools
21 Silent installation on Windows and Linux X X X X tools
22 Tool to analyze database statistics X X X X tools
23 Tool to analyze connections/transactions/memory consumption/IO operations X X X X tools
24 Recovery tools X X X X tools
25 Optimized configurations X X X X tools

Read detailed description of HQbird features here.

How can HQbird reduce costs for software development companies?

Let's see how HQbird can recoup its cost, and in some cases, even create a new source of revenue.

There are 3 main advantages of HQbird for enterprises:

  1. Speed: Speeding up Firebird databases
  2. Protection: various types of backups, backup to the cloud, replication (including “mirror” in the cloud), data protection from theft (using encryption)
  3. Mass maintenance and monitoring (1 admin per 1000 servers).
Click to open each section below and read details:

HQbird is faster than vanilla Firebird due to

  • a) better configuration,
  • b) improved core performance, features such as external connection pooling, blob optimizations, multi-threaded parallel operations, and other features,
  • c) optimizer improvements (including optional ones),
  • d) Eliminating the problem of accumulation of redundant and garbage versions of records through proper database maintenance,
  • e) Functionality extensions (streaming, functions for working with blobs, etc.).

Replica! The easy way to speed up the system is to create a read-only replica (mirror) of the main database and transfer there all reports and interactions with third-party applications (export to BI applications, for example).

HQbird provides tooling for organizing a native replica (which transmits DDL, has no problems with generators and, most importantly, unlike replication on triggers, has very high performance).

By simply installing HQbird instead of vanilla Firebird, without modifications to the application, many users immediately notice improved performance.

An even greater effect can be obtained if you use HQbird's advanced monitoring capabilities, identify problematic queries, configurations and applications, and fix them (with new indices, database structure’s optimization, configuration workarounds, and SQL optimization).

We (IBSurgeon) also offer a performance optimization service as an additional service.

The performance improvement is especially noticeable for large databases that have reached some of the bottlenecks and therefore cannot use the powerful hardware/VM effectively.

How does investing in database acceleration pay off?

  1. Increased customer satisfaction due to improved applications performance. Customers are not annoyed anymore by freezes and slowness of applications, they are stop losing money due to delays, and, therefore, stop planning to leave for competitors.
  2. Savings on purchasing hardware or renting virtual machines in the cloud. Direct savings on improving the underlying performance of the DBMS are especially noticeable in the case of VM rental, when instead of 32 cores, you can rent a VM with 16 cores.
  3. Savings for developing a new version of the application on a different, supposedly faster DBMS. Unfortunately, many application developers, having encountered a performance problem, begin to try to find a solution by migrating to another DBMS, far from exhausting the capabilities of Firebird, which is mistakenly considered as a “database for small projects”. HQbird is much cheaper, faster and easier solution than migration to other DBMS.

Many clients have lost their data due to server failures, problems with virtual machines and, especially, due to encryption viruses (ransomware). HQbird has all necessary backup solutions.

HQbird also offers solutions to the problems of physical theft of database files and unauthorized access to the database (for example, directly access data using development tools).

Backups in HQbird

HQbird implements several types of backups, based on gbak (multi-threaded verified record-level online backup) and incremental backup with nbackup (physical page-level online backup), with the ability to rotate backups.

Incremental backup has 3 options: a dump (a quick online copy of the database file, suitable for a combination with VM level backups and deduplication backup software), a simple nbackup schedule (within a week), and an extended schedule (CRON).

HQbird analyzes backup logs to ensure the success of the backup and optionally performs a test restore.

HQbird also can compress backup copies (including native compression since version 3.0) and can encrypt compressed copies.

Backups and Replicas in the cloud

HQbird can send backups to FTP or Amazon S3 (requires a plugin, available upon request).

The replication mechanism (starting from version 2.5 in HQbird) allows you to organize a replica copy of the database in the cloud within 30 minutes if you use ready-made images, for example, Digital Ocean, which already has a ready-made VM image with HQbird.

The publishing delay in this scenario will be around 40-60 seconds.

Database encryption

HQbird, starting from version 3.0, optionally supports database encryption in AES256 format.

The performance degradation of encrypted is small, depending on the Firebird settings and the performance of the server or virtual machine: with adequate settings 4-5%, in the worst scenario up to 20%.

HQbird also supports encrypted database backups.

To access the database through tools that do not support key transfer, there is a mechanism in HQbird to load an encryption key through the client library fbclient.dll.

For client applications on Windows without the source code, it is possible to enter a password to access an encrypted database without modifying the application - the dialog will be launched in the client library.

How does investing in protection against failures and data theft pay off?

A number of software development companies have implemented, using the backup functionality of HQbird, the additional service “Backup in the Cloud”, which they offer to their clients for a small additional fee.

In general, having a backup outside the office (in the cloud or in another office) is now the standard for all office applications.

However, it would be a big mistake to use file backup software and virtual machine backup software for database backups (any database, not just Firebird), since backing up database files requires a special approach.

A large number of clients use database encryption to comply with GDPR requirements.

Also, it is often a good idea to have an additional protection against theft of the database file and unauthorized access to the database.

HQbird Control Center

The HQbird Unlimited Subscription license includes the HQbird Control Center, a tool that centralizes monitoring data for all HQbird installations and allows you to see the status of hundreds of servers on one page.

By centralizing information about errors (backups, database corruption, lack of space, etc.) and performance issues, a development company can significantly reduce the cost of monitoring and maintaining a large number of servers, since even a single technical support employee can monitor a very large number of servers.

With HQbird Control Center, you do not need to connect to the console of each server to assess the situation, that is, whether the server or database requires any intervention, what warnings HQbird generated and what is recorded in firebird.log and replication.log, whether the backup was completed successfully completed, and whether there was any problem with free space or database corruption.

Bulk installation and update of HQbird

Installing and updating software on a large number of servers is always a difficult task, so the HQbird installer supports a “silent” installation mode and a “silent” update mode.

In the next version of HQbird (v2024), it will be also possible to mass convert between major versions: for example, simultaneously migrate from version 2.5 to 5.0 on hundreds of servers (of course, some preparation is required, since proper migration is not only a backup-restore).

Automatic maintenance

Firebird has always been known as a database management system that does not require an administrator. However, when the database grows to a size of 100-500-1000GB, standard maintenance tools are not good enough, and if you still want to save on a dedicated administrator, then you need to use HQbird.

HQbird implements the proper combination of sweep operations and “soft” shutdown of long-running writeable transactions, and allows to avoid frequent database backups/restores. With HQbird it is recommended to do backup/restore no more than once per year.

HQbird also implements a fully automatic backup-restore task, with all necessary precautions.

Speeding up backup and restoring by 4-6 times thanks to multi-threaded operations (available in HQbird from version 2.5, in vanilla Firebird from version 5.0) also reduces maintenance time.

HQbird also can reduce space requirements by compressing backups (starting with HQbird 3.0 and vanilla 4.0, native compression is available with the -zip option).

How does an investment in mass database maintenance pay off?

It's very simple — increase customer satisfaction and reduce technical support costs.

The Firebird Mass Migration Framework is an integrated suite of tools and guidelines designed to automate a substantial portion of the Firebird migration process. It provides a streamlined methodology for automating of Firebird migration workflow.

Utilization Scenarios for the Firebird Mass Migration Framework

Enterprises managing numerous databases across multiple servers and locations face the daunting challenge of complex migrations. The Framework delivers a proven, reliable solution for large-scale migrations, demonstrated in the successful transition of over 600 servers. It enables organizations to expediently migrate all servers in a fraction of the time typically required for manual migration efforts.

See more details here.

Additional protection against failures

  • High Availability. Starting from version HQBird 2.5, you can create a database replica, which can be located either on another server (for example, in the cloud), or locally, in order to create a “hot” backup. In the event of an emergency, data loss will be limited to the last few records and outage can be a few seconds (requires custom implementation).
  • Database recovery tools are included in the distribution: FirstAID, IBBackupSurgeon, IBUndelete

Cloud and Docker

  • HQbird is ready to work in the cloud - all key actions with databases can be carried out through the web console interface, the configuration is optimized for working on a VM.
  • There are ready-made images of HQbird on Digital Ocean (publicly available), SaveInCloud (Brazil) (publicly available), Amazon (available upon request), and a Docker image (available upon request).
  • HQbird has ready to use Docker images (supplied by request).

Streaming: sending changed data to Kafka, RabbitMQ, Mongo and JSON/text files

HQbird, starting from version 3.0, has the ability to asynchronously send table changes to external sinks, for example, RabbitMQ, Mongo, or just text files.

It is enough to configure HQbird by specifying which tables we want to track, which operations we want to send (INSERT/UPDATE/DELETE), and specify the data sink plugin.

This functionality works on top of the changes created by the replication engine, so all changes to all tables are included in it, regardless of the SQL command that caused the changes.

Access from PSQL (stored procedures, triggers, Execute Block) to other DBMS (MySQL, ODBC) through the Execute Statement On External mechanism

HQbird, starting from version 3.0, can access other databases, for example, MySQL, in order to execute SQL queries in an external database. Parameter passing and return data type matching are supported.

HQbird combines all the advantages of Firebird and the functionality needed to work with large databases with a heavy load.

If your company develops Firebird database applications and has more than 20 servers with fairly large databases, contact us.

We will help you build a solid foundation for effective use of the Firebird DBMS for years and terabytes to come.

More details about HQbird: