HQbird: Advanced FirebirdSQL

HQbird is not a “different” database in terms of compatibility with Firebird: there is no need to make a backup-restore when switching between HQbird and Firebird, no need to rewrite SQLs or change client applications.

Without any problems, you can install HQbird and the vanilla version of Firebird in parallel on the same server, work with the database file using HQbird, then switch to the vanilla Firebird, and vice versa.

HQbird's 100% compatibility with vanilla Firebird is the most important feature of HQbird!

Almost all functions that are developed for HQbird end up in Firebird within 1-2 versions: for example, replication appeared in HQbird for version 2.5, and it appeared in vanilla version 4.0, external connection pooling was developed in HQbird 3.0, and appeared in Firebird 4.0, multi-threaded backup, restore, sweep capabilities appeared starting with HQbird 2.5, and became available in Firebird 5.0, etc.

In addition to the new functionality, bugs fixed in HQbird are also fixed in the corresponding versions of vanilla Firebird.

Also, IBSurgeon Software provides public testing of Firebird and HQbird in terms of reliability and performance: the testing results are published on the website www.firebirdtest.com.

HQbird is faster than vanilla Firebird due to

• a) better configuration,
• b) improved core performance, features such as external connection pooling, blob optimizations, multi-threaded parallel operations, and other features,
• c) optimizer improvements (including optional ones),
• d) Eliminating the problem of accumulation of redundant and garbage versions of records through proper database maintenance,
• e) Functionality extensions (streaming, functions for working with blobs, etc.).

Replica! The easy way to speed up the system is to create a read-only replica (mirror) of the main database and transfer there all reports and interactions with third-party applications (export to BI applications, for example).

HQbird provides tooling for organizing a native replica (which transmits DDL, has no problems with generators and, most importantly, unlike replication on triggers, has very high performance).

By simply installing HQbird instead of vanilla Firebird, without modifications to the application, many users immediately notice improved performance.

An even greater effect can be obtained if you use HQbird's advanced monitoring capabilities, identify problematic queries, configurations and applications, and fix them (with new indices, database structure’s optimization, configuration workarounds, and SQL optimization).

We (IBSurgeon) also offer a performance optimization service as an additional service.

The performance improvement is especially noticeable for large databases that have reached some of the bottlenecks and therefore cannot use the powerful hardware/VM effectively.

How does investing in database acceleration pay off?

1. Increased customer satisfaction due to improved applications performance. Customers are not annoyed anymore by freezes and slowness of applications, they are stop losing money due to delays, and, therefore, stop planning to leave for competitors.
2. Savings on purchasing hardware or renting virtual machines in the cloud. Direct savings on improving the underlying performance of the DBMS are especially noticeable in the case of VM rental, when instead of 32 cores, you can rent a VM with 16 cores.
3. Savings for developing a new version of the application on a different, supposedly faster DBMS. Unfortunately, many application developers, having encountered a performance problem, begin to try to find a solution by migrating to another DBMS, far from exhausting the capabilities of Firebird, which is mistakenly considered as a “database for small projects”. HQbird is much cheaper, faster and easier solution than migration to other DBMS.

Many clients have lost their data due to server failures, problems with virtual machines and, especially, due to encryption viruses (ransomware). HQbird has all necessary backup solutions.

HQbird also offers solutions to the problems of physical theft of database files and unauthorized access to the database (for example, directly access data using development tools).

Backups in HQbird

HQbird implements several types of backups, based on gbak (multi-threaded verified record-level online backup) and incremental backup with nbackup (physical page-level online backup), with the ability to rotate backups.

Incremental backup has 3 options: a dump (a quick online copy of the database file, suitable for a combination with VM level backups and deduplication backup software), a simple nbackup schedule (within a week), and an extended schedule (CRON).

HQbird analyzes backup logs to ensure the success of the backup and optionally performs a test restore.

HQbird also can compress backup copies (including native compression since version 3.0) and can encrypt compressed copies.

Backups and Replicas in the cloud

HQbird can send backups to FTP or Amazon S3 (requires a plugin, available upon request).

The replication mechanism (starting from version 2.5 in HQbird) allows you to organize a replica copy of the database in the cloud within 30 minutes if you use ready-made images, for example, Digital Ocean, which already has a ready-made VM image with HQbird.

The publishing delay in this scenario will be around 40-60 seconds.

Database encryption

HQbird, starting from version 3.0, optionally supports database encryption in AES256 format.

The performance degradation of encrypted is small, depending on the Firebird settings and the performance of the server or virtual machine: with adequate settings 4-5%, in the worst scenario up to 20%.

HQbird also supports encrypted database backups.

To access the database through tools that do not support key transfer, there is a mechanism in HQbird to load an encryption key through the client library fbclient.dll.

For client applications on Windows without the source code, it is possible to enter a password to access an encrypted database without modifying the application - the dialog will be launched in the client library.

How does investing in protection against failures and data theft pay off?

A number of software development companies have implemented, using the backup functionality of HQbird, the additional service “Backup in the Cloud”, which they offer to their clients for a small additional fee.

In general, having a backup outside the office (in the cloud or in another office) is now the standard for all office applications.

However, it would be a big mistake to use file backup software and virtual machine backup software for database backups (any database, not just Firebird), since backing up database files requires a special approach.

A large number of clients use database encryption to comply with GDPR requirements.

Also, it is often a good idea to have an additional protection against theft of the database file and unauthorized access to the database.

HQbird Control Center

The HQbird Unlimited Subscription license includes the HQbird Control Center, a tool that centralizes monitoring data for all HQbird installations and allows you to see the status of hundreds of servers on one page.

By centralizing information about errors (backups, database corruption, lack of space, etc.) and performance issues, a development company can significantly reduce the cost of monitoring and maintaining a large number of servers, since even a single technical support employee can monitor a very large number of servers.

With HQbird Control Center, you do not need to connect to the console of each server to assess the situation, that is, whether the server or database requires any intervention, what warnings HQbird generated and what is recorded in firebird.log and replication.log, whether the backup was completed successfully completed, and whether there was any problem with free space or database corruption.

Bulk installation and update of HQbird

Installing and updating software on a large number of servers is always a difficult task, so the HQbird installer supports a “silent” installation mode and a “silent” update mode.

In the next version of HQbird (v2024), it will be also possible to mass convert between major versions: for example, simultaneously migrate from version 2.5 to 5.0 on hundreds of servers (of course, some preparation is required, since proper migration is not only a backup-restore).

Automatic maintenance

Firebird has always been known as a database management system that does not require an administrator. However, when the database grows to a size of 100-500-1000GB, standard maintenance tools are not good enough, and if you still want to save on a dedicated administrator, then you need to use HQbird.

HQbird implements the proper combination of sweep operations and “soft” shutdown of long-running writeable transactions, and allows to avoid frequent database backups/restores. With HQbird it is recommended to do backup/restore no more than once per year.

HQbird also implements a fully automatic backup-restore task, with all necessary precautions.

Speeding up backup and restoring by 4-6 times thanks to multi-threaded operations (available in HQbird from version 2.5, in vanilla Firebird from version 5.0) also reduces maintenance time.

HQbird also can reduce space requirements by compressing backups (starting with HQbird 3.0 and vanilla 4.0, native compression is available with the -zip option).

How does an investment in mass database maintenance pay off?

It's very simple — increase customer satisfaction and reduce technical support costs.

Utilization Scenarios for the Firebird Mass Migration Framework

See more details here.

Additional protection against failures

• High Availability. Starting from version HQBird 2.5, you can create a database replica, which can be located either on another server (for example, in the cloud), or locally, in order to create a “hot” backup. In the event of an emergency, data loss will be limited to the last few records and outage can be a few seconds (requires custom implementation).
• Database recovery tools are included in the distribution: FirstAID, IBBackupSurgeon, IBUndelete

Cloud and Docker

• HQbird is ready to work in the cloud - all key actions with databases can be carried out through the web console interface, the configuration is optimized for working on a VM.
• There are ready-made images of HQbird on Digital Ocean (publicly available), SaveInCloud (Brazil) (publicly available), Amazon (available upon request), and a Docker image (available upon request).
• HQbird has ready to use Docker images (supplied by request).

Streaming: sending changed data to Kafka, RabbitMQ, Mongo and JSON/text files

HQbird, starting from version 3.0, has the ability to asynchronously send table changes to external sinks, for example, RabbitMQ, Mongo, or just text files.

It is enough to configure HQbird by specifying which tables we want to track, which operations we want to send (INSERT/UPDATE/DELETE), and specify the data sink plugin.

This functionality works on top of the changes created by the replication engine, so all changes to all tables are included in it, regardless of the SQL command that caused the changes.

Access from PSQL (stored procedures, triggers, Execute Block) to other DBMS (MySQL, ODBC) through the Execute Statement On External mechanism

HQbird, starting from version 3.0, can access other databases, for example, MySQL, in order to execute SQL queries in an external database. Parameter passing and return data type matching are supported.