Alright! So Database Engine is nothing but a library that takes care of the disk storage and CRUD operations, like create, update and delete on the actual disk. This can be as simple as a key-value store, or as rich and complex as full support ACID and transactions with foreign keys.
When you insert some values into a table, there are a lot of layers and the lowest layer comprises of taking your data and flushing it to a disk, whether it’s an SSD or a normal hard drive (spinning wheel, right?). Most of the databases have merged this with the management system (DBMS), but recently this has started being divided into two parts.
- Actual work to store on the disk.
- Features that get built on top of the engine like server, replication, isolation, store procedures, etc.
So if you want to write a new database, you don’t have to start from scratch, just pick up an engine and then write your own database on top of it. You can design your DB to fit a single good use case rather than using a general-purpose database out there. For example, you’re storing logs, which requires better “writes” than “reads” as you’ll rarely look at them. Sometimes database engine is also referred to as Storage Engine.
Some DBMS gives you the flexibility to switch engines like MySQL & MariaDB whereas some come with a built-in engine that you can’t change like Postgres. There are pros and cons to each, so you need to understand the use case and then make a decision.
Here’s a link to Uber’s Engineering blog on why they shifted from Postgres to MySQL: