5 Good Habits of Embedded Database Developers

Is data management important for you? Do you need to store, manage and distribute data and make your system communicate with other embedded systems and smart devices? The fast-paced world of IT is always in motion. Every day, more and more businesses – from the smallest to Fortune 500 corporations – realize the need for systems equipped with a strong database. This helps them have data available when and where it is needed, whether for a near real-time transactions or long-term storage.

Author: Sasan Montaseri, ITTIA, www.ittia.com

When it comes to embedded systems and mobile devices, developers face a very important choice between flat files and an embedded database, as any future change can be very risky and expensive. However, to meet deadlines and satisfy hardware restrictions, many developers build custom solutions from scratch, preferring a quick, simple implementation to a scalable one. As the product line evolves, these solutions can become very complex and difficult to reconcile with new requirements.

Flat files – whether in text or binary format – have been widely customized for data management in devices’ built-in applications. However, the text file solution comes with number of limitations, such as:

• Although easy to write and create, even the slightest change requires the entire file to be rewritten, which in turn makes files susceptible to data loss.
• The extra wear caused by repeated erasing and writing of a large text file to flash memory will eventually wear out and thus reduce the total lifetime of flash media.
• Since the entire file must be read into memory to efficiently search for data, they cannot be very large because of the time required to write changes.
• Flat files also have a limited data management life cycle offering, as devices and embedded systems are becoming more aware of each other and other systems.

Common problems with binary files are:

• Binary files can be divided into smaller pieces used independently, but special tools must be developed to use these files outside the application.
• Binary files provide some protection against corruption of the entire file, but can still lose data when changes are only partially written before an unexpected power failure.
• Binary files make it more difficult to store variable-width data.
• If data is copied directly from memory to a binary flat file, it is not easy to open the file on another architecture.

Since flat files are not the best option, the next question is to learn what an embedded database is at all, and what is the difference between an embedded database and an enterprise database.

To put it in a nutshell, an embedded database is a software library used by application developers to store data. The library adds database features such as transaction logging, scalable index algorithms, and isolated concurrency to the application. Unlike enterprise databases, an embedded database is distributed with the application and is not installed separately by end-users. Embedded databases are especially well-suited for special-purpose devices and embedded systems with limited resources and a dedicated user interface.

Modern embedded devices are now responsible for storing more data than ever before. Selecting an available off-the-shelf database is a good alternative that allows developers to focus on the application’s business logic and leave data management to solutions available in the market.

Generally, developers of embedded systems and mobile devices have the habit of taking five major points into consideration when it comes to choosing a database. Their chosen database should:

1. be embedded directly in the application, completely hiding the database from end-user and greatly simplifying management and maintenance. Also, it should have a code footprint less than a megabyte to easily fit on an embedded device and be customized to satisfy strict footprint requirements.
2. be developed from the ground up to recognize and solve issues with data management on modern embedded systems and devices, rather than having been developed as a consulting project for a single application and repackaged for use in other applications later on.
3. be designed as cross-platform RDBMS using the standard relational data model to ensure that careful attention has been paid to algorithm selection and memory utilization in order to accommodate footprint restrictions.
4. store data in a portable format – whether it be in a local file, or shared across a network – that can be accessed directly using table cursors or with SQL queries.
5. be modular, allowing features such as shared access and even the SQL engine to be omitted to reduce footprint, and must come with dynamic schema alteration to allow applications to add and remove tables and columns to a live database.

This provides the application all the features of a relational database (RDBMS) without the overhead and complexity of a traditional database management system. Some of these features can be described as:

•  Relational Database Functionality
•  Small Footprint
•  Linked Library
•  Lower Price & TCO
• Cross-platform Portability
•  Shorter Time to Market
•  Shorter Sales Cycle

Benefits include:

• Data can be continuously synchronized without interrupting normal use.
• Use of transaction recovery to both protect against corruption and ensure that data is completely consistent after a power failure.
• Using a database gives an application a consistent architecture for persistent data storage, making it easy to add new features and migrate the application code to a new environment.
• Database is a good long-term investment because it lets applications scale through the entire life cycle of the application.
  Sending and receiving data and running custom queries is greatly simplified by the database API.
• Database-driven applications use row-level locking to interact with other systems and share data without interrupting normal usage.
• An embedded database that understands the structure of data can automatically perform the necessary conversions when a database is shared between devices, either by copying the database file itself or using network communications to access the database.

Final Thoughts

Embedded database gives software developers the edge they need to deploy sooner, cut costs, and differentiate themselves from the competition. Every application that manipulates information must face at least one of the problems addressed by database: reliable persistent storage, high-performance search, and safe data sharing. An embedded database library, like ITTIA DB SQL, is an out-of-the-box solution to all of these common data storage problems. Database simplifies application development, and rather than building separate solutions for each of these problems, the application developer uses a standard framework for accessing and modifying data. Furthermore, reconciling these features with each other in a single application is not trivial and can become a great distraction to developing the business logic of the device.

With a database supporting standard features like SQL, the application can offer features that would otherwise be impractical to develop. Database makes it possible to rapidly develop feature-rich applications that will be able to scale to meet the demands of the future.