Guest Post: Relational Databases - the Exception to DevOps Rules

PGConf US, the largest official gathering of the PostgreSQL open source community, will be held this year at the New York Marriott, Brooklyn Bridge, from April 18 - 20 (http://www.pgconf.us). Alongside the conference, the Postgres community is also celebrating the database’s 20th birthday with a party open to the broader New York tech community, on Tuesday, April 19, starting at 6.30pm at the Brookyn Bridge Marriott: http://www.meetup.com/postgresql-3/events/229101241/

Scott Mead (shown) is a Sr. Architect and founding member at the Open Source Consulting Group (OpenSCG) and will be Speaking about “PostgreSQL DevOps with Chef & Puppet” at the upcoming conferenence. We reached out to him for a guest post on his speaking topic, and you'll find it here.

 Relational Databases: The Exception to DevOps Rules

By Scott Mead

The use of configuration management tools like Chef and Puppet is changing the way we think about system deployment. They allow us to start new cloud instances on demand or create new test environments as required, but how should relational databases be handled?

There is a ton of buzz around NoSQL databases. While many people now claim that the relational database is dead, they have been around for decades because they solve a real business need.

Relational databases are different than other pieces of your infrastructure,  because of how they solved that business need. Your web servers, cache servers, and queue servers are all essentially stateless where you can just add more nodes and start leveraging them, but your relational database holds that state. It is holding your sales orders, your inventory, your financials and more.

Configuration management tools are capable of rolling out a full database deployment, but it requires a significant amount of time to create the cookbook or manifest to fully configure the database. Some of the tasks needed to roll out a production database are:

1.  
   1. Install dependencies
   2. Create user accounts
   3. Setup the storage
   4. Install the database
   5. Create a backup schedule
   6. Configure high availability
   7. Tune the database
   8. Secure the database
   9. Install the database schema
   10. Load the data

When thinking about configuration management, it is important to note that many of these tasks are frequently different from database to database: the database backing your customer facing website has very different requirements from your internal accounting system which is very different from your data warehouse.

You may have development or test environments for your databases, but unlike your web and application servers, you will only have a single instance of your accounting database. Fully rolling out your database using configuration management is a lot of overhead for something you may only do once.

Configuration management should be used to enforce best practices and corporate policies across your relational databases. Tools like Chef or Puppet are ideally suited to ensure all database servers have the binaries installed in the same directory structure and running with the latest patches installed. They can also be used to conform to corporate security policies. Unlike tasks like installing your database schema, these types of task should be mostly consistent across all databases in your environment and your scripts would be leveraged many times.

Other tasks such as configuring high availability are very specific to a single environment. To conserve budget, you are only likely to configure your production database for your customer facing website for 99.999% uptime. Whereas your development environment may not have any high availability configured at all. Writing the necessary code to configure high availability is complex and once you are in production, you may never need to run it again. Tasks like these may never get the ROI to warrant using a configuration management tool when strong documentation will do.

When planning out your DevOps strategy and thinking about your configuration management tool, plan on your relational database being the exception in your infrastructure. Configuration management should be used for your database environment, but fully automating its deployment is often a wasted effort. That special service is unique for a very good reason.

About Scott Mead:

Scott Mead is a Sr. Architect and founding member at the Open Source Consulting Group (OpenSCG) and will be Speaking about “PostgreSQL DevOps with Chef & Puppet” at the upcoming PostgreSQL Conferenence in New York: http://www.pgconf.us/2016/event/141/postgresql-devops-with-chef-puppet/

Scott has 12 years of experience with the PostgreSQL database engine including, migrations to / from commercial and open-source RDBMS, application development, large-scale (500+ node) database administration as well as code contributions to the PostgreSQL core engine.