Chapter 2. Hiera

The history of Puppet is an interesting example of how best practices have evolved with time, following new usage patterns and contributions from the community.

Once people started to write manifests and express the desired state of their systems with Puppet's DSL, they found themselves placing custom variables and parameters that expressed various resources of their infrastructures (IP addresses, hostnames, paths, URLs, names, properties, lists of objects, and so on) inside the code used to create the needed resource types.

At times, variables were used to classify and categorize nodes (systems' roles, operational environments, and so on); at other times, facts (such as $::operatingsystem) were used to provide resources with the right names and paths according to the underlying OS.

Variables could be defined in different places; they could be set via an ENC, inside node declarations, or inside classes.

There wasn't (and actually, still there isn't) any strict rule on how and where the user's data could be placed, but the general outcome was that we often found ourselves having our custom data defined inside our manifests.

Now, in my very personal and definitely non-orthodox opinion, this is not necessarily or inherently a bad thing; seeing the data we provide when we define our resources gives us a clearer visibility on how things are done and doesn't compel us to look in different places in order to understand what our code is doing.

Nevertheless, such an approach may be fit for relatively simple setups where we don't need to cope with large chunks of data that might come from different sources and change a lot according to different factors.

Also, we might need to have different people working on Puppet who write the code and design its logic and who need to apply configurations, mostly dealing with data.

More generally, the concept of separating data from code is a well-established and sane development practice, which also makes sense in the Puppet world.

The person who faced this issue in the most resolutive way is R.I. Pienaar. First, he developed the extlookup function (included in Puppet core for a long time), which allows users to read data from external CSV files. Then, he took a further step and developed Hiera, a key-value lookup tool, where data used by our manifests can be placed and evaluated differently according to a custom hierarchy from different data sources.

One of the greatest features of Hiera is its modular pluggable design that allows the usage of different backends that may retrieve data from different sources, such as YAML or JSON files, Puppet classes, MySQL, Redis, REST services, and more.

In this chapter, we will cover the following topics: