Extending pillar data in saltstack

If you've worked with pillars in saltstack, you'll have undoubtedly encounter the issue of being unable to extend data in pillars in different files.

This is easily done in states using include and extend but unfortunately, those do not exist for pillars (or not the way you think they do).

This was a problem for me when I wanted services to define iptables rules they need set.

Let's create a problem for us. Let's say you have the following pillar:

pillar/users/init.sls:

users:  
  - bob
  - martha

{% if grains['role'] == 'webserver' %}
include:  
  - users.webusers
{% endif %}

pillar/users/webusers.sls:

users:  
  - webadmin

If you check the pillar data of some webserver, for example: salt webserver1 pillar.items you'll see this:

router1:  
    ----------
    users:
        ----------
        - webadmin
    ...

Not the ideal outcome. Fortunately there is a workaround and it is how I solved the above problem for configuring iptables for NFP.

A workaround to the above problem would be as follow:

pillar/users/init.sls:

users.default:  
  - bob
  - martha

{% if grains['role'] == 'webserver' %}
include:  
  - users.webusers
{% endif %}

pillar/users/webusers.sls:

users.webusers:  
  - webadmin

Running salt webserver1 pillar.items will give you this:

router1:  
    ----------
    users.default:
        ----------
        - bob
        - martha
    users.webusers:
        ----------
        - webadmin
    ...

But how do we loop through them in a salt state? Simple, we loop through pillar and search for items that begin with 'users.' like so:

salt/users/init.sls:

{% for value in pillar %}
{% if 'users.' in value %}
{% for user in pillar[value] %}
{{user}}:
  user:
    - present:
{% endfor%}
{% endif %}
{% endfor %}

Kinda ugly but it works. The upside is that now any pillar can define users that it requres. So the following pillar for nginx:

pillar/nginx/init.sls:

#something to do with nginx

users.nginx:  
  - nginx

Would define a new user for nginx that would be automatically created by the above state. Now each service pillar can define the user it requres. This could also be used to define iptables rules for each service inside each service's pillar data.

You can see the above solution being used to configure iptables here:

Pillar: https://github.com/nfp-projects/salt-pillars/tree/master/iptables
State: https://github.com/nfp-projects/salt-states/tree/master/iptables

Show Comments