Parcourir la source

Merge branch 'master' into pr/fix/minion_dependencis

pr/fix/minion_dependencis
Petr Michalec il y a 7 ans
Parent
révision
51ae48f8fa
21 fichiers modifiés avec 368 ajouts et 187 suppressions
  1. +8
    -2
      .kitchen.yml
  2. +1
    -1
      .travis.yml
  3. +93
    -57
      README.rst
  4. +3
    -24
      salt/control/virt.sls
  5. +0
    -47
      salt/files/_pki.conf
  6. +9
    -0
      salt/files/_reactor.conf
  7. +0
    -37
      salt/files/_virt.conf
  8. +2
    -0
      salt/files/master.conf
  9. +0
    -2
      salt/map.jinja
  10. +28
    -1
      salt/master/env.sls
  11. +3
    -0
      salt/master/init.sls
  12. +17
    -0
      salt/master/reactor.sls
  13. +95
    -2
      salt/meta/salt.yml
  14. +0
    -11
      salt/minion/ca.sls
  15. +1
    -1
      salt/minion/cert.sls
  16. +1
    -1
      salt/minion/grains.sls
  17. +42
    -0
      salt/minion/service.sls
  18. +13
    -0
      tests/integration/master-default/formulas_spec.rb
  19. +1
    -1
      tests/pillar/master_acl.sls
  20. +51
    -0
      tests/pillar/master_formulas.sls
  21. +0
    -0
      tests/pillar/master_log_handler.sls

+ 8
- 2
.kitchen.yml Voir le fichier

@@ -15,6 +15,12 @@ provisioner:
formula: salt
grains:
noservices: true
vendor_repo:
- type: apt
url: http://apt-mk.mirantis.com/trusty
key_url: http://apt-mk.mirantis.com/public.gpg
components: salt
distribution: testing
state_top:
base:
"*":
@@ -103,8 +109,8 @@ suites:
- master_ssh_key
- master_ssh_password
- master_ssh_root
- master_handler
- minion_pki_cert
- master_formulas
pillars-from-files:
minion_pki.sls: tests/pillar/minion_pki_ca.sls
minion_pki_cert.sls: tests/pillar/minion_pki_cert.sls
@@ -113,7 +119,7 @@ suites:
master_ssh_key.sls: tests/pillar/master_ssh_minion_key.sls
master_ssh_password.sls: tests/pillar/master_ssh_minion_password.sls
master_ssh_root.sls: tests/pillar/master_ssh_minion_root.sls
master_handler.sls: tests/pillar/master_custom_handler.sls
master_formulas.sls: tests/pillar/master_formulas.sls

- name: control-default
provisioner:

+ 1
- 1
.travis.yml Voir le fichier

@@ -27,7 +27,7 @@ before_script:

script:
- test ! -e .kitchen.yml || bundle exec kitchen converge || true
- test ! -e .kitchen.yml || bundle exec kitchen verify
- test ! -e .kitchen.yml || bundle exec kitchen verify -t tests/integration

notifications:
webhooks:

+ 93
- 57
README.rst Voir le fichier

@@ -1,24 +1,29 @@

====
Salt
====
============
Salt Formula
============

Salt is a new approach to infrastructure management. Easy enough to get running in minutes, scalable enough to manage tens of thousands of servers, and fast enough to communicate with them in seconds.
Salt is a new approach to infrastructure management. Easy enough to get
running in minutes, scalable enough to manage tens of thousands of servers,
and fast enough to communicate with them in seconds.

Salt delivers a dynamic communication bus for infrastructures that can be used for orchestration, remote execution, configuration management and much more.
Salt delivers a dynamic communication bus for infrastructures that can be used
for orchestration, remote execution, configuration management and much more.


Sample Metadata
===============

Sample pillars
==============

Salt master
-----------

Salt master with base production environment and pillar tree as metadata backend
Salt master with base formulas and pillar metadata backend

.. literalinclude:: tests/pillar/master_single_pillar.sls
:language: yaml

Salt master with reclass ENC as metadata backend
Salt master with reclass ENC metadata backend

.. literalinclude:: tests/pillar/master_single_reclass.sls
:language: yaml
@@ -43,7 +48,7 @@ Salt master with preset minions
minions:
- name: 'node1.system.location.domain.com'

Salt master with alternative installation source and version (optional) - pip
Salt master with pip based installation (optional)

.. code-block:: yaml

@@ -55,7 +60,7 @@ Salt master with alternative installation source and version (optional) - pip
engine: pip
version: 2016.3.0rc2

Salt master with specified formula to install through apt-get
Install formula through system package management

.. code-block:: yaml

@@ -65,9 +70,21 @@ Salt master with specified formula to install through apt-get
...
environment:
prd:
keysone:
keystone:
source: pkg
name: salt-formula-keystone
nova:
source: pkg
name: salt-formula-keystone
version: 0.1+0~20160818133412.24~1.gbp6e1ebb
postresql:
source: pkg
name: salt-formula-postgresql
version: purged

Formula keystone is installed latest version and the formulas without version are installed in one call to aptpkg module.
If the version attribute is present sls iterates over formulas and take action to install specific version or remove it.
The version attribute may have these values ``[latest|purged|removed|<VERSION>]``.

Clone master branch of keystone formula as local feature branch

@@ -102,40 +119,7 @@ Salt master with specified formula refs (for example for Gerrit review)
address: https://git.openstack.org/openstack/salt-formula-keystone
revision: refs/changes/56/123456/1

Salt syndic: Master of masters

.. code-block:: yaml

salt:
master:
enabled: true
order_masters: True

Salt syndic: Lower master

.. code-block:: yaml

salt:
syndic:
enabled: true
master:
host: master-of-master-host
timeout: 5

Salt syndic: Lower master with multi-master of masters

.. code-block:: yaml

salt:
syndic:
enabled: true
masters:
- host: master-of-master-host1
- host: master-of-master-host2
timeout: 5


Salt master with custom handlers
Salt master with logging handlers

.. code-block:: yaml

@@ -161,7 +145,7 @@ Salt master with custom handlers
host: 127.0.0.1
port: 9999

Salt master peer for remote certificate sign.
Salt master peer setup for remote certificate signing

.. code-block:: yaml

@@ -171,8 +155,63 @@ Salt master peer for remote certificate sign.
".*":
- x509.sign_remote_certificate

Salt proxy
----------
Configure verbosity of state output (used for `salt` command)

.. code-block:: yaml

salt:
master:
state_output: changes

Salt Reactor system configuration

.. code-block:: yaml

salt:
master:
reactor:
salt/minion/*/start:
- salt://reactor/minion-started.sls


Salt syndic
-----------

The master of masters

.. code-block:: yaml

salt:
master:
enabled: true
order_masters: True

Lower syndicated master

.. code-block:: yaml

salt:
syndic:
enabled: true
master:
host: master-of-master-host
timeout: 5

Syndicated master with multiple master of masters

.. code-block:: yaml

salt:
syndic:
enabled: true
masters:
- host: master-of-master-host1
- host: master-of-master-host2
timeout: 5


Salt-minion proxy
-----------------

Salt proxy pillar

@@ -267,7 +306,7 @@ Salt minion with graphing dependencies
.. literalinclude:: tests/pillar/minion_graph.sls
:language: yaml

Salt minion behind http proxy
Salt minion behind HTTP proxy

.. code-block:: yaml

@@ -277,15 +316,12 @@ Salt minion behind http proxy
host: 127.0.0.1
port: 3128

PKI CA
~~~~~~

Salt minion with PKI CA
Salt minion with PKI certificate authority (CA)

.. literalinclude:: tests/pillar/minion_pki_ca.sls
:language: yaml

Salt minion with PKI certificate
Salt minion using PKI certificate

.. literalinclude:: tests/pillar/minion_pki_cert.sls
:language: yaml
@@ -326,8 +362,8 @@ Debug LIBCLOUD for salt-cloud connection
export LIBCLOUD_DEBUG=/dev/stderr; salt-cloud --list-sizes provider_name --log-level all


Read more
=========
More Information
================

* http://salt.readthedocs.org/en/latest/
* https://github.com/DanielBryan/salt-state-graph

+ 3
- 24
salt/control/virt.sls Voir le fichier

@@ -8,34 +8,13 @@ salt_control_virt_packages:
pkg.installed:
- names: {{ control.virt_pkgs }}

{% if grains.oscodename == 'trusty' %}
{#- This tool is not available in newer releases #}
update-guestfs-appliance:
cmd.wait:
- watch:
- pkg: salt_control_virt_packages

{#
{%- for package in control.virt_pips %}

{{ package }}:
pip.installed:
- require:
- pkg: salt_control_virt_packages

{%- endfor %}
#}

/etc/salt/minion.d/_virt.conf:
file.managed:
- source: salt://salt/files/_virt.conf
- user: root
- group: root
- template: jinja
- require:
- pkg: salt_control_virt_packages
{%- if not grains.get('noservices', False) %}
- watch_in:
- service: salt_minion_service
{%- endif %}
{%- endif %}

{%- for cluster_name, cluster in control.cluster.iteritems() %}


+ 0
- 47
salt/files/_pki.conf Voir le fichier

@@ -1,47 +0,0 @@
{%- from "salt/map.jinja" import minion with context %}

x509_signing_policies:
{%- for ca_name,ca in minion.ca.items() %}
{%- for signing_policy_name, signing_policy in ca.signing_policy.iteritems() %}
{{ ca_name }}_{{ signing_policy_name }}:
- minions: '{{ signing_policy.minions }}'
- signing_private_key: /etc/pki/ca/{{ ca_name }}/ca.key
- signing_cert: /etc/pki/ca/{{ ca_name }}/ca.crt
{%- if ca.country is defined %}
- C: {{ ca.country }}
{%- endif %}
{%- if ca.state is defined %}
- ST: {{ ca.state }}
{%- endif %}
{%- if ca.locality is defined %}
- L: {{ ca.locality }}
{%- endif %}
{%- if ca.organization is defined %}
- O: {{ ca.organization }}
{%- endif %}
{%- if ca.organization_unit is defined %}
- OU: {{ ca.organization_unit }}
{%- endif %}
{%- if signing_policy.type == 'v3_edge_cert_client' %}
- basicConstraints: "CA:FALSE"
- keyUsage: "critical digitalSignature,nonRepudiation,keyEncipherment"
- extendedKeyUsage: "critical clientAuth"
{%- elif signing_policy.type == 'v3_edge_cert_server' %}
- basicConstraints: "CA:FALSE"
- keyUsage: "critical digitalSignature,nonRepudiation,keyEncipherment"
- extendedKeyUsage: "critical,serverAuth"
{%- elif signing_policy.type == 'v3_intermediate_ca' %}
- basicConstraints: "CA:TRUE"
- keyUsage: "critical cRLSign,keyCertSign"
{%- elif signing_policy.type == 'v3_edge_ca' %}
- basicConstraints: "CA:TRUE,pathlen:0"
- keyUsage: "critical cRLSign,keyCertSign"
{%- elif signing_policy.type == 'v3_edge_cert_open' %}
- basicConstraints: "CA:FALSE"
{%- endif %}
- subjectKeyIdentifier: hash
- authorityKeyIdentifier: keyid,issuer:always
- days_valid: {{ ca.days_valid.certificate }}
- copypath: /etc/pki/ca/{{ ca_name }}/certs/
{%- endfor %}
{%- endfor %}

+ 9
- 0
salt/files/_reactor.conf Voir le fichier

@@ -0,0 +1,9 @@
{% from "salt/map.jinja" import master with context %}

reactor:
{%- for event, reactors in master.reactor.items() %}
- {{ event }}:
{%- for reactor in reactors %}
- {{ reactor }}
{%- endfor %}
{%- endfor %}

+ 0
- 37
salt/files/_virt.conf Voir le fichier

@@ -1,37 +0,0 @@
{% from "salt/map.jinja" import control with context %}

{%- if control.net_profile is defined or control.disk_profile is defined %}
virt:
{%- if control.net_profile is defined %}
nic:
{%- for item_name, item in control.net_profile.iteritems() %}
{{ item_name }}:
{%- for iface_name, iface in item.iteritems() %}
{{ iface_name }}:
{%- if iface.bridge is defined %}
bridge: {{ iface.bridge }}
{%- endif %}
{%- if iface.network is defined %}
network: {{ iface.network }}
{%- endif %}
{%- if iface.model is defined %}
model: {{ iface.model }}
{%- endif %}
{%- endfor %}
{%- endfor %}
{%- endif %}
{%- if control.disk_profile is defined %}
disk:
{%- for item_name, item in control.disk_profile.iteritems() %}
{{ item_name }}:
{%- for disk_name, disk in item.iteritems() %}
- {{ disk }}:
{%- if disk.size is defined %}
size: {{ disk.size }}
{%- endif %}
{%- endfor %}
{%- endfor %}
{%- endif %}
{%- endif %}

virt.images: /var/lib/libvirt/images

+ 2
- 0
salt/files/master.conf Voir le fichier

@@ -4,6 +4,8 @@
worker_threads: {{ master.worker_threads }}
timeout: {{ master.command_timeout }}

state_output: {{ master.get('state_output', 'changes') }}

{%- if master.system is defined %}

file_roots:

+ 0
- 2
salt/map.jinja Voir le fichier

@@ -89,8 +89,6 @@ default:
Arch:
pkgs:
- salt-zmq
grains_validity_pkgs:
- python-yaml
Debian:
pkgs:
- salt-minion

+ 28
- 1
salt/master/env.sls Voir le fichier

@@ -136,20 +136,47 @@ salt_env_{{ environment_name }}_dirs:
{%- endif %}

{%- set _formula_pkgs = [] %}
{%- set _formula_pkgs_with_version = [] %}
{%- for formula_name, formula in environment.get('formula', {}).iteritems() %}
{%- if formula.source == 'pkg' %}
{%- if formula.version is defined %}
{%- do _formula_pkgs_with_version.append(formula) %}
{%- else %}
{%- do _formula_pkgs.append(formula.name) %}
{%- endif %}
{%- endif %}
{%- endfor %}

{% if _formula_pkgs|length > 1 %}

salt_master_{{ environment_name }}_pkg_formulas:
pkg.latest:
- pkgs:
{%- for pkg in _formula_pkgs %}
- {{ pkg }}
{%- endfor %}
- refresh: True
- cache_valid_time: 300
{% endif %}

{% if _formula_pkgs_with_version|length > 1 %}
{%- for formula in _formula_pkgs_with_version %}
salt_master_{{ environment_name }}_pkg_formula_{{ formula.name }}:
{%- if formula.version == 'latest' %}
pkg.latest:
- refresh: True
- cache_valid_time: 300
{%- elif formula.version == 'purged' %}
pkg.purged:
{%- elif formula.version == 'removed' %}
pkg.removed:
{%- else %}
pkg.installed:
- version: {{ formula.version }}
- refresh: True
- cache_valid_time: 300
{% endif %}
- name: {{ formula.name }}
{%- endfor %}

{% endif %}


+ 3
- 0
salt/master/init.sls Voir le fichier

@@ -1,5 +1,8 @@
include:
- salt.master.service
{%- if pillar.salt.master.reactor is defined %}
- salt.master.reactor
{%- endif %}
- salt.master.env
- salt.master.pillar
- salt.master.minion

+ 17
- 0
salt/master/reactor.sls Voir le fichier

@@ -0,0 +1,17 @@
{%- from "salt/map.jinja" import master with context %}
{%- if master.enabled %}

include:
- salt.master.service

/etc/salt/master.d/_reactor.conf:
file.managed:
- source: salt://salt/files/_reactor.conf
- user: root
- template: jinja
- require:
- {{ master.install_state }}
- watch_in:
- service: salt_master_service

{%- endif %}

+ 95
- 2
salt/meta/salt.yml Voir le fichier

@@ -1,10 +1,103 @@
orchestrate:
master:
priority: 60
minion:
minion:
priority: 70
control:
control:
priority: 400
require:
- salt: salt.master

minion:
{%- if pillar.get('salt', {}).get('minion', {}).get('ca') %}
pki:
{%- from "salt/map.jinja" import minion with context %}
x509_signing_policies:
{%- for ca_name,ca in minion.ca.items() %}
{%- for signing_policy_name, signing_policy in ca.signing_policy.iteritems() %}
{{ ca_name }}_{{ signing_policy_name }}:
- minions: '{{ signing_policy.minions }}'
- signing_private_key: /etc/pki/ca/{{ ca_name }}/ca.key
- signing_cert: /etc/pki/ca/{{ ca_name }}/ca.crt
{%- if ca.country is defined %}
- C: {{ ca.country }}
{%- endif %}
{%- if ca.state is defined %}
- ST: {{ ca.state }}
{%- endif %}
{%- if ca.locality is defined %}
- L: {{ ca.locality }}
{%- endif %}
{%- if ca.organization is defined %}
- O: {{ ca.organization }}
{%- endif %}
{%- if ca.organization_unit is defined %}
- OU: {{ ca.organization_unit }}
{%- endif %}
{%- if signing_policy.type == 'v3_edge_cert_client' %}
- basicConstraints: "CA:FALSE"
- keyUsage: "critical digitalSignature,nonRepudiation,keyEncipherment"
- extendedKeyUsage: "critical clientAuth"
{%- elif signing_policy.type == 'v3_edge_cert_server' %}
- basicConstraints: "CA:FALSE"
- keyUsage: "critical digitalSignature,nonRepudiation,keyEncipherment"
- extendedKeyUsage: "critical,serverAuth"
{%- elif signing_policy.type == 'v3_intermediate_ca' %}
- basicConstraints: "CA:TRUE"
- keyUsage: "critical cRLSign,keyCertSign"
{%- elif signing_policy.type == 'v3_edge_ca' %}
- basicConstraints: "CA:TRUE,pathlen:0"
- keyUsage: "critical cRLSign,keyCertSign"
{%- elif signing_policy.type == 'v3_edge_cert_open' %}
- basicConstraints: "CA:FALSE"
{%- endif %}
- subjectKeyIdentifier: hash
- authorityKeyIdentifier: keyid,issuer:always
- days_valid: {{ ca.days_valid.certificate }}
- copypath: /etc/pki/ca/{{ ca_name }}/certs/
{%- endfor %}
{%- endfor %}
{%- endif %}

{%- if pillar.salt.control is defined and pillar.salt.control.virt_enabled is defined %}
virt:
{% from "salt/map.jinja" import control with context %}
{%- if control.net_profile is defined or control.disk_profile is defined %}
virt:
{%- if control.net_profile is defined %}
nic:
{%- for item_name, item in control.net_profile.iteritems() %}
{{ item_name }}:
{%- for iface_name, iface in item.iteritems() %}
{{ iface_name }}:
{%- if iface.bridge is defined %}
bridge: {{ iface.bridge }}
{%- endif %}
{%- if iface.network is defined %}
network: {{ iface.network }}
{%- endif %}
{%- if iface.model is defined %}
model: {{ iface.model }}
{%- endif %}
{%- endfor %}
{%- endfor %}
{%- endif %}
{%- if control.disk_profile is defined %}
disk:
{%- for item_name, item in control.disk_profile.iteritems() %}
{{ item_name }}:
{%- for disk_name, disk in item.iteritems() %}
- {{ disk }}:
{%- if disk.size is defined %}
size: {{ disk.size }}
{%- endif %}
{%- endfor %}
{%- endfor %}
{%- endif %}
{%- endif %}
virt.images: /var/lib/libvirt/images
{%- endif %}

{#-
vim: syntax=jinja
-#}

+ 0
- 11
salt/minion/ca.sls Voir le fichier

@@ -4,17 +4,6 @@
include:
- salt.minion.service

/etc/salt/minion.d/_pki.conf:
file.managed:
- source: salt://salt/files/_pki.conf
- template: jinja
- require:
- {{ minion.install_state }}
{%- if not grains.get('noservices', False) %}
- watch_in:
- service: salt_minion_service
{%- endif %}

{%- for ca_name,ca in minion.ca.iteritems() %}

/etc/pki/ca/{{ ca_name }}/certs:

+ 1
- 1
salt/minion/cert.sls Voir le fichier

@@ -192,7 +192,7 @@ salt_cert_{{ cacerts_dir }}/{{ cacert_file }}:
file.managed:
- name: {{ cacerts_dir }}/{{ cacert_file }}
- contents: |
{{ ca_cert | indent(8) }}
{{ ca_cert|replace(' ', '')|indent(8) }}
- makedirs: True
- show_changes: True
- follow_symlinks: True

+ 1
- 1
salt/minion/grains.sls Voir le fichier

@@ -27,7 +27,7 @@ salt_minion_grains_files:
{%- macro load_support_file() %}{% include support_fragment_file ignore missing %}{% endmacro %}
{%- set support_yaml = load_support_file()|load_yaml %}

{%- if support_yaml %}
{%- if support_yaml and support_yaml.get('grain', {}) %}
{%- for name, grain in support_yaml.get('grain', {}).iteritems() %}
salt_minion_grain_{{ service_name }}_{{ name }}:
file.managed:

+ 42
- 0
salt/minion/service.sls Voir le fichier

@@ -39,6 +39,38 @@ salt_minion_dependency_packages:
- service: salt_minion_service
{%- endif %}

{%- for service_name, service in pillar.items() %}
{%- set support_fragment_file = service_name+'/meta/salt.yml' %}
{%- macro load_support_file() %}{% include support_fragment_file ignore missing %}{% endmacro %}
{%- set support_yaml = load_support_file()|load_yaml %}

{%- if support_yaml and support_yaml.get('minion', {}) %}
{%- for name, conf in support_yaml.get('minion', {}).iteritems() %}
salt_minion_config_{{ service_name }}_{{ name }}:
file.managed:
- name: /etc/salt/minion.d/_{{ name }}.conf
- contents: |
{{ conf|yaml(False)|indent(8) }}
{%- if not grains.get('noservices', False) %}
- watch_in:
- cmd: salt_minion_service_restart
{%- endif %}
- require:
- {{ minion.install_state }}

salt_minion_config_{{ service_name }}_{{ name }}_validity_check:
cmd.wait:
- name: python -c "import yaml; stream = file('/etc/salt/minion.d/_{{ name }}.conf', 'r'); yaml.load(stream); stream.close()"
- watch:
- file: salt_minion_config_{{ service_name }}_{{ name }}
{%- if not grains.get('noservices', False) %}
- require_in:
- cmd: salt_minion_service_restart
{%- endif %}
{%- endfor %}
{%- endif %}
{%- endfor %}

{%- if not grains.get('noservices', False) %}
salt_minion_service:
service.running:
@@ -47,6 +79,16 @@ salt_minion_service:
- require:
- pkg: salt_minion_packages
- pkg: salt_minion_dependency_packages

{#- Restart salt-minion if needed but after all states are executed #}
salt_minion_service_restart:
cmd.wait:
- name: 'while true; do salt-call saltutil.running|grep fun: && continue; salt-call --local service.restart {{ minion.service }}; break; done'
- shell: /bin/bash
- bg: true
- require:
- service: salt_minion_service

{%- endif %}

salt_minion_sync_all:

+ 13
- 0
tests/integration/master-default/formulas_spec.rb Voir le fichier

@@ -0,0 +1,13 @@

%w(keepalived haproxy libvirt ntp openssh).each do |f|
describe package("salt-formula-#{f}") do
it { should be_installed }
end
end

%w(mysql postgresql).each do |f|
describe package("salt-formula-#{f}") do
it { should_not be_installed }
end
end


+ 1
- 1
tests/pillar/master_acl.sls Voir le fichier

@@ -7,7 +7,7 @@ linux:
salt:
master:
command_timeout: 5
worker_threads: 2
worker_threads: 3
enabled: true
source:
engine: pkg

+ 51
- 0
tests/pillar/master_formulas.sls Voir le fichier

@@ -0,0 +1,51 @@
git:
client:
enabled: true
linux:
system:
enabled: true
salt:
master:
enabled: true
pillar:
source:
engine: local
environment:
prd:
formula:
keepalived:
source: pkg
name: salt-formula-keepalived
haproxy:
source: pkg
name: salt-formula-haproxy
libvirt:
source: pkg
name: salt-formula-libvirt
version: latest
ntp:
source: pkg
name: salt-formula-ntp
version: latest
openssh:
source: pkg
name: salt-formula-openssh
version: latest
mysql:
source: pkg
name: salt-formula-mysql
version: purged
postgresql:
source: pkg
name: salt-formula-postgresql
version: removed
dev:
formula:
aptly:
source: git
address: 'https://github.com/salt-formulas/salt-formula-aptly.git'
revision: master
bind:
source: git
address: 'https://github.com/salt-formulas/salt-formula-bind.git'
revision: master

tests/pillar/master_custom_handler.sls → tests/pillar/master_log_handler.sls Voir le fichier


Chargement…
Annuler
Enregistrer