This change adds support for installation of other formula dependencies
during run of salt.minion state. Due to this, other states are able to
run successfuly on first run because their modules' dependencies are
already satisfied and loaded.

Change-Id: I6e3149b808be641125b82c650f98cd09f79dd631

Change-Id: I77a710ad93a3cc46b721a5b3f1e45603151b0761

PROD-16375

Change-Id: I758d38701cdf0e1cc6fa99c38192de2f9052468c

Change-Id: Idfe764a5b8106e02614eb3eab78895629a6fc570

Change-Id: I1d09c5297440a6469cfaabbfc67d1f1c11decece

Exposing CA keys in a mine creates a security flaw, thus such
should be avoided.
This change removes code responsible for putting and retrieving
CA key from a mine and changes the ca.sls state to allow configuring
where CA cert and its key would be generated as well as their owners.

Fixes PROD-13439

Change-Id: I6d78b13dcb3754c51606edd7e2d8158e128244a4

Issues:
* cmd.wait doesn't work correctly with bg=True
* cmd.wait will be deprecated in next releases [1]
* watch/watch_in work differently than onchanges/onchanges_in as some
* module override mod_watch()

This patch does:
* Replaces cmd.wait to cmd.run with onchanges.
* On any config file changes salt_minion_service_restart will be
  triggered so salt master will get state immediately because of bg=True
* salt_minion_service is used only for service enablement

[1] https://docs.saltstack.com/en/latest/ref/states/all/salt.states.cmd.html

Change-Id: I7b87b3614708b861e1767566426c7a67c337ba01
Signed-off-by: Sergii Golovatiuk <sgolovatiuk@mirantis.com>

In cases  when a service whants to generate and sign a certificate
it requires a CA key along with a CA cert itself.
For example, Octavia needs it for signing a certificate it generates
for a newly spawned amphora.

This change add sending a CA key to the mine from where it can be
extracted in the cert.sls state.
Also allow managing permissions for a CA cert and key retrieved
from the mine.

Related PROD: PROD-11933

Change-Id: I911effb4a63ae048e348ed04b7aca33998e359aa

In case of trust_salt_ca usage, the salt.minion.cert state
generates broken certs body due to a space replacing.

Change-Id: Id49e42807ddbc2addaf59a4c4541b720bbf87527

Currently the CI job fails with module not found errror.
AttributeError: 'module' object has no attribute 'check_refresh

The reason is new version of salt tries to call
salt.utils.pkg.check_refresh which is not available in 2016.11.3 but is
available in 2016.11.6.
Reference:
1. https://github.com/saltstack/salt/blob/v2016.11.3/salt/states/pkg.py
2. https://github.com/saltstack/salt/blob/v2016.11.6/salt/states/pkg.py#L1819

The fix proposed here is to use pkg.installed instead of pkg.latest.
It's not a good idea to always update salt whenever the salt state is
run, this may introduce failures unknown to the user. There is a pipeline
to update packages which should be used for updating to latest.
pkg.latest generally isn't a good way to write idempotent formulas. See
official salt docs: "Generally it is better for the installed function
to be used, as latest will update the package whenever a new package is
available."

Change-Id: I8da5c36c1613e54768993080f2514afc920c49f8

1. Remove implicit creation of "local_trusted_symlink".
   To install a system-wide certificates the linux.system.cert
   state or 'trusted_ca_minion' option must be used.

2. A ca-cert file may exist on a file-system and
   not be pulled from mine. So, in this case
   the following state be incorrect:

   - watch:
     - x509: ca_file

   To support this case, we need to replace `watch` statement
   with `watch_in`.

Change-Id: If41d050b56913d72da1ef7981f30780fec5d6d95

Change-Id: Ie6cf0a8361524b594b60119b25f6d336b6797ece

This reverts commit 6b2a592dc9.

1:
In case of trust_salt_ca usage, the salt.minion.cert state
generates broken certs body due to a space replacing:

-----BEGINCERTIFICATE-----
  MIIFzzCCA7egAwIBAgIITiyuuFgl1S4wDQYJKoZIhvcNAQELBQAwSjELMAkGA1UE
  BhMCY3oxFzAVBgNVBAMMDlNhbHQgT
  ....

To fix it the "x509.pem_managed" is used.

2:
If a symlink to CA cert is already exists, then the state is failing.
The force=True (replace) is used now to avoid the issue.

Change-Id: I4a2bd7c882e179560657a3dc7edf18c7e5835492

Change-Id: I73a4a14734651f6483fae169a2290e9c5b377d76

Change-Id: I7b4dc2001e07b047964f4ebbbbe0b23db7819566

Change-Id: Ie1f1397817b47f299107e4f44dfb4c5ffe1c010c

Change-Id: I6f1292779858c45f9cf6f4caf3657ee000b2cf06

Change-Id: I1e1269268a81d8cd01b5fe9328f63e8bf85e941b

We can have this failing because of bad mine data:
salt['mine.get'](cert.host, 'x509.get_pem_entries')

Without this change, dependency between salt_minion_cert_*_all and
ca_file is just ignored and salt_minion_cert_*_all state fails because
it can't find appropriate file.

Change-Id: I2a5dd12e08159bf110ff0d9879ebf0ad5d9d97c1

Change-Id: I2aba1213b1dda46aee929b8ea583c41316e3eb0b

It was failing with:
	Rendering SLS 'base:salt.minion.cert' failed: Conflicting ID 'salt_ca_certificates_packages'

ca-certificates installation should be probably moved out of the loop in
the future.

Change-Id: I26aeae62cc1c1d407d36d1d6bf101db073d9e601

Change-Id: I89cec95e87db52fd59a84d57c485d8c938711ef3

Unfortunately this is not idempotent, however we surely want to sync
everything when salt.minion state is executed.

Change-Id: I0faaf606b57dbd7d009156abfe50d2e5f350190e

Change-Id: I1eaa9cf7a7f861cc5de604e03fba8a74436d99ad

This error occures when multiple certs signed by same CA are requested
on minion.

Change-Id: I6b20ab4e1795298c94f55fdc61af99f933d8491c

First run is made during salt-master cloud-init and thus it is onchanges
is not suitable here because ca.crt file is already generated.

It was responsible for

    [CRITICAL] Rendering SLS 'base:salt.minion.cert' failed: Jinja variable 'dict object' has no attribute 'cfg01.mk20-lab-advanced.local'
    [ERROR   ] Data passed to highstate outputter is not a valid highstate return: {'local': ["Rendering SLS 'base:salt.minion.cert' failed: Jinja variable 'dict object' has no attribute 'cfg01.mk20-lab-advanced.local'"]}

- allow defining custom key/cert path
- ensure key/cert directories
- set key/cert permissions by metadata

source:engine metadata created - defaults to pkg
installation, pip installation alternative added