hiera-eyaml-2.0.2/0000755000004100000410000000000012333514275013745 5ustar www-datawww-datahiera-eyaml-2.0.2/Rakefile0000644000004100000410000000003412333514275015407 0ustar www-datawww-datarequire "bundler/gem_tasks" hiera-eyaml-2.0.2/Gemfile.lock0000644000004100000410000000201012333514275016160 0ustar www-datawww-dataGEM remote: https://rubygems.org/ specs: CFPropertyList (2.2.7) aruba (0.5.4) childprocess (>= 0.3.6) cucumber (>= 1.1.1) rspec-expectations (>= 2.7.0) builder (3.2.2) childprocess (0.5.3) ffi (~> 1.0, >= 1.0.11) cucumber (1.3.14) builder (>= 2.1.2) diff-lcs (>= 1.1.3) gherkin (~> 2.12) multi_json (>= 1.7.5, < 2.0) multi_test (>= 0.1.1) diff-lcs (1.2.5) facter (2.0.1) CFPropertyList (~> 2.2.6) ffi (1.9.3) gherkin (2.12.2) multi_json (~> 1.3) hiera (1.3.2) json_pure hiera-eyaml-plaintext (0.6) highline (1.6.21) json_pure (1.8.1) multi_json (1.10.0) multi_test (0.1.1) puppet (3.5.1) facter (> 1.6, < 3) hiera (~> 1.0) json_pure rgen (~> 0.6.5) rake (10.3.1) rgen (0.6.6) rspec-expectations (2.14.5) diff-lcs (>= 1.1.3, < 2.0) trollop (2.0) PLATFORMS ruby DEPENDENCIES aruba hiera-eyaml-plaintext highline puppet rake trollop hiera-eyaml-2.0.2/bin/0000755000004100000410000000000012333514275014515 5ustar www-datawww-datahiera-eyaml-2.0.2/bin/eyaml0000755000004100000410000000052112333514275015550 0ustar www-datawww-data#!/usr/bin/env ruby require 'rubygems' require 'hiera/backend/eyaml/CLI' require 'hiera/backend/eyaml/plugins' require 'hiera/backend/eyaml/encryptors/pkcs7' # Register all plugins Hiera::Backend::Eyaml::Encryptors::Pkcs7.register Hiera::Backend::Eyaml::Plugins.find Hiera::Backend::Eyaml::CLI.parse Hiera::Backend::Eyaml::CLI.executehiera-eyaml-2.0.2/Gemfile0000644000004100000410000000026712333514275015245 0ustar www-datawww-datasource 'https://rubygems.org/' gem 'highline' gem 'trollop' group :development do gem "aruba" gem "hiera-eyaml-plaintext" gem "puppet" end group :test do gem "rake" end hiera-eyaml-2.0.2/LICENSE.txt0000644000004100000410000000206712333514275015575 0ustar www-datawww-data The MIT License (MIT) Copyright (c) 2013 Tom Poulton Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. hiera-eyaml-2.0.2/hiera-eyaml.gemspec0000644000004100000410000000160312333514275017507 0ustar www-datawww-data# -*- encoding: utf-8 -*- lib = File.expand_path('../lib', __FILE__) $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib) require 'hiera/backend/eyaml' Gem::Specification.new do |gem| gem.name = "hiera-eyaml" gem.version = Hiera::Backend::Eyaml::VERSION gem.description = "Hiera backend for decrypting encrypted yaml properties" gem.summary = "OpenSSL Encryption backend for Hiera" gem.author = "Tom Poulton" gem.license = "MIT" gem.homepage = "http://github.com/TomPoulton/hiera-eyaml" gem.files = `git ls-files`.split($/).reject { |file| file =~ /^features.*$/ } gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) } gem.test_files = gem.files.grep(%r{^(test|spec|features)/}) gem.require_paths = ["lib"] gem.add_dependency('trollop', '>= 2.0') gem.add_dependency('highline', '>= 1.6.19') end hiera-eyaml-2.0.2/.travis.yml0000644000004100000410000000026012333514275016054 0ustar www-datawww-datalanguage: ruby rvm: - "1.8.7-p374" - "1.9.2" - "1.9.3" before_install: - sudo apt-get update - sudo apt-get install expect script: bundle exec cucumber -f progress hiera-eyaml-2.0.2/tools/0000755000004100000410000000000012333514275015105 5ustar www-datawww-datahiera-eyaml-2.0.2/tools/regem.sh0000755000004100000410000000034612333514275016546 0ustar www-datawww-data#!/bin/bash gem uninstall hiera-eyaml --executables RAKE_OUT=`rake build` echo ${RAKE_OUT} VERSION=`echo ${RAKE_OUT} | awk '{print $2}'` echo Installing version: ${VERSION} ... gem install pkg/hiera-eyaml-${VERSION}.gem eyaml -v hiera-eyaml-2.0.2/lib/0000755000004100000410000000000012333514275014513 5ustar www-datawww-datahiera-eyaml-2.0.2/lib/hiera/0000755000004100000410000000000012333514275015603 5ustar www-datawww-datahiera-eyaml-2.0.2/lib/hiera/backend/0000755000004100000410000000000012333514275017172 5ustar www-datawww-datahiera-eyaml-2.0.2/lib/hiera/backend/eyaml.rb0000644000004100000410000000175212333514275020633 0ustar www-datawww-dataclass Hiera module Backend module Eyaml VERSION = "2.0.2" DESCRIPTION = "Hiera-eyaml is a backend for Hiera which provides OpenSSL encryption/decryption for Hiera properties" class RecoverableError < StandardError end def self.subcommand= command @@subcommand = command end def self.subcommand @@subcommand end def self.default_encryption_scheme= new_encryption @@default_encryption_scheme = new_encryption end def self.default_encryption_scheme @@default_encryption_scheme ||= "PKCS7" @@default_encryption_scheme end def self.verbosity_level= new_verbosity_level @@debug_level = new_verbosity_level end def self.verbosity_level @@debug_level ||= 1 @@debug_level end def self.subcommands= commands @@subcommands = commands end def self.subcommands @@subcommands end end end end hiera-eyaml-2.0.2/lib/hiera/backend/eyaml/0000755000004100000410000000000012333514275020301 5ustar www-datawww-datahiera-eyaml-2.0.2/lib/hiera/backend/eyaml/subcommand.rb0000644000004100000410000001041112333514275022753 0ustar www-datawww-datarequire 'base64' require 'yaml' # require 'hiera/backend/eyaml/subcommands/unknown_command' class Hiera module Backend module Eyaml class Subcommand class << self attr_accessor :global_options, :options, :helptext end @@global_options = [ {:name => :encrypt_method, :description => "Override default encryption and decryption method (default is PKCS7)", :short => 'n', :default => "pkcs7"}, {:name => :version, :description => "Show version information"}, {:name => :verbose, :description => "Be more verbose", :short => 'v'}, {:name => :quiet, :description => "Be less verbose", :short => 'q'}, {:name => :help, :description => "Information on how to use this command", :short => 'h'} ] def self.load_config_file config_file=ENV['EYAML_CONFIG'] || "#{ENV['HOME']}/.eyaml/config.yaml" begin config = YAML.load_file(config_file) Utils::info "Loaded config from #{config_file}" config rescue Errno::ENOENT, IndexError {} end end def self.all_options options = @@global_options.dup options += self.options if self.options options += Plugins.options # merge in defaults from configuration file config_file = self.load_config_file options.map!{ | opt| key_name = "#{opt[:name]}" if config_file.has_key? key_name opt[:default] = config_file[key_name] opt else opt end } options end def self.attach_option opt self.suboptions += opt end def self.find commandname = "unknown_command" begin require "hiera/backend/eyaml/subcommands/#{commandname.downcase}" rescue Exception => e require "hiera/backend/eyaml/subcommands/unknown_command" return Hiera::Backend::Eyaml::Subcommands::UnknownCommand end command_module = Module.const_get('Hiera').const_get('Backend').const_get('Eyaml').const_get('Subcommands') command_class = Utils.find_closest_class :parent_class => command_module, :class_name => commandname command_class || Hiera::Backend::Eyaml::Subcommands::UnknownCommand end def self.parse me = self options = Trollop::options do version "Hiera-eyaml version " + Hiera::Backend::Eyaml::VERSION.to_s banner ["eyaml #{me.prettyname}: #{me.description}", me.helptext, "Options:"].compact.join("\n\n") me.all_options.each do |available_option| skeleton = {:description => "", :short => :none} skeleton.merge! available_option opt skeleton[:name], skeleton[:desc] || skeleton[:description], #legacy plugins :short => skeleton[:short], :default => skeleton[:default], :type => skeleton[:type] end stop_on Eyaml.subcommands end if options[:verbose] Hiera::Backend::Eyaml.verbosity_level += 1 end if options[:quiet] Hiera::Backend::Eyaml.verbosity_level = 0 end if options[:encrypt_method] Hiera::Backend::Eyaml.default_encryption_scheme = options[:encrypt_method] end options end def self.validate args args end def self.description "no description" end def self.helptext "Usage: eyaml #{self.prettyname} [options]" end def self.execute raise StandardError, "This command is not implemented yet (#{self.to_s.split('::').last})" end def self.prettyname Utils.snakecase self.to_s.split('::').last end def self.hidden? false end end end end end hiera-eyaml-2.0.2/lib/hiera/backend/eyaml/utils.rb0000644000004100000410000001225512333514275021773 0ustar www-datawww-datarequire 'highline/import' require 'tempfile' require 'fileutils' class Hiera module Backend module Eyaml class Utils def self.read_password ask("Enter password: ") {|q| q.echo = "*" } end def self.confirm? message result = ask("#{message} (y/N): ") if result.downcase == "y" or result.downcase == "yes" true else false end end def self.camelcase string return string if string !~ /_/ && string =~ /[A-Z]+.*/ string.split('_').map{|e| e.capitalize}.join end def self.snakecase string return string if string !~ /[A-Z]/ string.split(/(?=[A-Z])/).collect {|x| x.downcase}.join("_") end def self.find_editor editor = ENV['EDITOR'] editor ||= %w{ /usr/bin/sensible-editor /usr/bin/editor /usr/bin/vim /usr/bin/vi }.collect {|e| e if FileTest.executable? e}.compact.first raise StandardError, "Editor not found. Please set your EDITOR env variable" if editor.nil? editor end def self.secure_file_delete args file = File.open(args[:file], 'r+') num_bytes = args[:num_bytes] [0xff, 0x55, 0xaa, 0x00].each do |byte| file.seek(0, IO::SEEK_SET) num_bytes.times { file.print(byte.chr) } file.fsync end File.delete args[:file] end def self.write_tempfile data_to_write file = Tempfile.open('eyaml_edit') path = file.path file.close! file = File.open(path, "w") file.puts data_to_write file.close path end def self.write_important_file args filename = args[ :filename ] content = args[ :content ] mode = args[ :mode ] if File.file? "#{filename}" raise StandardError, "User aborted" unless Utils::confirm? "Are you sure you want to overwrite \"#{filename}\"?" end open( "#{filename}", "w" ) do |io| io.write(content) end File.chmod( mode, filename ) unless mode.nil? end def self.ensure_key_dir_exists key_file key_dir = File.dirname key_file unless File.directory? key_dir begin FileUtils.mkdir_p key_dir Utils::info "Created key directory: #{key_dir}" rescue raise StandardError, "Cannot create key directory: #{key_dir}" end end end def self.find_closest_class args parent_class = args[ :parent_class ] class_name = args[ :class_name ] constants = parent_class.constants candidates = [] constants.each do | candidate | candidates << candidate.to_s if candidate.to_s.downcase == class_name.downcase end if candidates.count > 0 parent_class.const_get candidates.first else nil end end def self.require_dir classdir num_class_hierarchy_levels = self.to_s.split("::").count - 1 root_folder = File.dirname(__FILE__) + "/" + Array.new(num_class_hierarchy_levels).fill("..").join("/") class_folder = root_folder + "/" + classdir Dir[File.expand_path("#{class_folder}/*.rb")].uniq.each do |file| # puts "Requiring file: #{file}" require file end end def self.find_all_subclasses_of args parent_class = args[ :parent_class ] constants = parent_class.constants candidates = [] constants.each do | candidate | candidates << candidate.to_s.split('::').last if parent_class.const_get(candidate).class.to_s == "Class" end candidates end def self.hiera? "hiera".eql? Eyaml::Options[:source] end def self.structure_message messageinfo message = {:from => "hiera-eyaml-core"} case messageinfo.class.to_s when 'Hash' message.merge!(messageinfo) else message.merge!({:msg => messageinfo.to_s}) end end def self.warn messageinfo message = self.structure_message messageinfo message = "[#{message[:from]}] !!! #{message[:msg]}" if self.hiera? Hiera.warn message else STDERR.puts message end end def self.info messageinfo message = self.structure_message messageinfo message = "[#{message[:from]}] #{message[:msg]}" if self.hiera? Hiera.debug message if Eyaml.verbosity_level > 0 else STDERR.puts message if Eyaml.verbosity_level > 0 end end def self.debug messageinfo message = self.structure_message messageinfo message = "[#{message[:from]}] #{message[:msg]}" if self.hiera? Hiera.debug message if Eyaml.verbosity_level > 1 else STDERR.puts message if Eyaml.verbosity_level > 1 end end end end end end hiera-eyaml-2.0.2/lib/hiera/backend/eyaml/plugins.rb0000644000004100000410000000320212333514275022304 0ustar www-datawww-datarequire 'rubygems' class Hiera module Backend module Eyaml class Plugins @@plugins = [] @@commands = [] @@options = [] def self.register_options args options = args[ :options ] plugin = args[ :plugin ] options.each do |name, option_hash| new_option = {:name => "#{plugin}_#{name}"} new_option.merge! option_hash @@options << new_option end end def self.options @@options end def self.find this_version = Gem::Version.create(Hiera::Backend::Eyaml::VERSION) index = Gem::VERSION >= "1.8.0" ? Gem::Specification : Gem.source_index [index].flatten.each do |source| specs = Gem::VERSION >= "1.6.0" ? source.latest_specs(true) : source.latest_specs specs.each do |spec| next if @@plugins.include? spec dependency = spec.dependencies.find { |d| d.name == "hiera-eyaml" } next if dependency && !dependency.requirement.satisfied_by?( this_version ) file = nil if Gem::VERSION >= "1.8.0" file = spec.matches_for_glob("**/eyaml_init.rb").first else file = Gem.searcher.matching_files(spec, "**/eyaml_init.rb").first end next unless file @@plugins << spec load file end end @@plugins end def self.plugins @@plugins end def self.commands @@commands end end end end end hiera-eyaml-2.0.2/lib/hiera/backend/eyaml/options.rb0000644000004100000410000000147512333514275022330 0ustar www-datawww-dataclass Hiera module Backend module Eyaml class Options def self.[]= key, value @@options ||= {} @@options[ key.to_sym ] = value end def self.[] key @@options ||= {} @@options[ key.to_sym ] end def self.set hash @@options = {} hash.each do |k, v| @@options[ k.to_sym ] = v end end def self.debug Utils::debug "Dump of eyaml tool options dict:" Utils::debug "--------------------------------" @@options.each do |k, v| Utils::debug sprintf "%18s %-18s = %18s %-18s", "(#{k.class.name})", k.to_s, "(#{v.class.name})", v.to_s end Utils::debug "--------------------------------" end end end end end hiera-eyaml-2.0.2/lib/hiera/backend/eyaml/encryptor.rb0000644000004100000410000000436212333514275022660 0ustar www-datawww-datarequire 'base64' require 'hiera/backend/eyaml/utils' class Hiera module Backend module Eyaml class Encryptor class << self attr_accessor :options attr_accessor :tag end def self.find encryption_scheme = nil encryption_scheme = Eyaml.default_encryption_scheme if encryption_scheme.nil? require "hiera/backend/eyaml/encryptors/#{encryption_scheme.downcase}" encryptor_module = Module.const_get('Hiera').const_get('Backend').const_get('Eyaml').const_get('Encryptors') encryptor_class = Utils.find_closest_class :parent_class => encryptor_module, :class_name => encryption_scheme raise StandardError, "Could not find hiera-eyaml encryptor: #{encryption_scheme}. Try gem install hiera-eyaml-#{encryption_scheme.downcase} ?" if encryptor_class.nil? encryptor_class end def self.encode binary_string Base64.encode64(binary_string).strip end def self.decode string Base64.decode64(string) end def self.encrypt *args raise StandardError, "encrypt() not defined for encryptor plugin: #{self}" end def self.decrypt *args raise StandardError, "decrypt() not defined for decryptor plugin: #{self}" end protected def self.plugin_classname self.to_s.split("::").last.downcase end def self.register Hiera::Backend::Eyaml::Plugins.register_options :options => self.options, :plugin => plugin_classname end def self.option name Eyaml::Options[ "#{plugin_classname}_#{name}" ] || self.options[ "#{plugin_classname}_#{name}" ] end def self.hiera? Utils::hiera? end def self.format_message msg "[eyaml_#{plugin_classname}]: #{msg}" end def self.debug msg Utils::debug :from => plugin_classname, :msg => msg end def self.info msg Utils::info :from => plugin_classname, :msg => msg end def self.warn msg Utils::warn :from => plugin_classname, :msg => msg end end end end end hiera-eyaml-2.0.2/lib/hiera/backend/eyaml/parser/0000755000004100000410000000000012333514275021575 5ustar www-datawww-datahiera-eyaml-2.0.2/lib/hiera/backend/eyaml/parser/parser.rb0000644000004100000410000000476412333514275023431 0ustar www-datawww-datarequire 'strscan' require 'hiera/backend/eyaml/parser/token' require 'hiera/backend/eyaml/parser/encrypted_tokens' class Hiera module Backend module Eyaml module Parser class ParserFactory def self.encrypted_parser enc_string = EncStringTokenType.new() enc_block = EncBlockTokenType.new() Parser.new([enc_string, enc_block]) end def self.decrypted_parser dec_string = DecStringTokenType.new() dec_block = DecBlockTokenType.new() Parser.new([dec_string, dec_block]) end def self.hiera_backend_parser enc_hiera = EncHieraTokenType.new() Parser.new([enc_hiera]) end end class Parser attr_reader :token_types def initialize(token_types) @token_types = token_types end def parse text parse_scanner(StringScanner.new(text)).reverse end def parse_scanner s if s.eos? [] else # Check if the scanner currently matches a regex current_match = @token_types.find { |token_type| s.match?(token_type.regex) } token = if current_match.nil? # No regex matches here. Find the earliest match. next_match_indexes = @token_types.map { |token_type| next_match = s.check_until(token_type.regex) if next_match.nil? nil else next_match.length - s.matched.length end }.reject { |i| i.nil? } non_match_size = if next_match_indexes.length == 0 s.rest_size else next_match_indexes.min end non_match = s.peek(non_match_size) # advance scanner s.pos = s.pos + non_match_size NonMatchToken.new(non_match) else # A regex matches so create a token and do a recursive call with the advanced scanner current_match.create_token s.scan(current_match.regex) end self.parse_scanner(s) << token end end end end end end endhiera-eyaml-2.0.2/lib/hiera/backend/eyaml/parser/token.rb0000644000004100000410000000206312333514275023243 0ustar www-datawww-dataclass Hiera module Backend module Eyaml module Parser class TokenType attr_reader :regex @regex def create_token string raise 'Abstract method called' end end class Token attr_reader :match def initialize(match) @match = match end def to_encrypted(args={}) raise 'Abstract method called' end def to_decrypted(args={}) raise 'Abstract method called' end def to_plain_text raise 'Abstract method called' end def to_s "#{self.class.name}:#{@match}" end end class NonMatchToken < Token def initialize(non_match) super(non_match) end def to_encrypted(args={}) @match end def to_decrypted(args={}) @match end def to_plain_text @match end end end end end endhiera-eyaml-2.0.2/lib/hiera/backend/eyaml/parser/encrypted_tokens.rb0000644000004100000410000001204512333514275025504 0ustar www-datawww-datarequire 'hiera/backend/eyaml/parser/token' require 'hiera/backend/eyaml/utils' require 'hiera/backend/eyaml/encryptor' require 'hiera/backend/eyaml' class Hiera module Backend module Eyaml module Parser class EncToken < Token attr_reader :format, :cipher, :encryptor, :indentation, :plain_text, :id def self.encrypted_value(format, encryption_scheme, cipher, match, indentation = '') decryptor = Encryptor.find encryption_scheme plain_text = decryptor.decrypt( decryptor.decode cipher ) EncToken.new(format, plain_text, decryptor, cipher, match, indentation) end def self.decrypted_value(format, plain_text, encryption_scheme, match, id, indentation = '') encryptor = Encryptor.find encryption_scheme cipher = encryptor.encode( encryptor.encrypt plain_text ) id_number = id.nil? ? nil : id.gsub(/\(|\)/, "").to_i EncToken.new(format, plain_text, encryptor, cipher, match, indentation, id_number) end def initialize(format, plain_text, encryptor, cipher, match = '', indentation = '', id = nil) @format = format @plain_text = plain_text @encryptor = encryptor @cipher = cipher @indentation = indentation @id = id super(match) end def to_encrypted(args={}) label = args[:label] label_string = label.nil? ? '' : "#{label}: " format = args[:format].nil? ? @format : args[:format] case format when :block # strip any white space @cipher = @cipher.gsub(/ /m, "") # normalize indentation ciphertext = @cipher.gsub(/\n/, "\n" + @indentation) chevron = (args[:use_chevron].nil? || args[:use_chevron]) ? ">\n" : '' "#{label_string}#{chevron}" + @indentation + "ENC[#{@encryptor.tag},#{ciphertext}]" when :string ciphertext = @cipher.gsub(/\n/, "") "#{label_string}ENC[#{@encryptor.tag},#{ciphertext}]" else raise "#{@format} is not a valid format" end end def to_decrypted(args={}) label = args[:label] label_string = label.nil? ? '' : "#{label}: " format = args[:format].nil? ? @format : args[:format] index = args[:index].nil? ? '' : "(#{args[:index]})" case format when :block chevron = (args[:use_chevron].nil? || args[:use_chevron]) ? ">\n" : '' "#{label_string}#{chevron}" + indentation + "DEC#{index}::#{@encryptor.tag}[" + @plain_text + "]!" when :string "#{label_string}DEC#{index}::#{@encryptor.tag}[" + @plain_text + "]!" else raise "#{@format} is not a valid format" end end def to_plain_text @plain_text end end class EncTokenType < TokenType def create_enc_token(match, type, enc_comma, cipher, indentation = '') encryption_scheme = enc_comma.nil? ? Eyaml.default_encryption_scheme : enc_comma.split(",").first EncToken.encrypted_value(type, encryption_scheme, cipher, match, indentation) end end class EncHieraTokenType < EncTokenType def initialize @regex = /ENC\[(\w+,)?([a-zA-Z0-9\+\/ =\n]+?)\]/ @string_token_type = EncStringTokenType.new() end def create_token(string) @string_token_type.create_token(string.gsub(/[ \n]/, '')) end end class EncStringTokenType < EncTokenType def initialize @regex = /ENC\[(\w+,)?([a-zA-Z0-9\+\/=]+?)\]/ end def create_token(string) md = @regex.match(string) self.create_enc_token(string, :string, md[1], md[2]) end end class EncBlockTokenType < EncTokenType def initialize @regex = />\n(\s*)ENC\[(\w+,)?([a-zA-Z0-9\+\/ =\n]+?)\]/ end def create_token(string) md = @regex.match(string) self.create_enc_token(string, :block, md[2], md[3], md[1]) end end class DecStringTokenType < TokenType def initialize @regex = /DEC(\(\d+\))?::(\w+)\[(.+?)\]\!/m end def create_token(string) md = @regex.match(string) EncToken.decrypted_value(:string, md[3], md[2], string, md[1]) end end class DecBlockTokenType < TokenType def initialize @regex = />\n(\s*)DEC(\(\d+\))?::(\w+)\[(.+?)\]\!/m end def create_token(string) md = @regex.match(string) EncToken.decrypted_value(:block, md[4], md[3], string, md[2], md[1]) EncToken.decrypted_value(:block, md[4], md[3], string, md[2], md[1]) end end end end end end hiera-eyaml-2.0.2/lib/hiera/backend/eyaml/subcommands/0000755000004100000410000000000012333514275022614 5ustar www-datawww-datahiera-eyaml-2.0.2/lib/hiera/backend/eyaml/subcommands/unknown_command.rb0000644000004100000410000000174212333514275026342 0ustar www-datawww-datarequire 'hiera/backend/eyaml/subcommand' class Hiera module Backend module Eyaml module Subcommands class UnknownCommand < Eyaml::Subcommand class << self attr_accessor :original_command end @@original_command = "unknown" def self.options [] end def self.description "Unknown command (#{@@original_command})" end def self.execute subcommands = Eyaml.subcommands puts <<-EOS Unknown subcommand#{ ": " + Eyaml.subcommand if Eyaml.subcommand } Usage: eyaml Please use one of the following subcommands or help for more help: #{Eyaml.subcommands.sort.collect {|command| command_class = Subcommands.const_get(Utils.camelcase command) command unless command_class.hidden? }.compact.join(", ")} EOS end def self.hidden? true end end end end end end hiera-eyaml-2.0.2/lib/hiera/backend/eyaml/subcommands/recrypt.rb0000644000004100000410000000271612333514275024637 0ustar www-datawww-datarequire 'hiera/backend/eyaml/subcommand' require 'hiera/backend/eyaml/options' require 'hiera/backend/eyaml/parser/parser' class Hiera module Backend module Eyaml module Subcommands class Recrypt < Subcommand def self.options [] end def self.description "recrypt an eyaml file" end def self.helptext "Usage: eyaml recrypt [options] " end def self.validate options Trollop::die "You must specify an eyaml file" if ARGV.empty? options[:source] = :eyaml options[:eyaml] = ARGV.shift options[:input_data] = File.read options[:eyaml] options end def self.execute encrypted_parser = Parser::ParserFactory.encrypted_parser tokens = encrypted_parser.parse Eyaml::Options[:input_data] decrypted_input = tokens.each_with_index.to_a.map{|(t,index)| t.to_decrypted :index => index}.join decrypted_parser = Parser::ParserFactory.decrypted_parser edited_tokens = decrypted_parser.parse(decrypted_input) encrypted_output = edited_tokens.map{ |t| t.to_encrypted }.join filename = Eyaml::Options[:eyaml] File.open("#{filename}", 'w') { |file| file.write encrypted_output } nil end end end end end end hiera-eyaml-2.0.2/lib/hiera/backend/eyaml/subcommands/help.rb0000644000004100000410000000175212333514275024076 0ustar www-datawww-datarequire 'hiera/backend/eyaml/subcommand' require 'hiera/backend/eyaml' class Hiera module Backend module Eyaml module Subcommands class Help < Subcommand def self.options [] end def self.description "this page" end def self.execute puts <<-EOS Welcome to eyaml #{Eyaml::VERSION} Usage: eyaml subcommand [global-opts] [subcommand-opts] Available subcommands: #{Eyaml.subcommands.collect {|command| command_class = Subcommands.const_get(Utils.camelcase command) sprintf "%15s: %-65s", command.downcase, command_class.description unless command_class.hidden? }.compact.join("\n")} For more help on an individual command, use --help on that command Installed Plugins: #{Plugins.plugins.collect {|plugin| "\t" + plugin.name.split("hiera-eyaml-").last }.join("\n")} EOS end def self.hidden? true end end end end end endhiera-eyaml-2.0.2/lib/hiera/backend/eyaml/subcommands/version.rb0000644000004100000410000000171212333514275024627 0ustar www-datawww-datarequire 'hiera/backend/eyaml/subcommand' require 'hiera/backend/eyaml' class Hiera module Backend module Eyaml module Subcommands class Version < Subcommand def self.options [] end def self.description "show version information" end def self.execute plugin_versions = {} puts <<-EOS Version info hiera-eyaml (core): #{Eyaml::VERSION} EOS Plugins.plugins.each do |plugin| plugin_shortname = plugin.name.split("hiera-eyaml-").last plugin_version = begin Encryptor.find(plugin_shortname)::VERSION.to_s rescue "unknown (is plugin compatible with eyaml 2.0+ ?)" end puts " hiera-eyaml-#{plugin_shortname} (gem): #{plugin_version}" end nil end end end end end endhiera-eyaml-2.0.2/lib/hiera/backend/eyaml/subcommands/encrypt.rb0000644000004100000410000000725012333514275024631 0ustar www-datawww-datarequire 'hiera/backend/eyaml/options' require 'hiera/backend/eyaml/parser/parser' require 'hiera/backend/eyaml/parser/encrypted_tokens' require 'hiera/backend/eyaml/subcommand' class Hiera module Backend module Eyaml module Subcommands class Encrypt < Subcommand def self.options [{:name => :password, :description => "Source input is a password entered on the terminal", :short => 'p'}, {:name => :string, :description => "Source input is a string provided as an argument", :short => 's', :type => :string}, {:name => :file, :description => "Source input is a regular file", :short => 'f', :type => :string}, {:name => :stdin, :description => "Source input is taken from stdin", :short => :none}, {:name => :eyaml, :description => "Source input is an eyaml file", :short => 'e', :type => :string}, {:name => :output, :description => "Output format of final result (examples, block, string)", :type => :string, :short => 'o', :default => "examples"}, {:name => :label, :description => "Apply a label to the encrypted result", :short => 'l', :type => :string} ] end def self.description "encrypt some data" end def self.validate options sources = [:password, :string, :file, :stdin, :eyaml].collect {|x| x if options[x]}.compact Trollop::die "You must specify a source" if sources.count.zero? Trollop::die "You can only specify one of (#{sources.join(', ')})" if sources.count > 1 options[:source] = sources.first options[:input_data] = case options[:source] when :password Utils.read_password when :string options[:string] when :file File.read options[:file] when :stdin STDIN.read when :eyaml File.read options[:eyaml] end options end def self.execute case Eyaml::Options[:source] when :eyaml parser = Parser::ParserFactory.decrypted_parser tokens = parser.parse(Eyaml::Options[:input_data]) encrypted = tokens.map{ |token| token.to_encrypted } encrypted.join else encryptor = Encryptor.find ciphertext = encryptor.encode( encryptor.encrypt(Eyaml::Options[:input_data]) ) token = Parser::EncToken.new(:block, Eyaml::Options[:input_data], encryptor, ciphertext, nil, ' ') case Eyaml::Options[:output] when "block" token.to_encrypted :label => Eyaml::Options[:label], :use_chevron => !Eyaml::Options[:label].nil?, :format => :block when "string" token.to_encrypted :label => Eyaml::Options[:label], :format => :string when "examples" string = token.to_encrypted :label => Eyaml::Options[:label] || 'string', :format => :string block = token.to_encrypted :label => Eyaml::Options[:label] || 'block', :format => :block "#{string}\n\nOR\n\n#{block}" else token.to_encrypted :format => :string end end end end end end end end hiera-eyaml-2.0.2/lib/hiera/backend/eyaml/subcommands/edit.rb0000644000004100000410000000725112333514275024073 0ustar www-datawww-datarequire 'hiera/backend/eyaml/utils' require 'hiera/backend/eyaml/options' require 'hiera/backend/eyaml/parser/parser' require 'hiera/backend/eyaml/subcommand' require 'highline/import' class Hiera module Backend module Eyaml module Subcommands class Edit < Subcommand def self.options [] end def self.description "edit an eyaml file" end def self.helptext "Usage: eyaml edit [options] " end def self.validate options Trollop::die "You must specify an eyaml file" if ARGV.empty? options[:source] = :eyaml options[:eyaml] = ARGV.shift options[:input_data] = File.read options[:eyaml] options end def self.execute encrypted_parser = Parser::ParserFactory.encrypted_parser tokens = encrypted_parser.parse Eyaml::Options[:input_data] decrypted_input = tokens.each_with_index.to_a.map{|(t,index)| t.to_decrypted :index => index}.join decrypted_file = Utils.write_tempfile decrypted_input editor = Utils.find_editor begin system "#{editor} #{decrypted_file}" status = $? raise StandardError, "File was moved by editor" unless File.file? decrypted_file edited_file = File.read decrypted_file raise StandardError, "Editor #{editor} has not exited?" unless status.exited? raise StandardError, "Editor did not exit successfully (exit code #{status.exitstatus}), aborting" unless status.exitstatus == 0 raise StandardError, "Edited file is blank" if edited_file.empty? if edited_file == decrypted_input Utils.info "No changes detected, exiting" else decrypted_parser = Parser::ParserFactory.decrypted_parser edited_tokens = decrypted_parser.parse(edited_file) # check that the tokens haven't been copy / pasted used_ids = edited_tokens.find_all{ |t| t.class.name =~ /::EncToken$/ and !t.id.nil? }.map{ |t| t.id } if used_ids.length != used_ids.uniq.length raise RecoverableError, "A duplicate DEC(ID) was found so I don't know how to proceed. This is probably because you copy and pasted a value - if you do this please delete the ID in parentheses" end # replace untouched values with the source values edited_denoised_tokens = edited_tokens.map{ |token| if token.class.name =~ /::EncToken$/ && !token.id.nil? old_token = tokens[token.id] if old_token.plain_text.eql? token.plain_text old_token else token end else token end } encrypted_output = edited_denoised_tokens.map{ |t| t.to_encrypted }.join filename = Eyaml::Options[:eyaml] File.open("#{filename}", 'w') { |file| file.write encrypted_output } end rescue RecoverableError => e Utils.info e if agree "Return to the editor to try again?" retry else raise e end ensure Utils.secure_file_delete :file => decrypted_file, :num_bytes => [edited_file.length, decrypted_input.length].max end nil end end end end end end hiera-eyaml-2.0.2/lib/hiera/backend/eyaml/subcommands/createkeys.rb0000644000004100000410000000103112333514275025273 0ustar www-datawww-datarequire 'hiera/backend/eyaml/subcommand' class Hiera module Backend module Eyaml module Subcommands class Createkeys < Subcommand def self.options [] end def self.description "create a set of keys with which to encrypt/decrypt eyaml data" end def self.execute encryptor = Encryptor.find Eyaml.default_encryption_scheme encryptor.create_keys nil end end end end end end hiera-eyaml-2.0.2/lib/hiera/backend/eyaml/subcommands/decrypt.rb0000644000004100000410000000462612333514275024623 0ustar www-datawww-datarequire 'hiera/backend/eyaml' require 'hiera/backend/eyaml/utils' require 'hiera/backend/eyaml/options' require 'hiera/backend/eyaml/parser/parser' require 'hiera/backend/eyaml/subcommand' class Hiera module Backend module Eyaml module Subcommands class Decrypt < Subcommand def self.options [{:name => :string, :description => "Source input is a string provided as an argument", :short => 's', :type => :string}, {:name => :file, :description => "Source input is a regular file", :short => 'f', :type => :string}, {:name => :eyaml, :description => "Source input is an eyaml file", :short => 'e', :type => :string}, {:name => :stdin, :description => "Source input is taken from stdin", :short => :none} ] end def self.description "decrypt some data" end def self.validate options sources = [:eyaml, :password, :string, :file, :stdin].collect {|x| x if options[x]}.compact Trollop::die "You must specify a source" if sources.count.zero? Trollop::die "You can only specify one of (#{sources.join(', ')})" if sources.count > 1 options[:source] = sources.first options[:input_data] = case options[:source] when :stdin STDIN.read when :string options[:string] when :file File.read options[:file] when :eyaml File.read options[:eyaml] end options end def self.execute parser = Parser::ParserFactory.encrypted_parser tokens = parser.parse(Eyaml::Options[:input_data]) case Eyaml::Options[:source] when :eyaml decrypted = tokens.map{ |token| token.to_decrypted } decrypted.join else decrypted = tokens.map{ |token| case token.class.name when /::EncToken$/ token.plain_text else token.match end } decrypted.join end end end end end end end hiera-eyaml-2.0.2/lib/hiera/backend/eyaml/CLI.rb0000644000004100000410000000262312333514275021240 0ustar www-datawww-datarequire 'trollop' require 'hiera/backend/eyaml' require 'hiera/backend/eyaml/utils' require 'hiera/backend/eyaml/plugins' require 'hiera/backend/eyaml/options' require 'hiera/backend/eyaml/subcommand' class Hiera module Backend module Eyaml class CLI def self.parse Utils.require_dir 'hiera/backend/eyaml/subcommands' Eyaml.subcommands = Utils.find_all_subclasses_of({ :parent_class => Hiera::Backend::Eyaml::Subcommands }).collect {|classname| Utils.snakecase classname} Eyaml.subcommand = ARGV.shift subcommand = case Eyaml.subcommand when nil ARGV.delete_if {true} "unknown_command" when /^\-/ ARGV.delete_if {true} "help" else Eyaml.subcommand end command_class = Subcommand.find subcommand options = command_class.parse options[:executor] = command_class options = command_class.validate options Eyaml::Options.set options Eyaml::Options.debug end def self.execute executor = Eyaml::Options[:executor] begin result = executor.execute puts result unless result.nil? rescue Exception => e Utils.warn e.message Utils.info e.backtrace.inspect end end end end end end hiera-eyaml-2.0.2/lib/hiera/backend/eyaml/commands.rb0000644000004100000410000000037212333514275022431 0ustar www-datawww-datarequire 'rubygems' class Hiera module Backend module Eyaml class Commands @@commands = [] def self.register end def self.commands @@commands end end end end endhiera-eyaml-2.0.2/lib/hiera/backend/eyaml/encryptors/0000755000004100000410000000000012333514275022511 5ustar www-datawww-datahiera-eyaml-2.0.2/lib/hiera/backend/eyaml/encryptors/pkcs7.rb0000644000004100000410000000735712333514275024101 0ustar www-datawww-datarequire 'openssl' require 'hiera/backend/eyaml/encryptor' require 'hiera/backend/eyaml/utils' require 'hiera/backend/eyaml/options' class Hiera module Backend module Eyaml module Encryptors class Pkcs7 < Encryptor self.options = { :private_key => { :desc => "Path to private key", :type => :string, :default => "./keys/private_key.pkcs7.pem" }, :public_key => { :desc => "Path to public key", :type => :string, :default => "./keys/public_key.pkcs7.pem" }, :subject => { :desc => "Subject to use for certificate when creating keys", :type => :string, :default => "/" }, } self.tag = "PKCS7" def self.encrypt plaintext public_key = self.option :public_key raise StandardError, "pkcs7_public_key is not defined" unless public_key public_key_pem = File.read public_key public_key_x509 = OpenSSL::X509::Certificate.new( public_key_pem ) cipher = OpenSSL::Cipher::AES.new(256, :CBC) OpenSSL::PKCS7::encrypt([public_key_x509], plaintext, cipher, OpenSSL::PKCS7::BINARY).to_der end def self.decrypt ciphertext public_key = self.option :public_key private_key = self.option :private_key raise StandardError, "pkcs7_public_key is not defined" unless public_key raise StandardError, "pkcs7_private_key is not defined" unless private_key private_key_pem = File.read private_key private_key_rsa = OpenSSL::PKey::RSA.new( private_key_pem ) public_key_pem = File.read public_key public_key_x509 = OpenSSL::X509::Certificate.new( public_key_pem ) pkcs7 = OpenSSL::PKCS7.new( ciphertext ) pkcs7.decrypt(private_key_rsa, public_key_x509) end def self.create_keys # Try to do equivalent of: # openssl req -x509 -nodes -days 100000 -newkey rsa:2048 -keyout privatekey.pem -out publickey.pem -subj '/' public_key = self.option :public_key private_key = self.option :private_key subject = self.option :subject key = OpenSSL::PKey::RSA.new(2048) Utils.ensure_key_dir_exists private_key Utils.write_important_file :filename => private_key, :content => key.to_pem, :mode => 0600 cert = OpenSSL::X509::Certificate.new() cert.subject = OpenSSL::X509::Name.parse(subject) cert.serial = 1 cert.version = 2 cert.not_before = Time.now cert.not_after = if 1.size == 8 # 64bit Time.now + 50 * 365 * 24 * 60 * 60 else # 32bit Time.at(0x7fffffff) end cert.public_key = key.public_key ef = OpenSSL::X509::ExtensionFactory.new ef.subject_certificate = cert ef.issuer_certificate = cert cert.extensions = [ ef.create_extension("basicConstraints","CA:TRUE", true), ef.create_extension("subjectKeyIdentifier", "hash"), ] cert.add_extension ef.create_extension("authorityKeyIdentifier", "keyid:always,issuer:always") cert.sign key, OpenSSL::Digest::SHA1.new Utils.ensure_key_dir_exists public_key Utils.write_important_file :filename => public_key, :content => cert.to_pem puts "Keys created OK" end end end end end endhiera-eyaml-2.0.2/lib/hiera/backend/eyaml_backend.rb0000644000004100000410000001000612333514275022272 0ustar www-datawww-datarequire 'hiera/backend/eyaml/encryptor' require 'hiera/backend/eyaml/utils' require 'hiera/backend/eyaml/options' require 'hiera/backend/eyaml/parser/parser' require 'hiera/filecache' require 'yaml' class Hiera module Backend class Eyaml_backend attr_reader :extension def initialize(cache = nil) debug("Hiera eYAML backend starting") @cache = cache || Filecache.new @extension = Config[:eyaml][:extension] || "eyaml" end def lookup(key, scope, order_override, resolution_type) answer = nil parse_options(scope) debug("Looking up #{key} in eYAML backend") Backend.datasources(scope, order_override) do |source| debug("Looking for data source #{source}") eyaml_file = Backend.datafile(:eyaml, scope, source, extension) || next next unless File.exists?(eyaml_file) data = @cache.read(eyaml_file, Hash) do |data| YAML.load(data) || {} end next if data.empty? next unless data.include?(key) # Extra logging that we found the key. This can be outputted # multiple times if the resolution type is array or hash but that # should be expected as the logging will then tell the user ALL the # places where the key is found. debug("Found #{key} in #{source}") # for array resolution we just append to the array whatever # we find, we then goes onto the next file and keep adding to # the array # # for priority searches we break after the first found data item new_answer = parse_answer(data[key], scope) case resolution_type when :array raise Exception, "Hiera type mismatch: expected Array and got #{new_answer.class}" unless new_answer.kind_of? Array or new_answer.kind_of? String answer ||= [] answer << new_answer when :hash raise Exception, "Hiera type mismatch: expected Hash and got #{new_answer.class}" unless new_answer.kind_of? Hash answer ||= {} answer = Backend.merge_answer(new_answer,answer) else answer = new_answer break end end return answer end private def debug(message) Hiera.debug("[eyaml_backend]: #{message}") end def decrypt(data) if encrypted?(data) debug("Attempting to decrypt") parser = Eyaml::Parser::ParserFactory.hiera_backend_parser tokens = parser.parse(data) decrypted = tokens.map{ |token| token.to_plain_text } plaintext = decrypted.join plaintext.chomp else data end end def encrypted?(data) /.*ENC\[.*?\]/ =~ data ? true : false end def parse_answer(data, scope, extra_data={}) if data.is_a?(Numeric) or data.is_a?(TrueClass) or data.is_a?(FalseClass) return data elsif data.is_a?(String) return parse_string(data, scope, extra_data) elsif data.is_a?(Hash) answer = {} data.each_pair do |key, val| interpolated_key = Backend.parse_string(key, scope, extra_data) answer[interpolated_key] = parse_answer(val, scope, extra_data) end return answer elsif data.is_a?(Array) answer = [] data.each do |item| answer << parse_answer(item, scope, extra_data) end return answer end end def parse_options(scope) Config[:eyaml].each do |key, value| parsed_value = Backend.parse_string(value, scope) Eyaml::Options[key] = parsed_value debug("Set option: #{key} = #{parsed_value}") end Eyaml::Options[:source] = "hiera" end def parse_string(data, scope, extra_data={}) decrypted_data = decrypt(data) Backend.parse_string(decrypted_data, scope, extra_data) end end end end hiera-eyaml-2.0.2/metadata.yml0000644000004100000410000000505212333514275016252 0ustar www-datawww-data--- !ruby/object:Gem::Specification name: hiera-eyaml version: !ruby/object:Gem::Version version: 2.0.2 platform: ruby authors: - Tom Poulton autorequire: bindir: bin cert_chain: [] date: 2014-05-06 00:00:00 Z dependencies: - !ruby/object:Gem::Dependency name: trollop prerelease: false requirement: &id001 !ruby/object:Gem::Requirement requirements: - - ">=" - !ruby/object:Gem::Version version: "2.0" type: :runtime version_requirements: *id001 - !ruby/object:Gem::Dependency name: highline prerelease: false requirement: &id002 !ruby/object:Gem::Requirement requirements: - - ">=" - !ruby/object:Gem::Version version: 1.6.19 type: :runtime version_requirements: *id002 description: Hiera backend for decrypting encrypted yaml properties email: executables: - eyaml extensions: [] extra_rdoc_files: [] files: - .gitignore - .travis.yml - Gemfile - Gemfile.lock - LICENSE.txt - PLUGINS.md - README.md - Rakefile - bin/eyaml - hiera-eyaml.gemspec - lib/hiera/backend/eyaml.rb - lib/hiera/backend/eyaml/CLI.rb - lib/hiera/backend/eyaml/commands.rb - lib/hiera/backend/eyaml/encryptor.rb - lib/hiera/backend/eyaml/encryptors/pkcs7.rb - lib/hiera/backend/eyaml/options.rb - lib/hiera/backend/eyaml/parser/encrypted_tokens.rb - lib/hiera/backend/eyaml/parser/parser.rb - lib/hiera/backend/eyaml/parser/token.rb - lib/hiera/backend/eyaml/plugins.rb - lib/hiera/backend/eyaml/subcommand.rb - lib/hiera/backend/eyaml/subcommands/createkeys.rb - lib/hiera/backend/eyaml/subcommands/decrypt.rb - lib/hiera/backend/eyaml/subcommands/edit.rb - lib/hiera/backend/eyaml/subcommands/encrypt.rb - lib/hiera/backend/eyaml/subcommands/help.rb - lib/hiera/backend/eyaml/subcommands/recrypt.rb - lib/hiera/backend/eyaml/subcommands/unknown_command.rb - lib/hiera/backend/eyaml/subcommands/version.rb - lib/hiera/backend/eyaml/utils.rb - lib/hiera/backend/eyaml_backend.rb - sublime_text/README.md - sublime_text/eyaml.sublime-package - sublime_text/eyaml.syntax_definition.json - tools/regem.sh homepage: http://github.com/TomPoulton/hiera-eyaml licenses: - MIT metadata: {} post_install_message: rdoc_options: [] require_paths: - lib required_ruby_version: !ruby/object:Gem::Requirement requirements: - &id003 - ">=" - !ruby/object:Gem::Version version: "0" required_rubygems_version: !ruby/object:Gem::Requirement requirements: - *id003 requirements: [] rubyforge_project: rubygems_version: 2.0.14 signing_key: specification_version: 4 summary: OpenSSL Encryption backend for Hiera test_files: [] hiera-eyaml-2.0.2/PLUGINS.md0000644000004100000410000000026512333514275015413 0ustar www-datawww-dataPLUGINS ======= Take a look at the skeleton project hiera-eyaml-plaintext, for a bare-bones demo plugin that you can copy and make into your own encryption plugin for hiera-eyaml. hiera-eyaml-2.0.2/.gitignore0000644000004100000410000000012612333514275015734 0ustar www-datawww-data.idea *.iml *.gradle keys/*.pem pkg/ tmp/ .DS_Store .rvmrc .ruby-version .ruby-gemset hiera-eyaml-2.0.2/sublime_text/0000755000004100000410000000000012333514275016451 5ustar www-datawww-datahiera-eyaml-2.0.2/sublime_text/eyaml.syntax_definition.json0000644000004100000410000001524312333514275024215 0ustar www-datawww-data{ "name": "EYAML", "scopeName": "source.eyaml", "fileTypes": ["eyaml"], "foldingStartMarker": "^[^#]\\s*.*:(\\s*\\[?| &.+)?$", "foldingStopMarker": "^\\s*$|^\\s*\\}|^\\s*\\]|^\\s*\\)", "keyEquivalent": "^~Y", "repository": { "erb": { "end": "%>", "begin": "<%+(?!>)=?", "patterns": [ { "match": "(#).*?(?=%>)", "captures": { "1": { "name": "punctuation.definition.comment.ruby" } }, "name": "comment.line.number-sign.ruby" }, { "include": "source.ruby.rails" } ], "captures": { "0": { "name": "punctuation.section.embedded.ruby" } }, "name": "source.ruby.rails.embedded.html" }, "escaped_char": { "match": "\\\\.", "name": "constant.character.escape.yaml" } }, "patterns": [ { "include": "#erb" }, { "end": "^(?!^\\1)|^(?=\\1(-|\\w+\\s*:)|#)", "begin": "^(\\s*)(?:(-)|(?:(-\\s*)?(\\w+\\s*(:))))\\s*(\\||>)", "beginCaptures": { "3": { "name": "punctuation.definition.entry.yaml" }, "4": { "name": "entity.name.tag.yaml" }, "5": { "name": "punctuation.separator.key-value.yaml" }, "2": { "name": "punctuation.definition.entry.yaml" } }, "patterns": [ { "include": "#erb" } ], "name": "string.unquoted.block.yaml" }, { "match": "(?:(?:(-\\s*)?(\\w+\\s*(:)))|(-))\\s*((0(x|X)[0-9a-fA-F]*)|(([0-9]+\\.?[0-9]*)|(\\.[0-9]+))((e|E)(\\+|-)?[0-9]+)?)(L|l|UL|ul|u|U|F|f)?\\s*$", "captures": { "3": { "name": "punctuation.separator.key-value.yaml" }, "4": { "name": "punctuation.definition.entry.yaml" }, "1": { "name": "punctuation.definition.entry.yaml" }, "2": { "name": "entity.name.tag.yaml" } }, "name": "constant.numeric.yaml" }, { "match": "(?:(?:(-\\s*)?(\\w+\\s*(:)))|(-))\\s*(?:((\")[^\"]*(\"))|((')[^']*('))|([^,{}&#\\[\\]]+))\\s*", "captures": { "7": { "name": "punctuation.definition.string.end.yaml" }, "3": { "name": "punctuation.separator.key-value.yaml" }, "11": { "name": "string.unquoted.yaml" }, "4": { "name": "punctuation.definition.entry.yaml" }, "8": { "name": "string.quoted.single.yaml" }, "9": { "name": "punctuation.definition.string.begin.yaml" }, "5": { "name": "string.quoted.double.yaml" }, "1": { "name": "punctuation.definition.entry.yaml" }, "6": { "name": "punctuation.definition.string.begin.yaml" }, "10": { "name": "punctuation.definition.string.end.yaml" }, "2": { "name": "entity.name.tag.yaml" } }, "name": "string.unquoted.yaml" }, { "match": "(?:(?:(-\\s*)?(\\w+\\s*(:)))|(-))\\s*([0-9]{4}-[0-9]{2}-[0-9]{2})\\s*$", "captures": { "3": { "name": "punctuation.separator.key-value.yaml" }, "4": { "name": "punctuation.definition.entry.yaml" }, "1": { "name": "punctuation.definition.entry.yaml" }, "2": { "name": "entity.name.tag.yaml" } }, "name": "constant.other.date.yaml" }, { "match": "(\\w.*?)(:)\\s*((\\!\\!)omap)?", "captures": { "3": { "name": "keyword.other.omap.yaml" }, "4": { "name": "punctuation.definition.keyword.yaml" }, "1": { "name": "entity.name.tag.yaml" }, "2": { "name": "punctuation.separator.key-value.yaml" } }, "name": "meta.tag.yaml" }, { "match": "(\\&|\\*)\\w.*?$", "captures": { "1": { "name": "punctuation.definition.variable.yaml" } }, "name": "variable.other.yaml" }, { "end": "\"", "begin": "\"", "beginCaptures": { "0": { "name": "punctuation.definition.string.begin.yaml" } }, "patterns": [ { "include": "#escaped_char" }, { "include": "#erb" } ], "endCaptures": { "0": { "name": "punctuation.definition.string.end.yaml" } }, "name": "string.quoted.double.yaml" }, { "end": "'", "begin": "'", "beginCaptures": { "0": { "name": "punctuation.definition.string.begin.yaml" } }, "patterns": [ { "include": "#escaped_char" }, { "include": "#erb" } ], "endCaptures": { "0": { "name": "punctuation.definition.string.end.yaml" } }, "name": "string.quoted.single.yaml" }, { "end": "`", "begin": "`", "beginCaptures": { "0": { "name": "punctuation.definition.string.begin.yaml" } }, "patterns": [ { "include": "#escaped_char" }, { "include": "#erb" } ], "endCaptures": { "0": { "name": "punctuation.definition.string.end.yaml" } }, "name": "string.interpolated.yaml" }, { "match": "(\\<\\<): ((\\*).*)$", "captures": { "3": { "name": "punctuation.definition.keyword.yaml" }, "1": { "name": "entity.name.tag.yaml" }, "2": { "name": "keyword.operator.merge-key.yaml" } }, "name": "keyword.operator.merge-key.yaml" }, { "match": "( |\t)+$", "disabled": "1", "name": "invalid.deprecated.trailing-whitespace.yaml" }, { "match": "(? ]`4 [ [3}ȩKvǸS8tɵa'pO"Ai" +[@jfg|]쏋6FCcIƴs}[c>f:I'WmMڗbp*?BEw%>,1|G֬y3J@'}&gMl= QbGjMK ,gR!"7aD&K Xx&J3~YAM K P@dDqŝ 3&aMSj6Lº= =,^UxJJT)  Z "epSAi`o~p| W9ǰ$۷5 9J$#vD28~GsG `S;~eAVcydXb\". B:v|qX*Vy}ZR[j/T"' ,s2G^*T8PI,l̼.B;=;vC U'UZnDշ."nRb" NO+05#¥%P(t\H}"Mn_eQU :1IQ@"&M=ZNL~dAoGsex-8Ũ7㬝Ư "Sz06l@8P@+/ pD5E5so!d?(C%^ER/L\KԔ\PKeF9(j6l (KF$0aLx.^~O1cY9_TeQڹaCew*zur)t^fba z}̴ _wK:*k~]MW"]o>ADI$u$&3^J-"{N sC ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv NBTZfOlPvMlAesyr4bUY4I5XeVbVk38XKxeriH69EFAD4CahIZlC8lkE/uDh jJGQfh052eonkungHIcuGKY/5sEbbZl/qufjAtp/ufor15VBJtsXt17tXP4y l5ZP119Fwq8xiREGOL0lVvFYJz2hZc1ppPCNG5lwuLnTekXN/OazNYpf4CMd /HjZFXwcXRtTlzewJLc+/gox2IfByQRhsI/AgogRfYQKocZgFb/DOZoXR7wm IZGeunzwhqfmEtGiqpvJJQ5wVRdzJVpTnANBA5qxeA==] ``` To edit this you can use the command `eyaml edit important.eyaml` which will decrypt the file, fire up an editor with the decrypted values and re-encrypt any edited values when you exit the editor. This tool makes editing your encrypted files as simple as clear text files. Setup ----- ### Installing hiera-eyaml $ gem install hiera-eyaml ### Generate keys The first step is to create a pair of keys: $ eyaml createkeys This creates a public and private key with default names in the default location. (./keys) #### Storing the keys securely when using Puppet Since the point of using this module is to securely store sensitive information, it's important to store these keys securely. If using Hiera with Puppet, Your puppetmaster will need to access these keys to perform decryption when the puppet agent runs on a remote node. So for this reason, a suggested location might be to store them in `/etc/puppet/secure/keys` or `/var/lib/puppet/keys` depending on your setup. The permissions for this folder should allow the puppet user (normally 'puppet') execute access to the keys directory, read only access to the keys themselves and restrict everyone else: $ chown -R puppet:puppet /etc/puppet/secure/keys $ chmod -R 0500 /etc/puppet/secure/keys $ chmod 0400 /etc/puppet/secure/keys/*.pem $ ls -lha /etc/puppet/secure/keys -r-------- 1 puppet puppet 1.7K Sep 24 16:24 private_key.pkcs7.pem -r-------- 1 puppet puppet 1.1K Sep 24 16:24 public_key.pkcs7.pem ### Encryption To encrypt something, you only need the public_key, so distribute that to people creating hiera properties $ eyaml encrypt -f filename # Encrypt a file $ eyaml encrypt -s 'hello there' # Encrypt a string $ eyaml encrypt -p # Encrypt a password (prompt for it) Use the -l parameter to pass in a label for the encrypted value, $ eyaml encrypt -l 'some_easy_to_use_label' -s 'yourSecretString' ### Decryption To decrypt something, you need the public_key and the private_key. To test decryption you can also use the eyaml tool if you have both keys $ eyaml decrypt -f filename # Decrypt a file $ eyaml decrypt -s 'ENC[PKCS7,.....]' # Decrypt a string ### Editing eyaml files Once you have created a few eyaml files, with a mixture of encrypted and non-encrypted properties, you can edit the encrypted values in place, using the special edit mode of the eyaml utility. Edit mode opens a decrypted copy of the eyaml file in your `$EDITOR` and will encrypt and modified values when you exit the editor. $ eyaml edit filename.eyaml # Edit an eyaml file in place When editing eyaml files, you will see that the unencrypted plaintext is marked to allow the eyaml tool to identify each encrypted block, along with the encryption method. This is used to make sure that the block is encrypted again only if the clear text value has changed, and is encrypted using the original encryption mechanism (see plugable encryption later). A decrypted file might look like this: ```yaml --- plain-property: You can see me cipher-property : > DEC(1)::PKCS7[You can't see me]! environments: development: host: localhost password: password production: host: prod.org.com password: > DEC(2)::PKCS7[securepassword]! things: - thing 1 - - nested thing 1.0 - > DEC(3)::PKCS7[secure nested thing 1.1]! - - nested thing 2.0 - nested thing 2.1 ``` Whilst editing you can delete existing values and add new one using the same format (as below). Note that it is important to omit the number in brackets for new values. If any duplicate IDs are found then the re-encryption process will be abandoned by the eyaml tool. some_new_key: DEC::PKCS7[a new value to encrypt]! Hiera ----- To use eyaml with hiera and puppet, first configure hiera.yaml to use the eyaml backend ```yaml --- :backends: - eyaml - yaml :hierarchy: - %{environment} - common :yaml: :datadir: '/etc/puppet/hieradata' :eyaml: :datadir: '/etc/puppet/hieradata' # If using the pkcs7 encryptor (default) :pkcs7_private_key: /path/to/private_key.pkcs7.pem :pkcs7_public_key: /path/to/public_key.pkcs7.pem ``` Then, edit your hiera yaml files, and insert your encrypted values. The default eyaml file extension is .eyaml, however this can be configured in the :eyaml block to set :extension, ```yaml :eyaml: :extension: 'yaml' ``` *Important Note:* The eyaml backend will not parse internally json formatted yaml files, whereas the regular yaml backend will. You'll need to ensure any existing yaml files using json format are converted to syntactically correct yaml format. ```yaml --- plain-property: You can see me cipher-property : > ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv NBTZfOlPvMlAesyr4bUY4I5XeVbVk38XKxeriH69EFAD4CahIZlC8lkE/uDh jJGQfh052eonkungHIcuGKY/5sEbbZl/qufjAtp/ufor15VBJtsXt17tXP4y l5ZP119Fwq8xiREGOL0lVvFYJz2hZc1ppPCNG5lwuLnTekXN/OazNYpf4CMd /HjZFXwcXRtTlzewJLc+/gox2IfByQRhsI/AgogRfYQKocZgFb/DOZoXR7wm IZGeunzwhqfmEtGiqpvJJQ5wVRdzJVpTnANBA5qxeA==] environments: development: host: localhost password: password production: host: prod.org.com password: > ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv NBTZfOlPvMlAesyr4bUY4I5XeVbVk38XKxeriH69EFAD4CahIZlC8lkE/uDh jJGQfh052eonkungHIcuGKY/5sEbbZl/qufjAtp/ufor15VBJtsXt17tXP4y l5ZP119Fwq8xiREGOL0lVvFYJz2hZc1ppPCNG5lwuLnTekXN/OazNYpf4CMd /HjZFXwcXRtTlzewJLc+/gox2IfByQRhsI/AgogRfYQKocZgFb/DOZoXR7wm IZGeunzwhqfmEtGiqpvJJQ5wVRdzJVpTnANBA5qxeA==] things: - thing 1 - - nested thing 1.0 - > ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv NBTZfOlPvMlAesyr4bUY4I5XeVbVk38XKxeriH69EFAD4CahIZlC8lkE/uDh jJGQfh052eonkungHIcuGKY/5sEbbZl/qufjAtp/ufor15VBJtsXt17tXP4y l5ZP119Fwq8xiREGOL0lVvFYJz2hZc1ppPCNG5lwuLnTekXN/OazNYpf4CMd /HjZFXwcXRtTlzewJLc+/gox2IfByQRhsI/AgogRfYQKocZgFb/DOZoXR7wm IZGeunzwhqfmEtGiqpvJJQ5wVRdzJVpTnANBA5qxeA==] - - nested thing 2.0 - nested thing 2.1 ``` Configuration file for eyaml ---------------------------- Default parameters for the eyaml command line tool can be provided by creating a configuration YAML file. The location of the file defaults to `~/.eyaml/config.yaml` but can be overriden by setting `EYAML_CONFIG` environment variable. The file takes any long form argument that you can provide on the command line. For example, to override the pkcs7 keys: ```yaml --- pkcs7_private_key: '~/keys/eyaml/private_key.pkcs7.pem' pkcs7_public_key: '~/keys/eyaml/public_key.pkcs7.pem' ``` Or to override to use GPG by default: ```yaml --- encrypt_method: 'gpg' gpg_gnupghome: '~/alternative_gnupghome' gpg_recipients: 'sihil@example.com,gtmtech@example.com,tpoulton@example.com' ``` Pluggable Encryption -------------------- hiera-eyaml backend is pluggable, so that further encryption types can be added as separate gems to the general mechanism which hiera-eyaml uses. Hiera-eyaml ships with one default mechanism of 'pkcs7', the encryption type widely used to sign smime email messages. Other encryption types (if the gems for them have been loaded) can be specified using the following formats: ENC[PKCS7,SOME_ENCRYPTED_VALUE] # a PKCS7 encrypted value ENC[GPG,SOME_ENCRYPTED_VALUE] # a GPG encrypted value (hiera-eyaml-gpg) ... etc ... When editing eyaml files, you will see that the unencrypted plaintext is marked in such a way as to identify the encryption method. This is so that the eyaml tool knows to encrypt it back using the correct method afterwards: some_key: DEC(1)::PKCS7[very secret password]! ### Encryption plugins This is a list of available plugins: - [hiera-eyaml-gpg](https://github.com/sihil/hiera-eyaml-gpg) - Provide GPG encryption - [hiera-eyaml-plaintext](https://github.com/gtmtech/hiera-eyaml-plaintext) - This is a no-op encryption plugin that simply base64 encodes the values. It exists as an example plugin to create your own and to do integration tests on hiera-eyaml. **THIS SHOULD NOT BE USED IN PRODUCTION** Notes ----- If you do not specify an encryption method within ENC[] tags, it will be assumed to be PKCS7 Also remember that after encrypting your sensitive properties, if anyone has access to your git source, they will see what the property was in previous commits before you encrypted. It's recommended that you roll any passwords when switching from unencrypted to encrypted properties. eg, Developers having write access to a DEV branch will be able to read/view the contents of the PRD branch, as per the design of GIT. Github has a great guide on removing sensitive data from repos here: https://help.github.com/articles/remove-sensitive-data Troubleshooting --------------- ### Installing from behind a corporate/application proxy $ export HTTP_PROXY=http://yourcorporateproxy:3128/ $ export HTTPS_PROXY=http://yourcorporateproxy:3128/ then run your install $ gem install hiera-eyaml Issues ------ If you have found a bug then please raise an issue here on github. Some of us hang out on #hiera-eyaml on freenode, please drop by if you want to say hi or have a question. Tests ----- In order to run the tests, simply run `cucumber` in the top level directory of the project. You'll need to have a few requirements installed: * `expect` (via yum/apt-get or system package) * `aruba` (gem) * `cucumber` (gem) * `puppet` (gem) Authors ------- - [Tom Poulton](http://github.com/TomPoulton) - Initial author. eyaml backend. - [Geoff Meakin](http://github.com/gtmtech) - Major contributor. eyaml command, tests, CI - [Simon Hildrew](http://github.com/sihil) - Contributor. eyaml edit sub command. - [Robert Fielding](http://github.com/rooprob) - Contributor. eyaml recrypt sub command.