|  |
| |
| Go configuration with fangs! |
| |
| Many Go projects are built using Viper including: |
| |
| * [Hugo](http://gohugo.io) |
| * [EMC RexRay](http://rexray.readthedocs.org/en/stable/) |
| * [Imgur's Incus](https://github.com/Imgur/incus) |
| * [Nanobox](https://github.com/nanobox-io/nanobox)/[Nanopack](https://github.com/nanopack) |
| * [Docker Notary](https://github.com/docker/Notary) |
| * [BloomApi](https://www.bloomapi.com/) |
| * [doctl(https://github.com/digitalocean/doctl) |
| |
| [](https://travis-ci.org/spf13/viper) [](https://gitter.im/spf13/viper?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) |
| |
| |
| ## What is Viper? |
| |
| Viper is a complete configuration solution for go applications including 12 factor apps. It is designed |
| to work within an application, and can handle all types of configuration needs |
| and formats. It supports: |
| |
| * setting defaults |
| * reading from JSON, TOML, YAML, HCL, and Java properties config files |
| * live watching and re-reading of config files (optional) |
| * reading from environment variables |
| * reading from remote config systems (etcd or Consul), and watching changes |
| * reading from command line flags |
| * reading from buffer |
| * setting explicit values |
| |
| Viper can be thought of as a registry for all of your applications |
| configuration needs. |
| |
| ## Why Viper? |
| |
| When building a modern application, you don’t want to worry about |
| configuration file formats; you want to focus on building awesome software. |
| Viper is here to help with that. |
| |
| Viper does the following for you: |
| |
| 1. Find, load, and unmarshal a configuration file in JSON, TOML, YAML, HCL, or Java properties formats. |
| 2. Provide a mechanism to set default values for your different |
| configuration options. |
| 3. Provide a mechanism to set override values for options specified through |
| command line flags. |
| 4. Provide an alias system to easily rename parameters without breaking existing |
| code. |
| 5. Make it easy to tell the difference between when a user has provided a |
| command line or config file which is the same as the default. |
| |
| Viper uses the following precedence order. Each item takes precedence over the |
| item below it: |
| |
| * explicit call to Set |
| * flag |
| * env |
| * config |
| * key/value store |
| * default |
| |
| Viper configuration keys are case insensitive. |
| |
| ## Putting Values into Viper |
| |
| ### Establishing Defaults |
| |
| A good configuration system will support default values. A default value is not |
| required for a key, but it's useful in the event that a key hasn’t been set via |
| config file, environment variable, remote configuration or flag. |
| |
| Examples: |
| |
| ```go |
| viper.SetDefault("ContentDir", "content") |
| viper.SetDefault("LayoutDir", "layouts") |
| viper.SetDefault("Taxonomies", map[string]string{"tag": "tags", "category": "categories"}) |
| ``` |
| |
| ### Reading Config Files |
| |
| Viper requires minimal configuration so it knows where to look for config files. |
| Viper supports JSON, TOML, YAML, HCL, and Java Properties files. Viper can search multiple paths, but |
| currently a single Viper instance only supports a single configuration file. |
| Viper does not default to any configuration search paths leaving defaults decision |
| to an application. |
| |
| Here is an example of how to use Viper to search for and read a configuration file. |
| None of the specific paths are required, but at least one path should be provided |
| where a configuration file is expected. |
| |
| ```go |
| viper.SetConfigName("config") // name of config file (without extension) |
| viper.AddConfigPath("/etc/appname/") // path to look for the config file in |
| viper.AddConfigPath("$HOME/.appname") // call multiple times to add many search paths |
| viper.AddConfigPath(".") // optionally look for config in the working directory |
| err := viper.ReadInConfig() // Find and read the config file |
| if err != nil { // Handle errors reading the config file |
| panic(fmt.Errorf("Fatal error config file: %s \n", err)) |
| } |
| ``` |
| |
| ### Watching and re-reading config files |
| |
| Viper supports the ability to have your application live read a config file while running. |
| |
| Gone are the days of needing to restart a server to have a config take effect, |
| viper powered applications can read an update to a config file while running and |
| not miss a beat. |
| |
| Simply tell the viper instance to watchConfig. |
| Optionally you can provide a function for Viper to run each time a change occurs. |
| |
| **Make sure you add all of the configPaths prior to calling `WatchConfig()`** |
| |
| ```go |
| viper.WatchConfig() |
| viper.OnConfigChange(func(e fsnotify.Event) { |
| fmt.Println("Config file changed:", e.Name) |
| }) |
| ``` |
| |
| ### Reading Config from io.Reader |
| |
| Viper predefines many configuration sources such as files, environment |
| variables, flags, and remote K/V store, but you are not bound to them. You can |
| also implement your own required configuration source and feed it to viper. |
| |
| ```go |
| viper.SetConfigType("yaml") // or viper.SetConfigType("YAML") |
| |
| // any approach to require this configuration into your program. |
| var yamlExample = []byte(` |
| Hacker: true |
| name: steve |
| hobbies: |
| - skateboarding |
| - snowboarding |
| - go |
| clothing: |
| jacket: leather |
| trousers: denim |
| age: 35 |
| eyes : brown |
| beard: true |
| `) |
| |
| viper.ReadConfig(bytes.NewBuffer(yamlExample)) |
| |
| viper.Get("name") // this would be "steve" |
| ``` |
| |
| ### Setting Overrides |
| |
| These could be from a command line flag, or from your own application logic. |
| |
| ```go |
| viper.Set("Verbose", true) |
| viper.Set("LogFile", LogFile) |
| ``` |
| |
| ### Registering and Using Aliases |
| |
| Aliases permit a single value to be referenced by multiple keys |
| |
| ```go |
| viper.RegisterAlias("loud", "Verbose") |
| |
| viper.Set("verbose", true) // same result as next line |
| viper.Set("loud", true) // same result as prior line |
| |
| viper.GetBool("loud") // true |
| viper.GetBool("verbose") // true |
| ``` |
| |
| ### Working with Environment Variables |
| |
| Viper has full support for environment variables. This enables 12 factor |
| applications out of the box. There are four methods that exist to aid working |
| with ENV: |
| |
| * `AutomaticEnv()` |
| * `BindEnv(string...) : error` |
| * `SetEnvPrefix(string)` |
| * `SetEnvReplacer(string...) *strings.Replacer` |
| |
| _When working with ENV variables, it’s important to recognize that Viper |
| treats ENV variables as case sensitive._ |
| |
| Viper provides a mechanism to try to ensure that ENV variables are unique. By |
| using `SetEnvPrefix`, you can tell Viper to use add a prefix while reading from |
| the environment variables. Both `BindEnv` and `AutomaticEnv` will use this |
| prefix. |
| |
| `BindEnv` takes one or two parameters. The first parameter is the key name, the |
| second is the name of the environment variable. The name of the environment |
| variable is case sensitive. If the ENV variable name is not provided, then |
| Viper will automatically assume that the key name matches the ENV variable name, |
| but the ENV variable is IN ALL CAPS. When you explicitly provide the ENV |
| variable name, it **does not** automatically add the prefix. |
| |
| One important thing to recognize when working with ENV variables is that the |
| value will be read each time it is accessed. Viper does not fix the value when |
| the `BindEnv` is called. |
| |
| `AutomaticEnv` is a powerful helper especially when combined with |
| `SetEnvPrefix`. When called, Viper will check for an environment variable any |
| time a `viper.Get` request is made. It will apply the following rules. It will |
| check for a environment variable with a name matching the key uppercased and |
| prefixed with the `EnvPrefix` if set. |
| |
| `SetEnvReplacer` allows you to use a `strings.Replacer` object to rewrite Env |
| keys to an extent. This is useful if you want to use `-` or something in your |
| `Get()` calls, but want your environmental variables to use `_` delimiters. An |
| example of using it can be found in `viper_test.go`. |
| |
| #### Env example |
| |
| ```go |
| SetEnvPrefix("spf") // will be uppercased automatically |
| BindEnv("id") |
| |
| os.Setenv("SPF_ID", "13") // typically done outside of the app |
| |
| id := Get("id") // 13 |
| ``` |
| |
| ### Working with Flags |
| |
| Viper has the ability to bind to flags. Specifically, Viper supports `Pflags` |
| as used in the [Cobra](https://github.com/spf13/cobra) library. |
| |
| Like `BindEnv`, the value is not set when the binding method is called, but when |
| it is accessed. This means you can bind as early as you want, even in an |
| `init()` function. |
| |
| The `BindPFlag()` method provides this functionality. |
| |
| Example: |
| |
| ```go |
| serverCmd.Flags().Int("port", 1138, "Port to run Application server on") |
| viper.BindPFlag("port", serverCmd.Flags().Lookup("port")) |
| ``` |
| |
| The use of [pflag](https://github.com/spf13/pflag/) in Viper does not preclude |
| the use of other packages that use the [flag](https://golang.org/pkg/flag/) |
| package from the standard library. The pflag package can handle the flags |
| defined for the flag package by importing these flags. This is accomplished |
| by a calling a convenience function provided by the pflag package called |
| AddGoFlagSet(). |
| |
| Example: |
| |
| ```go |
| package main |
| |
| import ( |
| "flag" |
| "github.com/spf13/pflag" |
| ) |
| |
| func main() { |
| pflag.CommandLine.AddGoFlagSet(flag.CommandLine) |
| pflag.Parse() |
| ... |
| } |
| ``` |
| |
| #### Flag interfaces |
| |
| Viper provides two Go interfaces to bind other flag systems if you don't use `Pflags`. |
| |
| `FlagValue` represents a single flag. This is a very simple example on how to implement this interface: |
| |
| ```go |
| type myFlag struct {} |
| func (f myFlag) IsChanged() { return false } |
| func (f myFlag) Name() { return "my-flag-name" } |
| func (f myFlag) ValueString() { return "my-flag-value" } |
| func (f myFlag) ValueType() { return "string" } |
| ``` |
| |
| Once your flag implements this interface, you can simply tell Viper to bind it: |
| |
| ```go |
| viper.BindFlagValue("my-flag-name", myFlag{}) |
| ``` |
| |
| `FlagValueSet` represents a group of flags. This is a very simple example on how to implement this interface: |
| |
| ```go |
| type myFlagSet struct { |
| flags []myFlag |
| } |
| |
| func (f myFlagSet) VisitAll(fn func(FlagValue)) { |
| for _, flag := range flags { |
| fn(flag) |
| } |
| } |
| ``` |
| |
| Once your flag set implements this interface, you can simply tell Viper to bind it: |
| |
| ```go |
| fSet := myFlagSet{ |
| flags: []myFlag{myFlag{}, myFlag{}}, |
| } |
| viper.BindFlagValues("my-flags", fSet) |
| ``` |
| |
| ### Remote Key/Value Store Support |
| |
| To enable remote support in Viper, do a blank import of the `viper/remote` |
| package: |
| |
| `import _ "github.com/spf13/viper/remote"` |
| |
| Viper will read a config string (as JSON, TOML, YAML or HCL) retrieved from a path |
| in a Key/Value store such as etcd or Consul. These values take precedence over |
| default values, but are overridden by configuration values retrieved from disk, |
| flags, or environment variables. |
| |
| Viper uses [crypt](https://github.com/xordataexchange/crypt) to retrieve |
| configuration from the K/V store, which means that you can store your |
| configuration values encrypted and have them automatically decrypted if you have |
| the correct gpg keyring. Encryption is optional. |
| |
| You can use remote configuration in conjunction with local configuration, or |
| independently of it. |
| |
| `crypt` has a command-line helper that you can use to put configurations in your |
| K/V store. `crypt` defaults to etcd on http://127.0.0.1:4001. |
| |
| ```bash |
| $ go get github.com/xordataexchange/crypt/bin/crypt |
| $ crypt set -plaintext /config/hugo.json /Users/hugo/settings/config.json |
| ``` |
| |
| Confirm that your value was set: |
| |
| ```bash |
| $ crypt get -plaintext /config/hugo.json |
| ``` |
| |
| See the `crypt` documentation for examples of how to set encrypted values, or |
| how to use Consul. |
| |
| ### Remote Key/Value Store Example - Unencrypted |
| |
| ```go |
| viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001","/config/hugo.json") |
| viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop" |
| err := viper.ReadRemoteConfig() |
| ``` |
| |
| ### Remote Key/Value Store Example - Encrypted |
| |
| ```go |
| viper.AddSecureRemoteProvider("etcd","http://127.0.0.1:4001","/config/hugo.json","/etc/secrets/mykeyring.gpg") |
| viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop" |
| err := viper.ReadRemoteConfig() |
| ``` |
| |
| ### Watching Changes in etcd - Unencrypted |
| |
| ```go |
| // alternatively, you can create a new viper instance. |
| var runtime_viper = viper.New() |
| |
| runtime_viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001", "/config/hugo.yml") |
| runtime_viper.SetConfigType("yaml") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop" |
| |
| // read from remote config the first time. |
| err := runtime_viper.ReadRemoteConfig() |
| |
| // unmarshal config |
| runtime_viper.Unmarshal(&runtime_conf) |
| |
| // open a goroutine to watch remote changes forever |
| go func(){ |
| for { |
| time.Sleep(time.Second * 5) // delay after each request |
| |
| // currently, only tested with etcd support |
| err := runtime_viper.WatchRemoteConfig() |
| if err != nil { |
| log.Errorf("unable to read remote config: %v", err) |
| continue |
| } |
| |
| // unmarshal new config into our runtime config struct. you can also use channel |
| // to implement a signal to notify the system of the changes |
| runtime_viper.Unmarshal(&runtime_conf) |
| } |
| }() |
| ``` |
| |
| ## Getting Values From Viper |
| |
| In Viper, there are a few ways to get a value depending on the value's type. |
| The following functions and methods exist: |
| |
| * `Get(key string) : interface{}` |
| * `GetBool(key string) : bool` |
| * `GetFloat64(key string) : float64` |
| * `GetInt(key string) : int` |
| * `GetString(key string) : string` |
| * `GetStringMap(key string) : map[string]interface{}` |
| * `GetStringMapString(key string) : map[string]string` |
| * `GetStringSlice(key string) : []string` |
| * `GetTime(key string) : time.Time` |
| * `GetDuration(key string) : time.Duration` |
| * `IsSet(key string) : bool` |
| |
| One important thing to recognize is that each Get function will return a zero |
| value if it’s not found. To check if a given key exists, the `IsSet()` method |
| has been provided. |
| |
| Example: |
| ```go |
| viper.GetString("logfile") // case-insensitive Setting & Getting |
| if viper.GetBool("verbose") { |
| fmt.Println("verbose enabled") |
| } |
| ``` |
| ### Accessing nested keys |
| |
| The accessor methods also accept formatted paths to deeply nested keys. For |
| example, if the following JSON file is loaded: |
| |
| ```json |
| { |
| "host": { |
| "address": "localhost", |
| "port": 5799 |
| }, |
| "datastore": { |
| "metric": { |
| "host": "127.0.0.1", |
| "port": 3099 |
| }, |
| "warehouse": { |
| "host": "198.0.0.1", |
| "port": 2112 |
| } |
| } |
| } |
| |
| ``` |
| |
| Viper can access a nested field by passing a `.` delimited path of keys: |
| |
| ```go |
| GetString("datastore.metric.host") // (returns "127.0.0.1") |
| ``` |
| |
| This obeys the precedence rules established above; the search for the root key |
| (in this example, `datastore`) will cascade through the remaining configuration |
| registries until found. The search for the sub-keys (`metric` and `host`), |
| however, will not. |
| |
| For example, if the `metric` key was not defined in the configuration loaded |
| from file, but was defined in the defaults, Viper would return the zero value. |
| |
| On the other hand, if the primary key was not defined, Viper would go through |
| the remaining registries looking for it. |
| |
| Lastly, if there exists a key that matches the delimited key path, its value |
| will be returned instead. E.g. |
| |
| ```json |
| { |
| "datastore.metric.host": "0.0.0.0", |
| "host": { |
| "address": "localhost", |
| "port": 5799 |
| }, |
| "datastore": { |
| "metric": { |
| "host": "127.0.0.1", |
| "port": 3099 |
| }, |
| "warehouse": { |
| "host": "198.0.0.1", |
| "port": 2112 |
| } |
| } |
| } |
| |
| GetString("datastore.metric.host") //returns "0.0.0.0" |
| ``` |
| |
| ### Extract sub-tree |
| |
| Extract sub-tree from Viper. |
| |
| For example, `viper` represents: |
| |
| ```json |
| app: |
| cache1: |
| max-items: 100 |
| item-size: 64 |
| cache2: |
| max-items: 200 |
| item-size: 80 |
| ``` |
| |
| After executing: |
| |
| ```go |
| subv := viper.Sub("app.cache1") |
| ``` |
| |
| `subv` represents: |
| |
| ```json |
| max-items: 100 |
| item-size: 64 |
| ``` |
| |
| Suppose we have: |
| |
| ```go |
| func NewCache(cfg *Viper) *Cache {...} |
| ``` |
| |
| which creates a cache based on config information formatted as `subv`. |
| Now it's easy to create these 2 caches separately as: |
| |
| ```go |
| cfg1 := viper.Sub("app.cache1") |
| cache1 := NewCache(cfg1) |
| |
| cfg2 := viper.Sub("app.cache2") |
| cache2 := NewCache(cfg2) |
| ``` |
| |
| ### Unmarshaling |
| |
| You also have the option of Unmarshaling all or a specific value to a struct, map, |
| etc. |
| |
| There are two methods to do this: |
| |
| * `Unmarshal(rawVal interface{}) : error` |
| * `UnmarshalKey(key string, rawVal interface{}) : error` |
| |
| Example: |
| |
| ```go |
| type config struct { |
| Port int |
| Name string |
| PathMap string `mapstructure:"path_map"` |
| } |
| |
| var C config |
| |
| err := Unmarshal(&C) |
| if err != nil { |
| t.Fatalf("unable to decode into struct, %v", err) |
| } |
| ``` |
| |
| ## Viper or Vipers? |
| |
| Viper comes ready to use out of the box. There is no configuration or |
| initialization needed to begin using Viper. Since most applications will want |
| to use a single central repository for their configuration, the viper package |
| provides this. It is similar to a singleton. |
| |
| In all of the examples above, they demonstrate using viper in it's singleton |
| style approach. |
| |
| ### Working with multiple vipers |
| |
| You can also create many different vipers for use in your application. Each will |
| have it’s own unique set of configurations and values. Each can read from a |
| different config file, key value store, etc. All of the functions that viper |
| package supports are mirrored as methods on a viper. |
| |
| Example: |
| |
| ```go |
| x := viper.New() |
| y := viper.New() |
| |
| x.SetDefault("ContentDir", "content") |
| y.SetDefault("ContentDir", "foobar") |
| |
| //... |
| ``` |
| |
| When working with multiple vipers, it is up to the user to keep track of the |
| different vipers. |
| |
| ## Q & A |
| |
| Q: Why not INI files? |
| |
| A: Ini files are pretty awful. There’s no standard format, and they are hard to |
| validate. Viper is designed to work with JSON, TOML or YAML files. If someone |
| really wants to add this feature, I’d be happy to merge it. It’s easy to specify |
| which formats your application will permit. |
| |
| Q: Why is it called “Viper”? |
| |
| A: Viper is designed to be a [companion](http://en.wikipedia.org/wiki/Viper_(G.I._Joe)) |
| to [Cobra](https://github.com/spf13/cobra). While both can operate completely |
| independently, together they make a powerful pair to handle much of your |
| application foundation needs. |
| |
| Q: Why is it called “Cobra”? |
| |
| A: Is there a better name for a [commander](http://en.wikipedia.org/wiki/Cobra_Commander)? |