Steve Francia | d2f75b4 | 2015-11-02 11:07:43 -0500 | [diff] [blame] | 1 |  |
Steve Francia | f98d9bf | 2014-04-02 07:33:33 -0700 | [diff] [blame] | 2 | |
Steve Francia | d2f75b4 | 2015-11-02 11:07:43 -0500 | [diff] [blame] | 3 | Go configuration with fangs! |
The Gitter Badger | 2763b90 | 2015-05-01 20:02:50 +0000 | [diff] [blame] | 4 | |
Steve Francia | dd66c89 | 2016-02-08 17:01:43 -0500 | [diff] [blame^] | 5 | Many Go projects are built using Viper including: |
| 6 | |
| 7 | * [Hugo](http://gohugo.io) |
| 8 | * [EMC RexRay](http://rexray.readthedocs.org/en/stable/) |
| 9 | * [Imgur's Incus](https://github.com/Imgur/incus) |
| 10 | * [Nanobox](https://github.com/nanobox-io/nanobox)/[Nanopack](https://github.com/nanopack) |
| 11 | * [Docker Notary](https://github.com/docker/Notary) |
| 12 | * [BloomApi](https://www.bloomapi.com/) |
| 13 | * [DOIt](https://github.com/bryanl/doit) |
| 14 | |
Steve Francia | d2f75b4 | 2015-11-02 11:07:43 -0500 | [diff] [blame] | 15 | [](https://travis-ci.org/spf13/viper) [](https://gitter.im/spf13/viper?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) |
| 16 | |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 17 | |
| 18 | ## What is Viper? |
| 19 | |
Steve Francia | d2f75b4 | 2015-11-02 11:07:43 -0500 | [diff] [blame] | 20 | Viper is a complete configuration solution for go applications including 12 factor apps. It is designed |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 21 | to work within an application, and can handle all types of configuration needs |
| 22 | and formats. It supports: |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 23 | |
| 24 | * setting defaults |
ryanwalls | 30ce444 | 2016-01-30 11:44:53 -0700 | [diff] [blame] | 25 | * reading from JSON, TOML, YAML, HCL, and Java properties config files |
spf13 | e37b56e | 2015-11-09 23:22:04 -0500 | [diff] [blame] | 26 | * live watching and re-reading of config files (optional) |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 27 | * reading from environment variables |
Jonathan Boulle | ce08532 | 2015-11-25 11:51:57 -0800 | [diff] [blame] | 28 | * reading from remote config systems (etcd or Consul), and watching changes |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 29 | * reading from command line flags |
oliveagle | 2a7f7f4 | 2015-05-11 12:33:35 +0800 | [diff] [blame] | 30 | * reading from buffer |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 31 | * setting explicit values |
| 32 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 33 | Viper can be thought of as a registry for all of your applications |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 34 | configuration needs. |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 35 | |
| 36 | ## Why Viper? |
| 37 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 38 | When building a modern application, you don’t want to worry about |
Anthony Fok | 0c5f3e2 | 2015-03-07 03:52:13 -0700 | [diff] [blame] | 39 | configuration file formats; you want to focus on building awesome software. |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 40 | Viper is here to help with that. |
| 41 | |
| 42 | Viper does the following for you: |
| 43 | |
ryanwalls | 30ce444 | 2016-01-30 11:44:53 -0700 | [diff] [blame] | 44 | 1. Find, load, and unmarshal a configuration file in JSON, TOML, YAML, HCL, or Java properties formats. |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 45 | 2. Provide a mechanism to set default values for your different |
Anthony Fok | 0c5f3e2 | 2015-03-07 03:52:13 -0700 | [diff] [blame] | 46 | configuration options. |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 47 | 3. Provide a mechanism to set override values for options specified through |
| 48 | command line flags. |
| 49 | 4. Provide an alias system to easily rename parameters without breaking existing |
| 50 | code. |
| 51 | 5. Make it easy to tell the difference between when a user has provided a |
| 52 | command line or config file which is the same as the default. |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 53 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 54 | Viper uses the following precedence order. Each item takes precedence over the |
| 55 | item below it: |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 56 | |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 57 | * explicit call to Set |
| 58 | * flag |
| 59 | * env |
| 60 | * config |
| 61 | * key/value store |
| 62 | * default |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 63 | |
| 64 | Viper configuration keys are case insensitive. |
| 65 | |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 66 | ## Putting Values into Viper |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 67 | |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 68 | ### Establishing Defaults |
| 69 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 70 | A good configuration system will support default values. A default value is not |
Derek Parker | cc3dd55 | 2015-12-03 11:47:09 -0800 | [diff] [blame] | 71 | required for a key, but it's useful in the event that a key hasn’t been set via |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 72 | config file, environment variable, remote configuration or flag. |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 73 | |
| 74 | Examples: |
| 75 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 76 | ```go |
| 77 | viper.SetDefault("ContentDir", "content") |
| 78 | viper.SetDefault("LayoutDir", "layouts") |
| 79 | viper.SetDefault("Taxonomies", map[string]string{"tag": "tags", "category": "categories"}) |
| 80 | ``` |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 81 | |
| 82 | ### Reading Config Files |
| 83 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 84 | Viper requires minimal configuration so it knows where to look for config files. |
ryanwalls | 30ce444 | 2016-01-30 11:44:53 -0700 | [diff] [blame] | 85 | Viper supports JSON, TOML, YAML, HCL, and Java Properties files. Viper can search multiple paths, but |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 86 | currently a single Viper instance only supports a single configuration file. |
Vlad Didenko | a7ef020 | 2015-11-06 13:43:15 -0600 | [diff] [blame] | 87 | Viper does not default to any configuration search paths leaving defaults decision |
| 88 | to an application. |
| 89 | |
| 90 | Here is an example of how to use Viper to search for and read a configuration file. |
| 91 | None of the specific paths are required, but at least one path should be provided |
spf13 | e37b56e | 2015-11-09 23:22:04 -0500 | [diff] [blame] | 92 | where a configuration file is expected. |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 93 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 94 | ```go |
| 95 | viper.SetConfigName("config") // name of config file (without extension) |
| 96 | viper.AddConfigPath("/etc/appname/") // path to look for the config file in |
| 97 | viper.AddConfigPath("$HOME/.appname") // call multiple times to add many search paths |
Vlad Didenko | a7ef020 | 2015-11-06 13:43:15 -0600 | [diff] [blame] | 98 | viper.AddConfigPath(".") // optionally look for config in the working directory |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 99 | err := viper.ReadInConfig() // Find and read the config file |
| 100 | if err != nil { // Handle errors reading the config file |
| 101 | panic(fmt.Errorf("Fatal error config file: %s \n", err)) |
| 102 | } |
| 103 | ``` |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 104 | |
spf13 | e37b56e | 2015-11-09 23:22:04 -0500 | [diff] [blame] | 105 | ### Watching and re-reading config files |
| 106 | |
| 107 | Viper supports the ability to have your application live read a config file while running. |
| 108 | |
| 109 | Gone are the days of needing to restart a server to have a config take effect, |
| 110 | viper powered applications can read an update to a config file while running and |
| 111 | not miss a beat. |
| 112 | |
| 113 | Simply tell the viper instance to watchConfig. |
| 114 | Optionally you can provide a function for Viper to run each time a change occurs. |
| 115 | |
| 116 | **Make sure you add all of the configPaths prior to calling `WatchConfig()`** |
| 117 | |
| 118 | ```go |
| 119 | viper.WatchConfig() |
| 120 | viper.OnConfigChange(func(e fsnotify.Event) { |
| 121 | fmt.Println("Config file changed:", e.Name) |
| 122 | }) |
| 123 | ``` |
| 124 | |
oliveagle | 0d75ece | 2015-05-15 19:36:05 +0800 | [diff] [blame] | 125 | ### Reading Config from io.Reader |
oliveagle | 2a7f7f4 | 2015-05-11 12:33:35 +0800 | [diff] [blame] | 126 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 127 | Viper predefines many configuration sources such as files, environment |
| 128 | variables, flags, and remote K/V store, but you are not bound to them. You can |
| 129 | also implement your own required configuration source and feed it to viper. |
oliveagle | 2a7f7f4 | 2015-05-11 12:33:35 +0800 | [diff] [blame] | 130 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 131 | ```go |
oliveagle | 2a7f7f4 | 2015-05-11 12:33:35 +0800 | [diff] [blame] | 132 | viper.SetConfigType("yaml") // or viper.SetConfigType("YAML") |
| 133 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 134 | // any approach to require this configuration into your program. |
oliveagle | 2a7f7f4 | 2015-05-11 12:33:35 +0800 | [diff] [blame] | 135 | var yamlExample = []byte(` |
| 136 | Hacker: true |
| 137 | name: steve |
| 138 | hobbies: |
| 139 | - skateboarding |
| 140 | - snowboarding |
| 141 | - go |
| 142 | clothing: |
| 143 | jacket: leather |
| 144 | trousers: denim |
| 145 | age: 35 |
| 146 | eyes : brown |
| 147 | beard: true |
| 148 | `) |
| 149 | |
oliveagle | 0d75ece | 2015-05-15 19:36:05 +0800 | [diff] [blame] | 150 | viper.ReadConfig(bytes.NewBuffer(yamlExample)) |
oliveagle | 2a7f7f4 | 2015-05-11 12:33:35 +0800 | [diff] [blame] | 151 | |
| 152 | viper.Get("name") // this would be "steve" |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 153 | ``` |
oliveagle | 2a7f7f4 | 2015-05-11 12:33:35 +0800 | [diff] [blame] | 154 | |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 155 | ### Setting Overrides |
| 156 | |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 157 | These could be from a command line flag, or from your own application logic. |
| 158 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 159 | ```go |
| 160 | viper.Set("Verbose", true) |
| 161 | viper.Set("LogFile", LogFile) |
| 162 | ``` |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 163 | |
| 164 | ### Registering and Using Aliases |
| 165 | |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 166 | Aliases permit a single value to be referenced by multiple keys |
| 167 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 168 | ```go |
| 169 | viper.RegisterAlias("loud", "Verbose") |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 170 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 171 | viper.Set("verbose", true) // same result as next line |
| 172 | viper.Set("loud", true) // same result as prior line |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 173 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 174 | viper.GetBool("loud") // true |
| 175 | viper.GetBool("verbose") // true |
| 176 | ``` |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 177 | |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 178 | ### Working with Environment Variables |
| 179 | |
| 180 | Viper has full support for environment variables. This enables 12 factor |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 181 | applications out of the box. There are four methods that exist to aid working |
| 182 | with ENV: |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 183 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 184 | * `AutomaticEnv()` |
| 185 | * `BindEnv(string...) : error` |
| 186 | * `SetEnvPrefix(string)` |
| 187 | * `SetEnvReplacer(string...) *strings.Replacer` |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 188 | |
Anthony Fok | 0c5f3e2 | 2015-03-07 03:52:13 -0700 | [diff] [blame] | 189 | _When working with ENV variables, it’s important to recognize that Viper |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 190 | treats ENV variables as case sensitive._ |
| 191 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 192 | Viper provides a mechanism to try to ensure that ENV variables are unique. By |
| 193 | using `SetEnvPrefix`, you can tell Viper to use add a prefix while reading from |
| 194 | the environment variables. Both `BindEnv` and `AutomaticEnv` will use this |
| 195 | prefix. |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 196 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 197 | `BindEnv` takes one or two parameters. The first parameter is the key name, the |
| 198 | second is the name of the environment variable. The name of the environment |
| 199 | variable is case sensitive. If the ENV variable name is not provided, then |
| 200 | Viper will automatically assume that the key name matches the ENV variable name, |
| 201 | but the ENV variable is IN ALL CAPS. When you explicitly provide the ENV |
| 202 | variable name, it **does not** automatically add the prefix. |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 203 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 204 | One important thing to recognize when working with ENV variables is that the |
| 205 | value will be read each time it is accessed. Viper does not fix the value when |
| 206 | the `BindEnv` is called. |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 207 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 208 | `AutomaticEnv` is a powerful helper especially when combined with |
| 209 | `SetEnvPrefix`. When called, Viper will check for an environment variable any |
| 210 | time a `viper.Get` request is made. It will apply the following rules. It will |
| 211 | check for a environment variable with a name matching the key uppercased and |
| 212 | prefixed with the `EnvPrefix` if set. |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 213 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 214 | `SetEnvReplacer` allows you to use a `strings.Replacer` object to rewrite Env |
| 215 | keys to an extent. This is useful if you want to use `-` or something in your |
| 216 | `Get()` calls, but want your environmental variables to use `_` delimiters. An |
| 217 | example of using it can be found in `viper_test.go`. |
Chance Zibolski | 03fb74b | 2015-03-06 11:21:17 -0800 | [diff] [blame] | 218 | |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 219 | #### Env example |
| 220 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 221 | ```go |
| 222 | SetEnvPrefix("spf") // will be uppercased automatically |
| 223 | BindEnv("id") |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 224 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 225 | os.Setenv("SPF_ID", "13") // typically done outside of the app |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 226 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 227 | id := Get("id") // 13 |
| 228 | ``` |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 229 | |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 230 | ### Working with Flags |
| 231 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 232 | Viper has the ability to bind to flags. Specifically, Viper supports `Pflags` |
| 233 | as used in the [Cobra](https://github.com/spf13/cobra) library. |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 234 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 235 | Like `BindEnv`, the value is not set when the binding method is called, but when |
| 236 | it is accessed. This means you can bind as early as you want, even in an |
| 237 | `init()` function. |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 238 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 239 | The `BindPFlag()` method provides this functionality. |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 240 | |
| 241 | Example: |
| 242 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 243 | ```go |
| 244 | serverCmd.Flags().Int("port", 1138, "Port to run Application server on") |
| 245 | viper.BindPFlag("port", serverCmd.Flags().Lookup("port")) |
| 246 | ``` |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 247 | |
Sanjay Bhandari | 5c53af5 | 2015-12-04 15:07:55 -0500 | [diff] [blame] | 248 | The use of [pflag](https://github.com/spf13/pflag/) in Viper does not preclude |
| 249 | the use of other packages that use the [flag](https://golang.org/pkg/flag/) |
| 250 | package from the standard library. The pflag package can handle the flags |
| 251 | defined for the flag package by importing these flags. This is accomplished |
| 252 | by a calling a convenience function provided by the pflag package called |
| 253 | AddGoFlagSet(). |
| 254 | |
| 255 | Example: |
| 256 | |
| 257 | ```go |
| 258 | package main |
| 259 | |
| 260 | import ( |
| 261 | "flag" |
| 262 | "github.com/spf13/pflag" |
| 263 | ) |
| 264 | |
| 265 | func main() { |
| 266 | pflag.CommandLine.AddGoFlagSet(flag.CommandLine) |
| 267 | pflag.Parse() |
| 268 | ... |
| 269 | } |
| 270 | ``` |
| 271 | |
David Calavera | c8c6312 | 2015-12-18 18:34:45 -0500 | [diff] [blame] | 272 | #### Flag interfaces |
| 273 | |
| 274 | Viper provides two Go interfaces to bind other flag systems if you don't use `Pflags`. |
| 275 | |
| 276 | `FlagValue` represents a single flag. This is a very simple example on how to implement this interface: |
| 277 | |
| 278 | ```go |
| 279 | type myFlag struct {} |
| 280 | func (f myFlag) IsChanged() { return false } |
| 281 | func (f myFlag) Name() { return "my-flag-name" } |
| 282 | func (f myFlag) ValueString() { return "my-flag-value" } |
| 283 | func (f myFlag) ValueType() { return "string" } |
| 284 | ``` |
| 285 | |
| 286 | Once your flag implements this interface, you can simply tell Viper to bind it: |
| 287 | |
| 288 | ```go |
| 289 | viper.BindFlagValue("my-flag-name", myFlag{}) |
| 290 | ``` |
| 291 | |
| 292 | `FlagValueSet` represents a group of flags. This is a very simple example on how to implement this interface: |
| 293 | |
| 294 | ```go |
| 295 | type myFlagSet struct { |
| 296 | flags []myFlag |
| 297 | } |
| 298 | |
| 299 | func (f myFlagSet) VisitAll(fn func(FlagValue)) { |
| 300 | for _, flag := range flags { |
| 301 | fn(flag) |
| 302 | } |
| 303 | } |
| 304 | ``` |
| 305 | |
| 306 | Once your flag set implements this interface, you can simply tell Viper to bind it: |
| 307 | |
| 308 | ```go |
| 309 | fSet := myFlagSet{ |
| 310 | flags: []myFlag{myFlag{}, myFlag{}}, |
| 311 | } |
| 312 | viper.BindFlagValues("my-flags", fSet) |
| 313 | ``` |
| 314 | |
Brian Ketelsen | 51da30f | 2014-10-27 15:32:46 -0400 | [diff] [blame] | 315 | ### Remote Key/Value Store Support |
bep | be5ff3e | 2015-05-30 21:28:33 +0200 | [diff] [blame] | 316 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 317 | To enable remote support in Viper, do a blank import of the `viper/remote` |
| 318 | package: |
bep | be5ff3e | 2015-05-30 21:28:33 +0200 | [diff] [blame] | 319 | |
David Symonds | 493be7d | 2015-10-26 14:36:42 +1100 | [diff] [blame] | 320 | `import _ "github.com/spf13/viper/remote"` |
bep | be5ff3e | 2015-05-30 21:28:33 +0200 | [diff] [blame] | 321 | |
patdhlk | 79971f1 | 2015-12-13 21:47:41 +0100 | [diff] [blame] | 322 | Viper will read a config string (as JSON, TOML, YAML or HCL) retrieved from a path |
Jonathan Boulle | ce08532 | 2015-11-25 11:51:57 -0800 | [diff] [blame] | 323 | in a Key/Value store such as etcd or Consul. These values take precedence over |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 324 | default values, but are overridden by configuration values retrieved from disk, |
Brian Ketelsen | 51da30f | 2014-10-27 15:32:46 -0400 | [diff] [blame] | 325 | flags, or environment variables. |
| 326 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 327 | Viper uses [crypt](https://github.com/xordataexchange/crypt) to retrieve |
| 328 | configuration from the K/V store, which means that you can store your |
| 329 | configuration values encrypted and have them automatically decrypted if you have |
| 330 | the correct gpg keyring. Encryption is optional. |
Brian Ketelsen | 51da30f | 2014-10-27 15:32:46 -0400 | [diff] [blame] | 331 | |
| 332 | You can use remote configuration in conjunction with local configuration, or |
Chance Zibolski | 1258332 | 2015-03-06 11:21:50 -0800 | [diff] [blame] | 333 | independently of it. |
Brian Ketelsen | 51da30f | 2014-10-27 15:32:46 -0400 | [diff] [blame] | 334 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 335 | `crypt` has a command-line helper that you can use to put configurations in your |
| 336 | K/V store. `crypt` defaults to etcd on http://127.0.0.1:4001. |
Brian Ketelsen | 51da30f | 2014-10-27 15:32:46 -0400 | [diff] [blame] | 337 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 338 | ```bash |
| 339 | $ go get github.com/xordataexchange/crypt/bin/crypt |
| 340 | $ crypt set -plaintext /config/hugo.json /Users/hugo/settings/config.json |
| 341 | ``` |
Brian Ketelsen | 51da30f | 2014-10-27 15:32:46 -0400 | [diff] [blame] | 342 | |
| 343 | Confirm that your value was set: |
| 344 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 345 | ```bash |
| 346 | $ crypt get -plaintext /config/hugo.json |
| 347 | ``` |
Brian Ketelsen | 51da30f | 2014-10-27 15:32:46 -0400 | [diff] [blame] | 348 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 349 | See the `crypt` documentation for examples of how to set encrypted values, or |
| 350 | how to use Consul. |
Brian Ketelsen | 51da30f | 2014-10-27 15:32:46 -0400 | [diff] [blame] | 351 | |
| 352 | ### Remote Key/Value Store Example - Unencrypted |
| 353 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 354 | ```go |
| 355 | viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001","/config/hugo.json") |
ryanwalls | 30ce444 | 2016-01-30 11:44:53 -0700 | [diff] [blame] | 356 | viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop" |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 357 | err := viper.ReadRemoteConfig() |
| 358 | ``` |
Brian Ketelsen | 51da30f | 2014-10-27 15:32:46 -0400 | [diff] [blame] | 359 | |
| 360 | ### Remote Key/Value Store Example - Encrypted |
| 361 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 362 | ```go |
| 363 | viper.AddSecureRemoteProvider("etcd","http://127.0.0.1:4001","/config/hugo.json","/etc/secrets/mykeyring.gpg") |
ryanwalls | 30ce444 | 2016-01-30 11:44:53 -0700 | [diff] [blame] | 364 | viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop" |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 365 | err := viper.ReadRemoteConfig() |
| 366 | ``` |
Brian Ketelsen | 51da30f | 2014-10-27 15:32:46 -0400 | [diff] [blame] | 367 | |
Jonathan Boulle | ce08532 | 2015-11-25 11:51:57 -0800 | [diff] [blame] | 368 | ### Watching Changes in etcd - Unencrypted |
oliveagle | 2a7f7f4 | 2015-05-11 12:33:35 +0800 | [diff] [blame] | 369 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 370 | ```go |
| 371 | // alternatively, you can create a new viper instance. |
| 372 | var runtime_viper = viper.New() |
oliveagle | 2a7f7f4 | 2015-05-11 12:33:35 +0800 | [diff] [blame] | 373 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 374 | runtime_viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001", "/config/hugo.yml") |
ryanwalls | 30ce444 | 2016-01-30 11:44:53 -0700 | [diff] [blame] | 375 | runtime_viper.SetConfigType("yaml") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop" |
oliveagle | 2a7f7f4 | 2015-05-11 12:33:35 +0800 | [diff] [blame] | 376 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 377 | // read from remote config the first time. |
| 378 | err := runtime_viper.ReadRemoteConfig() |
oliveagle | 2a7f7f4 | 2015-05-11 12:33:35 +0800 | [diff] [blame] | 379 | |
Matt Surabian | 1967d93 | 2015-08-23 23:40:56 -0400 | [diff] [blame] | 380 | // unmarshal config |
| 381 | runtime_viper.Unmarshal(&runtime_conf) |
oliveagle | 2a7f7f4 | 2015-05-11 12:33:35 +0800 | [diff] [blame] | 382 | |
James Mintram | 3041a43 | 2015-10-22 16:25:52 +0100 | [diff] [blame] | 383 | // open a goroutine to watch remote changes forever |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 384 | go func(){ |
| 385 | for { |
| 386 | time.Sleep(time.Second * 5) // delay after each request |
spf13 | e37b56e | 2015-11-09 23:22:04 -0500 | [diff] [blame] | 387 | |
James Mintram | 3041a43 | 2015-10-22 16:25:52 +0100 | [diff] [blame] | 388 | // currently, only tested with etcd support |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 389 | err := runtime_viper.WatchRemoteConfig() |
| 390 | if err != nil { |
| 391 | log.Errorf("unable to read remote config: %v", err) |
| 392 | continue |
| 393 | } |
spf13 | e37b56e | 2015-11-09 23:22:04 -0500 | [diff] [blame] | 394 | |
| 395 | // unmarshal new config into our runtime config struct. you can also use channel |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 396 | // to implement a signal to notify the system of the changes |
Matt Surabian | 1967d93 | 2015-08-23 23:40:56 -0400 | [diff] [blame] | 397 | runtime_viper.Unmarshal(&runtime_conf) |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 398 | } |
| 399 | }() |
| 400 | ``` |
Brian Ketelsen | 51da30f | 2014-10-27 15:32:46 -0400 | [diff] [blame] | 401 | |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 402 | ## Getting Values From Viper |
| 403 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 404 | In Viper, there are a few ways to get a value depending on the value's type. |
Chance Zibolski | 1258332 | 2015-03-06 11:21:50 -0800 | [diff] [blame] | 405 | The following functions and methods exist: |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 406 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 407 | * `Get(key string) : interface{}` |
| 408 | * `GetBool(key string) : bool` |
| 409 | * `GetFloat64(key string) : float64` |
| 410 | * `GetInt(key string) : int` |
| 411 | * `GetString(key string) : string` |
| 412 | * `GetStringMap(key string) : map[string]interface{}` |
| 413 | * `GetStringMapString(key string) : map[string]string` |
| 414 | * `GetStringSlice(key string) : []string` |
| 415 | * `GetTime(key string) : time.Time` |
| 416 | * `GetDuration(key string) : time.Duration` |
| 417 | * `IsSet(key string) : bool` |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 418 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 419 | One important thing to recognize is that each Get function will return a zero |
| 420 | value if it’s not found. To check if a given key exists, the `IsSet()` method |
| 421 | has been provided. |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 422 | |
| 423 | Example: |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 424 | ```go |
| 425 | viper.GetString("logfile") // case-insensitive Setting & Getting |
| 426 | if viper.GetBool("verbose") { |
| 427 | fmt.Println("verbose enabled") |
| 428 | } |
| 429 | ``` |
Kiril Zvezdarov | 2e47d9e | 2015-05-02 14:20:33 -0400 | [diff] [blame] | 430 | ### Accessing nested keys |
| 431 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 432 | The accessor methods also accept formatted paths to deeply nested keys. For |
| 433 | example, if the following JSON file is loaded: |
Kiril Zvezdarov | 2e47d9e | 2015-05-02 14:20:33 -0400 | [diff] [blame] | 434 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 435 | ```json |
Kiril Zvezdarov | 2e47d9e | 2015-05-02 14:20:33 -0400 | [diff] [blame] | 436 | { |
| 437 | "host": { |
| 438 | "address": "localhost", |
| 439 | "port": 5799 |
| 440 | }, |
| 441 | "datastore": { |
| 442 | "metric": { |
| 443 | "host": "127.0.0.1", |
| 444 | "port": 3099 |
| 445 | }, |
| 446 | "warehouse": { |
| 447 | "host": "198.0.0.1", |
| 448 | "port": 2112 |
| 449 | } |
| 450 | } |
| 451 | } |
| 452 | |
| 453 | ``` |
| 454 | |
| 455 | Viper can access a nested field by passing a `.` delimited path of keys: |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 456 | |
| 457 | ```go |
Kiril Zvezdarov | 2e47d9e | 2015-05-02 14:20:33 -0400 | [diff] [blame] | 458 | GetString("datastore.metric.host") // (returns "127.0.0.1") |
| 459 | ``` |
| 460 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 461 | This obeys the precedence rules established above; the search for the root key |
| 462 | (in this example, `datastore`) will cascade through the remaining configuration |
| 463 | registries until found. The search for the sub-keys (`metric` and `host`), |
| 464 | however, will not. |
Kiril Zvezdarov | 2e47d9e | 2015-05-02 14:20:33 -0400 | [diff] [blame] | 465 | |
| 466 | For example, if the `metric` key was not defined in the configuration loaded |
| 467 | from file, but was defined in the defaults, Viper would return the zero value. |
| 468 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 469 | On the other hand, if the primary key was not defined, Viper would go through |
| 470 | the remaining registries looking for it. |
Kiril Zvezdarov | 2e47d9e | 2015-05-02 14:20:33 -0400 | [diff] [blame] | 471 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 472 | Lastly, if there exists a key that matches the delimited key path, its value |
| 473 | will be returned instead. E.g. |
Kiril Zvezdarov | 2e47d9e | 2015-05-02 14:20:33 -0400 | [diff] [blame] | 474 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 475 | ```json |
Kiril Zvezdarov | 2e47d9e | 2015-05-02 14:20:33 -0400 | [diff] [blame] | 476 | { |
| 477 | "datastore.metric.host": "0.0.0.0", |
| 478 | "host": { |
| 479 | "address": "localhost", |
| 480 | "port": 5799 |
| 481 | }, |
| 482 | "datastore": { |
| 483 | "metric": { |
| 484 | "host": "127.0.0.1", |
| 485 | "port": 3099 |
| 486 | }, |
| 487 | "warehouse": { |
| 488 | "host": "198.0.0.1", |
| 489 | "port": 2112 |
| 490 | } |
| 491 | } |
| 492 | } |
| 493 | |
| 494 | GetString("datastore.metric.host") //returns "0.0.0.0" |
| 495 | ``` |
| 496 | |
Lei Feng | 105e3d0 | 2015-12-24 19:44:44 +0800 | [diff] [blame] | 497 | ### Extract sub-tree |
| 498 | |
| 499 | Extract sub-tree from Viper. |
| 500 | |
| 501 | For example, `viper` represents: |
| 502 | |
| 503 | ```json |
| 504 | app: |
| 505 | cache1: |
| 506 | max-items: 100 |
| 507 | item-size: 64 |
| 508 | cache2: |
| 509 | max-items: 200 |
| 510 | item-size: 80 |
| 511 | ``` |
| 512 | |
| 513 | After executing: |
| 514 | |
| 515 | ```go |
| 516 | subv := viper.Sub("app.cache1") |
| 517 | ``` |
| 518 | |
| 519 | `subv` represents: |
| 520 | |
| 521 | ```json |
| 522 | max-items: 100 |
| 523 | item-size: 64 |
| 524 | ``` |
| 525 | |
| 526 | Suppose we have: |
| 527 | |
| 528 | ```go |
| 529 | func NewCache(cfg *Viper) *Cache {...} |
| 530 | ``` |
| 531 | |
| 532 | which creates a cache based on config information formatted as `subv`. |
| 533 | Now it's easy to create these 2 caches separately as: |
| 534 | |
| 535 | ```go |
| 536 | cfg1 := viper.Sub("app.cache1") |
| 537 | cache1 := NewCache(cfg1) |
| 538 | |
| 539 | cfg2 := viper.Sub("app.cache2") |
| 540 | cache2 := NewCache(cfg2) |
| 541 | ``` |
| 542 | |
Matt Surabian | 1967d93 | 2015-08-23 23:40:56 -0400 | [diff] [blame] | 543 | ### Unmarshaling |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 544 | |
Matt Surabian | 1967d93 | 2015-08-23 23:40:56 -0400 | [diff] [blame] | 545 | You also have the option of Unmarshaling all or a specific value to a struct, map, |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 546 | etc. |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 547 | |
| 548 | There are two methods to do this: |
| 549 | |
Matt Surabian | 1967d93 | 2015-08-23 23:40:56 -0400 | [diff] [blame] | 550 | * `Unmarshal(rawVal interface{}) : error` |
| 551 | * `UnmarshalKey(key string, rawVal interface{}) : error` |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 552 | |
| 553 | Example: |
| 554 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 555 | ```go |
| 556 | type config struct { |
| 557 | Port int |
| 558 | Name string |
Abhinandan | 823bc13 | 2015-12-03 21:44:42 +0530 | [diff] [blame] | 559 | PathMap string `mapstructure:"path_map"` |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 560 | } |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 561 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 562 | var C config |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 563 | |
Matt Surabian | 1967d93 | 2015-08-23 23:40:56 -0400 | [diff] [blame] | 564 | err := Unmarshal(&C) |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 565 | if err != nil { |
| 566 | t.Fatalf("unable to decode into struct, %v", err) |
| 567 | } |
| 568 | ``` |
spf13 | 54fed16 | 2014-12-22 21:45:56 -0500 | [diff] [blame] | 569 | |
| 570 | ## Viper or Vipers? |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 571 | |
| 572 | Viper comes ready to use out of the box. There is no configuration or |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 573 | initialization needed to begin using Viper. Since most applications will want |
| 574 | to use a single central repository for their configuration, the viper package |
| 575 | provides this. It is similar to a singleton. |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 576 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 577 | In all of the examples above, they demonstrate using viper in it's singleton |
| 578 | style approach. |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 579 | |
| 580 | ### Working with multiple vipers |
| 581 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 582 | You can also create many different vipers for use in your application. Each will |
| 583 | have it’s own unique set of configurations and values. Each can read from a |
| 584 | different config file, key value store, etc. All of the functions that viper |
| 585 | package supports are mirrored as methods on a viper. |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 586 | |
| 587 | Example: |
| 588 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 589 | ```go |
| 590 | x := viper.New() |
| 591 | y := viper.New() |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 592 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 593 | x.SetDefault("ContentDir", "content") |
| 594 | y.SetDefault("ContentDir", "foobar") |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 595 | |
Maxime Horcholle | 1e6a237 | 2015-07-16 14:16:31 +0200 | [diff] [blame] | 596 | //... |
| 597 | ``` |
spf13 | cacbc4c | 2014-12-07 02:03:49 +0100 | [diff] [blame] | 598 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 599 | When working with multiple vipers, it is up to the user to keep track of the |
| 600 | different vipers. |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 601 | |
| 602 | ## Q & A |
| 603 | |
| 604 | Q: Why not INI files? |
| 605 | |
Anthony Fok | 0c5f3e2 | 2015-03-07 03:52:13 -0700 | [diff] [blame] | 606 | A: Ini files are pretty awful. There’s no standard format, and they are hard to |
| 607 | validate. Viper is designed to work with JSON, TOML or YAML files. If someone |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 608 | really wants to add this feature, I’d be happy to merge it. It’s easy to specify |
| 609 | which formats your application will permit. |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 610 | |
Anthony Fok | 0c5f3e2 | 2015-03-07 03:52:13 -0700 | [diff] [blame] | 611 | Q: Why is it called “Viper”? |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 612 | |
jackspirou | 2abb1be | 2015-07-19 12:52:04 -0500 | [diff] [blame] | 613 | A: Viper is designed to be a [companion](http://en.wikipedia.org/wiki/Viper_(G.I._Joe)) |
| 614 | to [Cobra](https://github.com/spf13/cobra). While both can operate completely |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 615 | independently, together they make a powerful pair to handle much of your |
| 616 | application foundation needs. |
| 617 | |
Anthony Fok | 0c5f3e2 | 2015-03-07 03:52:13 -0700 | [diff] [blame] | 618 | Q: Why is it called “Cobra”? |
spf13 | a54c415 | 2014-04-05 15:44:26 -0400 | [diff] [blame] | 619 | |
Anthony Fok | 0c5f3e2 | 2015-03-07 03:52:13 -0700 | [diff] [blame] | 620 | A: Is there a better name for a [commander](http://en.wikipedia.org/wiki/Cobra_Commander)? |