Keeping Chef logs quiet
When you’re looking at Chef logs in production, there are a handful of things you care about.
- Which recipes are running?
- What changed since the last run?
- Did anything break?
- How long did the run take?
Chef shows you those things out of the box, but there’s a lot of noise mixed in. When you’re running Chef every ten minutes on every server, logs fill up fast with with information about state that hasn’t changed.
Fortunately, the Chef client has a setting you can use to keep things quiet. Just stick these lines in your client.rb file:
log_level :info
verbose_logging false
Keeping the log level at INFO means Chef will still show you the run list for the node and how long the run took. Plus, you’ll see custom messages your recipes output with the log resource.
Setting verbose logging to false (the default value is nil) means Chef will
only show you what’s changed on a node during that run. Instead of seeing events
like “Processing service[nginx] start” you’ll only see a log entry when the
Nginx service changes to a starting state.
Trimming your Chef logs down to show you just what’s changed on a server also makes it easier to set up LogStash filters that alert on abnormalities. Instead of having to grep and grok around the noise, you can cut right to the chase.
Logs become a lot more useful when they only show you the things you care about.