Elastic Watcher in Five Minutes

X-Pack exposes REST APIs that can be used by the UI components and can be called directly to configure and access X-Pack features. Watcher APIs is one such feature that we are going to discuss in this blog.

Elasticsearch has astounding documentation for the majority of its APIs. But, when you are running short of time, you’re looking to just try it out quickly. This five-minutes read will assist you on that front.

PUT _xpack/watcher/watch/watch_name
{   
  “trigger” : {},   
  “input” : {},   
  “condition” : { },   
  “transform” : { },   
  “actions” : {}
}

This is a simple elastic call that can be executed in Kibana console. This call will create an Elastic Watcher with the name “watch_name”. Here, we will discuss each element separately:

Trigger

Determines when the watch execution process should start. You can either provide a cron expression or specify a period in terms of intervals.

"trigger" : {
   "schedule" : {
      "interval" : "5m"
   }
}

This is an interval based trigger that will execute in a five-minute interval once the Watcher is created.

Input

Input is the source from where we fetch the data and load it into the execution context. The result from this input is called a “watcher payload” or “context payload”. Watcher supports four types of inputs:

• simple: load static data into the execution context
• search: load the results of a search into the execution context
• http: load the results of an HTTP request into the execution context
• chain: use a series of inputs to load data into the execution context

Condition

It is used to decide whether we should invoke a Watcher action or not. Default is true.

"condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gt": 0
      }
    }
  }

Transformation

A “Transform” processes and changes the payload in the watch execution context to prepare it for the watch actions.

Action

It is executed when conditions are met. Eg. Send an email, send a slack message, call a rest service, add loggers, etc.,

"actions": {
    "log": {
      "logging": {
        "text": "We got the expected error"
      }
    }
  }

Watcher APIs

1.Create Watcher

PUT _xpack/watcher/watch/watch_name
{   
  \\Body 
}

2. Get Watcher

GET _xpack/watcher/watch/watch_name

3. Activate/Deactivate watcher

PUT _xpack/watcher/watch/watch_name/_activate/

PUT _xpack/watcher/watch/watch_name/_deactivate

4. Execute Watcher

PUT _xpack/watcher/watch/watch_name/_execute

So our final Watcher looks something like this:

PUT _xpack/watcher/watch/log_error_watch
{
  "trigger": {
    "schedule": {
      "interval": "10m"
    }
  },
  "input": {
    "search": {
      "request": {
        "indices": ["logs"],
        "body": {
          "query": {
            "match": {
              "message": "error"
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gt": 0
      }
    }
  },
  "actions": {
    "log": {
      "logging": {
        "text": "We got the expected error"
      }
    }
  }
}

Highlights about the Watcher definition above:

• This Watcher will execute once every 10 minutes
• It will look for a particular log with the specified status in the input i.e. the search query
• In the event that the condition is fulfilled, it will execute the action
• In the action, we simply add a logger in the elastic log, which you can check in the elastic console

This should get you started with Watchers.

Below, I have provided a sample Watcher which will execute different kinds of actions. You can refer to them and create a new one with your desired requirements.

(Note: Please check the below sample queries to add a new logs index and feed it with data)

Steps to follow

Step 1: Add a new index logs

PUT logs

Step 2: Add mapping for the newly added index

PUT logs/_mapping/event
{
  "properties": 
    {
      "request":     { "type": "keyword" },
      "status_code": { "type": "keyword" },
      "message":     { "type": "keyword" },
      "timestamp":   { "type": "date"    }
    }
}

Step 3: Add a document to this index which will satisfy the Watcher condition

POST logs/event
{
  "timestamp" : "2015-05-17T18:12:07.613Z",
  "request" : "GET index.html",
  "status_code" : 404,
  "message" : "Error: File not found"
}

Step 4: Execute the Watcher

PUT _xpack/watcher/watch/log_error_watch/_execute

A more advanced version is noted below. But for this, you might need to perform some basic setup, like configuring an email account/slack account (Mail & Slack), web-hook implementations for elastic, etc.

PUT _xpack/watcher/watch/log_error_watch
{
  "trigger": {
    "schedule": {
      "interval": "10m"
    }
  },
  "input": {
    "search": {
      "request": {
        "indices": ["logs"],
        "body": {
          "query": {
            "match": {
              "message": "error"
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gt": 0
      }
    }
  },
  "actions": {
    "log": {
      "logging": {
        "text": "We got the expected error"
      }
    },
    "email_action": {
      "email": {
        "to": "test@gmail.com",
        "subject": "Encountered  errors",
        "body": "Too many 404 error in the system, see attached data",
        "attachments": {
          "attached_data": {
            "data": {
              "format": "json"
            }
          }
        }
      }
    },
    "webhook_action": {
      "webhook": {
        "method": "POST",
        "host": "localhost",
        "port": 1234,
        "path": "/",
        "body": "Encountered  errors"
      }
    },
     "notify-slack": {
      "throttle_period": "5m",
      "slack": {
        "message": {
          "to": [
            "#admins",
            "@chief-admin"
          ],
          "text": "Encountered   errors in the last 5 minutes (facepalm)"
        }
      }
    }
  }
}    

If you are looking for Cloud based services for your business needs, don’t forget to check out our Cloud services solutions.