Shipping Logs to Elasticsearch Servers

Prerequisites

• This feature only supports Elasticsearch version 7 and above.

Configuring Sisense to Send Logs

Sisense enables you to ship log files to your Elasticsearch server to index your data. You can use these logs by themselves or ship them in addition to your existing logs.

If you installed Sisense with the parameter external_monitoring set to true in the configuration YAML file, Sisense sends your logs to a Sisense account at Logz.io for data processing and analysis. With some additional parameters in your configuration YAML file, you can send these logs to Elasticsearch.

To configure Sisense to send logs to Elasticsearch:

1. In your configuration YAML file, modify and add the following values in the configuration file (cluster_config.yaml for example):

   external_monitoring: true
   external_log_provider: elasticsearch
   elasticsearch_host: 10.50.31.144 (can be DNS address as well)
   elasticsearch_port: 31009 (the port that Elasticsearch is used to listening to)
   elasticsearch_user: sisense (User with access to Elasticsearch)
   elasticsearch_password: sisense
   elasticsearch_sslVerify: true (In case the Elasticsearch server is behind a trusted SSL, otherwise, change it to false to skip verification)
   

   

2. After you have configured all the relevant settings, Initializing Sisense on Linux Sisense and activate it. After a few minutes, a Sisense index will be created in your Elasticsearch server.

Increasing Elasticsearch Index Limits

Elasticsearch’s default index limit is set to 1000. To accept all Sisense logs,y our created index should be increased to 2000.

To increase the limit:

1. In Elasticsearch’s Dev Tools, enter the following code and run it:

   PUT sns-sisense-2020.11.25/_settings
   {
      “Index.mapping.total_fields.limit”: 2000
   }
   

   

2. In Kibana, create an index pattern so the logs will be displayed, select Index Patterns > Create index pattern.

3. Provide a name for the new index pattern. One pattern should match all the indexes. You can use wildcards.

   

   

4. Open Kibana > Discover to see the logs.