Shipping Logs to Elasticsearch Servers
  • 18 May 2022
  • 1 Minute to read
  • Contributors
  • Dark
    Light
  • PDF

Shipping Logs to Elasticsearch Servers

  • Dark
    Light
  • PDF

Prerequisites

  • This feature only supports Elasticsearch version 7 and above.

Configuring Sisense to Send Logs

Sisense enables you to ship log files to your Elasticsearch server to index your data. You can use these logs by themselves or ship them in addition to your existing logs.

If you installed Sisense with the parameter external_monitoring set to true in the configuration YAML file, Sisense sends your logs to a Sisense account at Logz.io for data processing and analysis. With some additional parameters in your configuration YAML file, you can send these logs to Elasticsearch.

To configure Sisense to send logs to Elasticsearch:

  1. In your configuration YAML file, modify and add the following values in the configuration file (cluster_config.yaml for example):

    external_monitoring: true
    external_log_provider: elasticsearch
    elasticsearch_host: 10.50.31.144 (can be DNS address as well)
    elasticsearch_port: 31009 (the port that Elasticsearch is used to listening to)
    elasticsearch_user: sisense (User with access to Elasticsearch)
    elasticsearch_password: sisense
    elasticsearch_sslVerify: true (In case the Elasticsearch server is behind a trusted SSL, otherwise, change it to false to skip verification)
    

    image.png

  2. After you have configured all the relevant settings, Initializing Sisense on Linux Sisense and activate it. After a few minutes, a Sisense index will be created in your Elasticsearch server.

Increasing Elasticsearch Index Limits

Elasticsearch’s default index limit is set to 1000. To accept all Sisense logs,y our created index should be increased to 2000.

To increase the limit:

  1. In Elasticsearch’s Dev Tools, enter the following code and run it:

    PUT sns-sisense-2020.11.25/_settings
    {
       “Index.mapping.total_fields.limit”: 2000
    }
    

    image.png

  2. In Kibana, create an index pattern so the logs will be displayed, select Index Patterns > Create index pattern.

  3. Provide a name for the new index pattern. One pattern should match all the indexes. You can use wildcards.

    image.png

    image.png

  4. Open Kibana > Discover to see the logs.
    image.png


Was this article helpful?