Deploying Sisense on EC2 with FSX for Lustre

Amazon FSx for Lustre provides a high-performance file system optimized for fast processing of workloads such as that of Sisense. Using AWS CloudFormation, you can launch a cluster of worker nodes on Amazon EC2, and then launch Sisense onto your cluster.

Prerequisites

Deploying Sisense

  1. Create a new instance support by Sisense.
  2. In your new instance, enter the following command to download the Sisense tar.gz file.
    wget [sisense-linux-deployment-link]
    Contact Sisense to receive the latest Linux archive file.
  3. Extracting the tar.gz file:
    tar zxf [sisense-linux-deployment-package-name]
  4. Navigate to the directory where you extracted the tar.gz file.
    cd sisense-[sisense-version]
  5. Edit the config.yaml file.
    vim cluster_config.yaml
    After running this command, the parameters of the cluster_config.yaml file are displayed.
  6. In cluster_config.yaml file, insert values for the following parameters:
    Note: If the following parameters are not included in your cluster_config.yaml file, you should include them and assign the values as described in the table below:
    • cloud_provider
    • fsx_file_system_id
    • fsx_region
      ParametersValue

      k8s_nodes

      K8S node/nodes are the set of machines that will be used to run Sisense.

      The installation machine is used only during installation to run the installation scripts. The Installation machine can be one of the K8S nodes, but it can also be a different machine (remote installation).

      This parameter defines the IP address of your internal and external IP addresses.

      If you are implementing a multi-node cluster deployment, define the following for each node: 

      node: Your node's AWS private DNS name.

      internal IP: Enter the addresses that define the location of your nodes within your Kubernetes cluster.

      external IP: address defines how your cluster can be accessed from outside, or the DNS name of the node. It can also be the same value as the internal IP in case this is the only IP used to access the node.

      roles: Define the role of the node. There are two possible values: application, query and build.

      remote_installation

      If you are running your installation from a Bastion machine, part of the K8S nodes, or remotely, enter true.

      update

      If you are upgrading Sisense, enter yes, otherwise, keep this value as the default.  

      If you enter yes, Sisense skips the Kubernetes installation. If your Kuberenetes cluster is already configured and running, keep yes to skip cluster installation.

      Staging and dev environments can run on the same cluster as a separate namespace. Make sure you have enough storage in the GlusterFS for a second namespace. For installing on a separate namespace,the value of namespace_name needs to be updated.

      offline_installer

      Set to true when you are using an offline installer.

      docker_registry

      Enter the address of your server.

      application_dns_name

      Enter your DNS name. The default is the first node of your external IP.

      If you have not defined a secure connection (No SSL), Sisense uses the external IP of your first node when accessing Sisense.

      linux_user

      Enter the name of your Linux user. This user must not be the "root" user, but should have sudo privileges, and all the other privileges as a root user.

      ssh_key

      If you have a secure connection to your server, enter the SSH key of the Linux user-defined in sisense_user. The SSH key should be in .pem format.

      storage_type

      Enter fsx. For Amazon EKS, enter fsx for using Amazon FSx luster. You must enter values for fsx_subnet_id and fsx_security_group_id.

      fsx_file_system_id

      Enter your EFS AWS region.

      fsx_region

      Enter your Amazon FSx luster file system ID.

      sisense_disk_size

      If you have implemented GlusterFS for storage, enter the amount of disk space in gigabytes to be allocated for Sisense.

      Important! You need to provide enough space to support your Sisense ElastiCube models, at least two times the amount of data in all ElastiCubes. GlusterFS is set for two replicas so assume the same logical disk is in two of the physical disks you set in the disk_volume_device.

      Every namespace takes additional logical disk space from the physical disks. If you have not allocated enough space, the installation will fail.

      You can use the following equation:

      sisense_disk_size = 50GB x 3. (In a three node deployment)

      If you allocate space for the application DB and configuration DB (the values of mongodb_disk_size and zookeeper_disk_size) this should be considered as well. If you enter 150GB as the value, this allocates 50GB in a 3 node deployment minus 3 times (the values of mongodb_disk_size and zookeeper_disk_size).

      Sisense also recommends that you specify an additional 5GB free space for the value of sisense_disk_size.

      mongodb_disk_size

      The amount of disk space allocated for the Sisense application database.

      This value should be multiplied by the number of nodes your deployment has.

      It is recommended to leave the default of 3 GB. If only metadata is stored in the MongoDB, there is no need to increase the size.

      zookeeper_disk_size

      The amount of disk space allocated for the ZooKeeper service.

      This value should be multiplied by the number of nodes your deployment has.

      It is recommended to leave the default of 1 GB. If only metadata is stored in Zookeeper, there is no need to increase the size.

      namespace_name

      Enter the name of the Kubernetes namespace.

      If you have multiple deployments, for example, for a development and production environment, you should have a unique namespace for each deployment.

      In addition, for multiple deployments, each should have a unique gateway_port value, and for each deployment after the first, the value of update should be set to true.

      Note: Kubernetes Ports should be released (Non-listening mode)

      gateway_port

      Enter the port of the API gateway for your deployment.

      If you are not implementing SSL, this will be the port used to connect to Sisense.

      is_ssl

      Enter true for secure connections to Sisense. Enter false if you have not implemented SSL.

      If you enter true, see Setting Up SSL in Sisense Linux for more information.

      ssl_key_path

      If you connect to your server securely, enter the SSL keypath.

      When SSL is defined, the Sisense API Gateway Port will be 443 and not the value defined in gateway_port.

      ssl_cer_path

      If you connect to your server securely, enter the SSL certificate path (.cer file).

      weave_enabled

      Enter true if your cluster uses Azure virtual machines or any other place in your deployment does not support IP in IP network protocol. The default network plugin is Calico,. Sisense supports the Weave network plugin for communication between nodes. You must include your Weave password in the value for weave_password below.

      Enabling weave will change the network protocol between the nodes to secured vXLAN

      If the weave_password is empty, the vXLAN will not be encrypted.

      new_node

      If you have already provisioned Sisense and have a working cluster, you can add additional nodes. Enter true to add new nodes. The new nodes should be defined under k8s_nodes above in addition to your existing nodes. Adding new nodes does not affect your existing nodes.

      remove_node

      Enter true to remove a node from your cluster. Enter the node to be removed in the value of node_to_remove below.

      nodes_to_remove

      Enter the name of a node to be removed. You can remove multiple nodes by entering each node separated by a comma; node_1, node_2.

      recover_kubernetes

      Enter true if one of your nodes is broken. Sisense removes the node and recovers it.

      http_proxy

      If you must use a web proxy, set this to the unsecured proxy value.

      https_proxy

      If you must use a web proxy, set this to the secured proxy value.

      no_proxy

      If you must use a web proxy, to exclude certain addresses from your proxy, enter the IP addresses separated by a comma.

      You need to include the internal IP address of the host in the list.

      internal_monitoring

      Enter false to disable a Grafana dashboard that is supported by Prometheus for monitoring your deployment.

      For more information, see Monitoring Sisense in Linux.

      external_monitoring

      Enter false to disable external monitoring with your Logz.io account. If you set this value to true, provide values for sisense_ownerid and logz_key.

      For more information, see Monitoring Sisense in Linux.

      uninstall_cluster

      Enter true to remove Kubernetes infrastructure and Sisense services.

      uninstall_sisense

      Enter true to uninstall Sisense services, but leave your Kubernetes infrastructure in case it's needed in the future. This can also be used if you need to remove Sisense from your own cloud-based cluster without impacting the cluster itself. Only Sisense the application is removed.

      remove_user_data

      Enter true to delete all user data. This deletes your ElastiCube models, application database, message broker, and add-ons.

  7. Run the configuration script.
    ./sisense.sh cluster_config.yaml
    Your configuration settings are displayed with a message to confirm that you want to deploy Sisense with these settings.
  8. Enter Yes to confirm that you want to deploy Sisense. Enter No to abort the deployment. If you entered Yes, Sisense will be deployed. If there are any issues, you can view the installation logs here: [installation-dir]/sisense-ansible.log. When this installation is complete a list of endpoints is displayed for accessing Sisense and managing your deployment. The URLs are listed below. In addition, you can run the following command to return the URL to access Sisense.
    kubectl cluster-info
    This displays the URL of your Sisense application. You can enter this address into your browser to access Sisense. To verify that all your services are running as expected, you can enter the URL of Sisense with the port and /app/test to the end of the address in your browser. This displays the status of each of your services. For example, 0.0.0.0:PORT/app/test.