Instana Install & Upgrade Guide Single VM Setup by Stanctl
Instana Install & Upgrade Guide Single VM Setup by Stanctl
Instana Install & Upgrade Guide Single VM Setup by Stanctl
Table of Contents
OVERVIEW ................................................................................................................................... 2
VM SPECS ...................................................................................................................................... 2
ARCHITECTURE ................................................................................................................................ 3
ANNEX....................................................................................................................................... 17
This doc focuses on option #3, which is “3. On-prem: Single-Node with bundled K3s-based
Kubernetes, aka Standard Edition”, by the newly built “stanctl” CLI, based on the GA’ed
version v1.0.0, on either Ubuntu 20.04 or RHEL 8.x, with both online and air-gapped modes.
Please check out the doc here for all supported Operating Systems, which are evolving very
fast. And other Operating Systems may just technically work too, but just “not yet QA’ed”.
VM Specs
Currently, there are two profiles provided out of the box, which require different VM specs:
Profile vCPU Memory Purposes
demo 16 64GB Demo, test, POC
production 24 96GB Production, heavy POC
Instana, or APM in general, is IO intensive, separate and fast storage volumes are required
for the storage capacity of the desired data directories / mount points. As an initial size, it’s
recommended as below:
Profile Data Metrics Analytics Objects Total
demo 150G 300G 500G 250G 1.2T
production 500G 1000G 1200G 1000G 3.7T
Please note that in the actual POC, we may use less disks than abovementioned specs, by
ignoring the pre-flight check, “stanctl up --skip-preflight-check” so that you may run POC
with say 700G disks – but you have to ensure that the given disks are sufficient, of course.
Since this is just a quick experiment, I used the default root disk based IBM Cloud’s classic
VM, where the size is 100G and the IOPS is 10k+, which is good enough.
Architecture
All components are containerized and deployed on the bundled “K3s” – a lightweight
Kubernetes distribution by CNCF.
1. Spinning up the VM
Omitted, you may simply get one from TechZone.
2. Preparing
Firstly, let’s perform such prerequisite checks within the Instana Server VM.
Please refer to “Prepare the additional disks” section in Instana official doc, here, when
you’re preparing external disks for the actual mount points.
Or simply create some data folders for simplicity purposes. For example:
• Ture/set some required Linux kernel parameters for the sake of performance.
# vm.swappiness
echo "vm.swappiness=0" | sudo tee -a /etc/sysctl.d/99-stanctl.conf && \
sudo sysctl -p /etc/sysctl.d/99-stanctl.conf
# fs.inotify.max_user_instances
echo "fs.inotify.max_user_instances=8192" | sudo tee -a /etc/sysctl.d/99-stanctl.conf && \
sudo sysctl -p /etc/sysctl.d/99-stanctl.conf
You may have another check after reboot on the transparent hugepages (THP) config and
make sure it’s set as “never”:
$ cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
Otherwise you may need to follow Red Hat’s doc here, if you’re with RHEL, to go through
some extra steps:
[vm]
transparent_hugepages=never
EOF
• The TLS cert is optional but if you want to generate and use your own, refer to section of
Annex -> TLS Certs for details.
3.1 On Ubuntu
$ stanctl --version
stanctl version 1.0.0 (commit=0088a63c15f68bbdb9de21bc69d25d6798ab4cc7, date=2024-03-28T07:35:40Z)
There are sub-commands that we can break down the Instana backend installation process
but the simplest way to spin up the complete Instana backend, including datastore
components, is to run “stanctl up”:
$ stanctl up
Important note: there are some pre-flight checks including the checks of the volume sizes of
the mount points. If you’re running small POCs and just simply trying on TechZone, which
typically has 100G disk only, you may skip such checks by enabling the flag of “--skip-
preflight-check”, like this:
$ stanctl up --skip-preflight-check
This will kick off the guided installation process, and what we need to prepare and decide
are just a few items:
• Installation type: we should pick “demo” if the resource specs are 16c64G.
• Instana download key: this is part of the license.
• Instana sales key: this is part of the license.
• Domain name / FQDN: for simplicity, we can simply pick <IP>.nip.io, which will
become a good FQDN that can automatically resolve the IP specified.
• Tenant name: Instana has built-in multi-tenancy support, type your desired Tenant
name here.
• Unit name: Under the tenant, there can be 1 or multiple Units, where the actually
data will be reside in for isolation and management purposes.
• TLS cert: as mentioned, we can prepare beforehand or simply let the CLI to generate
for us.
$ stanctl up --skip-preflight-check
The overall process takes about 15-30 minutes, mainly depending on your server’s
download speed for the container images.
Please also note that, the “stanctl” CLI well supports .env so you may simply put all
necessary variables into such a “.env” file to make the installation process fully quiet and
streamlined without such prompts for answers.
For example, we can create a “.env” file under current folder like below and then run
“stanctl up” command without the need to go through an interactive process:
STANCTL_DOWNLOAD_KEY=**********************
STANCTL_SALES_KEY=**********************
STANCTL_CORE_BASE_DOMAIN=FQDN, e.g.162.133.113.15.nip.io
STANCTL_UNIT_TENANT_NAME=Tenant Name, e.g. ibm
STANCTL_UNIT_UNIT_NAME=Tenant Unit Name, e.g. prod
STANCTL_UNIT_INITIAL_ADMIN_PASSWORD=**********
STANCTL_VOLUME_ANALYTICS=/mnt/instana/analytics
STANCTL_VOLUME_DATA=/mnt/instana/data
STANCTL_VOLUME_METRICS=/mnt/instana/metrics
STANCTL_VOLUME_OBJECTS=/mnt/instana/objects
EOF
$ stanctl up --skip-preflight-check -q
5. Post Actions
The command will take a few seconds to complete and after a few minutes, the Instana
backend will be displayed on the Infrastructure View:
And since it’s installed on the Kubernetes, the cluster will be automatically monitored as
well. Navigate to Kubernetes menu (or Platforms -> Kubernetes if you have more platforms
monitored):
5.2 Enable Beta / Extensible Features
Since the Instana is now deployed on Kubernetes – well, K3s is just another Kubernetes
distribution – all beta / extensible features can be enabled. So there is great feature parity
compared to Instana’s SaaS offering.
Please keep in mind that, the “demo” profile is meant to have only the core features when
running in a 16c64g VM. As you can see based on below commands, that the 16c64g VM has
high utilization already, at about 85% CPU and 86% memory at request, after installing the
Instana backend and the agent for self-monitoring:
# K3s has built-in kubectl but an alias is recommended to simplify the command
$ alias k="k3s kubectl"
$ k get node
NAME STATUS ROLES AGE VERSION
itz-550004ghs4-w63m Ready control-plane,master 37m v1.29.0+k3s1
But if you have sufficient resources than the bare minimum, it’s extremely easy to enable
extensible and configurable features by properly setting the desired feature flags. Please
refer to this page for the extensible features, while keep evolving to be more and better.
# Enable Logging
$ stanctl backend apply --core-feature-flags feature.logging.enabled=true
5.3 Others
The bundled K3s has built-in kubectl CLI. We can refer it by typing “k3s kubectl”, or to make
it even simpler, we can create aliases:
$ source ~/.bashrc
From now on, we can use k to call kubectl command. For example:
$ k get node
NAME STATUS ROLES AGE VERSION
itz-550004ghs4-rtma.dte.demo.ibmcloud.com Ready control-plane,master 21m v1.29.1+k3s1
Installation Process – Air-gapped Mode
The “stanctl” supports not only online mode but also the air-gapped mode, which makes it a
perfect tool for on-prem environment.
Let me assume that you’ve installed the “stanctl” CLI by following step 3 of the “online
mode”.
The packaging process would take some time, 1 hour or so, depending on your download
speed – it took me ~50mins on TechZone:
------------------------------------------
Air-gapped package successfully exported!
File: instana-airgapped.tar.gz
------------------------------------------
By default, the packaged file will be generated in the current folder as “instana-
airgapped.tar.gz”, you may specify the desired folder by flag of “-o, --output-dir string”.
$ ls -l
total 16440776
-rw-rw-r--. 1 itzuser itzuser 16835349234 Mar 17 23:12 instana-airgapped.tar.gz
Now, you may simply cover this file over to the targeted VM in the air-gapped env.
1. Spinning up the VM
(same as online mode)
2. Preparing
(same as online mode, but Internet access is NOT needed)
Please note that the “stanctl” CLI is packaged into the downloaded file so we can extract it
from that file and move it to the proper place. For example:
# Verify it
$ stanctl --version
stanctl version 1.0.0 (commit=0088a63c15f68bbdb9de21bc69d25d6798ab4cc7, date=2024-03-28T07:35:40Z)
4. Importing packages
We now can import the downloaded packages:
stanctl air-gapped import -f instana-airgapped.tar.gz
$ stanctl up --air-gapped
Important note: there are some pre-flight checks including the checks of the volume sizes of
the mount points. If you’re running small POCs and just simply trying on TechZone, which
typically has 100G disk only, you may skip such checks by enabling the flag of “--skip-
preflight-check”, like this:
This will kick off the guided installation process, and what we need to prepare and decide
are just a few items:
• Installation type: we should pick “demo” if the resource specs are 16c64G.
• Instana download key: this is part of the license.
• Instana sales key: this is part of the license.
• Domain name / FQDN: for simplicity, we can simply pick <IP>.nip.io, which will
become a good FQDN that can automatically resolve the IP specified.
• Tenant name: Instana has built-in multi-tenancy support, type your desired Tenant
name here.
• Unit name: Under the tenant, there can be 1 or multiple Units, where the actually
data will be reside in for isolation and management purposes.
• TLS cert: as mentioned, we can prepare beforehand or simply let the CLI to generate
for us.
****************************************************************
* Successfully installed Instana Self-Hosted Standard Edition! *
* *
* URL: https://162.133.113.4.nip.io *
* Username: admin@instana.local *
****************************************************************
Please also note that, the “stanctl” CLI well supports “.env” so you may simply put all
necessary variables into such a “.env” file to make the installation process fully quiet and
streamlined without such prompts for answers.
For example, we can create a “.env” file under current folder like below and then run
“stanctl up” command without the need to go through an interactive process:
STANCTL_DOWNLOAD_KEY=**********************
STANCTL_SALES_KEY=**********************
STANCTL_CORE_BASE_DOMAIN=FQDN, e.g.162.133.113.15.nip.io
STANCTL_UNIT_TENANT_NAME=Tenant Name, e.g. ibm
STANCTL_UNIT_UNIT_NAME=Tenant Unit Name, e.g. prod
STANCTL_UNIT_INITIAL_ADMIN_PASSWORD=**********
STANCTL_VOLUME_ANALYTICS=/mnt/instana/analytics
STANCTL_VOLUME_DATA=/mnt/instana/data
STANCTL_VOLUME_METRICS=/mnt/instana/metrics
STANCTL_VOLUME_OBJECTS=/mnt/instana/objects
EOF
6. Post Actions
(same as online mode – please refer to the respective section for steps)
Annex
TLS Certs
TLS cert is optional as the “stanctl” CLI will generate if you’re okay with the generated one.
In the real world, you either can use your official one generated by CA, or just a self-signed
one by a simple process as illustrated below:
# TLS
$ curl -sLO https://github.com/FiloSottile/mkcert/releases/download/v1.4.3/mkcert-v1.4.3-linux-amd64
$ chmod +x mkcert-v1.4.3-linux-amd64 && sudo mv mkcert-v1.4.3-linux-amd64 /usr/local/bin/mkcert
$ ls
tls.crt tls.key
Important Node: The TLS cert generated by mkcert does’t have CN, some legacy
components connecting to Instana TLS endpoints powered by this TLS might have some
issues. In this case, use cfssl instead.
Resource Utilization
Currently, the “stanctl” is targeting two types of VM specs, as mentioned in above “VM
Specs” section.
Profile vCPU Memory Purposes
demo 16 64GB Demo, test, POC
production 24 96GB Production, heavy POC
$ k get node
NAME STATUS ROLES AGE VERSION
instana-0 Ready control-plane,master 39m v1.29.2+k3s1
$ k describe no/instana-0
...
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 13720m (85%) 7400m (46%)
memory 58072100Ki (88%) 55576792Ki (84%)
ephemeral-storage 0 (0%) 0 (0%)
hugepages-1Gi 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
As you can see, with the default setup plus the self-monitored agent installed:
• The CPU requests are at 85%;
• The memory requests are at 88%.
Clean Up
It’s possible to clean up the complete Instana setup.