How to configure Read access
AWS Clusters (EKS, KOps on EC2) - AWS Container Insights
To set up Container Insights on Amazon EKS, you can either follow the AWS installation guidelines or proceed with the following steps:
Set up the command line environment to access the EKS cluster control plane. The easiest method is to launch an AWS CloudShell in your preferred region. You can use this link to run AWS CloudShell.
Set the cluster name and region variables. Replace ‘region-code’ with the AWS Region your cluster is in and ‘cluster-name’ with the name of your cluster:
export CLUSTER=cluster-name
export REGION=region-codeInstall AWS OTEL Agent by running this command:
curl https://uniskai-eu-templates.s3.amazonaws.com/eks/install-otel-container-insights.sh | bash -s
After a successful installation, it may take up to 15 minutes for the OTEL Agent to spin up and export performance metrics.
Return to Uniskai and click on "I carried out the instruction"
or close the pop-up "Connect K8s cluster" and refresh your account in Uniskai to see updated cost reports and recommendations.
Do not forget to refresh your account!
Here you can find the button to refresh the account:
The script for automating Container Insights installation is ‘install-otel-container-insights.sh.’
The script for automating Container Insights installation is ‘install-otel-container-insights.sh.’
PROFILE=
POLICY=arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy
USE_IAM_SERVICE_ACCOUNTS=
LOG_RETENTION_DAYS=7
if [ -z "$REGION" ]
then
region_arg=""
else
region_arg="--region $REGION"
fi
if [ -z "$PROFILE" ]
then
profile_arg=""
else
profile_arg="--profile $PROFILE"
fi
aws_version=$(aws --version | cut -d " " -f1 | cut -d/ -f2)
aws_major_version=$(echo $aws_version | cut -d. -f1)
aws_minor_version=$(echo $aws_version | cut -d. -f2)
if [ $aws_major_version -lt 2 ] || [ $aws_major_version -eq 2 ] && [ $aws_minor_version -lt 8 ]
then
echo "WARNING: Outdated AWS CLI version detected - $aws_version, the script might fail."
echo "Follow these instructions to update your AWS CLI version: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html#getting-started-install-instructions"
fi
if [ -z "$USE_IAM_SERVICE_ACCOUNTS" ]
then
node_roles=()
echo "Fetch node IAM roles from EKS Nodegroups"
node_groups=$(aws eks list-nodegroups --cluster-name $CLUSTER $region_arg $profile_arg --query "nodegroups" --output text)
for node_group in ${node_groups[@]}; do
node_role=$(aws eks describe-nodegroup --cluster-name $CLUSTER $region_arg $profile_arg --nodegroup-name $node_group --query "nodegroup.nodeRole" --output text)
node_roles+=($node_role)
done
echo "Fetch node IAM instance roles from EC2 instances"
ec2_instance_profile_ids=($(aws ec2 describe-instances $region_arg $profile_arg --filter Name=tag-key,Values=kubernetes.io/cluster/$CLUSTER --query "Reservations[*].Instances[*].IamInstanceProfile.Id" --output text | sed 's/\s/\n/g' | sed -r '/^\s*$/d' | sort -u))
for profile_id in ${ec2_instance_profile_ids[@]}; do
echo "Fetch node IAM roles for EC2 instance profile with ID $profile_id"
instance_roles=$(aws iam list-instance-profiles $profile_arg --query "InstanceProfiles[?InstanceProfileId=='$profile_id'].Roles[*].Arn" --output text)
node_roles+=(${instance_roles[@]})
done
echo "Fetch node IAM instance profiles from auto-scaling groups"
lt_instance_profile_names=()
launch_template_ids=($(aws autoscaling describe-auto-scaling-groups $region_arg $profile_arg --filter Name=tag-key,Values=kubernetes.io/cluster/$CLUSTER --query "AutoScalingGroups[*].MixedInstancesPolicy.LaunchTemplate.LaunchTemplateSpecification.LaunchTemplateId" --output text | sed 's/\s/\n/g' | sed -r '/^\s*$/d' | sort -u))
launch_template_ids+=($(aws autoscaling describe-auto-scaling-groups $region_arg $profile_arg --filter Name=tag-key,Values=kubernetes.io/cluster/$CLUSTER --query "AutoScalingGroups[*].LaunchTemplate.LaunchTemplateId" --output text | sed 's/\s/\n/g' | sed -r '/^\s*$/d' | sort -u))
for launch_template_id in ${launch_template_ids[@]}; do
lt_instance_profile_names+=($(aws ec2 describe-launch-template-versions $region_arg $profile_arg --launch-template-id ${launch_template_id} --versions $Default $Latest --query "LaunchTemplateVersions[*].LaunchTemplateData.IamInstanceProfile.Name" --output text | sed 's/\s/\n/g' | sort -u))
done
for profile_name in ${lt_instance_profile_names[@]}; do
echo "Fetch node IAM roles for EC2 instance profile named $profile_name"
instance_roles=$(aws iam list-instance-profiles $profile_arg --query "InstanceProfiles[?InstanceProfileName=='$profile_name'].Roles[*].Arn" --output text)
node_roles+=(${instance_roles[@]})
done
unique_node_roles=($(for role in "${node_roles[@]}"; do echo "${role}"; done | sort -u))
for node_role in ${unique_node_roles[@]}; do
role_name="${node_role#*/}"
echo "Attach IAM policy $POLICY to role $role_name"
aws iam attach-role-policy \
--policy-arn $POLICY \
--role-name $role_name \
$profile_arg
done
else
echo "Check if the cluster has an ODIC provider."
oidc_id=$(aws eks describe-cluster --name $CLUSTER $region_arg $profile_arg --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5)
if [[ $(aws iam list-open-id-connect-providers | grep $oidc_id) ]]
then
echo "You already have an IAM OIDC provider associated with your cluster."
else
echo "No IAM OIDC identity provider found. Creating one for your cluster."
eksctl utils associate-iam-oidc-provider --cluster $CLUSTER --approve
fi
eksctl create iamserviceaccount \
--name aws-otel-sa \
--namespace aws-otel-eks \
--cluster $CLUSTER \
--role-name "$CLUSTER-aws-otel-sa" \
--attach-policy-arn $POLICY \
--approve
fi
if [ ! -z "$LOG_RETENTION_DAYS" ]
then
log_group_name="/aws/containerinsights/$CLUSTER/performance"
log_group_created=$(aws logs describe-log-groups --log-group-name-prefix $log_group_name $region_arg $profile_arg --query 'length(logGroups[])' --output text)
if [[ $log_group_created -eq 0 ]]
then
echo "Create log group $log_group_name"
aws logs create-log-group --log-group-name $log_group_name $region_arg $profile_arg
fi
echo "Set log group $log_group_name retention to $LOG_RETENTION_DAYS days"
aws logs put-retention-policy --log-group-name $log_group_name --retention-in-days $LOG_RETENTION_DAYS $region_arg $profile_arg
fi
ami_ids=($(aws ec2 describe-instances --filters "Name=tag-key,Values=kubernetes.io/cluster/$CLUSTER" $region_arg $profile_arg --query 'Reservations[*].Instances[*].ImageId' --output text | sort -u))
for launch_template_id in ${launch_template_ids[@]}; do
ami_ids+=($(aws ec2 describe-launch-template-versions $region_arg $profile_arg --launch-template-id ${launch_template_id} --versions $Default $Latest --query "LaunchTemplateVersions[*].LaunchTemplateData.ImageId" --output text | sed 's/\s/\n/g' | sort -u))
done
ami_names=($(aws ec2 describe-images $region_arg $profile_arg --image-ids ${ami_ids[@]} --query 'Images[*].Name' --output text))
echo "Detected AMIs: ${ami_names[@]}"
pattern="^bottlerocket-aws-k8s-1\.2[0-9]"
bottlerocket_amis_count=0
other_amis_count=0
for ami_name in ${ami_names[@]}; do
if [[ $ami_name =~ $pattern ]]; then
((bottlerocket_amis_count++))
else
((other_amis_count++))
fi
done
echo "Update kubeconfig"
aws eks update-kubeconfig --name $CLUSTER $region_arg $profile_arg
if [[ $bottlerocket_amis_count -gt 0 && $other_amis_count -eq 0 ]]; then
echo "Bottlerocket AMI detected, installing Amazon OTEL Agent with dockershim.sock patch."
echo "Bottlerocket overrides containerd's socket to be dockershim.sock: https://github.com/bottlerocket-os/bottlerocket/issues/1126#issuecomment-698045504"
echo "Issue description: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ContainerInsights-troubleshooting.html#ContainerInsights-troubleshooting-bottlerocket"
curl -o otel-container-insights-infra.yaml https://uniskai-eu-templates.s3.amazonaws.com/eks/otel-container-insights-infra.yaml
sed -i 's/path: \/run\/containerd\/containerd.sock/path: \/run\/dockershim.sock/g' otel-container-insights-infra.yaml
echo "Applying Amazon OTEL Agent manifest with /run/dockershim.sock patch"
kubectl apply -f otel-container-insights-infra.yaml
elif [[ $bottlerocket_amis_count -gt 0 && $other_amis_count -gt 0 ]]; then
echo "Mix of BottleRocket and other AMI types detected."
echo "If you ever face an issue where billing data appears, but grouping by workload properties (Service, Controller, etc.) does not work,"
echo "and the cluster view is empty; please try to install Amazon OTEL Agent with dockershim.sock patch"
echo "Details: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ContainerInsights-troubleshooting.html#ContainerInsights-troubleshooting-bottlerocket"
echo "Installing Amazon OTEL Agent"
curl https://uniskai-eu-templates.s3.amazonaws.com/eks/otel-container-insights-infra.yaml | kubectl apply -f -
else
echo "Installing Amazon OTEL Agent"
curl https://uniskai-eu-templates.s3.amazonaws.com/eks/otel-container-insights-infra.yaml | kubectl apply -f -
fi
Troubleshooting
If you encounter a timeout error during agent installation:
Unable to connect to the server: dial tcp 196.255.255.255:443: i/o timeout
, follow these steps.Use your preferred method to connect to the cluster control plane API (e.g., VPN or bastion-host).
Check cluster availability using a simple kubectl command, e.g., kubectl get nodes.
If there is no billing data and the cluster view is not updated after installing the Amazon OTEL Agent and refreshing the account in Uniskai, ensure.
The agent can send log events to CloudWatch Logs by checking if the log group
/aws/containerinsights/$CLUSTER/performance
has been created and has recently added events.Verify that pods in the
aws-otel-eks
namespace are running.
If billing data appears but grouping by workload properties (Service, Controller, etc.) does not work, and the cluster view is empty, check the pod logs of the
aws-otel-eks/aws-otel-eks-ci
daemon set. There is a known issue withbottlerocket/aws-k8s-1.22
AMI that prevents the OTEL Agent from reading pod metrics from the container runtime. To fix it, update thehostPath.path
property in thecontainerdsock
volume in the daemonset specification to/run/dockershim.sock
.
Azure AKS cluster
You can connect your AKS cluster to Uniskai by allowing direct access to the cluster API.
Check the authentication and authorization method in the cluster configuration and select the appropriate setup manual. Uniskai currently supports these modes:
az role assignment create --assignee-object-id <PRINCIPAL_ID> --assignee-principal-type ServicePrincipal --role "Azure Kubernetes Service Cluster Monitoring User" --scope "<CLUSTER_ID>"
Replace <PRINCIPAL_ID> with the service principal ID (found in the account details on the Account Manager page) and <CLUSTER_ID> with your cluster ID
(/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RG_NAME>/providers/Microsoft.ContainerService/managedClusters/<NAME>
)
Alternatively, follow these steps:
On the cluster page in the Azure Portal, select Access Control (IAM) and choose Add → Add role assignment.
Select the role with the desired access level and click Next:
Read-only access (monitoring access) - Azure Kubernetes Service Cluster Monitoring User
Read-write access - Azure Kubernetes Service Cluster User Role
3. Search for Uniskai service principles and add them to the member list.
4. Create the role assignment.
Configuring Editor (Read/Write) Access for Uniskai Agent
Uniskai Agent
The Uniskai Agent brings platform features to Kubernetes clusters without compromising security.
To install the agent, establish a reader connection first and launch the installation command provided by the platform for read/write connection.
Find more info about creating a SchedulePolicy Resource in Kubernetes with Kubesitter here.
AWS EKS Cluster with Public API Access
Set up the command line environment by running AWS CloudShell in your preferred region. Use this link to run AWS CloudShell.
Create or update a
kubeconfig
file for your cluster. Replaceregion-code
with the AWS Region your cluster is in, and replacecluster-name
with the name of your cluster:aws eks update-kubeconfig --region region-code --name cluster-name
Create
ClusterRole
andClusterRoleBinding
for Uniskai to allow read-only access to the cluster API:kubectl apply -f https://uniskai-eu-templates.s3.amazonaws.com/eks/uniskai-reader-role-v2.yaml
(Optional) Create
ClusterRole
andClusterRoleBinding
for Uniskai to allow resource modification access to the cluster API:kubectl apply -f https://uniskai-eu-templates.s3.amazonaws.com/eks/uniskai-modifier-role.yaml
Open the
aws-auth
ConfigMap
for editing:kubectl edit -n kube-system configmap/aws-auth
Add the mappings to the
aws-auth
ConfigMap
to include the Uniskai user or role with the read-only permissions assigned. Do not replace any existing mappings. Obtain account connection details from the Uniskai Account Manager window.If a role was used to connect the AWS account to Uniskai (connection type is
Cross-account role
), update themapRoles
ield while replacing ‘ACCOUNT_ID’ with the cluster AWS account ID and ‘ARN_ROLE’ with the IAM role name:apiVersion: v1
data:
mapRoles: |
- groups:
- uniskai-reader-group
- uniskai-modifier-group
rolearn: "arn:aws:iam::ACCOUNT_ID:role/ARN_ROLE"
username: "ARN_ROLE"If a user was used to connect the AWS account to Uniskai (connection type is
Access key
), update themapUsers
field while replacing ‘ACCOUNT_ID’ with the cluster AWS account ID and ‘IAM_USER’ with the IAM username:apiVersion: v1
data:
mapUsers: |
- groups:
- uniskai-reader-group
- uniskai-modifier-group
userarn: "arn:aws:iam::ACCOUNT_ID:role/IAM_USER"
username: "IAM_USER"
Save the file and exit the editor.
When using Nano (AWS CloudShell’s default editor), press the Ctrl+X combination to exit the editor, press Y to approve changes and then press Enter to save the file.
When using Vim, press the Esc key and type the
:wq
command.
(Optional) Enable Metrics API in your cluster to enable usage visualization, cost reports, and recommendations by installing a metrics-server:
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml