Skip to main content
All CollectionsUniskai and Kubernetes
Connecting Kubernetes Clusters
Connecting Kubernetes Clusters

Configuring Read Access For AWS Clusters (EKS, KOps on EC2) - AWS Container Insights

Updated over a week ago

How to configure Read access

AWS Clusters (EKS, KOps on EC2) - AWS Container Insights


To set up Container Insights on Amazon EKS, you can either follow the AWS installation guidelines or proceed with the following steps:

  1. Set up the command line environment to access the EKS cluster control plane. The easiest method is to launch an AWS CloudShell in your preferred region. You can use this link to run AWS CloudShell.

  2. Set the cluster name and region variables. Replace ‘region-code’ with the AWS Region your cluster is in and ‘cluster-name’ with the name of your cluster: 

    export CLUSTER=cluster-name 
    export REGION=region-code

  3. Install AWS OTEL Agent by running this command:

    curl https://uniskai-eu-templates.s3.amazonaws.com/eks/install-otel-container-insights.sh | bash -s

  4. After a successful installation, it may take up to 15 minutes for the OTEL Agent to spin up and export performance metrics.

  5. Return to Uniskai and click on "I carried out the instruction"

    or close the pop-up "Connect K8s cluster" and refresh your account in Uniskai to see updated cost reports and recommendations.

Do not forget to refresh your account!

Here you can find the button to refresh the account:

The script for automating Container Insights installation is ‘install-otel-container-insights.sh.’

PROFILE=

POLICY=arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy

USE_IAM_SERVICE_ACCOUNTS=

LOG_RETENTION_DAYS=7

if [ -z "$REGION" ]

then

region_arg=""

else

region_arg="--region $REGION"

fi

if [ -z "$PROFILE" ]

then

profile_arg=""

else

profile_arg="--profile $PROFILE"

fi

aws_version=$(aws --version | cut -d " " -f1 | cut -d/ -f2)

aws_major_version=$(echo $aws_version | cut -d. -f1)

aws_minor_version=$(echo $aws_version | cut -d. -f2)

if [ $aws_major_version -lt 2 ] || [ $aws_major_version -eq 2 ] && [ $aws_minor_version -lt 8 ]

then

echo "WARNING: Outdated AWS CLI version detected - $aws_version, the script might fail."

echo "Follow these instructions to update your AWS CLI version: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html#getting-started-install-instructions"

fi

if [ -z "$USE_IAM_SERVICE_ACCOUNTS" ]

then

node_roles=()

echo "Fetch node IAM roles from EKS Nodegroups"

node_groups=$(aws eks list-nodegroups --cluster-name $CLUSTER $region_arg $profile_arg --query "nodegroups" --output text)

for node_group in ${node_groups[@]}; do

node_role=$(aws eks describe-nodegroup --cluster-name $CLUSTER $region_arg $profile_arg --nodegroup-name $node_group --query "nodegroup.nodeRole" --output text)

node_roles+=($node_role)

done

echo "Fetch node IAM instance roles from EC2 instances"

ec2_instance_profile_ids=($(aws ec2 describe-instances $region_arg $profile_arg --filter Name=tag-key,Values=kubernetes.io/cluster/$CLUSTER --query "Reservations[*].Instances[*].IamInstanceProfile.Id" --output text | sed 's/\s/\n/g' | sed -r '/^\s*$/d' | sort -u))

for profile_id in ${ec2_instance_profile_ids[@]}; do

echo "Fetch node IAM roles for EC2 instance profile with ID $profile_id"

instance_roles=$(aws iam list-instance-profiles $profile_arg --query "InstanceProfiles[?InstanceProfileId=='$profile_id'].Roles[*].Arn" --output text)

node_roles+=(${instance_roles[@]})

done

echo "Fetch node IAM instance profiles from auto-scaling groups"

lt_instance_profile_names=()

launch_template_ids=($(aws autoscaling describe-auto-scaling-groups $region_arg $profile_arg --filter Name=tag-key,Values=kubernetes.io/cluster/$CLUSTER --query "AutoScalingGroups[*].MixedInstancesPolicy.LaunchTemplate.LaunchTemplateSpecification.LaunchTemplateId" --output text | sed 's/\s/\n/g' | sed -r '/^\s*$/d' | sort -u))

launch_template_ids+=($(aws autoscaling describe-auto-scaling-groups $region_arg $profile_arg --filter Name=tag-key,Values=kubernetes.io/cluster/$CLUSTER --query "AutoScalingGroups[*].LaunchTemplate.LaunchTemplateId" --output text | sed 's/\s/\n/g' | sed -r '/^\s*$/d' | sort -u))

for launch_template_id in ${launch_template_ids[@]}; do

lt_instance_profile_names+=($(aws ec2 describe-launch-template-versions $region_arg $profile_arg --launch-template-id ${launch_template_id} --versions $Default $Latest --query "LaunchTemplateVersions[*].LaunchTemplateData.IamInstanceProfile.Name" --output text | sed 's/\s/\n/g' | sort -u))

done

for profile_name in ${lt_instance_profile_names[@]}; do

echo "Fetch node IAM roles for EC2 instance profile named $profile_name"

instance_roles=$(aws iam list-instance-profiles $profile_arg --query "InstanceProfiles[?InstanceProfileName=='$profile_name'].Roles[*].Arn" --output text)

node_roles+=(${instance_roles[@]})

done

unique_node_roles=($(for role in "${node_roles[@]}"; do echo "${role}"; done | sort -u))

for node_role in ${unique_node_roles[@]}; do

role_name="${node_role#*/}"

echo "Attach IAM policy $POLICY to role $role_name"

aws iam attach-role-policy \

--policy-arn $POLICY \

--role-name $role_name \

$profile_arg

done

else

echo "Check if the cluster has an ODIC provider."

oidc_id=$(aws eks describe-cluster --name $CLUSTER $region_arg $profile_arg --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5)

if [[ $(aws iam list-open-id-connect-providers | grep $oidc_id) ]]

then

echo "You already have an IAM OIDC provider associated with your cluster."

else

echo "No IAM OIDC identity provider found. Creating one for your cluster."

eksctl utils associate-iam-oidc-provider --cluster $CLUSTER --approve

fi

eksctl create iamserviceaccount \

--name aws-otel-sa \

--namespace aws-otel-eks \

--cluster $CLUSTER \

--role-name "$CLUSTER-aws-otel-sa" \

--attach-policy-arn $POLICY \

--approve

fi

if [ ! -z "$LOG_RETENTION_DAYS" ]

then

log_group_name="/aws/containerinsights/$CLUSTER/performance"

log_group_created=$(aws logs describe-log-groups --log-group-name-prefix $log_group_name $region_arg $profile_arg --query 'length(logGroups[])' --output text)

if [[ $log_group_created -eq 0 ]]

then

echo "Create log group $log_group_name"

aws logs create-log-group --log-group-name $log_group_name $region_arg $profile_arg

fi

echo "Set log group $log_group_name retention to $LOG_RETENTION_DAYS days"

aws logs put-retention-policy --log-group-name $log_group_name --retention-in-days $LOG_RETENTION_DAYS $region_arg $profile_arg

fi

ami_ids=($(aws ec2 describe-instances --filters "Name=tag-key,Values=kubernetes.io/cluster/$CLUSTER" $region_arg $profile_arg --query 'Reservations[*].Instances[*].ImageId' --output text | sort -u))

for launch_template_id in ${launch_template_ids[@]}; do

ami_ids+=($(aws ec2 describe-launch-template-versions $region_arg $profile_arg --launch-template-id ${launch_template_id} --versions $Default $Latest --query "LaunchTemplateVersions[*].LaunchTemplateData.ImageId" --output text | sed 's/\s/\n/g' | sort -u))

done

ami_names=($(aws ec2 describe-images $region_arg $profile_arg --image-ids ${ami_ids[@]} --query 'Images[*].Name' --output text))

echo "Detected AMIs: ${ami_names[@]}"

pattern="^bottlerocket-aws-k8s-1\.2[0-9]"

bottlerocket_amis_count=0

other_amis_count=0

for ami_name in ${ami_names[@]}; do

if [[ $ami_name =~ $pattern ]]; then

((bottlerocket_amis_count++))

else

((other_amis_count++))

fi

done

echo "Update kubeconfig"

aws eks update-kubeconfig --name $CLUSTER $region_arg $profile_arg

if [[ $bottlerocket_amis_count -gt 0 && $other_amis_count -eq 0 ]]; then

echo "Bottlerocket AMI detected, installing Amazon OTEL Agent with dockershim.sock patch."

echo "Bottlerocket overrides containerd's socket to be dockershim.sock: https://github.com/bottlerocket-os/bottlerocket/issues/1126#issuecomment-698045504"

echo "Issue description: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ContainerInsights-troubleshooting.html#ContainerInsights-troubleshooting-bottlerocket"

curl -o otel-container-insights-infra.yaml https://uniskai-eu-templates.s3.amazonaws.com/eks/otel-container-insights-infra.yaml

sed -i 's/path: \/run\/containerd\/containerd.sock/path: \/run\/dockershim.sock/g' otel-container-insights-infra.yaml

echo "Applying Amazon OTEL Agent manifest with /run/dockershim.sock patch"

kubectl apply -f otel-container-insights-infra.yaml

elif [[ $bottlerocket_amis_count -gt 0 && $other_amis_count -gt 0 ]]; then

echo "Mix of BottleRocket and other AMI types detected."

echo "If you ever face an issue where billing data appears, but grouping by workload properties (Service, Controller, etc.) does not work,"

echo "and the cluster view is empty; please try to install Amazon OTEL Agent with dockershim.sock patch"

echo "Details: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ContainerInsights-troubleshooting.html#ContainerInsights-troubleshooting-bottlerocket"

echo "Installing Amazon OTEL Agent"

curl https://uniskai-eu-templates.s3.amazonaws.com/eks/otel-container-insights-infra.yaml | kubectl apply -f -

else

echo "Installing Amazon OTEL Agent"

curl https://uniskai-eu-templates.s3.amazonaws.com/eks/otel-container-insights-infra.yaml | kubectl apply -f -

fi

Troubleshooting

  1. If you encounter a timeout error during agent installation: Unable to connect to the server: dial tcp 196.255.255.255:443: i/o timeout , follow these steps.

    • Use your preferred method to connect to the cluster control plane API (e.g., VPN or bastion-host).

    • Check cluster availability using a simple kubectl command, e.g., kubectl get nodes.

  2. If there is no billing data and the cluster view is not updated after installing the Amazon OTEL Agent and refreshing the account in Uniskai, ensure.

    1. The agent can send log events to CloudWatch Logs by checking if the log group /aws/containerinsights/$CLUSTER/performance has been created and has recently added events.

    2. Verify that pods in the aws-otel-eks namespace are running.

  3. If billing data appears but grouping by workload properties (Service, Controller, etc.) does not work, and the cluster view is empty, check the pod logs of the aws-otel-eks/aws-otel-eks-ci daemon set. There is a known issue with bottlerocket/aws-k8s-1.22 AMI that prevents the OTEL Agent from reading pod metrics from the container runtime. To fix it, update the hostPath.path property in the containerdsock volume in the daemonset specification to /run/dockershim.sock.

Azure AKS cluster

You can connect your AKS cluster to Uniskai by allowing direct access to the cluster API.

  1. Check the authentication and authorization method in the cluster configuration and select the appropriate setup manual. Uniskai currently supports these modes:

    1. Local accounts with Kubernetes RBAC

    2. To connect the AKS cluster with K8s RBAC authorization, provide permissions for Uniskai to list cluster user credentials. Run the following command in Azure CloudShell: 

az role assignment create --assignee-object-id <PRINCIPAL_ID> --assignee-principal-type ServicePrincipal --role "Azure Kubernetes Service Cluster Monitoring User" --scope "<CLUSTER_ID>"

Replace <PRINCIPAL_ID> with the service principal ID (found in the account details on the Account Manager page) and <CLUSTER_ID> with your cluster ID

(/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RG_NAME>/providers/Microsoft.ContainerService/managedClusters/<NAME>)

Alternatively, follow these steps:

  1. On the cluster page in the Azure Portal, select Access Control (IAM) and choose Add → Add role assignment.

  2. Select the role with the desired access level and click Next:

    1. Read-only access (monitoring access) - Azure Kubernetes Service Cluster Monitoring User

    2. Read-write access - Azure Kubernetes Service Cluster User Role

3. Search for Uniskai service principles and add them to the member list.

4. Create the role assignment.

Configuring Editor (Read/Write) Access for Uniskai Agent

Uniskai Agent

The Uniskai Agent brings platform features to Kubernetes clusters without compromising security.

To install the agent, establish a reader connection first and launch the installation command provided by the platform for read/write connection.

Find more info about creating a SchedulePolicy Resource in Kubernetes with Kubesitter here.

AWS EKS Cluster with Public API Access

  1. Set up the command line environment by running AWS CloudShell in your preferred region. Use this link to run AWS CloudShell.

  2. Create or update a kubeconfig file for your cluster. Replace region-code with the AWS Region your cluster is in, and replace cluster-name with the name of your cluster:

    aws eks update-kubeconfig --region region-code --name cluster-name

  3. Create ClusterRole and ClusterRoleBinding for Uniskai to allow read-only access to the cluster API:

    kubectl apply -f https://uniskai-eu-templates.s3.amazonaws.com/eks/uniskai-reader-role-v2.yaml

  4. (Optional) Create ClusterRole and ClusterRoleBinding for Uniskai to allow resource modification access to the cluster API: 

    kubectl apply -f https://uniskai-eu-templates.s3.amazonaws.com/eks/uniskai-modifier-role.yaml

  5. Open the aws-auth ConfigMap for editing:

    kubectl edit -n kube-system configmap/aws-auth

  6. Add the mappings to the aws-auth ConfigMap to include the Uniskai user or role with the read-only permissions assigned. Do not replace any existing mappings. Obtain account connection details from the Uniskai Account Manager window.

    • If a role was used to connect the AWS account to Uniskai (connection type is Cross-account role), update the mapRoles ield while replacing ‘ACCOUNT_ID’ with the cluster AWS account ID and ‘ARN_ROLE’ with the IAM role name:

      apiVersion: v1
      data:
        mapRoles: |
          - groups:
            - uniskai-reader-group
            - uniskai-modifier-group
            rolearn: "arn:aws:iam::ACCOUNT_ID:role/ARN_ROLE"
            username: "ARN_ROLE"

    • If a user was used to connect the AWS account to Uniskai (connection type is Access key), update the mapUsers field while replacing ‘ACCOUNT_ID’ with the cluster AWS account ID and ‘IAM_USER’ with the IAM username:

      apiVersion: v1
      data:
        mapUsers: |
          - groups:
            - uniskai-reader-group
            - uniskai-modifier-group
            userarn: "arn:aws:iam::ACCOUNT_ID:role/IAM_USER"
            username: "IAM_USER"

  7. Save the file and exit the editor.

    1. When using Nano (AWS CloudShell’s default editor), press the Ctrl+X combination to exit the editor, press Y to approve changes and then press Enter to save the file.

    2. When using Vim, press the Esc key and type the :wq command.

  8. (Optional) Enable Metrics API in your cluster to enable usage visualization, cost reports, and recommendations by installing a metrics-server: 

    kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Related Articles
Uniskai and Kubernetes
How to switch AWS account role to Read/write with Cloudfomation stack
Step-by-step connecting Kubernetes Cluster — AWS
Step-by-step connecting Kubernetes Cluster for Azure
Switch cluster from Read only to Read/Write connection
Did this answer your question?