Skip to main content
All CollectionsUniskai and Kubernetes
Step-by-step connecting Kubernetes Cluster for AWS
Step-by-step connecting Kubernetes Cluster for AWS

Configuring Read Access For AWS Clusters (EKS, KOps on EC2) - AWS Container Insights

Updated over a week ago

Step 1: Select the Cloudk8s tab on the left menu.

Step 2: Click on the switcher of the desired cluster.

Step 3: The modal window appears.

Configuring Read-Only Access for Uniskai Agent

Step 4.1: Click on the "read only" button.

Step 4.2: The guide link appears. Click on "Learn more" right to the OTEL agent

Step 4.3: To set up Container Insights on Amazon EKS, you can either follow the AWS installation guidelines or proceed with the following steps:

  1. Set up the command line environment to access the EKS cluster control plane. The easiest method is to launch an AWS CloudShell in your preferred region. You can use this link to run AWS CloudShell.

  2. Set the cluster name and region variables. Replace ‘region-code’ with the AWS Region your cluster is in and ‘cluster-name’ with the name of your cluster:

    export CLUSTER=cluster-name 
    export REGION=region-code

  3. Install AWS OTEL Agent by running this command:

    curl https://uniskai-eu-templates.s3.amazonaws.com/eks/install-otel-container-insights.sh | bash -s

  4. After a successful installation, the OTEL Agent may take up to 15 minutes to spin up and export performance metrics.

  5. Return to Uniskai and click on "I carried out the instruction"

    or close the pop-up "Connect K8s cluster" and refresh your account in Uniskai to see updated cost reports and recommendations.

Do not forget to refresh your account!

Here you can find the button to refresh the account:

The state is changed to read-only.

The script for automating Container Insights installation is ‘install-otel-container-insights.sh.’

PROFILE=

POLICY=arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy

USE_IAM_SERVICE_ACCOUNTS=

LOG_RETENTION_DAYS=7

if [ -z "$REGION" ]

then

region_arg=""

else

region_arg="--region $REGION"

fi

if [ -z "$PROFILE" ]

then

profile_arg=""

else

profile_arg="--profile $PROFILE"

fi

aws_version=$(aws --version | cut -d " " -f1 | cut -d/ -f2)

aws_major_version=$(echo $aws_version | cut -d. -f1)

aws_minor_version=$(echo $aws_version | cut -d. -f2)

if [ $aws_major_version -lt 2 ] || [ $aws_major_version -eq 2 ] && [ $aws_minor_version -lt 8 ]

then

echo "WARNING: Outdated AWS CLI version detected - $aws_version, the script might fail."

echo "Follow these instructions to update your AWS CLI version: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html#getting-started-install-instructions"

fi

if [ -z "$USE_IAM_SERVICE_ACCOUNTS" ]

then

node_roles=()

echo "Fetch node IAM roles from EKS Nodegroups"

node_groups=$(aws eks list-nodegroups --cluster-name $CLUSTER $region_arg $profile_arg --query "nodegroups" --output text)

for node_group in ${node_groups[@]}; do

node_role=$(aws eks describe-nodegroup --cluster-name $CLUSTER $region_arg $profile_arg --nodegroup-name $node_group --query "nodegroup.nodeRole" --output text)

node_roles+=($node_role)

done

echo "Fetch node IAM instance roles from EC2 instances"

ec2_instance_profile_ids=($(aws ec2 describe-instances $region_arg $profile_arg --filter Name=tag-key,Values=kubernetes.io/cluster/$CLUSTER --query "Reservations[*].Instances[*].IamInstanceProfile.Id" --output text | sed 's/\s/\n/g' | sed -r '/^\s*$/d' | sort -u))

for profile_id in ${ec2_instance_profile_ids[@]}; do

echo "Fetch node IAM roles for EC2 instance profile with ID $profile_id"

instance_roles=$(aws iam list-instance-profiles $profile_arg --query "InstanceProfiles[?InstanceProfileId=='$profile_id'].Roles[*].Arn" --output text)

node_roles+=(${instance_roles[@]})

done

echo "Fetch node IAM instance profiles from auto-scaling groups"

lt_instance_profile_names=()

launch_template_ids=($(aws autoscaling describe-auto-scaling-groups $region_arg $profile_arg --filter Name=tag-key,Values=kubernetes.io/cluster/$CLUSTER --query "AutoScalingGroups[*].MixedInstancesPolicy.LaunchTemplate.LaunchTemplateSpecification.LaunchTemplateId" --output text | sed 's/\s/\n/g' | sed -r '/^\s*$/d' | sort -u))

launch_template_ids+=($(aws autoscaling describe-auto-scaling-groups $region_arg $profile_arg --filter Name=tag-key,Values=kubernetes.io/cluster/$CLUSTER --query "AutoScalingGroups[*].LaunchTemplate.LaunchTemplateId" --output text | sed 's/\s/\n/g' | sed -r '/^\s*$/d' | sort -u))

for launch_template_id in ${launch_template_ids[@]}; do

lt_instance_profile_names+=($(aws ec2 describe-launch-template-versions $region_arg $profile_arg --launch-template-id ${launch_template_id} --versions $Default $Latest --query "LaunchTemplateVersions[*].LaunchTemplateData.IamInstanceProfile.Name" --output text | sed 's/\s/\n/g' | sort -u))

done

for profile_name in ${lt_instance_profile_names[@]}; do

echo "Fetch node IAM roles for EC2 instance profile named $profile_name"

instance_roles=$(aws iam list-instance-profiles $profile_arg --query "InstanceProfiles[?InstanceProfileName=='$profile_name'].Roles[*].Arn" --output text)

node_roles+=(${instance_roles[@]})

done

unique_node_roles=($(for role in "${node_roles[@]}"; do echo "${role}"; done | sort -u))

for node_role in ${unique_node_roles[@]}; do

role_name="${node_role#*/}"

echo "Attach IAM policy $POLICY to role $role_name"

aws iam attach-role-policy \

--policy-arn $POLICY \

--role-name $role_name \

$profile_arg

done

else

echo "Check if the cluster has an ODIC provider."

oidc_id=$(aws eks describe-cluster --name $CLUSTER $region_arg $profile_arg --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5)



if [[ $(aws iam list-open-id-connect-providers | grep $oidc_id) ]]

then

echo "You already have an IAM OIDC provider associated with your cluster."

else

echo "No IAM OIDC identity provider found. Creating one for your cluster."

eksctl utils associate-iam-oidc-provider --cluster $CLUSTER --approve

fi

eksctl create iamserviceaccount \

--name aws-otel-sa \

--namespace aws-otel-eks \

--cluster $CLUSTER \

--role-name "$CLUSTER-aws-otel-sa" \

--attach-policy-arn $POLICY \

--approve

fi

if [ ! -z "$LOG_RETENTION_DAYS" ]

then

log_group_name="/aws/containerinsights/$CLUSTER/performance"

log_group_created=$(aws logs describe-log-groups --log-group-name-prefix $log_group_name $region_arg $profile_arg --query 'length(logGroups[])' --output text)

if [[ $log_group_created -eq 0 ]]

then

echo "Create log group $log_group_name"

aws logs create-log-group --log-group-name $log_group_name $region_arg $profile_arg

fi

echo "Set log group $log_group_name retention to $LOG_RETENTION_DAYS days"

aws logs put-retention-policy --log-group-name $log_group_name --retention-in-days $LOG_RETENTION_DAYS $region_arg $profile_arg

fi

ami_ids=($(aws ec2 describe-instances --filters "Name=tag-key,Values=kubernetes.io/cluster/$CLUSTER" $region_arg $profile_arg --query 'Reservations[*].Instances[*].ImageId' --output text | sort -u))

for launch_template_id in ${launch_template_ids[@]}; do

ami_ids+=($(aws ec2 describe-launch-template-versions $region_arg $profile_arg --launch-template-id ${launch_template_id} --versions $Default $Latest --query "LaunchTemplateVersions[*].LaunchTemplateData.ImageId" --output text | sed 's/\s/\n/g' | sort -u))

done

ami_names=($(aws ec2 describe-images $region_arg $profile_arg --image-ids ${ami_ids[@]} --query 'Images[*].Name' --output text))

echo "Detected AMIs: ${ami_names[@]}"

pattern="^bottlerocket-aws-k8s-1\.2[0-9]"

bottlerocket_amis_count=0

other_amis_count=0

for ami_name in ${ami_names[@]}; do

if [[ $ami_name =~ $pattern ]]; then

((bottlerocket_amis_count++))

else

((other_amis_count++))

fi

done

echo "Update kubeconfig"

aws eks update-kubeconfig --name $CLUSTER $region_arg $profile_arg

if [[ $bottlerocket_amis_count -gt 0 && $other_amis_count -eq 0 ]]; then

echo "Bottlerocket AMI detected, installing Amazon OTEL Agent with dockershim.sock patch."

echo "Bottlerocket overrides containerd's socket to be dockershim.sock: https://github.com/bottlerocket-os/bottlerocket/issues/1126#issuecomment-698045504"

echo "Issue description: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ContainerInsights-troubleshooting.html#ContainerInsights-troubleshooting-bottlerocket"

curl -o otel-container-insights-infra.yaml https://uniskai-eu-templates.s3.amazonaws.com/eks/otel-container-insights-infra.yaml

sed -i 's/path: \/run\/containerd\/containerd.sock/path: \/run\/dockershim.sock/g' otel-container-insights-infra.yaml

echo "Applying Amazon OTEL Agent manifest with /run/dockershim.sock patch"

kubectl apply -f otel-container-insights-infra.yaml

elif [[ $bottlerocket_amis_count -gt 0 && $other_amis_count -gt 0 ]]; then

echo "Mix of BottleRocket and other AMI types detected."

echo "If you ever face an issue where billing data appears, but grouping by workload properties (Service, Controller, etc.) does not work,"

echo "and the cluster view is empty; please try to install Amazon OTEL Agent with dockershim.sock patch"

echo "Details: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ContainerInsights-troubleshooting.html#ContainerInsights-troubleshooting-bottlerocket"

echo "Installing Amazon OTEL Agent"

curl https://uniskai-eu-templates.s3.amazonaws.com/eks/otel-container-insights-infra.yaml | kubectl apply -f -

else

echo "Installing Amazon OTEL Agent"

curl https://uniskai-eu-templates.s3.amazonaws.com/eks/otel-container-insights-infra.yaml | kubectl apply -f -

fi

Troubleshooting

  1. If you encounter a timeout error during agent installation: Unable to connect to the server: dial tcp 196.255.255.255:443: i/o timeout , follow these steps.

    • You can use your preferred method to connect to the cluster control plane API (e.g., VPN or bastion-host).

    • Check cluster availability using a simple kubectl command, e.g., kubectl get nodes.

  2. If there is no billing data and the cluster view is not updated after installing the Amazon OTEL Agent and refreshing the account in Uniskai, please make sure.

    1. The agent can send log events to CloudWatch Logs by checking if the log group /aws/containerinsights/$CLUSTER/performance has been created and has recently added events.

    2. Verify that pods in the aws-otel-eks namespace is running.

  3. If billing data appears but grouping by workload properties (Service, Controller, etc.) does not work, and the cluster view is empty, check the pod logs of the aws-otel-eks/aws-otel-eks-ci daemon set. There is a known issue with bottlerocket/aws-k8s-1.22 AMI that prevents the OTEL Agent from reading pod metrics from the container runtime. To fix it, update the hostPath.path property in the containerdsock volume in the daemonset specification to /run/dockershim.sock.


Configuring Read/Write Access for Uniskai Agent

Step 5.1: Click on the read/write button

Step 5.2: The modal window with the connection script appears. Run the command in the command line

To install the agent, establish a reader connection first and launch the installation command provided by the platform for read/write connection.

Find more info about creating a SchedulePolicy Resource in Kubernetes with Kubesitter here.

AWS EKS Cluster with Public API Access

  1. Set up the command line environment by running AWS CloudShell in your preferred region. Use this link to run AWS CloudShell.

  2. Create or update a kubeconfig file for your cluster. Replace region-code with the AWS Region your cluster is in, and replace cluster-name with the name of your cluster:

    aws eks update-kubeconfig --region region-code --name cluster-name

  3. Create ClusterRole and ClusterRoleBinding for Uniskai to allow read-only access to the cluster API:

    kubectl apply -f https://uniskai-eu-templates.s3.amazonaws.com/eks/uniskai-reader-role-v2.yaml

  4. (Optional) Create ClusterRole and ClusterRoleBinding for Uniskai to allow resource modification access to the cluster API:

    kubectl apply -f https://uniskai-eu-templates.s3.amazonaws.com/eks/uniskai-modifier-role.yaml

  5. Open the aws-auth ConfigMap for editing:

    kubectl edit -n kube-system configmap/aws-auth

  6. Add the mappings to the aws-auth ConfigMap to include the Uniskai user or role with the read-only permissions assigned. Do not replace any existing mappings. Obtain account connection details from the Uniskai Account Manager window.

    • If a role was used to connect the AWS account to Uniskai (connection type is Cross-account role), update the mapRoles field while replacing ‘ACCOUNT_ID’ with the cluster AWS account ID and ‘ARN_ROLE’ with the IAM role name:

      apiVersion: v1
      data:
      mapRoles: |
      - groups:
      - uniskai-reader-group
      - uniskai-modifier-group
      rolearn: "arn:aws:iam::ACCOUNT_ID:role/ARN_ROLE"
      username: "ARN_ROLE"

    • If a user was used to connect the AWS account to Uniskai (connection type is Access key), update the mapUsers field while replacing ‘ACCOUNT_ID’ with the cluster AWS account ID and ‘IAM_USER’ with the IAM username:

      apiVersion: v1
      data:
      mapUsers: |
      - groups:
      - uniskai-reader-group
      - uniskai-modifier-group
      userarn: "arn:aws:iam::ACCOUNT_ID:role/IAM_USER"
      username: "IAM_USER"

  7. Save the file and exit the editor.

    1. When using Nano (AWS CloudShell’s default editor), press the Ctrl+X combination to exit the editor, press Y to approve changes and then press Enter to save the file.

    2. When using Vim, press the Esc key and type the :wq command.

  8. (Optional) Enable Metrics API in your cluster to enable usage visualization, cost reports, and recommendations by installing a metrics-server:

    kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Step 5.3: Return to Uniskai wait several minutes and click on "I ran the script"

or close the pop-up "Connect K8s cluster" and refresh your account in Uniskai to see updated cost reports and recommendations.

Do not forget to refresh your account!

Here you can find the button to refresh the account:

The state is changed to read/write

Switch from Read-Write connection to Read only connection

  1. When the Agent was connected

To change the connection type from read/write to read-only when an agent is connected, you need to remove the agent from the cluster. To do this, you can use the command.

kubectl delete -f https://uniskai-eu-templates.s3.eu-central-1.amazonaws.com/kubernetes-agent/deployment.yaml

  • Refresh your account on the platform.

  • The state is changed to the read-only.

2. If the EKS cluster is connected through the API

  • Open aws-auth map configuration for modification in kube-system namespace

  • Delete the uniskai-modifier-group group or the entire record (for complete disabling) with the cluster role from the mapRoles/mapUsers block

  • Refresh your account on the platform.

  • The state is changed to read-only.

Did this answer your question?