In the previous post we checked several EKS addons (Kubecost, Dynarace and Istio), but we still have others. In this post we will look at Teleport. Teleport is an identity-aware, multi-protocol access proxy which understands SSH, HTTPS, RDP, Kubernetes API, MySQL, MongoDB and PostgreSQL wire protocols.
Add-on installation
Teleport can be installed into your EKS clusted as add-on.
Once the add-on is installed, you need to find a load balancer, that was created:
$ kubectl get services -n teleport NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE teleport LoadBalancer 172.20.41.184 ac211ba11cc6a4340a2012236e7fc418-1390710022.us-east-1.elb.amazonaws.com 443:31946/TCP,3023:32150/TCP,3026:32689/TCP,3024:31432/TCP,3036:31808/TCP 44s
Then you need to add the load balancer endpoint to the ConfigMap „teleport“:
$ kubectl edit cm -n teleport teleport
apiVersion: v1
data:
teleport.yaml: |
teleport:
log:
severity: INFO
output: stderr
format:
output: text
extra_fields: ["timestamp","level","component","caller"]
auth_service:
enabled: true
cluster_name: teleport.teleport.svc.cluster.local
authentication:
type: "local"
local_auth: true
second_factor: "otp"
proxy_service:
public_addr: ['ac211ba11cc6a4340a2012236e7fc418-1390710022.us-east-1.elb.amazonaws.com', 'teleport.teleport.svc.cluster.local:443', '127.0.0.1:3080']
kube_listen_addr: 0.0.0.0:3026
mysql_listen_addr: 0.0.0.0:3036
enabled: true
ssh_service:
enabled: false
kind: ConfigMap
metadata:
creationTimestamp: "2023-02-27T12:10:47Z"
name: teleport
namespace: teleport
resourceVersion: "17380"
uid: 2da3f3b3-297f-45a8-b8b7-8d155d23ba76
Recreate all Teleport pods:
$ kubectl delete pods -n teleport --all pod "teleport-b99977958-6hwdw" deleted $ k get po -n teleport -w NAME READY STATUS RESTARTS AGE teleport-b99977958-tkrxk 1/1 Running 0 7s
Create a new „admin“ user. You will recieve a link for setting up the password and MFA (required).
$ kubectl exec -n teleport deployment/teleport -- tctl users add admin --roles=editor,access,auditor User "admin" has been created but requires a password. Share this URL with the user to complete user setup, link is valid for 1h: https://ac211ba11cc6a4340a2012236e7fc418-1390710022.us-east-1.elb.amazonaws.com:3080/web/invite/f9ba7c6013e29276700140fc0f042875
Server Access
Teleport Server Access consolidates SSH access across all environments, decreases configuration complexity, supports industry best practices and compliance while giving complete visibility over all sessions and events.
Teleport Server Access is designed for the following kinds of scenarios:
- When up to a vast number of clusters must be managed using the command-line (tsh) or programmatically (through the Teleport API) and you want to simplify your stack, security, and configuration complexity.
- When security team members must track and audit every user session.
- When Teleport users require a complete, dedicated, and secure SSH option (Teleport Node running in SSH mode) and more than a certificate authority (Teleport Auth) with proxy (Teleport Proxy).
- When resource and network security must be maximized: SSH certificates over secret keys, Two-Factor Authentication (2FA), Single Sign-On (SSO), and short-lived certificates.
Here is an example of manual adding a new VM. I chose Amazon Linux 2 EC2 instance, but Teleport supports various OSs:
A bootstrap script is generated with a short-term access token:
Script installs required packages and configure them:
sudo bash -c "$(curl -kfsSL https://ac211ba11cc6a4340a2012236e7fc418-1390710022.us-east-1.elb.amazonaws.com/scripts/3bd4fa2bd8b232302aa465f51ddee489/install-node.sh)" 2023-02-27 13:09:57 UTC [teleport-installer] TELEPORT_VERSION: 10.3.1 2023-02-27 13:09:57 UTC [teleport-installer] TARGET_HOSTNAME: ac211ba11cc6a4340a2012236e7fc418-1390710022.us-east-1.elb.amazonaws.com 2023-02-27 13:09:57 UTC [teleport-installer] TARGET_PORT: 3080 2023-02-27 13:09:57 UTC [teleport-installer] JOIN_TOKEN: 3bd4fa2bd8b232302aa465f51ddee489 2023-02-27 13:09:57 UTC [teleport-installer] CA_PIN_HASHES: sha256:47525f0632e6382036180cbeb9bd975383d3ca58fbb8f7f3f69023d2996c64d7 2023-02-27 13:09:57 UTC [teleport-installer] Checking TCP connectivity to Teleport server (ac211ba11cc6a4340a2012236e7fc418-1390710022.us-east-1.elb.amazonaws.com:3080) 2023-02-27 13:09:57 UTC [teleport-installer] Couldn't find nc, telnet or /dev/tcp to do a connection test 2023-02-27 13:09:57 UTC [teleport-installer] Going to blindly continue without testing connectivity 2023-02-27 13:09:57 UTC [teleport-installer] Detected host: linux-gnu, using Teleport binary type linux 2023-02-27 13:09:57 UTC [teleport-installer] Detected arch: x86_64, using Teleport arch amd64 2023-02-27 13:09:57 UTC [teleport-installer] Detected distro type: "centos rhel fedora" 2023-02-27 13:09:57 UTC [teleport-installer] Using Teleport distribution: rpm 2023-02-27 13:09:57 UTC [teleport-installer] Created temp dir /tmp/teleport-1ITr6q3fQS 2023-02-27 13:09:57 UTC [teleport-installer] Found 'yum' package manager, using it 2023-02-27 13:09:57 UTC [teleport-installer] Downloading Teleport rpm release 10.3.1 2023-02-27 13:09:57 UTC [teleport-installer] Running curl -fsSL --retry 5 --retry-delay 5 https://get.gravitational.com/teleport-10.3.1-1.x86_64.rpm 2023-02-27 13:09:57 UTC [teleport-installer] Downloading to /tmp/teleport-1ITr6q3fQS/teleport-10.3.1-1.x86_64.rpm 2023-02-27 13:10:07 UTC [teleport-installer] Downloaded file size: 112045092 bytes 2023-02-27 13:10:07 UTC [teleport-installer] Will use sha256sum to validate the checksum of the downloaded file 2023-02-27 13:10:08 UTC [teleport-installer] The downloaded file's checksum validated correctly 2023-02-27 13:10:08 UTC [teleport-installer] Installing Teleport release from /tmp/teleport-1ITr6q3fQS/teleport-10.3.1-1.x86_64.rpm using yum -y localinstall Loaded plugins: extras_suggestions, langpacks, priorities, update-motd Examining /tmp/teleport-1ITr6q3fQS/teleport-10.3.1-1.x86_64.rpm: teleport-10.3.1-1.x86_64 Marking /tmp/teleport-1ITr6q3fQS/teleport-10.3.1-1.x86_64.rpm to be installed Resolving Dependencies --> Running transaction check ---> Package teleport.x86_64 0:10.3.1-1 will be installed --> Finished Dependency Resolution amzn2-core/2/x86_64 | 3.7 kB 00:00:00 Dependencies Resolved ========================================================================== Package Arch Version Repository Size ========================================================================== Installing: teleport x86_64 10.3.1-1 /teleport-10.3.1-1.x86_64 370 M Transaction Summary ========================================================================== Install 1 Package Total size: 370 M Installed size: 370 M Downloading packages: Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : teleport-10.3.1-1.x86_64 1/1 Verifying : teleport-10.3.1-1.x86_64 1/1 Installed: teleport.x86_64 0:10.3.1-1 Complete! 2023-02-27 13:10:15 UTC [teleport-installer] Found: Teleport v10.3.1 git:v10.3.1-0-g2fa9454 go1.18.6 2023-02-27 13:10:15 UTC [teleport-installer] Writing Teleport node service config to /etc/teleport.yaml Wrote config to file "/etc/teleport.yaml". Now you can start the server. Happy Teleporting! 2023-02-27 13:10:15 UTC [teleport-installer] Host is using systemd 2023-02-27 13:10:15 UTC [teleport-installer] Starting Teleport via systemd. It will automatically be started whenever the system reboots. Created symlink from /etc/systemd/system/multi-user.target.wants/teleport.service to /usr/lib/systemd/system/teleport.service. Teleport has been started. View its status with 'sudo systemctl status teleport.service' View Teleport logs using 'sudo journalctl -u teleport.service' To stop Teleport, run 'sudo systemctl stop teleport.service' To start Teleport again if you stop it, run 'sudo systemctl start teleport.service' You can see this node connected in the Teleport web UI or 'tsh ls' with the name 'ip-10-0-151-153.ec2.internal' Find more details on how to use Teleport here: https://goteleport.com/docs/user-manual/
Choose the OS user that is present in the VM and will be used by Teleport:
And test a connectivity:
After that you can see the VM in the list:
You don’t need to register all EC2 instances manually, there is a native capability to do it automatically.
The Teleport Discovery Service can connect to Amazon EC2 and automatically discover and enroll EC2 instances matching configured labels. It will then execute an install script on these discovered instances using AWS Systems Manager that will install Teleport, start it and join the cluster.
You can get a CLI via your web browser:
or your local terminal:
$ tctl version Teleport v12.0.5 git:api/v12.0.5-0-g818318c go1.20.1 $ tsh version Teleport v12.0.5 git:api/v12.0.5-0-g818318c go1.20.1 $ tsh login --proxy=teleport.ait-demo1.com --user=admin Enter password for Teleport user admin: Enter an OTP code from a device: > Profile URL: https://teleport.ait-demo1.com:443 Logged in as: admin Cluster: teleport.ait-demo1.com Roles: access, editor Logins: ec2-user, -teleport-internal-join Kubernetes: enabled Valid until: 2023-03-05 09:23:03 +0200 EET [valid for 12h0m0s] Extensions: login-ip, permit-agent-forwarding, permit-port-forwarding, permit-pty, private-key-policy $ tsh ls Node Name Address Labels ----------------------------------------------------------------------------------- ip-10-0-151-153.ec2.internal <- Tunnel hostname=ip-10-0-151-153.ec2.internal,teleport.internal/resource-id=39162ea2-7e84-4be1-a417-d416f2d3b... $ tsh ssh ec2-user@ip-10-0-151-153.ec2.internal [ec2-user@ip-10-0-151-153 ~]$ ls -la total 12 drwx------ 3 ec2-user ec2-user 74 Feb 27 12:23 . drwxr-xr-x 4 root root 38 Feb 27 13:08 .. -rw-r--r-- 1 ec2-user ec2-user 18 Jul 15 2020 .bash_logout -rw-r--r-- 1 ec2-user ec2-user 193 Jul 15 2020 .bash_profile -rw-r--r-- 1 ec2-user ec2-user 231 Jul 15 2020 .bashrc drwx------ 2 ec2-user ec2-user 29 Feb 27 12:23 .ssh
Other users can join the active session as „Observer“, „Moderator“ or „Peer“
- peer: Can join and collaborate in a session. They can view output and send input.
- moderator: Can join and watch a session. They can view output and forcefully terminate the session at will.
- observer: Can join and watch a session. They cannot control the session in any way.
Access logs usually contain information like:
- IP address or hostname of the client
- Timestamps of operations or access attempts
- Event metadata like “severity”
- Result of an attempted operation (success/failure)
- Log messaging output from the operation
It’s incredibly important to understand what types of data are being radiated and recorded by your system so that you can stay on top of your infrastructure.
Kubernetes access
Teleport provides secure access to Kubernetes clusters:
- Users can access Kubernetes clusters with Single Sign-On (SSO) providers like Okta and switch between clusters without logging in twice.
- Operators can implement granular role-based access controls, including limiting access to specific Kubernetes clusters or even specific pods within a namespace.
- Organizations can achieve compliance by recording kubectl sessions.
I chose EKS cluster for demo. You will need to install the Teleport agent Helm chart, values file will be generated for you:
Values file contains a shotr-term auth token:
$ helm install teleport-agent teleport/teleport-kube-agent -f prod-cluster-values.yaml --create-namespace --namespace teleport $ kubectl get po teleport-agent-0 -n teleport NAME READY STATUS RESTARTS AGE teleport-agent-0 1/1 Running 0 28s
When your Kubernetes cluster is identified, you can connect:
$ tsh kube login demo1 Logged into Kubernetes cluster "demo1". Try 'kubectl version' to test the connection. $ kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE development loadbalancer-6df4b7468c-wzbmx 1/1 Running 0 17m development webapp-66b94c9c64-ms4r6 1/1 Running 0 17m kube-system aws-node-87f82 1/1 Running 0 5d9h kube-system aws-node-vq5ft 1/1 Running 0 5d9h kube-system coredns-79989457d9-jw88j 1/1 Running 0 5d9h kube-system coredns-79989457d9-p5p6j 1/1 Running 0 5d9h kube-system ebs-csi-controller-68fb47d4f5-4gmqt 6/6 Running 1 (5h42m ago) 5d9h kube-system ebs-csi-controller-68fb47d4f5-qtm2l 6/6 Running 0 5d9h kube-system ebs-csi-node-jkjd2 3/3 Running 0 5d9h kube-system ebs-csi-node-zclxv 3/3 Running 0 5d9h kube-system kube-proxy-5lsc6 1/1 Running 0 5d9h kube-system kube-proxy-gb5fb 1/1 Running 0 5d9h production loadbalancer-6df4b7468c-7khv6 1/1 Running 0 17m production webapp-66b94c9c64-p5n5c 1/1 Running 0 17m
EKS clusters can be registered in Teleport automatially as well
EKS Auto-Discovery can automatically discover any EKS cluster and enroll it in Teleport if its tags match the configured labels.
Teleport Kubernetes Auto-Discovery involves two components.
The first, the Discovery Service, is responsible for watching your cloud provider and checking if there are any new clusters or if there have been any modifications to previously discovered clusters. The second, the Kubernetes Service, monitors the clusters created by the Discovery Service. It proxies communications between users and the API servers of these clusters.
Database Access
Teleport can provide secure connections to your databases while improving both access control and visibility.
Some of the things you can do with Database Access:
- Enable users to retrieve short-lived database certificates using a Single Sign-On flow, thus maintaining their organization-wide identity.
- Configure role-based access controls for databases and implement custom Access Request workflows.
- Capture database activity in the Teleport audit log.
Teleport supports many types of databases:
- Active Directory SQL Server: Connect Microsoft SQL Server with Active Directory authentication.
- AWS DynamoDB: Connect AWS DynamoDB.
- AWS ElastiCache & MemoryDB: Connect AWS ElastiCache or AWS MemoryDB for Redis database.
- AWS RDS & Aurora: Connect AWS RDS or Aurora PostgreSQL, MariaDB or MySQL database.
- AWS RDS Proxy: Connect AWS RDS Proxy instances to Teleport.
- AWS Redshift: Connect AWS Redshift database.
- AWS Redshift Serverless: Connect to AWS Redshift serverless.
- AWS Keyspaces (Apache Cassandra): Connect to an AWS Keyspaces database.
- Azure PostgreSQL & MySQL: Connect Azure PostgreSQL or MySQL.
- Azure Cache for Redis: Connect Azure Cache for Redis.
- Azure SQL Server: Connect Azure SQL Server with Azure Active Directory authentication.
- GCP Cloud SQL MySQL: Connect GCP Cloud SQL MySQL database.
- GCP Cloud SQL PostgreSQL: Connect GCP Cloud SQL PostgreSQL database.
- MongoDB Atlas: Connect MongoDB Atlas cluster.
- Self-hosted CockroachDB: Connect self-hosted CockroachDB database.
- Self-hosted Elasticsearch
- Self-hosted MongoDB: Connect self-hosted MongoDB database.
- Self-hosted MySQL & MariaDB: Connect self-hosted MySQL or MariaDB database.
- Self-hosted PostgreSQL: Connect self-hosted PostgreSQL database.
- Self-hosted Redis Cluster: Connect a self-hosted Redis Cluster.
- Self-hosted Redis: Connect self-hosted Redis.
- Self-Hosted Cassandra & ScyllaDB: Connect self-hosted Cassandra or ScyllaDB.
- Snowflake: Connect Snowflake.
For this demo I chose AWS RDS for MySQL. A high-level diagram if the following:
Add the database as all other services:
Choose the required type:
Enter the DB connection endpoint, AWS account ID and RDS Resource ID:
The generated bootstrap script should be executed on the host that has a network connectivity wiyh your RDS:
RDS IAM Authentication will be used for login, so the intermediate host must have such permissions:
DB user and database should be created in advance. AWSAuthenticationPlugin should be enabled and appropriate permissions granted
CREATE USER demouser IDENTIFIED WITH AWSAuthenticationPlugin AS 'RDS'; GRANT ALL ON `%`.* TO 'demouser'@'%';
Then you can use your local terminal to connect to the database, even if it is deployed in a private subnet.
$ tsh db connect demodb --db-user=demouser --db-name=demodb mysql> show databases; +--------------------+ | Database | +--------------------+ | demodb | | information_schema | | mysql | | performance_schema | | sys | +--------------------+ 5 rows in set (0.13 sec)
Pricing
There are two pricing models, Community and Enterprise:
Conclusion
Teleport is the identity-native infrastructure access platform for engineers and machines. By replacing insecure secrets like passwords, keys and tokens with true identity based on biometrics and security modules, Teleport delivers phishing-proof zero trust for every engineer and service connected to your global infrastructure. The open-source Teleport Access Platform consolidates connectivity, authentication, authorization and audit into a single source of truth for access policy across your entire infrastructure while delivering a frictionless developer experience. Teleport replaces VPNs, shared credentials, secrets vaults and legacy PAM solutions, improving security and engineering productivity.
Available in the AWS Marketplace as EKS add-on
