Problem statement
Centralized network security may be challenging but absolutely required by some companies. Auditors might need evidence, that network traffic undergoes an inspection, and the tool/appliance that fulfills this function is strictly isolated and protected. We can achieve this by placing the tool in the dedicated AWS account, according to the best security practices. Palo Alto VM-series firewall can act as an appliance, complemented by AWS Gateway Load Balancer for high-availability capabilities and Transit Gateway as a shared connection point for spoke VPCs.
Solution overview
In previous posts, like this and this, we already demonstrated a multi-account AWS environment with a central networking account that contains different shared components, e.g. IP management system, shared Transit Gateway with the Serverless Transit Network Orchestrator, VPN connection, etc. In this post, we will take a look at how to deploy a highly-available firewall behind a Gateway Load Balancer and combine it with the current Transit Gateway configuration.
Palo Alto VM-series firewall can protect your network and filter Infress/Egress traffic. The security VPC, firewall, and Transit Gateway reside in the central networking account within a multi-account AWS environment (Landing Zone). Spoke VPCs will be connected to the central Transit Gateway (with a default route to it), traffic will go through TGW, Gateway Load Balancer to one of VMs. If the traffic is allowed, it’s forwarded to the internet through Nat Gateway.
Here is a high-level explanation of the architecture and packet flow.
The outbound traffic flow is following:
- A server, located in the Production VPC, sends a packet to the internet. The first route table on the way is a subnet RT with a default route 0.0.0.0/0 to the Transit Gateway.
- It goes to the “Production” TGW VPC attachment. The “Spoke Route Table” is associated with it. The default route 0.0.0.0/0 targets to the “Security VPC attachment”.
- The next hop will be to one of the “TGW attachment” subnet, which is attached to the TGW. The subnet is associated with the route table, where a default route 0.0.0.0/0 points to the VPC endpoint for a Gateway Load Balancer.
- The Gateway Load Balancer receives the packet a forwards it to a healthy VM firewall (data interface). Palo Alto VM performs Layer-7 inspection. If the traffic is allowed, it sends it back to the Load Balancer. (keep in mind that the manemanagementerface is not used for data traffic. It is used only for SSH or HTTPS connection to the web UI for configuration purposes).
- Once the Load Balancer receives back traffic, it returns it to the original endpoint (GWLBe subnet). How to get the internet here, look at the route table, the default route 0.0.0.0/0 goes to the NAT Gateway.
- The NAT Gateway is deployed in the “NATGW subnet”. The next hop in the route table is 0.0.0.0/0 via Internet Gateway.
- Then the traffic goes back from the internet to the NAT Gateway. Now the destination IP address is in the Production VPC 10.128.12.0/24. NAT Gateway subnet is associated with the route table, where the route will be 10.0.0.0/8 to the Gateway Load Balancer endpoint.
- Traffic comes to the Gateway Load Balancer and is automatically forwarded to the Palo Alto VM-series firewall (data interface).
- The firewall inspects the traffic and sends it back to the Load Balancer (to the original VPC endpoint).
- Traffic comes to the “GWLBe subnet”, and according to the route table entry 10.0.0.0/8 goes to the Transit Gateway.
- Traffic comes to the Transit Gateway attachment “Security-VPC”, and checks a route table where routes for all attached VPCs are already propagated. The next route is 10.128.12.0/24 to the TGW attachment “Production-VPC”
Deployment and configuration
Security VPC
The following CloudFormation template can be used to deploy the Security VPC, provided above. VPC has 5 types of subnets from the diagram + one extra subnet type, one for OpenVPN Access Server that should be public and connected to the Transit Gateway. The template also creates a Gateway Load Balancer with VPC endpoints.
AWSTemplateFormatVersion: '2010-09-09' Description: Creates VPC, 6 types of subnets, InternetGateway, security groups, and route tables Parameters: projectName: Type: String AllowedPattern: "^[a-zA-Z][a-zA-Z0-9_-]*$" Default: '' projectEnv: Type: String Description: Environment type. Default: 'SECURITY' ExternalLogBucket: Description: '(Optional) Name of an S3 bucket where you want to store flow logs. If you leave this empty, FlowLogs will not be configured' Type: String Default: '' TrafficType: Description: 'The type of traffic to log.' Type: String Default: ALL AllowedValues: - ACCEPT - REJECT - ALL vpcAssociateWith: Type: String Description: Associate a transit gateway route table (Flat, Isolated, Infrastructure, or On-premises) with a transit gateway attachment. Default: Flat AllowedValues: - Flat - Isolated - Infrastructure - On-premises ConstraintDescription: Must be one of (Flat, Isolated, Infrastructure, or On-premises) vpcPropagateTo: Type: String Description: Add a route from a route table to the attachment. The value can be one or more default route table names (Flat, Isolated, Infrastructure, or On-premises) Default: Flat, On-premises, Isolated, Infrastructure IMAPPoolID: Type: String Default: '' Description: The ID of an IPv4 IPAM pool you want to use for allocating this VPC's CIDR. GlobalInternalCidr: Type: String Description: Global internal CIDR block Conditions: EnableFlowLogs: !Not [!Equals [!Ref ExternalLogBucket, '']] ### VPC ### Resources: VPC: Type: "AWS::EC2::VPC" Properties: EnableDnsSupport: true EnableDnsHostnames: true Ipv4IpamPoolId: !Ref IMAPPoolID Ipv4NetmaskLength: 20 ### VPC Flow Logs ### FlowLogExternalBucket: Condition: EnableFlowLogs Type: 'AWS::EC2::FlowLog' Properties: LogDestination: !Sub 'arn:aws:s3:::${ExternalLogBucket}' LogDestinationType: s3 ResourceId: !Ref VPC ResourceType: 'VPC' TrafficType: !Ref TrafficType ### Internet Gateway ### IGW: Type: "AWS::EC2::InternetGateway" GatewayAttach: Type: "AWS::EC2::VPCGatewayAttachment" Properties: InternetGatewayId: !Ref IGW VpcId: !Ref VPC ### Subnets for NAT gateways ### NATSubnetA: Type: "AWS::EC2::Subnet" Properties: AvailabilityZone: !Select [0, !GetAZs ] CidrBlock: !Select [ 0, !Cidr [ !GetAtt VPC.CidrBlock, 12, 8 ]] MapPublicIpOnLaunch: true VpcId: !Ref VPC RouteTableNatA: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC SubnetRouteTableAssociateNatA: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: RouteTableId: !Ref RouteTableNatA SubnetId: !Ref NATSubnetA RouteDefaultNatA: Type: "AWS::EC2::Route" DependsOn: GatewayAttach Properties: DestinationCidrBlock: 0.0.0.0/0 GatewayId: !Ref IGW RouteTableId: !Ref RouteTableNatA RouteInternalNatA: Type: "AWS::EC2::Route" DependsOn: GwlbVpcEndpointA Properties: DestinationCidrBlock: !Ref GlobalInternalCidr VpcEndpointId: !Ref GwlbVpcEndpointA RouteTableId: !Ref RouteTableNatA NATSubnetB: Type: "AWS::EC2::Subnet" Properties: AvailabilityZone: !Select [1, !GetAZs ] CidrBlock: !Select [ 1, !Cidr [ !GetAtt VPC.CidrBlock, 12, 8 ]] MapPublicIpOnLaunch: true VpcId: !Ref VPC RouteTableNatB: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC SubnetRouteTableAssociateNatB: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: RouteTableId: !Ref RouteTableNatB SubnetId: !Ref NATSubnetB RouteDefaultNatB: Type: "AWS::EC2::Route" DependsOn: GatewayAttach Properties: DestinationCidrBlock: 0.0.0.0/0 GatewayId: !Ref IGW RouteTableId: !Ref RouteTableNatB RouteInternalNatB: Type: "AWS::EC2::Route" DependsOn: GwlbVpcEndpointB Properties: DestinationCidrBlock: !Ref GlobalInternalCidr VpcEndpointId: !Ref GwlbVpcEndpointB RouteTableId: !Ref RouteTableNatB ### Nat gateways ### NatGatewayA: DependsOn: GatewayAttach Type: "AWS::EC2::NatGateway" Properties: AllocationId: !GetAtt EIPNatGWa.AllocationId SubnetId: !Ref NATSubnetA NatGatewayB: DependsOn: GatewayAttach Type: "AWS::EC2::NatGateway" Properties: AllocationId: !GetAtt EIPNatGWb.AllocationId SubnetId: !Ref NATSubnetB VPCSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Default Security Group for VPC VpcId: Ref: VPC SecurityGroupIngress: - IpProtocol: "-1" CidrIp: !GetAtt VPC.CidrBlock SecurityGroupEgress: - IpProtocol: "-1" CidrIp: 0.0.0.0/0 SGBaseIngress: Type: 'AWS::EC2::SecurityGroupIngress' Properties: GroupId: !Ref VPCSecurityGroup IpProtocol: "-1" SourceSecurityGroupId: !GetAtt VPCSecurityGroup.GroupId ### Elastic IPs for NAT Gateways ### EIPNatGWa: DependsOn: GatewayAttach Type: "AWS::EC2::EIP" Properties: Domain: vpc EIPNatGWb: DependsOn: GatewayAttach Type: "AWS::EC2::EIP" Properties: Domain: vpc ### Subnets for management interface of firewall ### MGMTSubnetA: Type: "AWS::EC2::Subnet" Properties: AvailabilityZone: !Select [0, !GetAZs ] CidrBlock: !Select [ 2, !Cidr [ !GetAtt VPC.CidrBlock, 12, 8 ]] MapPublicIpOnLaunch: true VpcId: !Ref VPC RouteTableMgmtA: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC SubnetRouteTableAssociateMgmtA: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: RouteTableId: !Ref RouteTableMgmtA SubnetId: !Ref MGMTSubnetA RouteDefaultMgmtA: Type: "AWS::EC2::Route" DependsOn: GatewayAttach Properties: DestinationCidrBlock: 0.0.0.0/0 GatewayId: !Ref IGW RouteTableId: !Ref RouteTableMgmtA MGMTSubnetB: Type: "AWS::EC2::Subnet" Properties: AvailabilityZone: !Select [1, !GetAZs ] CidrBlock: !Select [ 3, !Cidr [ !GetAtt VPC.CidrBlock, 12, 8 ]] MapPublicIpOnLaunch: true VpcId: !Ref VPC RouteTableMgmtB: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC SubnetRouteTableAssociateMgmtB: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: RouteTableId: !Ref RouteTableMgmtB SubnetId: !Ref MGMTSubnetB RouteDefaultMgmtB: Type: "AWS::EC2::Route" DependsOn: GatewayAttach Properties: DestinationCidrBlock: 0.0.0.0/0 GatewayId: !Ref IGW RouteTableId: !Ref RouteTableMgmtB ### Gateway Load Balancer Endpoint subnets ### GWLBeSubnetA: Type: "AWS::EC2::Subnet" Properties: AvailabilityZone: !Select [0, !GetAZs ] CidrBlock: !Select [ 4, !Cidr [ !GetAtt VPC.CidrBlock, 12, 8 ]] MapPublicIpOnLaunch: false VpcId: !Ref VPC RouteTableGWLBeA: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC SubnetRouteTableAssociateGWLBeA: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: RouteTableId: !Ref RouteTableGWLBeA SubnetId: !Ref GWLBeSubnetA RouteDefaultGWLBeA: Type: "AWS::EC2::Route" Properties: DestinationCidrBlock: 0.0.0.0/0 NatGatewayId: !Ref NatGatewayA RouteTableId: !Ref RouteTableGWLBeA RouteInternalGWLBeA: Type: "AWS::EC2::Route" DependsOn: TGWSubnetsAttachment Properties: DestinationCidrBlock: !Ref GlobalInternalCidr TransitGatewayId: !ImportValue STNO-TGW-ID RouteTableId: !Ref RouteTableGWLBeA GWLBeSubnetB: Type: "AWS::EC2::Subnet" DependsOn: GWLBeSubnetA Properties: AvailabilityZone: !Select [1, !GetAZs ] CidrBlock: !Select [ 5, !Cidr [ !GetAtt VPC.CidrBlock, 12, 8 ]] MapPublicIpOnLaunch: false VpcId: !Ref VPC RouteTableGWLBeB: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC SubnetRouteTableAssociateGWLBeB: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: RouteTableId: !Ref RouteTableGWLBeB SubnetId: !Ref GWLBeSubnetB RouteDefaultGWLBeB: Type: "AWS::EC2::Route" Properties: DestinationCidrBlock: 0.0.0.0/0 NatGatewayId: !Ref NatGatewayB RouteTableId: !Ref RouteTableGWLBeB RouteInternalGWLBeB: Type: "AWS::EC2::Route" DependsOn: TGWSubnetsAttachment Properties: DestinationCidrBlock: !Ref GlobalInternalCidr TransitGatewayId: !ImportValue STNO-TGW-ID RouteTableId: !Ref RouteTableGWLBeB ### Subnets for openvpn server ### OpenVPNSubnetC: Type: "AWS::EC2::Subnet" DependsOn: GWLBeSubnetB Properties: AvailabilityZone: !Select [2, !GetAZs ] CidrBlock: !Select [ 6, !Cidr [ !GetAtt VPC.CidrBlock, 12, 8 ]] MapPublicIpOnLaunch: true VpcId: !Ref VPC RouteTableOpenVpnC: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC RouteDefaultOpenVpnC: Type: "AWS::EC2::Route" DependsOn: GatewayAttach Properties: DestinationCidrBlock: 0.0.0.0/0 GatewayId: !Ref IGW RouteTableId: !Ref RouteTableOpenVpnC RouteInternalOpenVpnC: Type: "AWS::EC2::Route" DependsOn: TGWSubnetsAttachment Properties: DestinationCidrBlock: !Ref GlobalInternalCidr TransitGatewayId: !ImportValue STNO-TGW-ID RouteTableId: !Ref RouteTableOpenVpnC SubnetRouteTableAssociateOpenVpnC: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: RouteTableId: !Ref RouteTableOpenVpnC SubnetId: !Ref OpenVPNSubnetC ### Firewall Data subnets ### DataSubnetA: Type: "AWS::EC2::Subnet" Properties: AvailabilityZone: !Select [0, !GetAZs ] CidrBlock: !Select [ 7, !Cidr [ !GetAtt VPC.CidrBlock, 12, 8 ]] MapPublicIpOnLaunch: false VpcId: !Ref VPC RouteTableDataA: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC SubnetRouteTableAssociateDataA: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: RouteTableId: !Ref RouteTableDataA SubnetId: !Ref DataSubnetA DataSubnetB: Type: "AWS::EC2::Subnet" Properties: AvailabilityZone: !Select [1, !GetAZs ] CidrBlock: !Select [ 8, !Cidr [ !GetAtt VPC.CidrBlock, 12, 8 ]] MapPublicIpOnLaunch: false VpcId: !Ref VPC RouteTableDataB: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC SubnetRouteTableAssociateDataB: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: RouteTableId: !Ref RouteTableDataB SubnetId: !Ref DataSubnetB ### Subnets for Transit gateway attachments ### TGWSubnetA: Type: "AWS::EC2::Subnet" Properties: AvailabilityZone: !Select [0, !GetAZs ] CidrBlock: !Select [ 9, !Cidr [ !GetAtt VPC.CidrBlock, 12, 8 ]] MapPublicIpOnLaunch: false VpcId: !Ref VPC RouteTableTgwA: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC SubnetRouteTableAssociateTgwA: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: RouteTableId: !Ref RouteTableTgwA SubnetId: !Ref TGWSubnetA RouteDefaultTgwA: Type: "AWS::EC2::Route" DependsOn: GwlbVpcEndpointA Properties: DestinationCidrBlock: 0.0.0.0/0 VpcEndpointId: !Ref GwlbVpcEndpointA RouteTableId: !Ref RouteTableTgwA TGWSubnetB: Type: "AWS::EC2::Subnet" Properties: AvailabilityZone: !Select [1, !GetAZs ] CidrBlock: !Select [ 10, !Cidr [ !GetAtt VPC.CidrBlock, 12, 8 ]] MapPublicIpOnLaunch: false VpcId: !Ref VPC RouteTableTgwB: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC SubnetRouteTableAssociateTgwB: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: RouteTableId: !Ref RouteTableTgwB SubnetId: !Ref TGWSubnetB RouteDefaultTgwB: Type: "AWS::EC2::Route" DependsOn: GwlbVpcEndpointB Properties: DestinationCidrBlock: 0.0.0.0/0 VpcEndpointId: !Ref GwlbVpcEndpointB RouteTableId: !Ref RouteTableTgwB TGWSubnetsAttachment: Type: AWS::EC2::TransitGatewayAttachment Properties: SubnetIds: - !Ref TGWSubnetA - !Ref TGWSubnetB - !Ref OpenVPNSubnetC TransitGatewayId: !ImportValue STNO-TGW-ID VpcId: !Ref VPC Gwlb: Type: AWS::ElasticLoadBalancingV2::LoadBalancer Properties: Name: Firewall-Gateway-LoadBalancer Type: gateway Subnets: - !Ref DataSubnetA - !Ref DataSubnetB Tags: - Key: Environment Value: !Ref projectEnv - Key: Managed_by Value: CloudFormation - Key: Project Value: !Ref projectName - Key: Tier Value: Private - Key: !Ref MAPtagKey Value: !Ref MAPtagValue GwlbEndpointService: Type: AWS::EC2::VPCEndpointService Properties: AcceptanceRequired: false GatewayLoadBalancerArns: - !Ref Gwlb GwlbVpcEndpointA: Type: AWS::EC2::VPCEndpoint DependsOn: GwlbEndpointService Properties: ServiceName: !Sub "com.amazonaws.vpce.${AWS::Region}.${GwlbEndpointService}" SubnetIds: - !Ref GWLBeSubnetA VpcEndpointType: GatewayLoadBalancer VpcId: !Ref VPC GwlbVpcEndpointB: Type: AWS::EC2::VPCEndpoint DependsOn: GwlbEndpointService Properties: ServiceName: !Sub "com.amazonaws.vpce.${AWS::Region}.${GwlbEndpointService}" SubnetIds: - !Ref GWLBeSubnetB VpcEndpointType: GatewayLoadBalancer VpcId: !Ref VPC
To ensure that the VM-Series firewall can inspect traffic that is routed between VPC attachments, you must enable appliance mode on the transit gateway VPC attachment for the security VPC containing the VM-Series firewall. You can enable appliance mode using the command:
aws ec2 modify-transit-gateway-vpc-attachment --transit-gateway-attachment-id <value> --options ApplianceModeSupport=enable
Firewall EC2 prerequisites
VM requires two subnets – one for management and one for data. (already created by CloudFormation)
- Create two security groups – one for firewall management and one for data.
- The management subnet security groups should allow HTTPS and SSH for management access.
- Ensure that the security group(s) in your data VPC allows GENEVE-encapsulated packets (UDP port 6081).
- The target group of the GWLB cannot use HTTP for health checks because the VM-Series firewall does not allow access with an unsecured protocol. Instead, use another protocol such as HTTPS or TCP. So we also add port 443 into the Security Group
Create a network interface (for VM
- Create a network interface (for VM management, in the Management subnet), that will be attached for the VM later.
- Attach the management security group to this “management” interface. Allow SSH and HTTPS.
Launch the VM-Series firewall
There are three PAYG bundles available. The licenses include are as follows:
Bundle 1: VM-Series + Threat Prevention + Premium Support.
Bundle 2: VM-Series + Threat Prevention + URL Filtering+DNS Security+Global Protect + WildFire + Premium Support.
Bundle 3: VM-Series + Advanced Threat Prevention + Advanced URL Filtering + DNS Security + Global Protect + WildFire + Premium Support.
If you have a license key, use BYOL AMI instead of bundles.
We used “Bundle 3” for the test.
- Select the Security VPC.
- Select the data subnet to attach to eth0.
- Add another network interface for eth1 to act as the management interface after the interface swap. Swapping interfaces requires a minimum of two ENIs (eth0 and eth1).
There is a limitation, GWLB can send traffic only to the Eth0 interface of the VM. By default, the Eth0 interface is a Management interface of the VM that can not be used for Data traffic. That’s why we need to swap interfaces and move the Management interface to Eth1.
- Expand the Advanced Details section and in the User data field enter as text to perform the interface swap during launch.
mgmt-interface-swap=enable plugin-op-commands=aws-gwlb-inspect:enable
- Select the Data Security Group for eth0 (data interface). Enable traffic on UDP port 6081.
- Create and assign an Elastic IP address (EIP) to the ENI used for management access (eth1) to the firewall.
- Configure a new administrative password for the firewall. You need to connect to the VM instance via SSH:
ssh -i <private_key.pem> admin@<public-ip_address> configure set mgt-config users admin password commit
- Login to the web UI (https://<management-interface-public-ip>
Configure the data-plane network interface as a Layer 3 interface on the firewall.
- Select Network – Interfaces – Ethernet.
Click the link for ethernet 1/1 and configure it as follows:
Interface Type: Layer3
On the Config tab, assign the interface to the default virtual router.
On the Config tab, expand the Security Zone drop-down and select New Zone. Define a new zone and leave the remaining fields with default values and then click OK.
On the IPv4 tab, select DHCP Client.
If using DHCP, select DHCP Client; the private IP address that you assigned to the ENI in the AWS management console will be automatically acquired.
On the Advanced tab, create a management profile to enable HTTP/HTTPS service as part of management profile creation and allow Health check probes from GWLB.
Create security policies to allow/deny traffic. Because the VM-Series treats traffic as intrazone when integrated with a GWLB, a default intrazone rule allows all traffic. It is a best practice to override the default intrazone rule with a deny action for traffic that does not match any of your other security policy rules.
In this example, the top-level rule denies all traffic. So PING is not possible.top-level
Once we remove the top-level “deny” rule, the “allow” rule becomes a top.
Pings work
You can see the traffic in the “Monitor” tab.
Conclusion
In this post, we demonstrated how to deploy a highly-available Palo Alto VM-series firewall appliance in a separate networking account with a Gateway Load Balancer and Transit gateway, and use it for egress traffic inspection from spoke VPCs, deployed in different accounts. Spoke VPCs don’t have Internet Gateway or NAT Gateways and send all traffic through the Transit Gateway and Palo Alto Firewall to the internet. This makes the environment more secure and provides a single place for configuring packet filtering rules. The AMI for the VM-Series firewall is available in the AWS Marketplace for both the Bring Your Own License (BYOL) and the usage-based pricing options.