Running Hybrid Cloud Patterns on AWS

Editor's Notes

1. Shiva What is Hybrid? Why Hybrid? Challenges and Best Practices How Hybrid? Connectivity Enterprise Integration Common Hybrid workloads Example hybrid workloads
2. Shiva What is Hybrid? Why Hybrid? Challenges and Best Practices How Hybrid? Connectivity Enterprise Integration Common Hybrid workloads Example hybrid workloads
3. Shiva Operating in hybrid model should be transparent to the end user.
4. Shiva It is not a question of why customers should move to the cloud. Cloud is the new normal. The question is why customers should run anything on physical infrastructure?! Are there any other reasons you see among your customers, on why they want to run Hybrid?
5. Shiva An ideal hybrid model should make the underlying providers transparent to the customer. Expensive - It is a lot more expensive because of the complexities involved and data movement across boundaries to run hybrid. Comparable services – You just might not have comparable services across various providers. The characteristics of similar services might be very different. For example the EBS volumes in AWS provide certain IOPS. But if you compare that directly to block storage from other providers, it might be very different because of the block sizes they are using. 1000 IOPs with a block size of 16KB, is very different from 1000 IOPS with block size of 64 KB, Transport delays – Network delays Customer is limited to the least common denominator – Because many other providers do not have a higher up the stack service, almost all hybrid environments are limited by the least common denominator, and operate at purely compute and storage and basic networking level. Degraded agility - Complex maintenance and operation - This is usually under estimated, and ends up in Degraded agility and other limitations Best practices Use each environment’s native services and features as much as possible – On AWS use native provisioning using Cloudformation, and monitoring using Cloudwatch, and notification using SNS. Even if you have other solutions in places, integrate the native tools in your operating model. Use cloud-native or made-for-the-cloud solutions/services - A lot of existing solutions/products and services are not natively designed for the cloud, and instead are retrofitted to the cloud. Databases are one example. The Oracles and SAPs of the world. Aurora is a enterprise grade database designed ground up for the cloud. F5s in AMP are a hot topic at the moment whi is not designed at the moment to run natively in the cloud.
6. Shiva What is Hybrid? Why Hybrid? Challenges and Best Practices How Hybrid? Connectivity Enterprise Integration Common Hybrid workloads Example hybrid workloads
7. Zoltak – Create a Hardware Virtual Private network between your data center and your VPC. Supported Customer Hardware & Options: Support Customer Devices https://aws.amazon.com/vpc/faqs/#C9 Internet-routable IP address (static) of the customer gateway's external interface. The value must be static and can't be behind a device performing network address translation (NAT). NAT (Optional) Border Gateway Protocol (BGP) Autonomous System Number (ASN) of the customer gateway, if you are creating a dynamically routed VPN connection. Private ASN 64512 - 65534 Amazon VPC supports 2-byte ASN numbers Internal network IP ranges that you want advertised over the VPN connection to the VPC. Redundant VPN Connections can be set up for failover. Use of a second customer gateway is required. VPC “Private RFC 1918 Address Space”’ – 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 Autonomous System Number - uniquely identifies each network on the Internet. Cost: $.12/GB of Traffic (depending on outbound data transfer per month)
8. Zoltak Reduced network transfer costs Improved application performance with predictable metrics Transferring large data sets Resiliency: Active/Active (BGP multipath). Network traffic is load balanced across both connections. If one connection becomes unavailable, all traffic is routed through the other. This is the default configuration. Active/Passive (failover). One connection is handling traffic, and the other is on standby. If the active connection becomes unavailable, all traffic is routed through the passive connection. Private Configuration: A new, unused VLAN tag that you select. A public or private BGP ASN. If you are using a public ASN, you must own it. If you are using a private ASN, it must be in the 65000 range. The network prefixes to advertise. Any advertised prefix must include only your ASN in the BGP AS-PATH. The virtual private gateway to connect to.  Public Configuration: A new, unused VLAN tag that you select. A public or private Border Gateway Protocol (BGP) Autonomous System Number (ASN). If you are using a public ASN, you must own it. If you are using a private ASN, it must be in the 65000 range. A unique CIDR for your interface IP addresses that does not overlap another CIDR announced via AWS Direct Connect. A unique CIDR range to announce via AWS Direct Connect that does not overlap another CIDR announced via AWS Direct Connect. Whether this connection will be paired with another AWS Direct Connect connection. If this connection will be paired with another AWS Direct Connect connection for redundancy, provide the other connection's connection ID, which you can find in the AWS Direct Connect console, and the pairing model for the connections, either active/passive (failover) or active/active (BGP multipath). Key Information: Each DX location is mapped to a single AWS region DX sessions are isolated, no inter-routing traverses DX border(unless EC2 is used/ or customer routers are interconnected) Customers cannot access to Internet directly from DX. Multiple “public” virtual interfaces are allowed from a single DX Connection Multiple “private” virtual interfaces VPC connections are allowed from a single DX Connection VLANs (virtual interfaces) can be tagged to different accounts. VPC “Private RFC 1918 Address Space” Reduced network transfer costs Improved application performance with predictable metrics Transferring large data sets Security and compliance Alternative to Internet-based IPSEC VPN
9. Zoltak Public Configuration: A new, unused VLAN tag that you select. A public or private Border Gateway Protocol (BGP) Autonomous System Number (ASN). If you are using a public ASN, you must own it. If you are using a private ASN, it must be in the 65000 range. A unique CIDR for your interface IP addresses that does not overlap another CIDR announced via AWS Direct Connect. A unique CIDR range to announce via AWS Direct Connect that does not overlap another CIDR announced via AWS Direct Connect. Whether this connection will be paired with another AWS Direct Connect connection. If this connection will be paired with another AWS Direct Connect connection for redundancy, provide the other connection's connection ID, which you can find in the AWS Direct Connect console, and the pairing model for the connections, either active/passive (failover) or active/active (BGP multipath).
10. Zoltak Customer Router Hardware Requirements: AWS Direct Connect require layer 2 single mode fiber, 1000BASE-LX (1310nm) for Gigabit Ethernet, or 10GBASE-LR (1310nm) for 10 Gigabit Ethernet. Support 802.1Q VLANs across this connection. Support Border Gateway Protocol (BGP) and BGP MD5 authentication. Optional support for bidirectional Forwarding Detection (BFD). Also available in speed as low as 50Mbps “This is done with APN partner and will be load sharing on the connection” Cost: $.30/hr for 1 Gbps & $2.25/hr for 10 Gbps | $219.6 per month or $1647 per month $.045 /GB of outbound data Resiliency: Active/Active (BGP multipath). Network traffic is load balanced across both connections. If one connection becomes unavailable, all traffic is routed through the other. This is the default configuration. Active/Passive (failover). One connection is handling traffic, and the other is on standby. If the active connection becomes unavailable, all traffic is routed through the passive connection. Private Configuration: A new, unused VLAN tag that you select. A public or private BGP ASN. If you are using a public ASN, you must own it. If you are using a private ASN, it must be in the 65000 range. The network prefixes to advertise. Any advertised prefix must include only your ASN in the BGP AS-PATH. The virtual private gateway to connect to.  Public Configuration: A new, unused VLAN tag that you select. A public or private Border Gateway Protocol (BGP) Autonomous System Number (ASN). If you are using a public ASN, you must own it. If you are using a private ASN, it must be in the 65000 range. A unique CIDR for your interface IP addresses that does not overlap another CIDR announced via AWS Direct Connect. A unique CIDR range to announce via AWS Direct Connect that does not overlap another CIDR announced via AWS Direct Connect. Whether this connection will be paired with another AWS Direct Connect connection. If this connection will be paired with another AWS Direct Connect connection for redundancy, provide the other connection's connection ID, which you can find in the AWS Direct Connect console, and the pairing model for the connections, either active/passive (failover) or active/active (BGP multipath). Key Information: Each DX location is mapped to a single AWS region DX sessions are isolated, no inter-routing traverses DX border(unless EC2 is used/ or customer routers are interconnected) Customers cannot access to Internet directly from DX. Multiple “public” virtual interfaces are allowed from a single DX Connection Multiple “private” virtual interfaces VPC connections are allowed from a single DX Connection VLANs (virtual interfaces) can be tagged to different accounts. VPC “Private RFC 1918 Address Space” Reduced network transfer costs Improved application performance with predictable metrics Transferring large data sets Security and compliance Alternative to Internet-based IPSEC VPN
11. Zoltak Customers concerned with getting the most out of their existing investments / use our DirectConnect service. This example shows how you can leverage your existing database / and application infrastructures, / while capturing the benefits of AWS with our Auto Scaling, / Elastic Load Balancing and / Elastic Cloud Computing services.
12. Shiva What is Hybrid? Why Hybrid? Challenges and Best Practices How Hybrid? Connectivity Enterprise Integration Common Hybrid workloads Example hybrid workloads
13. Shiva Integration with Federation Services and Active Directory. AWS whitepaper, reference architecture and Cloudformation template to set up a resilient, highly available AD and domain services in minutes
14. Shiva Use Cases: Enterprise/business customers starting a new Windows environment with AWS. Connect their on-premises environment to the cloud to use their existing credentials on AWS instances. Lab/Test environments. Isolation of credentials for contractors/temp workers Connect Directory: (Prerequisites) Connectivity to On-Premise datacenter IPSec VPN or Direct Connect IP address of on-premises DNS server Credentials for domain privileged user Creates a Connect SecurityGroup which is used on the customer side Connect Directory Functionality: Enables use of existing account and credentials on on-premises Active Directory domain. Connects your on-premises directory to AWS Apps apps and services such as Workspaces and Zocalo. Acts as a proxy of requests (ie. authentication, query/search) and sends them to the on-premises domain.) No data is stored on AWS. Connect Access URL: Globally unique ‘friendly’ identifier for AWS Directory Service Chosen by customer 1 unique access URL per Directory Used by Apps such as Zocalo to access their service or to the AWS Management Console. Names reserved for top Fortune 500 companies IAM Federation: Ability to use your on-premise or simple AD directory credentials to login into AWS management console. Map users or groups to IAM roles (new or existing). Use access URL of directory followed by /console (ie. https://test.awsapps.com/console). Highlights: Simple Use AWS management console or simple API calls to setup within minutes Managed Automates management tasks like backup or patch management Secure Accessible via your security groups within VPC only Compatible Continue using existing Active Directory tools (except Powershell AD module) Reliable Multi-Availability Zone by default, Automatic periodic snapshots Versatile Setup completely new or connect existing directory Choose from different sizes (Small or Large) Limitations: Directory with single sub tree (i.e. no multi-domain forests) Connect directory functions as proxy (no sync functionality) Windows Server 2008 R2 forest functional level No AD web services protocol (no ADAC or PowerShell) Only certain applications supported (No Exchange) Inability to change directory type after creation. No performance metrics available for customers.
15. Shiva We also allow you to secure access to your AWS resources using your identity management systems, either to provide single sign on to your AWS management console or federated access to APIs and recently the support center as well. You can build this federation using the AWS Security Token Service. However the easiest way to federate to AWS is using industry standard SAML2.0 integration; it’s supported by many common on-premises directories as well as a range of other external SAML2.0 compliant Identity providers. Auth0 - Auth0 enables identity delegation for AWS APIs (such as S3, EC2, and DynamoDB) so that developers can easily integrate authentication from any IdP with AWS' powerful IAM policies for fine-grained access control, along with SSO with the AWS management console using SAML. Ping Identity - Ping Identity is The Identity Security Company whose identity and access management platform gives enterprise customers and employees one-click access to any application from any device. To enable SAML-based SSO to AWS, configure AWS with PingFederate or with PingOne. Salesforce - Salesforce Identity provides open-standard identity and access management for web and mobile applications, through the simplicity, transparency, and trust of the Salesforce Platform. Learn more about how to configure Salesforce.com to use SAML to achieve SSO with AWS. Okta - Okta provides a comprehensive but flexible SSO solution that spans all of your web applications, whether they are in the cloud or behind the firewall. Learn more about how to configure Okta to use SAML to achieve SSO with AWS.
16. Shiva Account structure is an important design decision, both from an operational perspective and billing perspective. Account structure determines Billing structure Blast radius in case of compromise Service limits Alignment to organizational structure
17. Shiva Once you’ve got your resources secure and your identity management systems integrated you’ll want to start keeping track of what you are using. Every AWS resource or object can be described through an API call. For example I can get a list of all my running EC2 instances, what type they are, where they are running, which VPC there in, what security rules they have and a range of other information. And this information is dynamic, as you add resources or additional information about your resources it can be described. You can add your own information using tags, you get to specify what the tag names are and the tag value. For example I can define a set of custom tags for my EC2 instance, including the purpose and cost center; I can then use those tags to control access using Identity & Access Management, or maybe I want to use the Cost Centre tag to Allocate costs to different business units. Tagging is incredible powerful and can help you create granular charge back of the services running in AWS. Now you have the situation where you can describe every resource, assign custom information and with an API command dynamically generate an inventory of your AWS environment, not just a list of resources but also security information about those resources. integrate this into your centralized management systems and your CMDB will never be out of date again. There are also a range of emerging 3rd party tools that help you visualize your AWS resources in real-time, making use of the AWS APIs and providing invaluable insight to your operations teams Madeira's visualization technology can help engineers explain how the cloud works to their pointy-haired bosses, and can make AWS more accessible to people who have previously worked mostly within visual on-premise management environments. Janitor Monkey
18. Shiva
19. Shiva In the time you’ve spent with us today you could have deployed infrastructure and applications ready to serve your business with high levels of automation and simplicity using AWS. And we provide services to give operational insight into those resources; Amazon CloudWatch provides real-time insight into your AWS resources and allows you to integrate your own metrics. Those metrics can generate alarms when breached and can you can use the Amazon Simple Notification Service to send email alerts or make web services calls to your alerting systems. And your current server monitoring tools still work, you simply install them on your EC2 instances. Many of your existing tools already have integration into the AWS APIs; these include a number of the open source tools and commercial offerings including Microsoft. With system center integration you can monitor and manage your Windows infrastructure on AWS as you do today; And remember your established operational processes don't need to change, you simply have the opportunity to make them more agile and adapt them to the flexibility that the AWS platform offers.
20. Shiva Next lets take it a step further and find out how AWS can help you deliver a service catalog with real business value.Every object in AWS can be described through an API and objects in AWS can be grouped together and described as templates. For example you can create a template for a standardised environment defining the EC2 instances, security groups, network placement, databases, etc. These templates can be re-dployed as stacks <C>because templates are re-usable, standardised architectures, where we turn infrastructure into code. Stacks can be as simple as a single instance, or as complex as highly available multi-tier architecture.created and managed using the AWS CloudFormation service.
21. Shiva Lets take an example: Your marketing department wants a new highly available web application for a one month campaign, they select the service request for this from your catalog The requests goes into the normal procurement, delivery, installation, integration and release process; weeks later your infrastructure is available for you start the application configuration. From my own personal experience I would wait 8-12 weeks minimum to get base infrastructure. Now if you integrate your service desk or service catalog with AWS CloudFormation you can deploy your infrastructure within minutes of a request being approved. and when you are finished with the solution, simply archive to S3 and delete the stack. Ensuring that you can meet your business needs in the timeframe they require with all the security controls and standardisation that you expect.
22. Shiva
23. Shiva These tools / are enablers / to make your Hybrid architectures more achievable.   These tools assist you in your effort to move /, manage / and monitor your business workloads in AWS. These plug-ins / allow you to manage instances and services inside your AWS account. The Management Pack for SCOM / allows you to monitor and alert / upon the health and performance of your hybrid infrastructure.
24. Shiva What is Hybrid? Why Hybrid? Challenges and Best Practices How Hybrid? Connectivity Enterprise Integration Common Hybrid workloads Example hybrid workloads
25. Shiva
26. Shiva On-premise backup server with Amazon S3 Eliminate tape, hardware, off-site storage Reduce capital expense for backup infrastructure Alleviate worry about backup durability Never run out of backup capacity Data stored off-site, with high durability, in multiple locations AWS Storage Gateway VTL Virtual tape – Virtual tape is analogous to a physical tape cartridge. However, virtual tape data is stored in the AWS cloud. Like physical tapes, virtual tapes can be blank or can have data written on them. You can create virtual tapes either by using the AWS Storage Gateway console or programmatically by using the AWS Storage Gateway API. Each gateway can contain up to 1500 tapes or up to 150 TiB of total tape data at a time. The size of each virtual tape, which you can configure when you create the tape, is between 100 GiB and 2.5 TiB. Virtual tape library (VTL) – A VTL is analogous to a physical tape library available on-premises with robotic arms and tape drives, including the collection of virtual tapes stored within the library. Each gateway-VTL comes with one VTL. The virtual tapes that you create appear in your gateway's VTL. Tapes in the VTL are backed up by Amazon S3. As your backup software writes data to the gateway, the gateway stores data locally and then asynchronously uploads it to virtual tapes in your VTL—that is, Amazon Simple Storage Service (Amazon S3). Tape drive – A VTL tape drive is analogous to a physical tape drive that can perform I/O and seek operations on a tape. Each VTL comes with a set of 10 tape drives, which are available to your backup application as iSCSI devices. Media changer – A VTL media changer is analogous to a robot that moves tapes around in a physical tape library's storage slots and tape drives. Each VTL comes with one media changer, which is available to your backup application as an iSCSI device. Virtual tape shelf (VTS) – A VTS is analogous to an off-site tape holding facility. You can archive tapes from your gateway's VTL to the VTS and, if needed, retrieve tapes from the VTS back to your gateway's VTL. Archiving tapes – When your backup software ejects a tape, your gateway moves the tape to the VTS for long-term storage. The VTS is located in the AWS region in which you activated the gateway. Tapes in the VTS are stored in Amazon Glacier, an extremely low-cost storage service for data archiving and backup. For more information, go to Amazon Glacier. Retrieving tapes – Tapes archived to the VTS cannot be read directly. To read an archived tape, you must first retrieve it to your gateway-VTL either by using the AWS Storage Gateway console or by using the AWS Storage Gateway API. A retrieved tape will be available in your VTL in about 24 hours.
27. Shiva On-premise storage appliance with Amazon S3 Reduce capital expense for storage infrastructure Alleviate worry about storage durability Never run out of storage capacity Storage appliance integrated to Amazon S3 Data durably stored off-site in multiple locations Take advantage of advanced storage optimization options, block based de-duplication, compression, WAN acceleration
28. Shiva
29. Shiva
30. Shiva Why Hybrid deployment for Kellogs? Cloud is the default strategy for new projects Automation, orchestration, and self-provisioning of IT and HANA resources Shift from CapEx to OpEx Ability to reduce the overall project cycle with impact to the bottom line Hybrid scenario with AWS allowed Kellogg to control both the timing and extent of cloud deployment SAP infrastructure hosted in external cloud and on-premises; both run and supported fully by in-house personnel SEACO, worlds largest container leasing company just finished migration of their entire SAP business suite landscape which includes ERP, CRM, BW, Portal Content Server and Solution Manager. Assisted by UK based Lemongrass consulting. Initial setup of core infrastructure and network topology, followed by Dev/Test, and then a DR. Finally production was cut over via a DR mechanism.
31. Shiva http://highscalability.com/blog/2014/12/1/auth0-architecture-running-in-multiple-cloud-providers-and-r.html
32. Shiva