Vertex Macro | Financial Cloud Cloud · AWS Re:cap
AWS Re:cap 02: Goldman Sachs: Fast Track your applications onto Cloud - AWS Re:cap Q1/2023
Introduction by Ragini Lalwani:
● Leads public cloud onboarding and migration at Goldman Sachs.
● Wishes everyone a Happy Women's Day and Holi.
● Has worked at Goldman Sachs for almost 15 years, starting as an intern.
● Has experience with various cloud platforms: private, hybrid, and public.
● Discussion on Goldman Sachs' approach to ensuring rapid and secure cloud use by developers in a regulated industry.
Introduction by Aditya:
● Leads finance planning engineering teams and is the lead architect for FP&A at Goldman Sachs.
● Also has close to 15 years with the firm, starting with web app development, now focused on enterprise systems.
● Discussion on the firm's unique phase of cloud onboarding and the transition from on-prem to cloud.
● Will talk about onboarding experience with public cloud, secure tooling, acceleration of cloud onboarding, and key highlights and learnings.
Overview of Goldman Sachs:
● A leading financial institution offering a broad range of services including investment banking, securities, investment management, and consumer banking.
● Serves a large and diversified client base including corporations, financial institutions, governments, and individuals.
● Founded in 1869, headquartered in New York, with offices across major financial centers globally.
● Focus on accelerating Goldman Sachs' cloud journey with a security-first mindset.
Fast Track Section Overview:
● Emphasizes a customer-centric mindset using Amazon's "working backwards" process.
Problem Statement from the Customer's Perspective (Application Developer):
● New developers need to understand the firm's security approach, posture, and modules.
● Deployment requires passing through security reviews, which can be manual and time-consuming.
● Security reviews are required not only at the initial deployment but also for subsequent changes, leading to a potentially slow and iterative process.
● Developers may receive alerts after deployment indicating incorrect configurations, highlighting the need for preventive controls rather than detective controls.
● Developers, skilled in application development, may not have the expertise in cloud security, creating a dichotomy between development and security requirements.
Tenets for Addressing the Problem Statement:
● Implement guardrails to ensure secure-by-design deployments, automating the security baseline.
● Establish a clear shared responsibility model delineating the roles of the cloud service provider, Goldman Sachs Cloud teams, and application developers.
● Utilize cloud-native technologies to minimize the need for learning firm-specific tooling, allowing developers to leverage existing knowledge of AWS CDK or Kubernetes.
● Aim for self-service capabilities to reduce the need for ticket raising or manual reviews, enabling more developers to access the cloud without hurdles.
Introduction to Fast Track:
● Fast Track is the solution developed in response to the identified problem statement.
● It involves a technical deep dive into the implementation details.
● The goal is to provide a secure, efficient, and self-service-oriented cloud onboarding experience for developers.
Fast Track Overview:
● A fully managed service enabling Goldman Sachs developers to rapidly develop and deploy software to AWS.
● Enforces the firm's security, regulatory, and compliance requirements through guardrails.
● Enhances the security posture by programmatically and proactively enforcing security policies.
Key Features of Fast Track:
● Handles account creation and pipeline management behind the scenes.
● Users interact with Fast Track via CLI or API to create, delete, or share pipelines.
● Pipeline creation triggers an account provisioning workflow, creating a service account for AWS resources and a pipeline account for management.
● Users push code changes, which run through a deployment pipeline and are checked against guardrails.
● Guardrails enforce the firm's baseline security policy, ensuring resources meet security requirements before deployment.
Simplified User Journey:
● User creates a Fast Track pipeline and associates it with a GitLab repo.
● Writes infrastructure as code using AWS CDK.
● Pipeline synthesizes CDK code into CloudFormation templates.
● Guardrails run against these templates, and the policy engine determines if the deployment can proceed ("go") or not ("no go").
Guardrails Details:
● Guardrails follow a default-deny policy.
● Written using Open Policy Agent (OPA).
● Run against standard CloudFormation templates.
● Example guardrail: Ensures that malicious actors cannot access unencrypted objects.
Threat Modeling and Control Definition:
● Uses NIST 800-53 guidelines to identify threats for specific technologies.
● ISO 27001 is an international standard for information security management systems, while NIST SP 800-53 is a U.S. government standard for security and privacy controls. 2. ISO 27001 focuses on the management of information security, while NIST SP 800-53 focuses on the technical security controls.
● Example threat: Malicious actors can access an unencrypted object.
● Control definition: Encrypt the object.
● Controls are defined and used as OPA policies, reviewed, and unit tested by the security team.
Collaboration and Release of Controls:
● Collaboration between the Cloud enablement guardrail development team and security teams.
● Controls are released as guardrails, creating a secure environment but potentially posing hurdles for developers.
Accelerators:
● Built to work with guardrails, allowing developers to quickly deploy resources that comply with all guardrails.
● Example: Fast Track S3 bucket that works with all 10 guardrails blocking S3.
Sample Code and Starter Projects:
● Provided to help users bootstrap their cloud journey.
Fast Track Launch and Production Use:
● Fast Track has been launched and is available, with several applications in production and many more underway.
Organizational Alignment and Model:
● Goldman Sachs recognized the need for a first-class developer experience without compromising security.
● Fast Track was deemed a flagship project, with Cloud enablement leading the development in collaboration with technology risk teams.
● Guardrail development involved a mix of developers, cloud architects, and security engineers.
● A guardrail contribution model ensures that specific technology teams (e.g., storage engineering) review and contribute to relevant guardrails.
Response to Question on Organizational Structure:
● Acknowledges the typical siloed model in large enterprises.
● Emphasizes the need for a collaborative approach to balance developer experience and security.
● Highlights the importance of a mandate from management to foster cross-team collaboration.
● Notes the constraint that not all Cloud enablement team members are experts in every cloud service (e.g., S3).
● The mix of skills required for guardrail development and the contribution model for specialized reviews.
Clarification on Ownership and Responsibilities in the Cloud:
● In the cloud, the application team owns all their cloud resources, including computers and databases, which are deployed into their VPC.
● The database team is responsible for working with Cloud enablement to establish access control, policy management, and audit-related pieces.
● On-premises, the database team owns the databases, but in the cloud, the customer (application team) owns the databases running in AWS.
Summary of Organizational Model:
● A central Cloud team establishes the guardrails framework and is the authority around cloud development.
● The CIO, CTO, and Tech Risk Office sponsor new guardrail development and policies around cloud onboarding and governance.
● Associated teams (DB teams, IAM teams, etc.) work with the central Cloud team to onboard their services and policies.
● Application development teams implement their apps using Fast Track and may request additional services, which are then worked on with related teams.
Collaborative Model:
● Multiple teams play their part in the cloud onboarding process, ensuring a collaborative approach to cloud development and security.
Hands-On Approach for the First Application:
● The initial application on Fast Track required a more hands-on approach as the team was learning about the customer journey.
● Future applications aim to leverage existing guardrails, allowing customers to use the service independently.
● Investment in tutorials and documentation to support self-service usage.
Platform Team's Role:
● The platform team provides guidance and builds security controls (guardrails) for resources like Lambda, DynamoDB, and RDS.
● Developers are free to choose the resources they need, with the platform team helping to build the necessary security controls.
● Accelerators are provided to make it easier for developers to deploy resources safely.
Guardrails in Development vs. Production:
● The same guardrails are applied in both development (sandbox) and production environments.
● Guardrails are in blocking mode in all environments to maintain a consistent security posture.
● Fine-grained control allows for specific guardrail bypasses on a per-pipeline level, subject to tech risk review.
Technical Implementation of Guardrails:
● Guardrails are checked via an API that the pipeline interacts with.
● The pipeline synthesizes infrastructure as code (e.g., AWS CDK) into CloudFormation templates.
● These templates are then checked against the guardrails and run using Open Policy Agent (OPA).
● The policy engine determines if the deployment can proceed ("go") or not ("no go").
Technical Implementation of Guardrails (Continued)
● Additionally, an IDE plugin is available to provide developers with feedback sooner, without the need to deploy the pipeline.
● Developers can see the feedback as they develop in their IDE, helping them adhere to security standards early in the development process.
Question on NIST Implementation and Threat Database:
● The project started with NIST 800-53 as a baseline.
● The firm's cyber defense and Intel team keeps the team informed about new threats.
● Decisions on managing new threats are made based on the vulnerability level (critical or not) and integrated into the software development lifecycle (SDLC).
● Detective controls complement the preventative controls provided by the guardrails.
Introduction to FP&A (Finance Planning and Analysis):
● FP&A is responsible for firm-wide business planning, cost allocations, and driving financial insights.
● FP&A is pursuing a cloud-first strategy, aiming to move 100% of on-prem and new applications to the cloud by the end of 2024.
● Reasons for adopting the cloud include monthly compute usage peaks and troughs, limitations of current on-prem solutions (installed and run on an organization's own physical hardware and servers, located within their own facilities), and the desire to reduce engineering time spent on maintenance.
● FP&A has aggressively decoupled legacy on-prem infrastructure, with approximately 70% completed, and has built new applications with cloud considerations in mind over the past three years.
FP&A's Cloud Strategy and Adoption:
● FP&A has been building microservice-friendly, cloud-ready architectures and heavily investing in Infrastructure as Code (IaC) even on-prem.
● The adoption of the cloud is driven by the low cost of compute, increased productivity, and reduced build and maintenance cycles.
● Fast Track, with sponsorship from engineering leaders, became the strategic platform for the firm.
First Application: FBA Central:
● A simple gateway application was chosen as the first app to learn AWS and Fast Track.
● Developed in three to four months, went live with an MVP in September-October.
● Representative microservice app with Spring Boot APIs, deployed using ECS Fargate and DynamoDB, with a React.js UI.
Subsequent Applications:
● Built more complex applications with increased microservices, on-prem connectivity, and workflow solutions.
● Deployed the first app in a dedicated FP&A-specific VPC, a first for the firm.
● Despite increased application complexity, Fast Track enabled increased productivity and decreased delivery times.
● Teams became more skilled over time, further reducing timelines.
Summary:
● FP&A's cloud-first strategy, combined with Fast Track, has led to successful onboarding of applications with increasing complexity, improved productivity, and reduced delivery times.
FP&A's Cloud Journey and Future Focus:
● Deployed three cloud-native apps with eight plus services.
● Learned to build software, undergo cloud onboarding, tech risk reviews, architecture bar raising, and SDLC processes.
● Trained about half of FP&A engineers.
● Key focus for 2023: high-usage apps and data-intensive calculations, processing about 6 billion records daily.
● Will explore serverless offerings (Lambda, Step Functions) and data processing/analytics services (Glue, Athena, S3, Snowflake).
● Aim to build FP&A-centric accelerators using CDK constructs to further accelerate development.
Key Decisions and Success Factors:
● Formation of a core team of four developers from different regions to train scrum teams, accelerating development.
● Choice of Fast Track for secure cloud onboarding, dealing with sensitive data.
● Fast Track's guardrails are tech risk-approved and continuously executed at every pipeline, providing a secure and up-to-date profile.
● Fast Track's CDK accelerators and common patterns helped improve developer productivity.
Deployment and Learnings:
● Deployed the first simple app, FBA Central, using AWS components like DynamoDB, ECS Fargate, and ALB.
● Underwent rigorous tech risk reviews and architecture bar raising.
● Learned about gold live mechanisms, environment creation, observability, logging, and using CloudWatch and PagerDuty.
● Decommissioned the on-prem app, becoming fully cloud-native for FP&A.
Cloud Learnings:
● Cloud provides a broader and more sophisticated set of options to closely match application compute consumption needs.
● Encourages organizations starting their cloud journey to adapt the playbook and learn from FP&A's experience.
Importance of Design Choices:
● Engineers must understand the consequences of their design choices, which impact operational costs and maintenance.
● Serverless options offer lower costs, easy scaling, and low maintenance.
● Infrastructure as Code (IaC) allows for faster innovation and toggling between different compute and cost profiles.
Benefits of IaC:
● Enables faster innovation and optimization for cloud costs.
● Allows for different costs and compute profiles based on architecture choices.
● Business Continuity Planning (BCP) is more straightforward with cloud strategies compared to on-prem.
Increased Developer Productivity and Cost Optimization:
● Developer productivity increased with the ability to manage costs on a daily basis.
● The maturing ecosystem and Fast Track have facilitated the onboarding of applications.
Resources for Further Learning:
● Goldman Sachs Engineering developer blog for insights on cloud and related builds.
Clarification on Fast Track and Open Source:
● Fast Track itself has not been open-sourced.
● Goldman Sachs has an open-source team and has open-sourced various projects.
● The guardrails for Fast Track were initially built in partnership with AWS and have been shared back with AWS, who are using them as part of CFN Card.
● Goldman Sachs does not consider their guardrails a competitive advantage and is open to sharing them.
SkyPath: Cloud to On-Prem Integration:
● SkyPath is a connectivity solution that allows applications in different trust zones to connect.
● It addresses the challenge of connecting on-prem and cloud applications in a hybrid environment.
● SkyPath leverages a Service Identity Framework (SIF) to establish a unique identity for each endpoint before onboarding to SkyPath.
● This ensures that only endpoints with a clearly defined identity can connect, addressing potential identity challenges in the integration.
Service Identity Framework (SIF):
● SIF requires every endpoint to have a unique identity defined before it can onboard to SkyPath.
● This acts as a pseudo-cloud identity for on-prem services, ensuring secure and controlled access.
● Establishing a service identity is a prerequisite for enabling access to on-prem APIs via SkyPath.
Conclusion:
● The discussion highlights the importance of secure and controlled integration between on-prem and cloud environments, facilitated by solutions like SkyPath and the Service Identity Framework.