Reference https://cloud.google.com/architecture/setting-up-private-access-to-cloud-apis-through-vpn-tunnels

Cloud Dataflow is a fully-managed service for transforming and enriching data in stream (real time) and batch (historical) modes with equal reliability and expressiveness -- no more complex workarounds or compromises needed.
Reference: https://cloud.google.com/dataflow/

Explanation
https://cloud.google.com/iam/docs/understanding-service-accounts
Topic 6, TerramEarth Case 2
Company Overview
TerramEarth manufactures heavy equipment for the mining and agricultural industries. About 80% of their business is from mining and 20% from agriculture. They currently have over 500 dealers and service centers in
100 countries. Their mission is to build products that make their customers more productive.
Solution Concept
There are 20 million TerramEarth vehicles in operation that collect 120 fields of data per second. Data is stored locally on the vehicle and can be accessed for analysis when a vehicle is serviced. The data is downloaded via a maintenance port. This same port can be used to adjust operational parameters, allowing the vehicles to be upgraded in the field with new computing modules.
Approximately 200,000 vehicles are connected to a cellular network, allowing TerramEarth to collect data directly. At a rate of 120 fields of data per second with 22 hours of operation per day, TerramEarth collects a total of about 9 TB/day from these connected vehicles.
Existing Technical Environment
TerramEarth's existing architecture is composed of Linux and Windows-based systems that reside in a single
U.S. west coast based data center. These systems gzip CSV files from the field and upload via FTP, and place the data in their data warehouse. Because this process takes time, aggregated reports are based on data that is 3 weeks old.
With this data, TerramEarth has been able to preemptively stock replacement parts and reduce unplanned downtime of their vehicles by 60%. However, because the data is stale, some customers are without their vehicles for up to 4 weeks while they wait for replacement parts.
Business Requirements
Decrease unplanned vehicle downtime to less than 1 week.
Support the dealer network with more data on how their customers use their equipment to better position new products and services Have the ability to partner with different companies - especially with seed and fertilizer suppliers in the fast-growing agricultural business - to create compelling joint offerings for their customers.
Technical Requirements
Expand beyond a single datacenter to decrease latency to the American Midwest and east coast.
Create a backup strategy.
Increase security of data transfer from equipment to the datacenter.
Improve data in the data warehouse.
Use customer and equipment data to anticipate customer needs.
Application 1: Data ingest
A custom Python application reads uploaded datafiles from a single server, writes to the data warehouse.
Compute:
Windows Server 2008 R2
- 16 CPUs
- 128 GB of RAM
- 10 TB local HDD storage
Application 2: Reporting
An off the shelf application that business analysts use to run a daily report to see what equipment needs repair.
Only 2 analysts of a team of 10 (5 west coast, 5 east coast) can connect to the reporting application at a time.
Compute:
Off the shelf application. License tied to number of physical CPUs
- Windows Server 2008 R2
- 16 CPUs
- 32 GB of RAM
- 500 GB HDD
Data warehouse:
A single PostgreSQL server
- RedHat Linux
- 64 CPUs
- 128 GB of RAM
- 4x 6TB HDD in RAID 0
Executive Statement
Our competitive advantage has always been in the manufacturing process, with our ability to build better vehicles for lower cost than our competitors. However, new products with different approaches are constantly being developed, and I'm concerned that we lack the skills to undergo the next wave of transformations in our industry. My goals are to build our skills while addressing immediate market needs through incremental innovations.

Explanation
Note: The principle of least privilege and separation of duties are concepts that, although semantically different, are intrinsically related from the standpoint of security. The intent behind both is to prevent people from having higher privilege levels than they actually need
* Principle of Least Privilege: Users should only have the least amount of privileges required to perform their job and no more. This reduces authorization exploitation by limiting access to resources such as targets, jobs, or monitoring templates for which they are not authorized.
* Separation of Duties: Beyond limiting user privilege level, you also limit user duties, or the specific jobs they can perform. No user should be given responsibility for more than one related function. This limits the ability of a user to perform a malicious action and then cover up that action.
References: https://cloud.google.com/kms/docs/separation-of-duties
Topic 3, Dress4Win case study
Company Overview
Dress4win is a web-based company that helps their users organize and manage their personal wardrobe using a website and mobile application. The company also cultivates an active social network that connects their users with designers and retailers. They monetize their services through advertising, e-commerce, referrals, and a freemium app model.
Company Background
Dress4win's application has grown from a few servers in the founder's garage to several hundred servers and appliances in a colocated data center. However, the capacity of their infrastructure is now insufficient for the application's rapid growth. Because of this growth and the company's desire to innovate faster, Dress4win is committing to a full migration to a public cloud.
Solution Concept
For the first phase of their migration to the cloud, Dress4win is considering moving their development and test environments. They are also considering building a disaster recovery site, because their current infrastructure is at a single location. They are not sure which components of their architecture they can migrate as is and which components they need to change before migrating them.
Existing Technical Environment
The Dress4win application is served out of a single data center location.
* Databases:
* MySQL - user data, inventory, static data
* Redis - metadata, social graph, caching
* Application servers:
* Tomcat - Java micro-services
* Nginx - static content
* Apache Beam - Batch processing
* Storage appliances:
* iSCSI for VM hosts
* Fiber channel SAN - MySQL databases
* NAS - image storage, logs, backups
* Apache Hadoop/Spark servers:
* Data analysis
* Real-time trending calculations
* MQ servers:
* Messaging
* Social notifications
* Events
* Miscellaneous servers:
* Jenkins, monitoring, bastion hosts, security scanners
Business Requirements
* Build a reliable and reproducible environment with scaled parity of production.
* Improve security by defining and adhering to a set of security and Identity and Access Management (IAM) best practices for cloud.
* Improve business agility and speed of innovation through rapid provisioning of new resources.
* Analyze and optimize architecture for performance in the cloud.
* Migrate fully to the cloud if all other requirements are met.
Technical Requirements
* Evaluate and choose an automation framework for provisioning resources in cloud.
* Support failover of the production environment to cloud during an emergency.
* Identify production services that can migrate to cloud to save capacity.
* Use managed services whenever possible.
* Encrypt data on the wire and at rest.
* Support multiple VPN connections between the production data center and cloud environment.
CEO Statement
Our investors are concerned about our ability to scale and contain costs with our current infrastructure. They are also concerned that a new competitor could use a public cloud platform to offset their up-front investment and freeing them to focus on developing better features.
CTO Statement
We have invested heavily in the current infrastructure, but much of the equipment is approaching the end of its useful life. We are consistently waiting weeks for new gear to be racked before we can start new projects. Our traffic patterns are highest in the mornings and weekend evenings; during other times, 80% of our capacity is sitting idle.
CFO Statement
Our capital expenditure is now exceeding our quarterly projections. Migrating to the cloud will likely cause an initial increase in spending, but we expect to fully transition before our next hardware refresh cycle. Our total cost of ownership (TCO) analysis over the next 5 years puts a cloud strategy between 30 to 50% lower than our current model.