Mega Data Center Depends on Orchestration Software to Control Operations, Implement Microservices

Publisher Name :
Date: 01-May-2017
No. of pages: 115

The 2017 module has 115 pages and 74 tables and figures. Orchestration Software in the Mega Data Center is used to tie a fabric architecture together that fills up an entire building with 100,000 processors and 100,001 switches. The mega data center described in the study is effective because it leverages the economies of scale. This orchestration software infrastructure study module is part of a longer study that addresses the business issues connected with data center modernization. There are 26 module parts to the larger study comprised of detailed analysis of how new infrastructure layers will work to support management of vast quantities of data.

Business growth depends on technology spending. Intelligent, automated process, not manual labor systems are what speed business growth. We have had the situation in the data center where 93% of spending is just to keep current systems running, many of those plagued with manual input. Mega data centers change that pattern of IT manual process.

The Internet has grown by a factor of 100 over the past 10 years. To accommodate that growth, mega data centers have evolved to provide processing at scale. Facebook for one, has increased the corporate data center compute capacity by a factor of 1,000, virtually eliminating much manual process. Orchestration software is a key aspect of that process. To meet future demands on the Internet over the next 10 years, companies with that capacity need to increase capacity by the same amount again while the other companies struggle to catch up. Nobody really knows how to get to increasing compute capacity by another factor of 1,000.

Business growth depends on technology spending. Intelligent, automated process, not manual labor systems are what speed business growth. We have had the situation in the data center where 93% of spending is just to keep current systems running, many of those plagued with manual input. Mega data centers change that pattern of IT manual process.

Realigning data center cost structures is a core job of orchestration software. The enterprise data centers and many cloud infrastructure operations all have similar problems of being mired in administrative expense. Containers address that issue by creating vastly more efficient operations for data center infrastructure.

Lead author of the team that prepared the study, “The only way to realign cost structure is to automate infrastructure management and orchestration. Mega data centers automate server and connectivity management using orchestration software to manage multiple application containers. Other systems automate switching and storage, along with hypervisor, operating system, and virtual machine provisioning. “

As IT relies more on virtualization and cloud mega data center computing, the physical infrastructure is flexible and agile enough to support the virtual infrastructure. Comprehensive infrastructure management and orchestration is essential.

The Enterprise Data Center has become a bottleneck, it needs to be completely replaced. Category 5 and Category 6 Ethernet cable is spread throughout the existing enterprise data centers and is too slow to handle all the digital data coming through the data center. Cat 5 and Cat 6 Ethernet utilized by the servers to achieve data transport using that cable does not keep up with the data coming through the data center the way optical cable and optical transceivers do.

The existing servers and cable are a problem because they are too slow for modern systems. The cable is too slow to handle all the data coming at us in the new digital age, and the associated technology that operates at Ethernet category 5 and category 6 cable speeds is too slow as well, this is why the entire set of existing enterprise data centers is a bottleneck.

Mobile data traffic is set to increase by a factor of eight between 2015 and 2020. Growth is anticipated at 53 percent per year, faster than systems revenue or industry revenue.

The theme of this study is that the pace of data expansion creates the need for more modern means of managing data. There are some companies that are doing a better job, better than others of adapting to IT infrastructure to the wild influx of data.

The four superstar companies that are able to leverage IT to achieve growth, Microsoft, Google, Facebook, and the leader AWS all use Clos architecture. What is significant is that systems have to hit a certain scale before Clos networks work Clos networks are what work now for flexibility and supporting innovation in an affordable manner. There is no dipping your toe in to try the system to see if it will work, it will not and then the IT says, “We tried that, we failed,” but what the executive needs to understand is that scale matters. A little mega data center does not exist. Only scale works.

Many companies are using digital technology to create market disruption. Amazon, Uber, Google, IBM, and Microsoft represent companies using effective strategic positioning that protects the security of the data. As entire industries shift to the digital world, once buoyant companies are threatened with disappearing. A digital transformation represents an approach that enables organizations to drive changes in their business models and ecosystems leveraging cloud computing, and not just hyperscale systems but leveraging mega data centers. Just as robots make work more automated, so also cloud based communications systems implement the IoT digital connectivity transformation.

Companies Profiled

Market Leaders


  • Amazon

  • Microsoft

  • Google

  • Facebook

Mega Data Center Depends on Orchestration Software to Control Operations, Implement Microservices

Table of Contents

Orchestration Software: Executive Summary
Sea Change Series: Orchestration Software In The Mega Data Center
Sea Change Series: Orchestration Software in the Mega Data Center, Amazon, Google, Microsoft, Facebook
Aim to Realign IT Cost Structure 3
Internet Has Grown by a Factor of 100 Over The Past 10 Years 4
Table of Contents 5
Orchestration Software Automates Data Center Infrastructure 14
Software Containers 15
Orchestration Schedulers Manage Containers 16
Orchestration Software Supports Container Automation 17
Realigning Data Center Cost Structures 18
IT Relies On Replacing Virtual Machine: VM Virtualization 19
Microservices 20
Microservices Features 21
Microservices Modules 23
Difficulties with Virtual Machines 24
Hypervisor a Difficulty 25
Virtual Machines Use Bare Metal, Containers Use Orchestration Software 27
Bare Metal an Inefficient Use of Compute Resource 28
Bare Metal Less Efficient 28
Industry Uses Robots Because Manual Labor Is Slow And Error Prone 29
IT processes Replace Manual Labor 30
Mega Data Center Orchestration Software 31
Large Fabric Network Not The Kind Of Environment That Can Be Realistically Configured And Operated In A Manual Way32
To Automate the Data Center Fabric 33
Value of Data Center Fabric 33
Google Shift from Bare Metal To Container Controllers 34
Google Container Controller Shift From Bare Metal In A Mixed Workload And In Nested Compute Units 35
Google Kubernetes Groups Software Containers 36
Fabric Services Inside A Container 37
Architecting Microsoft Cloud 38
Microsoft Managed Clustering and Container Management: Docker and Mesos 39
Microsoft Azure Service Fabric 41
Microsoft Is Seen As The Overall Winner In The Move To Application Containers 42
Microsoft Dublin Cloud 2.0 Mega Data Center 43
Microsoft Data Center, Dublin, 550,000 Square Feet 43
Microsoft Dublin Center Operates at a Power Usage Effectiveness (PUE) of 1.25. 44
Microsoft Data Center Container Area in Chicago. 44
Microsoft 45
Advantages of Using Containers 46
Orchestration Software Used to Create Containers 47
Disadvantages of Using Containers 48
Advantages of Virtual Machines 49
Container Orchestration 50
Use of Containers Eliminate Manual Process 51
IT Pros Increasingly Turn to Chef and Puppet 52
Hardware Containers Do Not Scale 53
Facebook Data Center Positioning 54
IBM Data Center Orchestration Software Automates Application Integration 55
Docker Orchestration & Docker Swarm 56
Docker Container Platform 57
Common Feature Sets For Orchestration Tools 58
Not all Orchestrators Are Created Equal 59
AWS Cloud Container Adoption Criteria 60
AWS Cloud Adoption Methodology 63
AWS Cloud Adoption Framework 64
AWS Market Leader In Cloud Computing 65
Facebook Fabric and Node are Core Structures Leveraging Software Orchestration 66
Apache Mesos Orchestration Software 67
Google Kubernetes Container 68
Google Container Builder Step Toward Building Pluggable Components in a Pipeline 69
Google Programmable Access To Network Stack 70
Google Andromeda Software Defined Networking (SDN)- 71
Google Compute Engine Load Balancing 72
Google Compute Engine Load Balanced Requests Architecture 73
Google Scaling Of The Compute Engine Load Balancing 74
Google Compute Engine (GCE) TCP Stream Performance Improvements 75
Google Cloud Platform TCP Andromeda Throughput Advantages 76
Google Open Sourced Its Container Management System Called Kubernetes 77
Facebook 79
Ability To Move Fast And Support Rapid Growth At The Core Of Facebook Infrastructure Design Philosophy 80
The Right Type of Cloud: Mega Data Centers 82
AWS Has Been Able To Adapt To Change 83
Manual Labor Is Slow And Error Prone 84
Mega Data Center Orchestration Software 86
Amazon, Google, Microsoft, Facebook 87
Cloud 2.0 Mega Data Center Fabric Implementation 87
Fabric and Node are Core Structures Leveraging Software Orchestration 89
Multi-Threading, Dynamic Systems 91
Oracle Multi-Threading Mega Data Center 92
Orchestration Tools Manage A Cluster As A Single Deployment 93
Microservice Monitoring with Google Kubernetes 94
Docker Container 94
Cluster Functions and Pod Benefits 95
Mesosphere DC/OS an Open-Source Project Built on Apache Mesos 96
Mesosphere Enterprise DC/OS Orchestration Software 96
Mesosphere DC/OS Production Containers Uses 97
Mesosphere DC/OS Orchestration Software 97
Mesosphere DC/OS Extending Capabilities Within Container Orchestration 98
Mesosphere DC/OS Certification Compliance 98
Mesosphere Market Leadership Position 99
Mesosphere DC/OS Runs Data Services on One Single Platform 99
Cloud Computing Not Enough: Entire Warehouse Building As A Single Mega Data Center System 100
Red Hat Ansible 101
Red Hat Ansible Architecture, Agents, And Security 102
Red Hat Ansible Advanced Features 102
Red Hat / Ansible 103
Red Hat Ansible Tower 3 Job Run Metrics 104
Cisco Integrated Infrastructure Management 105
Cisco UCS Helps Manage Administrative Costs And Reign In Complexity 106
Mesosphere DC/OS: Mesos Features 108
Heart of DC/OS: Apache Mesos 109
DC/OS Implements Containers 110
WinterGreen Research, 111
WinterGreen Research Methodology 112

List of Figures

Mega Data Center Depends on Orchestration Software to Control Operations, Implement Microservices
Figure 1. Slow Growth Mode of Companies with Enterprise Data Centers 3
Figure 2. Mega Data Center Fabric Implementation 4
Figure 3. Business Innovation and Technology 14
Figure 4. Docker Orchestration Software Creates Containers 15
Figure 5. Docker Compose 16
Figure 6. Mesosphere Marathon 16
Figure 7. Google Kubernetes 16
Figure 8. Orchestration Software Supports Container Automation 17
Figure 9. Orchestration Software Decreases Data Center Cost Structure 18
Figure 10. Files Bundled into a Container 19
Figure 11. Microservices: Suite Of Independently Deployable Service Modules with a Unique Process And Well-Defined, Lightweight Communication Portal: Mechanism To Serve A Business Goal 20
Figure 12. Microservices Distinct Features: Taxi Hailing Example 21
Figure 13. Microservices Market Segments 22
Figure 14. Microservices Modules 23
Figure 15. Hypervisor Virtualization Operating System Interface 24
Figure 16. Hypervisor Virtualization Operating System Interface 25
Figure 17. Virtual Machines Less Efficient Than Containers 26
Figure 18. Difference Between Virtual Machines and Containers 26
Figure 19. Bare Metal Management Replaced by Container Controllers 27
Figure 20. Containers vs. VMs 28
Figure 21. Industrial Robots Eliminate Manual Labor 29
Figure 22. Industry Uses Robots To Replace Manual Labor 29
Figure 23. Data Centers Need The Precision and Automation Similar to that Provided by Multi-Step Sequential Task Industrial Robots 30
Figure 24. Mega Data Center Orchestration Software 31
Figure 25. Single-Fabric Data Center Network Architecture 32
Figure 26. Bare Metal Presents a Lot of Extra Parameters and Metrics, Significantly More than With Containers 34
Figure 27. Nested Compute Units 35
Figure 28. Kubernetes Orchestration Software Groups Containers That Make Up An Application Into Logical Units 36
Figure 29. Kubernetes Orchestration Software Functions 36
Figure 30. Container Features as it integrates with the Service Fabric Runtime 37
Figure 31. Microsoft Setting Up A Secure Service Fabric Cluster in Azure using the Azure Portal. 40
Figure 32. Microsoft Data Center, Dublin, 550,000 Sf 43
Figure 33. Container Area In The Microsoft Data Center In Chicago 44
Figure 34. Microsoft Cloud Network Features 45
Figure 35. Like Physical Containers on a Ship, Software Containers Bring Many Servers Densely Packed 46
Figure 36. Advantages of Using Containers 47
Figure 37. Software Orchestration Container Challenges 48
Figure 38. Manual Process 51
Figure 39. Containers Need Orchestration Software 51
Figure 40. Virtual Machine Data Center Management Tasks: 52
Figure 41. FaceBook Open Compute Project 53
Figure 42. Facebook Data Center Modernization Functions 54
Figure 43. Facebook Altoona Iowa Cloud 2.0 Mega Data Center 54
Figure 44. Manual Process for Application Integration Deployment 55
Figure 45. Feature Sets For Orchestration Tools 58
Figure 46. Issues for Orchestration Software 59
Figure 47. AWS Cloud Container Adoption Criteria 60
Figure 48. AWS Cloud Container 61
Figure 49. AWS Cloud Adoption Framework 62
Figure 50. AWS Market Leader In Cloud Computing 65
Figure 51. Description of the Orchestration Software 67
Figure 52. Advantages of Using the Container Builder Cloud Architecture as a Service: 69
Figure 53. Google Andromeda Cloud High-Level Architecture 70
Figure 54. Google Andromeda Software Defined Networking (SDN)-Based Substrate Functions 71
Figure 55. Google Andromeda Performance Factors Of The Underlying Network 72
Figure 56. Google Compute Engine Load Balanced Requests Architecture 73
Figure 57. Google Compute Engine Load Balancing 74
Figure 58. Google Cloud Platform TCP Andromeda Throughput Advantages 76
Figure 59. IoT: Open Source IoT High Level Platform, OpenStack and Kubernetes 78
Figure 60. Facebook DuPont Fabros Technology Ashburn, VA Data Center 81
Figure 61. Cloud 2.0 Mega Data Centers Support 1.5 Billion Facebook Users Worldwide. 82
Figure 62. AWS Market Leader In Cloud Computing 83
Figure 63. Data Centers Need The Precision and Automation Provided by Multi-Step Sequential Task Industrial Robots 85
Figure 64. Mega Data Center Orchestration Software Functions 86
Figure 65. Multiple Pathways Open To Processing Nodes In The Cloud 2.0 Mega Data Center Functions 90
Figure 66. Dynamic Load Balancing 91
Figure 67. Mesosphere Customer References 96
Figure 68. Mesosphere DC/OS Certification Compliance 98
Figure 69. Cloud Is Not Enough 100
Figure 70. Red hat Ansible Playbook Language Advanced Features 103
Figure 71. Red Hat Ansible Tower 3 Job Run Metrics 104 Cisco UCS Helps Manage Administrative Costs And Reign In Complexity 106
Figure 72. Cisco UCS Helps Manage Administrative Costs And Reign In Complexity 106
Figure 73. Cisco UCS Director Delivers Comprehensive Infrastructure Management and Orchestration 107
Figure 74. Mesosphere DC/OS: Mesos Features: 108
Figure 75. Native Mesos Containerizer Functions 110

  • Global Colocation Market Research Report 2024
    Published: 08-Jan-2024        Price: US 2900 Onwards        Pages: 106
    A Colocation center (or colo) offers dedicated facility, where a business can physically house their servers and other equipment. Businesses can rent The space, as opposed to locating their servers or other computing hardware in their offices, for greater network reliability and uptime. The global Colocation market was valued at US$ 51230 million in 2023 and is anticipated to reach US$ 110720 million by 2030, witnessing a CAGR of 11.5% during The forecast period 2024-2030. So......
  • Global Data Centre Market Research Report 2024
    Published: 08-Jan-2024        Price: US 2900 Onwards        Pages: 104
    A data center is a facility used to house computer systems and associated components, such as telecommunications and storage systems. It generally includes redundant or backup power supplies, redundant data communications connections, environmental controls (e.g., air conditioning, fire suppression) and various security devices. Large data centers are industrial scale operations using as much electricity as a small town. The global Data Centre market was valued at US$ 137500 million......
  • Global Data Virtualization Market Growth (Status and Outlook) 2024-2030
    Published: 04-Jan-2024        Price: US 3660 Onwards        Pages: 137
    According to our LPI (LP Information) latest study, the global Data Virtualization market size was valued at US$ 2604.1 million in 2023. With growing demand in downstream market, the Data Virtualization is forecast to a readjusted size of US$ 9922.7 million by 2030 with a CAGR of 21.1% during review period. The research report highlights the growth potential of the global Data Virtualization market. Data Virtualization are expected to show stable growth in the future market. However, pro......
  • Global Colocation Market Growth (Status and Outlook) 2024-2030
    Published: 04-Jan-2024        Price: US 3660 Onwards        Pages: 161
    According to our LPI (LP Information) latest study, the global Colocation market size was valued at US$ 50120 million in 2023. With growing demand in downstream market, the Colocation is forecast to a readjusted size of US$ 110300 million by 2030 with a CAGR of 11.9% during review period. The research report highlights the growth potential of the global Colocation market. Colocation are expected to show stable growth in the future market. However, product differentiation, reducing costs,......
  • Global Data Center IT Asset Disposition Market Growth (Status and Outlook) 2024-2030
    Published: 04-Jan-2024        Price: US 3660 Onwards        Pages: 110
    According to our LPI (LP Information) latest study, the global Data Center IT Asset Disposition market size was valued at US$ 10660 million in 2023. With growing demand in downstream market, the Data Center IT Asset Disposition is forecast to a readjusted size of US$ 17070 million by 2030 with a CAGR of 7.0% during review period. The research report highlights the growth potential of the global Data Center IT Asset Disposition market. Data Center IT Asset Disposition are expected to show......
  • Global Data Monetization Market Growth (Status and Outlook) 2024-2030
    Published: 04-Jan-2024        Price: US 3660 Onwards        Pages: 158
    According to our LPI (LP Information) latest study, the global Data Monetization market size was valued at US$ 1620.1 million in 2023. With growing demand in downstream market, the Data Monetization is forecast to a readjusted size of US$ 27280 million by 2030 with a CAGR of 49.7% during review period. The research report highlights the growth potential of the global Data Monetization market. Data Monetization are expected to show stable growth in the future market. However, product diff......
  • Global Modular Data Centers Market Growth (Status and Outlook) 2024-2030
    Published: 04-Jan-2024        Price: US 3660 Onwards        Pages: 125
    According to our LPI (LP Information) latest study, the global Modular Data Centers market size was valued at US$ million in 2023. With growing demand in downstream market, the Modular Data Centers is forecast to a readjusted size of US$ million by 2030 with a CAGR of % during review period. The research report highlights the growth potential of the global Modular Data Centers market. Modular Data Centers are expected to show stable growth in the future market. However, product differenti......
  • Global Micro-Mobile Data Center Market Growth (Status and Outlook) 2024-2030
    Published: 04-Jan-2024        Price: US 3660 Onwards        Pages: 129
    According to our LPI (LP Information) latest study, the global Micro-Mobile Data Center market size was valued at US$ 6922.1 million in 2023. With growing demand in downstream market, the Micro-Mobile Data Center is forecast to a readjusted size of US$ 38990 million by 2030 with a CAGR of 28.0% during review period. The research report highlights the growth potential of the global Micro-Mobile Data Center market. Micro-Mobile Data Center are expected to show stable growth in the future m......
  • Global Big Data Technology and Services Industry Research Report, In-depth Analysis of Current Status and Outlook of Key Countries 2023-2028
    Published: 04-Jan-2024        Price: US 3380 Onwards        Pages: 109
    Market Overview of Global Big Data Technology and Services market: According to our latest research, the global Big Data Technology and Services market looks promising in the next 5 years. As of 2022, the global Big Data Technology and Services market was estimated at USD XX million, and it's anticipated to reach USD XX million in 2028, with a CAGR of XX% during the forecast years. Big Data is a term used for a large collection of complex data that is difficult to process usi......
  • SERVICES
    Value for Money
    We believe in "optimum utilization of available budget and resources". While servicing our clients' (your) market research requirements, we keep the same approach in focus to help you get the best value for your $$s.
    Ever Growing Inventory
    Ranging from the smallest feasible / required data (datasheets, data facts, SWOT analysis, company profiles, etc) to full research reports that help you make decisions, our inventory is updated almost on a daily basis with the latest industry reports from domain experts that track more than 5000 niche sectors.
    One Stop Solution
    Need a custom research report on medical devices market? Require all available business intelligence on 3D printing industry? Exploring F&B sector of a particular country/region? RnRMarketResearch.com is your one-stop-solution to all market intelligence needs. We not only offer custom research and consulting services, we also "bundle" reports to meet your needs and help you fetch the data analysis you require for your business.
    Dedicated Client Engagement
    Not limited to only "finding" relevant reports for you, our client engagement team dedicates its efforts to understand your "business need" and accordingly maps available research data to help you move forward. Call "your" client engagement executive any time of your day and get your questions answered in order to make the correct business decision.
    Saving Time and Efforts
    Simply share your research requirement details with us and let us do all the hard work to find required intelligence for you. When you add up our "one stop solution" and "dedicated client engagement" services mentioned above, you obviously know the time and effort saving you do by working with us.
    Payment Flexibility
    Working with Fortune 500 organizations, we understand the importance of being flexible for payments. Share your payment terms with us and we will surely match up to them to ensure you get access to required business intelligence data without having to wait for the payment to be done.
    Post-Purchase Research Support
    Have questions after reading a report / datasheet bought through us? Not sure about the methodology used for data available in the research? Talk to us / Share your questions with us and if required, we will connect you with the analyst(s)/author(s) of the report(s) and ensure you get satisfactory answers for the same. Need more data / analysis / report(s) on the topic of your research/project? The RnRMarketResearch.com team is here for you 24X7 to support you with your post-purchase requirements. Subscription Offers & Packages (Get in touch with us for more details - [email protected] / +1 888 391 5441 )
    • Ad Hoc
    • Pay - as - you - go / Bucket Subscriptions
    • Fixed Cost for #of reports
    • Customize / Personalize as per your needs