Introduction to cloud computing and Microsoft Azure

intro to cloud computing - managed solution

Introduction to cloud computing and Microsoft Azure

Cloud computing overview

Cloud computing provides a modern alternative to the traditional on-premises datacenter. Public cloud vendors provide and manage all computing infrastructure and the underlying management software. These vendors provide a wide variety of cloud services. A cloud service in this case might be a virtual machine, a web server, or cloud-hosted database engine. As a cloud provider customer, you lease these cloud services on an as-needed basis. In doing so, you convert the capital expense of hardware maintenance into an operational expense. A cloud service also provides these benefits:
  •   Rapid deployment of large compute environments
  •   Rapid deallocation of systems that are no longer required
  •   Easy deployment of traditionally complex systems like load balancers
  •   Ability to provide flexible compute capacity or scale when needed
  •   More cost-effective computing environments
  •   Access from anywhere with a web-based portal or programmatic automation
  •   Cloud-based services to meet most compute and application needs
    With on-premises infrastructure, you have complete control over the hardware and software that is deployed. Historically, this has led to hardware procurement decisions that focus on scaling up. An example is purchasing a server with more cores to satisfy peak performance needs. Unfortunately, this infrastructure might be underutilized outside a demand window. With Azure, you can deploy only the infrastructure that you need, and adjust this up or down at any time. This leads to a focus on scaling out through the deployment of additional compute nodes to satisfy a performance need. Although this has consequences for the design of an appropriate software architecture, there is now ample proof that scaling out the commodity of cloud services is more cost-effective than scaling up through expensive hardware.
    Microsoft has deployed many Azure datacenters around the globe, with more planned. Additionally, Microsoft is increasing sovereign clouds in regions like China and Germany. Only the largest global enterprises can deploy datacenters in this manner, so using Azure makes it easy for enterprises of any size to deploy their services close to their customers.
    For small businesses, Azure allows for a low-cost entry point, with the ability to scale rapidly as demand for compute increases. This prevents a large up-front capital investment in infrastructure, and it provides the flexibility to architect and re-architect systems as needed. The use of cloud computing fits well with the scale-fast and fail-fast model of startup growth.

Types of cloud computing

Cloud computing is usually classified into three categories: SaaS, PaaS, and IaaS.

SaaS: Software as a service

SaaS is software that is centrally hosted and managed. It’s usually based on a multitenant architecture— a single version of the application is used for all customers. It can be scaled out to multiple instances to ensure the best performance in all locations. SaaS software typically is licensed through a monthly or annual subscription.
Microsoft Office 365 is a prototypical model of a SaaS offering. Subscribers pay a monthly or annual subscription fee, and they get Microsoft Exchange as a service (online and/or desktop Microsoft Outlook), storage as a service (Microsoft OneDrive), and the rest of the Microsoft Office suite (online, the desktop version, or both). Subscribers always get the most recent version. So you can have an Exchange server without having to purchase a server and install and support Exchange—the Exchange server is managed for you. Compared to installing and upgrading Office every year, this is much less expensive and requires much less effort to keep updated.

PaaS: Platform as a service

With PaaS, you deploy your application into an application-hosting environment that the cloud service vendor provides. The developer provides the application, and the PaaS vendor provides the ability to deploy and run it. This frees developers from infrastructure management so they can focus on development.
Azure provides several PaaS compute offerings, including the Web Apps feature of Azure App Service and Azure Cloud Services (web and worker roles). In either case, developers have multiple ways to deploy their application without knowing anything about the nuts and bolts that support it. Developers don’t have to create virtual machines (VMs), use Remote Desktop Protocol (RDP) to sign in to each one, or install the application. They just hit a button (or close to it), and the tools provided by Microsoft provision the VMs and then deploy and install the application on them.

IaaS: Infrastructure as a service

An IaaS cloud vendor runs and manages all physical compute resources and the required software to enable computer virtualization. A customer of this service deploys virtual machines in these hosted datacenters. Although the virtual machines are located in an offsite datacenter, the IaaS consumer has control over the configuration and management of them.
Azure includes several IaaS solutions, including Azure Virtual Machines, virtual machine scale sets, and related networking infrastructure. Azure Virtual Machines is a popular choice for initially migrating services to Azure because it enables a “lift and shift” migration model. You can configure a VM like the infrastructure currently running your services in your datacenter, and then migrate your software to the new VM. You might need to make configuration updates, such as URLs to other services or storage, but you can migrate many applications in this way.
Virtual machine scale sets are built on top of Azure Virtual Machines and provide an easy way to deploy clusters of identical VMs. Virtual machine scale sets also support autoscaling so that new VMs can be deployed automatically when required. This makes virtual machine scale sets an ideal platform to host higher-level microservice compute clusters, such as Azure Service Fabric and Azure Container Service.

Azure services

Azure offers many services in its cloud computing platform. These services include the following.
Compute services
Services for hosting and running application workload:
  •   Azure Virtual Machines—both Linux and Windows
  •   App Services (Web Apps, Mobile Apps, Logic Apps, API Apps, and Function Apps)
  •   Azure Batch (for large-scale parallel and batch compute jobs)
  •   Azure RemoteApp
  •   Azure Service Fabric
  •   Azure Container Service
    Data services
    Services for storing and managing data:
  •   Azure Storage (comprises the Azure Blob, Queue, Table, and File services)
  •   Azure SQL Database
  •   Azure DocumentDB
  •   Microsoft Azure StorSimple
  •   Azure Redis Cache Application services
    Services for building and operating applications:
  •   Azure Active Directory (Azure AD)
  •   Azure Service Bus for connecting distributed systems
  •   Azure HDInsight for processing big data
  •   Azure Scheduler
  •   Azure Media Services
    Network services
    Services for networking both within Azure and between Azure and on-premises datacenters:
  •   Azure Virtual Network
  •   Azure ExpressRoute
  •   Azure-provided DNS
  •   Azure Traffic Manager
  •   Azure Content Delivery Network

New Azure services help more people realize the possibilities of big data


New Azure services help more people realize the possibilities of big data

By T. K. “Ranga” Rengarajan as written on
This week in San Jose thousands of people are at Strata + Hadoop World to explore the technology and business of big data. As part of our participation in the conference, we are pleased to announce new and enhanced Microsoft data services: a preview of Azure HDInsight running on Linux, the general availability of Storm on HDInsight, the general availability of Azure Machine Learning, and the availability of Informatica technology on Azure.
These new services are part of our continued investment in a broad portfolio of solutions to unlock insights from data. They can help businesses dramatically improve their performance, enable governments to better serve their citizenry, or accelerate new advancements in science. Our goal is to make big data technology simpler and more accessible to the greatest number of people possible: big data pros, data scientists and app developers, but also everyday businesspeople and IT managers. Azure is at the center of our strategy, offering customers scale, simplicity and great economics. And we’re embracing open technologies, so people can use the tools, languages and platforms of their choice to pull the maximum value from their data.
Simply put, we want to bring big data to the mainstream.
Azure HDInsight, our Apache Hadoop-based service in the cloud, is a prime example. It makes it easy for customers to crunch petabytes of all types of data with fast, cost-effective scale on demand, as well as programming extensions so developers can use their favorite languages. Customers like Virginia Tech, Chr. Hanson, Mediatonic and many others are using it to find important data insights. And, today, we are announcing that customers can run HDInsight on Ubuntu clusters (the leading scale-out Linux), in addition to Windows, with simple deployment, a managed service level agreement and full technical support. This is particularly compelling for people that already use Hadoop on Linux on-premises like on Hortonworks Data Platform, because they can use common Linux tools, documentation, and templates and extend their deployment to Azure with hybrid cloud connections.











Storm for Azure HDInsight, generally available today, is another example of making big data simpler and more accessible. Storm is an open source stream analytics platform that can process millions of data “events” in real time as they are generated by sensors and devices. Using Storm with HDInsight, customers can deploy and manage applications for real-time analytics and Internet-of-Things scenarios in a few minutes with just a few clicks. Linkury is using HDInsight with Storm for its online monetization services, for example. We are also making Storm available for both .NET and Java and the ability to develop, deploy, and debug real-time Storm applications directly in Visual Studio. That helps developers to be productive in the environments they know best.
You can read this blog to learn about these and other updates we’re making to HDInsight to make Hadoop simpler and easier to use on Azure.
Azure Machine Learning, also generally available today, further demonstrates our commitment to help more people and organizations use the cloud to unlock the possibilities of data. It is a first-of-its-kind, managed cloud service for advanced analytics that makes it dramatically simpler for businesses to predict future trends with data. In mere hours, developers and data scientists can build and deploy apps to improve customer experiences, predict and prevent system failures, enhance operational efficiencies, uncover new technical insights, or a universe of other benefits. Such advanced analytics normally take weeks or months and require extensive investment in people, hardware and software to manage big data. Also, now developers – even those without data science training – can use the Machine Learning Marketplace to find APIs and finished services, such as recommendations, anomaly detection and forecasting, in order to deploy solutions quickly. Already customers like Pier 1, Carnegie Mellon, eSmart Systems, Mendeley and ThyssenKrupp are finding value in their data with Azure Machine Learning.

Azure Machine Learning reflects our support for open source. The Python programming language is a first class citizen in Azure Machine Learning Studio, along with R, the popular language of statisticians. New breakthrough algorithms, such as “Learning with Counts,” now allow customers to learn from terabytes of data. A new community gallery allows data scientists to share experiments via Twitter and LinkedIn, too. You can read more about these innovations and how customers are using Azure Machine Learning in this blog post.
Another key part of our strategy is to offer customers a wide range of partner solutions that build on and extend the benefits of Azure data services. Today, data integration leader Informatica is joining the growing ecosystem of partners in the Azure Marketplace. The Informatica Cloud agent is now available in Linux and Windows virtual machines on Azure. That will enable enterprise customers to create data pipelines from both on-premises systems and the cloud to Azure data services such as Azure HDInsight, Azure Machine Learning, Azure Data Factory and others, for management and analysis.
The value provided by our data services multiplies when customers use them together. A case in point is Ziosk, maker of the world’s first ordering, entertainment and pay-at-the table tablet. They are using Azure HDInsight, Azure Machine Learning, our Power BI analytics service and other Microsoft technologies to help restaurant chains like Chili’s drive guest satisfaction, frequency and advocacy with data from tabletop devices in 1,400 locations.
This week the big data world is focused on Strata + Hadoop World, a great event for the industry and community. It’s exciting to consider the new ideas and innovations happening around the world every day with data. Here at Microsoft, we’re thrilled to be part of it and to fuel that innovation with data solutions that give customers simple but powerful capabilities, using their choice of tools and platforms in the cloud.


FaST-LMM and Windows Azure Accelerate Genetics Research

FaST-LMM and Windows Azure Accelerate Genetics Research


Today, researchers can collect, store, and analyze tremendous volumes of data; however, technological and storage limitations can severely impede the speed at which they can analyze these data. A new algorithm that was developed by Microsoft Research, called FaST-LMM (Factored Spectrally Transformed Linear Mixed Models), runs on Windows Azure in the cloud and expedites analysis time—reducing processing periods from years to just days or hours. An early application of FaST-LMM and Windows Azure helps researchers analyze data for the genetic causes of common diseases.
Searching for DNA Clues to Disease
The Wellcome Trust in Cambridge, England, is researching the genetic causes of seven diseases—including hypertension, rheumatoid arthritis, and diabetes. The project involves searching for combinations of genomic information to gain insight into an individual’s likelihood to develop one of these diseases. With a database containing genetic information from 2,000 people and a shared set of approximately 13,000 controls for each of the seven diseases, they needed both massive storage and powerful computation capacity.
They are storing their vast database of genetic information in the Windows Azure cloud, instead of traditional hardware storage, which represents a profound shift in how big data are stored. ”We are taking on the challenge of taking what would be traditional high-performance computing, one of the hardest workloads to move to the cloud, and moving to the cloud,” observes Jeff Baxter, development lead in the Windows HPC team at Microsoft. “There’s a variety of both technical and business challenges, which makes it exciting and interesting.”
Exploring the Power of the Cloud
Resource management is one of the primary issues associated with big data: not only determining how many resources are required for the project, but also identifying the right type of resources—within the available budget. For example, running a large project on fewer machines might save on hardware costs but result in substantial project delays. Researchers must find a balance that will keep their project on track while working with available resources.
The FaST-LMM algorithm can analyze enormous datasets in less time than existing alternatives. Microsoft Research also has the infrastructure that is required to perform the computations, explains David Heckerman, distinguished scientist at Microsoft Research. With more CPUs dedicated to a job, computations that would ordinarily take years to finish can be completed in just hours.
For the Wellcome Trust project, the team’s available resources included a combination of Windows HPC Server, Windows Azure, and the FaST-LMM algorithm. The team knew that they had a powerful set of technologies. The question was, could it achieve the results required in the desired timeline?
“For this project, we would need to do about 125 compute years of work. We wanted to get that work done in about three days,” explains Baxter. By running FaST-LMM on Windows Azure, the team had access to tens of thousands of computer cores and an improved algorithm that was able to expedite the work. “You’re still doing hundreds of compute years of work,” he explains, “but with these resources, we can actually do hundreds of compute years in a couple of days.”
While the results were impressive, there was something that had an even bigger impact. “The most impressive thing was how quickly we could take this project from inception to actually completing it and generating new science,” Baxter notes. “This is stuff that, without both the improvements in the algorithms that the Microsoft Research guys had come up with and the ability for us to provide the tens to hundreds of thousands of cores, would have been infeasible.”
The Future for Big Data Research
The Wellcome Trust project is just the beginning of what could be a major shift in how research databases are stored and analyzed. “With this new, huge amount of data that’s coming online, we’re now able to find connections between our DNA and who we are that we could never find before,” Heckerman says. The ability to analyze that data more quickly, and with greater depth, could help scientists make faster breakthroughs in genetic research—and breakthroughs in critical genetic research. The FaST-LMM algorithm running on Windows Azure is helping to accelerate just such breakthroughs.