Introducing the Windows Azure Pack connector: two clouds, one portal
Introducing the Windows Azure Pack connector: two clouds, one portal
Microsoft IT has created a more seamless way to manage hybrid clouds. Now available for download, the Windows Azure Pack connector lets administrators and tenants use one portal to manage infrastructure as a service (IaaS) virtual machines in both private clouds and public Azure subscriptions.
We’ve heard from our partners—they’ve bought into the vision of the hybrid cloud. Until now, hybrid cloud administration has always required multiple portals. Microsoft IT set out to create a more seamless connection between private and public cloud environments and their management systems. We are pleased to provide the result as an open-source solution for everyone.
Partners provided great feedback on the idea, and told us that they need a more streamlined way to manage IaaS VMs on both private and public clouds. At the same time, they wanted strong oversight and governance. We also heard that partners are sensitive to their tenants. Simply put, “Don’t make my customers leave my portal!”
Microsoft IT responded to a cloud management challenge that was first recognized internally, realized that the solution resonated with a broader partner base, and then quickly created and released an open-source solution that scaled and engaged the larger community.
You’ve probably used a Windows Azure Pack portal to manage IaaS VMs on your private clouds. With the Windows Azure Pack connector, the portal has now been extended to create and manage public Azure subscriptions and VMs. The connector, which supports the Azure Resource Manager fabric, provides a “single pane of glass” administration experience. You create and manage both private cloud VMs and public Azure subscriptions and VMs.
With the Windows Azure Pack connector, administrators and tenants onboard and customize Azure subscriptions with an end-to-end installer and an automated installation test suite. Customizations for administrators include a variety of operating system images and different VM sizes. Tenants use the same portal to provision and configure Azure VMs.
Microsoft IT originally created the connector for internal use. We are pleased to extend the open-source connector to our partner network.
© 2016 Microsoft Corporation. All rights reserved. Microsoft and Windows are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners. This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.
Fire App Fights Wildfires with Data
Fire App Fights Wildfires with Data
Every second counts when combating a wildfire. Time lost can result in devastating loss of life or property. The University of the Aegean in Greece developed the VENUS-C Fire app—featuring Bing Maps, Microsoft Silverlight, and Windows Azure—to calculate and visualize the risk of wildfire ignition and to simulate fire propagation in the Greek island of Lesvos during its dry season. The university team generates a visualization of environmental factors each morning for the island’s fire management team, who then use the app to determine optimal resource allocation across the island for the day.
The fire app
The Fire app wildfire management software was designed by the Geography of Natural Disasters Laboratory at the University of the Aegean in Greece in 2011. Earlier, Microsoft Research partnered with the lab during the development phase, providing IT expertise, high-performance computing resources, and cloud computing infrastructure.
The app was built with functionality from multiple resources, giving it both technological depth and a visual interface that is accessible to non-technical users. "[The Fire app] nicely integrates Bing Maps, Microsoft Silverlight, and Windows Azure in a single system that allows users to be able to see the big picture of an emerging fire or the potential of an emerging fire," observes Dennis Gannon, director of Cloud Research Strategy for Microsoft Research Connections.
All of the Fire app data is stored in the cloud via Windows Azure. "You need a large cloud infrastructure such as Windows Azure to be able to bring these sources together," Gannon explains." The use of massive data analytics and machine learning is now the new frontier in many areas of science."
"With the cloud computing infrastructure, we were able to do business as we couldn’t do in the past," states Dr. Kostas Kalabokidis, associate professor, University of the Aegean. "[Windows Azure] is essential for us, because the cloud provides us with the necessary processing power and storage that is required. That means the real end users for the fire department do not need to have any huge processing power or storage capabilities locally."
Tracking risk factors daily
There are two distinct sets of users accessing Fire app daily during the dry season: the lab team, which loads new information into the tool in the morning; and the emergency responders, including the fire service, fire departments, and civil protection agencies that address wildfires on the island of Lesvos. They use the tool to view the data in a refined, graphical view. The process starts with the forecast.
"Every morning, our systems ask the Windows Azure cloud to provide approximately 20 virtual machines in order to process the available weather data," explains Dr. Christos Vasilakos, research associate, University of the Aegean. "It then stores the fire-risk outputs that the user needs to see and make the proper call. From the fire-risk menu, the end user can see for the next 120 hours or five days what will be the fire ignition risk for our study area." Additional information, including an animation of the weather for the next 120 hours, also can be accessed through the same menu.
The information is updated in the morning. The Fire Brigade of Greece uses the fire-risk data and fire simulations, together with weather forecast information, to inform the day's resource allocations. Based on the Fire app projections, personnel and fire trucks may be deployed throughout the island to areas that appear to be at particular risk that day.
The simulator also provides crucial information during fires. The firefighters who aren't dispatched to the fire use the Fire app at the station to create a wildfire simulation for the blaze. The team begins with the ignition point and pulls in other critical data to determine the fire’s potential path.
Fighting fire in the cloud
Cloud computing and storage are not merely integral to the Fire app; they are enabling significant advances throughout the research world.
"The data tsunami is changing everything in science. Every discipline is now confronted with it—a vast exploration of data that comes from instruments, from online sources, from the web, from social media," observes Gannon. "Analyzing this data can’t be done on a PC." Cloud computing, and the processing power that accompanies it, has made it possible for researchers to reduce processing job times from months to just hours.
Kypriotellis believes it has made a difference on the island. While wildfires do still break out, statistical evidence shows the department has been better prepared to respond to and control fires, preventing potential loss of life and property. He is hopeful that, one day, other firefighters will be able to add the tool to their arsenal as well.
FaST-LMM and Windows Azure Accelerate Genetics Research
FaST-LMM and Windows Azure Accelerate Genetics Research
Today, researchers can collect, store, and analyze tremendous volumes of data; however, technological and storage limitations can severely impede the speed at which they can analyze these data. A new algorithm that was developed by Microsoft Research, called FaST-LMM (Factored Spectrally Transformed Linear Mixed Models), runs on Windows Azure in the cloud and expedites analysis time—reducing processing periods from years to just days or hours. An early application of FaST-LMM and Windows Azure helps researchers analyze data for the genetic causes of common diseases.
Searching for DNA Clues to Disease
The Wellcome Trust in Cambridge, England, is researching the genetic causes of seven diseases—including hypertension, rheumatoid arthritis, and diabetes. The project involves searching for combinations of genomic information to gain insight into an individual’s likelihood to develop one of these diseases. With a database containing genetic information from 2,000 people and a shared set of approximately 13,000 controls for each of the seven diseases, they needed both massive storage and powerful computation capacity.
They are storing their vast database of genetic information in the Windows Azure cloud, instead of traditional hardware storage, which represents a profound shift in how big data are stored. ”We are taking on the challenge of taking what would be traditional high-performance computing, one of the hardest workloads to move to the cloud, and moving to the cloud,” observes Jeff Baxter, development lead in the Windows HPC team at Microsoft. “There’s a variety of both technical and business challenges, which makes it exciting and interesting.”
Exploring the Power of the Cloud
Resource management is one of the primary issues associated with big data: not only determining how many resources are required for the project, but also identifying the right type of resources—within the available budget. For example, running a large project on fewer machines might save on hardware costs but result in substantial project delays. Researchers must find a balance that will keep their project on track while working with available resources.
The FaST-LMM algorithm can analyze enormous datasets in less time than existing alternatives. Microsoft Research also has the infrastructure that is required to perform the computations, explains David Heckerman, distinguished scientist at Microsoft Research. With more CPUs dedicated to a job, computations that would ordinarily take years to finish can be completed in just hours.
For the Wellcome Trust project, the team’s available resources included a combination of Windows HPC Server, Windows Azure, and the FaST-LMM algorithm. The team knew that they had a powerful set of technologies. The question was, could it achieve the results required in the desired timeline?
“For this project, we would need to do about 125 compute years of work. We wanted to get that work done in about three days,” explains Baxter. By running FaST-LMM on Windows Azure, the team had access to tens of thousands of computer cores and an improved algorithm that was able to expedite the work. “You’re still doing hundreds of compute years of work,” he explains, “but with these resources, we can actually do hundreds of compute years in a couple of days.”
While the results were impressive, there was something that had an even bigger impact. “The most impressive thing was how quickly we could take this project from inception to actually completing it and generating new science,” Baxter notes. “This is stuff that, without both the improvements in the algorithms that the Microsoft Research guys had come up with and the ability for us to provide the tens to hundreds of thousands of cores, would have been infeasible.”
The Future for Big Data Research
The Wellcome Trust project is just the beginning of what could be a major shift in how research databases are stored and analyzed. “With this new, huge amount of data that’s coming online, we’re now able to find connections between our DNA and who we are that we could never find before,” Heckerman says. The ability to analyze that data more quickly, and with greater depth, could help scientists make faster breakthroughs in genetic research—and breakthroughs in critical genetic research. The FaST-LMM algorithm running on Windows Azure is helping to accelerate just such breakthroughs.