Microsoft announces ‘extensive commitment’ to Apache Spark

Microsoft is investing in Apache Spark to help power its data and analytic services like Azure, Cortana Intelligence Suite, Power BI, and Microsoft R Server.

Here are the ways Apache Spark will help improve Microsoft’s wide array of services:

• Spark for Azure HDInsight General Availability, previously announced as public preview, Spark for Azure HDInsight generally available today, and introducing a fully managed Spark service from Hortonworks that has been hardened for the enterprise and made simpler for you to use. You can also rely on the industry’s highest availability service level agreement for Spark at 99.9%. You can get value out of Spark immediately with out-of-the-box integration with Jupyter, the most popular open source notebook for data scientists.
• R Server for HDInsight in the cloud powered by Spark, previously announced as public preview, R Server for HDInsight will be generally available in the summer making the Spark integration available both on-premises and in the cloud. This makes it easy to move code and projects to the cloud with a few clicks and within a few minutes without buying hardware or hiring specialized operations teams typically associated with big data infrastructure.
• R Server for Hadoop on-premises now powered by Spark, as the leading solution in the world to run R at scale, R Server for Hadoop will support both Microsoft R and native Spark execution frameworks available in June. Combining R Server with Spark gives users the ability to run R functions over thousands of Spark nodes letting you train your models on data 1000x larger and 100x faster than was possible with open source R and nearly 2x faster than Spark’s own MLLib.
• Power BI support for Spark Streaming, previously announced with Power BI General Availability, Spark support in Power BI is now expanded with new support for Spark Streaming scenarios. This allows you to publish real-time events from Spark Streaming directly into one of the fastest growing visualization tools in the market today.

Joseph Sirosh, corporate vice president at Microsoft, will be making his keynote at this year’s Spark Summit in San Francisco on Wednesday, June 8, 2016. Sirosh will go into more details in his Spark Summit keynote about how Apache Spark will be integrated into Microsoft software and services.