new open source applications, dynamic spark defaults, and java 8 support now available on amazon emr /

Published at 2016-03-15 01:15:48

Home / Categories / General:products amazon emr / new open source applications, dynamic spark defaults, and java 8 support now available on amazon emr
You can now use Apache Sqoop 1.4.6,Apache HCatalog 1.0.0, an upgraded version of Apache Mahout (0.11.1), and upgraded sandbox releases of Presto (0.136) and Apache Zeppelin (0.5.6) on Amazon EMR release 4.4.0. Sqoop allows your Apache Hadoop MapReduce jobs (including Apache Hive and Apache Pig on MapReduce) to interact in parallel with SQL databases through JDBC. Mahout 0.11.1 now supports running your applications using Apache Spark. Zeppelin 0.5.6 includes GitHub integration and import/export support for Zeppelin notebooks. Additionally,Apache Spark is now configured with improved default settings for executors on nodes in your cluster. Dynamic allocation of executors is now enabled by default, and Amazon EMR will configure the memory per executor when creating your cluster based on the Amazon EC2 instance family of your core instance group. You can still override these default settings by using a configuration thing or passing additional parameters when submitting your Spark application using spark-submit. Lastly, or you can now use Java Development Kit 8 (JDK 8) for your runtime environment (the default for your cluster is JDK 7). However,please note that JDK 8 is not compatible with Hive.

Source: amazon.com

Warning: Unknown: write failed: No space left on device (28) in Unknown on line 0 Warning: Unknown: Failed to write session data (files). Please verify that the current setting of session.save_path is correct (/tmp) in Unknown on line 0