DeltaLake on EMR

  1. Installation of delta core jar: Like installation of other jars, delta-core_<version>.jar needs to be placed inside the jars folder under Spark home path in master node. For EMR cluster, it’s /usr/lib/spark/jars. This process also can be installed by adding the required code in a script file and executed as an EMR step.
  2. Required Configurations: The configurations can be set either in spark configurations file in master node (or) added as additional configuration in zeppelin interpreter (in case zeppelin notebook is used as development environment). Below are the two mandatory configurations required to access delta lake functionalities.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store