This article explains how to execute SQL statements on a dataframe in Azure Databricks notebook.
As a Data Engineer, SQL will always be my first love. So naturally when I learnt that within Databricks, I can create and run SQL statements on a dataframe without needing a SQL environment, I jumped on the opportunity. In this article, I will explain how to run SQL commands on a dataframe.
I will start by importing Koalas — a pandas API on Apache Spark.
