Hi, I am using below code in python to read data from a SQL table and copy results in a dataframe then push the results into a json document and save it in Azure Data Lake Storage Gen2.
https://docs.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-overview
jdbcHostname = "hostname"
jdbcDatabase = "databasename"
jdbcPort = 1413
jdbcUsername = "username"
jdbcPassword = "password"
jdbcUrl = "jdbc:sqlserver://{0}:{1};database={2}".format(jdbcHostname, jdbcPort, jdbcDatabase)
connectionProperties = {
"user" : jdbcUsername,
"password" : jdbcPassword,
"driver" : "com.microsoft.sqlserver.jdbc.SQLServerDriver"
}
pushdown_query = "(select * from hr.employee) emp"
df = spark.read.jdbc(url=jdbcUrl, table=pushdown_query, properties=connectionProperties)
display(df)
df.write.mode("overwrite").json("wasbs://<file_system>@<storage-account-name>.blob.core.windows.net/hr/emp")
The above code displays data in dataframe but does not create the folder and the json document. Could you please where I am doing it wrong.
Thank you



