I have two azure databricks tables,table one having 10 million rows and tabletwo having five thousand rows. there is common key - itemcode
The big table was created from parquet files and the other from Excel file using azure databricks GUI. Both table have columns as STRINGS.
when i run the query as -->
%sql
select * from tableone where itemcode = <some value> without quotes --- it returns the rows expected
but when i use a join as shown below it returns no rows, strange
select * from tableone join tabletwo on tableone.itemcode = tabletwo.itemcode
2) tried dataframe method same no results
sqlDF1 = spark.sql("SELECT * FROM tableone")
display(sqlDF1)
sqlDF1.printSchema()
sqlDF2 = spark.sql("SELECT * FROM tabletwo")
display(sqlDF2)
sqlDF2.printSchema()
df = sqlDF1.join(sqlDF2, on=['ItemBarcode'], how='inner')
display(df)






