question

AzeemK-2483 avatar image
0 Votes"
AzeemK-2483 asked MartinJaffer-MSFT edited

error while creating table in Databricks

I am creating a simple table in Azure Databricks , I am getting this error below

Error in SQL statement: AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.UnsupportedOperationException: Parquet does not support date. See HIVE-6384;





DROP TABLE IF EXISTS operations_staging.PendedClaims;
CREATE EXTERNAL TABLE IF NOT EXISTS operations_staging.PendedClaims STORED AS
PARQUET LOCATION '/dbfs/Operations_staging/PendedClaims' AS


select
group_num as GroupNumber,
group_name as GroupName,
member_id as MemberID,
claim_num as ClaimNumber,
claimant_last_name as MemberLastName,
claimant_first_name as MemberFirstName,
cast(incurred as date) as IncurredDate,
pending_amount as PendedAmount,
pend_code as PendedCode,
pend_code_description as PendedCodeDescription,
cast(received as date) as ReceivedDate,
state as State,
cast(report_date as date) as ReportDate,
age as ClaimAge,
member_enrolled as MemberEnrolled,
provider as ProviderName,
tax_id as TaxID,
claimtype as ClaimType,
hold_reason_code as HoldReasonCode,
hold_code_description as HoldReasonDescription,
hold_review_code as HoldDate,
hold_userid as HoldUserID,
subscriber_state as SubscriberState,
shell as OrgResponsibility,
provider_id as NPI,
bill_type as HoldReviewCode,
dx1 as dx1,
dx2 as dx2,
dx3 as dx3,
dx4 as dx4,
dx5 as dx5,
dx6 as dx6,
dx7 as dx7,
dx8 as dx8,
dx9 as dx9,
dx10 as dx10,
dx11 as dx11,
dx12 as dx12,
dx13 as dx13,
dx14 as dx14,
dx15 as dx15,
dx16 as dx16,
dx17 as dx17,
dx18 as dx18,
dx19 as dx19,
dx20 as dx20,
dx21 as dx21,
dx22 as dx22,
dx23 as dx23,
dx24 as dx24,
dx25 as dx25
from
loomis_2021.pu

azure-databricks
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Is your select query running fine?

select * from loomis_2021.pu

0 Votes 0 ·

Yeah Select query runs fine

0 Votes 0 ·

1 Answer

MartinJaffer-MSFT avatar image
0 Votes"
MartinJaffer-MSFT answered MartinJaffer-MSFT edited

Hello @AzeemK-2483 and welcome to Microsoft Q&A.

The error says parquet does not support the data type date. How about trying datetime or string?

like changing

 cast(received as date) as ReceivedDate,

to

 cast(received as datetime) as ReceivedDate,

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Databircks doesn't support datetime!!

0 Votes 0 ·

Oh, you are right. My mistake @AzeemK-2483 .

Looking into that error code HIVE-6384, it seems that Date should be supported. Since this message is popping up, I suspect the version of one of the components is old.
What runtimes are you using? Can you use a more recent one?


0 Votes 0 ·