Pyodbc install error in Azure Databricks: File not found error

Chung, Joshua 151

Running the code below in Azure Databricks

%sh
curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list > /etc/apt/sources.list.d/mssql-release.list 
apt-get update
ACCEPT_EULA=Y apt-get install msodbcsql17
apt-get -y install unixodbc-devhttps://packages.microsoft.com
sudo apt-get install python3-pip -y
pip3 install --upgrade pyodbc

At apt-get install msodbcsql17 step, I get the

Err:1 https://packages.microsoft.com/ubuntu/16.04/prod xenial/main amd64 msodbcsql17 amd64 17.7.2.1-1
404 Not Found [IP: 13.81.215.193 443]

Since I have not changed the code, I assume the package was somehow removed from the repository, but I'm not sure where else to go find the package.

Jaoua, Emna 16 Reputation points

2021-06-17T06:50:39.36+00:00

Hello,

the same happened for me as well. I tried with almost all Ubuntu versions 16.04, 18.04, 20.04 and 20.10. They are all not found.
D. Stefanoski 26 Reputation points

2021-06-17T07:28:34.19+00:00

same here on Ubuntu 20.04 trying to install msodbcsql17 on fresh installation, issue was first noticed on our side started around 2:15am UTC

curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add
curl https://packages.microsoft.com/config/ubuntu/20.04/prod.list > /etc/apt/sources.list.d/mssql-release.list
apt-get install -y msodbcsql17

E: Failed to fetch https://packages.microsoft.com/ubuntu/20.04/prod/pool/main/u/unixodbc/libodbc1_2.3.7_amd64.deb 404 Not Found [IP: 13.80.99.124 443]
Dagoberto Romer González 26 Reputation points

2021-06-17T07:34:37.893+00:00

Same here. Using Ubuntu 18.04 the driver according to apt-get is in https://packages.microsoft.com/ubuntu/18.04/prod/pool/main/m/msodbcsql17/msodbcsql17_17.7.2.1-1_amd64.deb

but looking at the repository it seems the directory pool under prod does not exist. Instead only dist exists. This is the same on the other versions 16/20

I guess Microsoft changed the structure of the repository or moved the drivers somewhere else and they haven't updated everything yet?
Tomáš 1 Reputation point

2021-06-17T08:29:34.64+00:00

The same here. First job - build CI/CD fails on this.
MaaNo 1 Reputation point

2021-06-17T08:31:44.327+00:00

Same error here since this morning
Francesco Caspani 1 Reputation point

2021-06-17T08:43:06.71+00:00

The same here. Job - build CI/CD fails on this.
Francesco Caspani 1 Reputation point

2021-06-17T08:45:11.217+00:00

Same errors
Loub Václav 1 Reputation point

2021-06-17T09:13:21.77+00:00

We are having same issue in all our CIs
at 'apt-get -y install mssql-tools'

we get:
After this operation, 1726 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian buster/main amd64 multiarch-support amd64 2.28-10 [215 kB]
Get:2 http://deb.debian.org/debian buster/main amd64 libltdl7 amd64 2.4.6-9 [390 kB]
Err:3 https://packages.microsoft.com/ubuntu/16.04/prod xenial/main amd64 libodbc1 amd64 2.3.7
404 Not Found [IP: 13.81.215.193 443}
Jussi 1 Reputation point

2021-06-17T09:21:04.113+00:00

Same here. Installation fails.
Jaoua, Emna 16 Reputation points

2021-06-17T09:27:50.37+00:00

Did anyone find another location for the msodbcsql17 package in Ubuntu 16.04 or 18.04 ?

Accepted answer

Chung, Joshua 151 Reputation points

2021-06-17T15:58:19.073+00:00

It's working for Europe now.
Please sign in to rate this answer.

2 people found this answer helpful.

0 comments No comments
Sign in to comment

4 additional answers

D. Stefanoski 26 Reputation points

2021-06-17T09:51:57.947+00:00

Look here, I just tested the broken url from various locations trough the world, it looks like Europe is being hit and updates are not propagated yet, le't give it some time until they sync properly! God I can't believe I am defending Microsoft :)

106601-screenshot-2021-06-17-at-114934.png
Please sign in to rate this answer.

2 people found this answer helpful.
Dagoberto Romer González 26 Reputation points

2021-06-17T09:58:47.423+00:00

Thanks for the clarification. This might explain why some conversations with support in India they've been able to see the files without a problem and then we (in Europe) fail to see the same. It's been a frustrating morning. It's probably a matter of time for the CDNs to catch up.

D. Stefanoski 26 Reputation points

2021-06-17T10:09:40.133+00:00

confirmed, colleague of mine was able to successfully build our docker from India. Let's hope it get's synced soon.

Sreeramulu Bonthala 11 Reputation points

2021-06-17T10:17:45.133+00:00

My Databricks job has been setup in Europe region and it has been failing since today morning. Thanks DagobertoRomerGonzlez and DStefanoski for sharing the information! This helped me getting clarity about the issue!

Benjamin david 1 Reputation point

2021-06-17T11:07:17.87+00:00

This sounds sensible to me now.

Sreeramulu Bonthala 11 Reputation points

2021-06-17T14:18:32.567+00:00

MSSQL driver is getting downloaded and working fine now!
Sign in to comment
PRADEEPCHEEKATLA-MSFT 78,576 Reputation points Microsoft Employee

2021-06-17T09:36:38.903+00:00
Hello All,

(UPDATE2): Looks like there is an outage in specific regions with the http://packages.microsoft.com and it's going to be back online asap.

In my region, I'm able to reach the packages repository.

---------------------------------------------

(UPDATE1): Even I tried reinstalling on a fresh cluster and able to successfully install msodbcsql17 on the cluster.

Proof on completion:

-----------------------------

@Chung, Joshua , @Jaoua, Emna , @D. Stefanoski , @Tomáš , @Dagoberto Romer González , @MaaNo , @Francesco Caspani ,

Welcome the Microsoft Q&A platform.

This could be an intermittent issue. I would suggest you re-try again and check if that works.

Now, I'm able to install the Pyodbc in Azure Databricks without any issue.

Installed on Databricks Runtime version : 8.2 (includes Apache Spark 3.1.1, Scala 2.12)

Hope this helps. Do let us know if you any further queries.

------------

Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.

Want a reminder to come back and check responses? Here is how to subscribe to a notification.
Please sign in to rate this answer.

1 person found this answer helpful.
Francesco Caspani 1 Reputation point

2021-06-17T09:40:00.46+00:00

IFor debian 9 not resolved yet

Tomas Romeiro 1 Reputation point

2021-06-17T09:43:56.84+00:00

Hi Prandeep,

The problem lies with fetching the msodbcsql17 driver from the repository and not with the PYODBC installation.

I can install PYODBC fine, but still no luck with the downloading the driver.

D. Stefanoski 26 Reputation points

2021-06-17T09:44:20.737+00:00

thanks for the update @PRADEEPCHEEKATLA-MSFT

we still receive the issue, most of our servers are in Europe datacenter so if they just started syncing the updates it might take a while for the changes to propagate trough the world

I will reply on resolved status as soon as we receive the updates

Cheers!

D

Dagoberto Romer González 26 Reputation points

2021-06-17T09:45:52.227+00:00

This is not a solution to this problem. The Microsoft repository still doesn't have the pool directory which is where the msodbcsql17 driver is pulled from. We can install Pyodbc no problem, you might have that driver installed on your cluster alreadry. Try it from a fresh cluster.

Editing my comment as it seems that in some parts of the world it works fine and others don't work.

Dagoberto Romer González 26 Reputation points

2021-06-17T09:47:24.993+00:00

Also I tried reinstalling on a fresh cluster and it still won't find the driver

Dagoberto Romer González 26 Reputation points

2021-06-17T10:13:15.627+00:00

Hey @PRADEEPCHEEKATLA-MSFT

It seems if you're not in Europe you have access to the files and will be able to install the library on a new cluster or on any linux terminal.
The problem seems to be the CDN where in Europe we still have a 404 on the http://packages.microsoft.com/ubuntu/18.04/prod/pool link where that driver will be fetched from. Seems like we have to wait for the CNDs in Europe to catch up.

Chakravarthi Lolla 26 Reputation points

2021-06-17T11:44:40.767+00:00

Really frustrating to see such high priority issue pending for these many hours. Our entire team is blocked because of this issue.
Sign in to comment
Francesco Caspani 1 Reputation point

2021-06-17T14:20:05.443+00:00

Now it works for debian 9.
Please sign in to rate this answer.

0 comments No comments
Sign in to comment
Benjamin david 1 Reputation point

2021-06-17T15:03:05.88+00:00

Now it works for ubuntu 16.04.. but it is quite slow while downloading

]1
Please sign in to rate this answer.

0 comments No comments
Sign in to comment