Authentification in cURL from Windows to Hadoop HTTPFS secured with Kerberos

Question

I want to load data from my local Windows machine to HDFS using HTTPFS with curl. The Hadoop cluster is secured with Kerberos.

How do I manage to get the authentication done?

When trying the following statement...

curl -k --negotiate -u : -i -X PUT "HTTP..."

it fails with

HTTP/1.1 401 Unauthorized
Server: Apache-Coyote/1.1
WWW-Authenticate: Negotiate
Set-Cookie: hadoop.auth=; Path=/; Secure; HttpOnly
Content-Type: text/html;charset=utf-8
Content-Length: 997
Date: Tue, 30 Jan 2018 12:15:30 GMT

HTTP/1.1 403 Forbidden
Server: Apache-Coyote/1.1
Set-Cookie: hadoop.auth=; Path=/; Secure; HttpOnly
Content-Type: text/html;charset=utf-8
Content-Length: 1149
Date: Tue, 30 Jan 2018 12:15:30 GMT

The curl version is:

curl 7.46.0 (x86_64-pc-win32) libcurl/7.46.0 OpenSSL/1.0.2e zlib/1.2.8 WinIDN libssh2/1.6.0
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL libz

I installed MIT Kerberos for Windows and it shows me a valid ticket. Unfortunately I don't know much about Kerberos which is why I am kind of stuck.

Any help or starting points for research are very much appreciated.

Answer 1

You don't need MIT Kerberos on Windows. Your curl has been compiled with SSPI, MS's variant of GSS-API. Enable --verbose to see wether a ticket is sent.

I highly assume that you have a network misconfig. Fire up Wireshark, filter for 'kerberos' and look for TGS-REQ and TGS-REP containing your designed SPN (is it registered?).

Curl works flawlessly with SPNEGO since version 7.38.0 on both Windows and Unix-like.

More over, if I see 403, authentication has passed, your account is just not entitled to access the resource.