Spark on Hadoop

Monitoring you Spark jobs

Spark jobs submitted to Yarn can be monitored through the web interface at http://hsrv02.pic.es:8088

You need to have a VPN and Kerberos authentication configured in your browser if you want to access it.

How to configure Kerberos authentication with Firefox

Tested in a Ubuntu 22 server with Firefox installed with apt. Configuring Kerberos authentication with other browser & OS combinations may be differ or not be supported. In particular, in Ubuntu 22, Firefox installed with snap won't work.

You need to have Kerberos client installed in your PC.

Update the "/etc/krb5.conf" file to contain PIC realm. Do not remove additional content in the "krb5.conf" file:

   [realms]
   	PIC.ES = {
   		kdc = ipa01.pic.es:88
   		master_kdc = ipa01.pic.es:88
   		admin_server = ipa01.pic.es:749
   		kpasswd_server = ipa01.pic.es:464
   		default_domain = pic.es
   	}
    
   [domain_realm]
     .pic.es = PIC.ES
     pic.es = PIC.ES
     hsrv03.pic.es = PIC.ES

Update the Firefox configuration

• Open Firefox
• Access the configuration in "about:config" in the navigation bar
• look for the options with the "negotiate" text and set these values:
  • network.negotiate-auth.allow-non-fqdn: true
  • network.negotiate-auth.delegation-uris: pic.es
  • network.negotiate-auth.trusted-uris: pic.es