Troubleshooting the slow startup of spark local programs

Test today spark When working locally , It is found that the program has to wait a long time every time it starts , Can start to enter the startup program normally , Seriously disturbed Debug The rhythm of . Then start ～～

add Yourkit Start parameter of , among tracing The parameter indicates that it starts when starting tracing Grade CPU profiling, Other kinds profiling The level can be adjusted as needed .

spark.driver.extraJavaOptions    -agentpath:/Applications/YourKit-Java-Profiler-2019.8.app/Contents/Resources/bin/mac/libyjpagent.dylib=exceptions=disable,delay=100,tracing

Look at the stack of program startup , The one who found the card is still java Of net Package tool class , By the way, I took a look at the source code , Then write a test class to test whether it is really stuck here .

Local test procedure ：

object IpTest extends Logging {
    
  def main(args: Array[String]): Unit = {
    
    logInfo("begin getCanonicalHostName")
    val address = InetAddress.getLocalHost
    println(address.getCanonicalHostName)
    logInfo("end getCanonicalHostName")
  }
}

20/03/04 00:21:53 INFO IpTest: begin getCanonicalHostName
192.168.0.194
20/03/04 00:22:23 INFO IpTest: end getCanonicalHostName

The test results were as expected .

Think about it. , I saw that the top layer of the stack turned out to be Inet6AddressImpl Implementation class of .

It must feel like DNS There is something wrong with the parsing service .
Enter the router , close IPv6 No effect .
There is no way , Force to specify in the router DNS Resolution server , Get it done ～～

20/03/04 00:51:59 INFO IpTest: begin getCanonicalHostName
192.168.0.194
20/03/04 00:51:59 INFO IpTest: end getCanonicalHostName