天天看點

spark sqlContext異常

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/sql/types/DataTypeParser at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:763) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) at java.net.URLClassLoader.access$100(URLClassLoader.java:73) at java.net.URLClassLoader$1.run(URLClassLoader.java:368) at java.net.URLClassLoader$1.run(URLClassLoader.java:362) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:361) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.apache.spark.sql.SQLContext.<init>(SQLContext.scala:172) at com.aron.test.test_gbdt_spark$.main(test_gbdt_spark.scala:18) at com.aron.test.test_gbdt_spark.main(test_gbdt_spark.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) Caused by: java.lang.ClassNotFoundException: org.apache.spark.sql.types.DataTypeParser at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 20 more

異常狀态:java.lang.ClassNotFoundException: org.apache.spark.sql.types.DataTypeParser

說明:報錯的第18行代碼為 val sql = new SQLContext(sc)

解決方案:spark SQL的版本

還是谷歌出來的方法!!百度了一下午,坑爹啊!!!!

具體參考問題:http://stackoverflow.com/questions/37030013/noclassdeffounderror-org-apache-spark-sql-types-genericarraydata

繼續閱讀