spark sql 報錯ClassLoaderResolver for class "" gave error on creation : {1}

在spark sql支援hive時，我們在idea 上編譯不會出錯，在生成jar包時缺報錯

]  Error creating validator of type org.datanucleus.properties.CorePropertyValidator

ClassLoaderResolver for class "" gave error on creation : {1}

。。。。。

最主要的提示在這裡

org.datanucleus.exceptions.NucleusUserException: Persistence process has been specified to use a ClassLoaderResolver of name "datanucleus" yet this has not been found by the DataNucleus plugin mechanism. Please check your CLASSPATH and plugin specification.

已經說明是plugin.xml檔案的問題，在檢視datanucleus.的三個jar包時發現确實有三個plugin.xml,把他們打開時确發現他們三個的内容完全不一樣，很明顯，在生成jar 包時這三個檔案必不可少，而檢視spark sql項目生成的jar包時隻有這三個plugin.xml的一部分，意思就是說：三個plugin.xml并沒合并！datanucleus并不能正常使用！

解決：必須将這三個plugin,xml手動合并，可以在這裡下載下傳https://download.csdn.net/download/qq_38426934/10616844

在生成jar包時需要用打包插件聲明datanucleus這個類

<build>

        <plugins>

            <plugin>

        <groupId>org.apache.maven.plugins</groupId>

        <artifactId>maven-shade-plugin</artifactId>

        <executions>

            <execution>

                <phase>package</phase>

                <goals>

                    <goal>shade</goal>

                </goals>

                <configuration>

                    <transformers>

                        <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">

                            <manifestEntries>

                                <Main-Class>com.test.RDDToDataset</Main-Class>

                                <Premain-Class>org.datanucleus.enhancer.DataNucleusClassFileTransformer</Premain-Class>

                                <Bundle-SymbolicName>org.datanucleus;singleton:=true</Bundle-SymbolicName>

                            </manifestEntries>

                        </transformer>

                        <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">

                            <resource>reference.conf</resource>

                        </transformer>

                        <transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>

                    </transformers>

                    <filters>

                        <filter>

                            <artifact>*:*</artifact>

                            <excludes>

                                <exclude>META-INF/*.SF</exclude>

                                <exclude>META-INF/*.DSA</exclude>

                                <exclude>META-INF/*.RSA</exclude>

                            </excludes>

                        </filter>

                    </filters>

                </configuration>

            </execution>

        </executions>

    </plugin>

    </plugins>

    </build>

如果用sbt道理一樣可以直接修改 MANIFEST.MF為

Manifest-Version: 1.0

Bundle-SymbolicName: org.datanucleus;singleton:=true

Premain-Class: org.datanucleus.enhancer.DataNucleusClassFileTransforme

 r

Built-By: Administrator

Build-Jdk: 1.8.0_101

Main-Class: com.test.RDDToDataset

可以看出關鍵就是

Bundle-SymbolicName: org.datanucleus;singleton:=true

Premain-Class: org.datanucleus.enhancer.DataNucleusClassFileTransforme

 r對datanucleus的聲明