jar package conflict building design book

. background

In the actual development process, using Maven to manage JAR has brought a lot of convenience to our development, and we don't need to download the JAR packages one by one, we only need to configure a POM configuration file, and write the corresponding coordinates and warehouse address. But the jar conflict is no problem and has not been solved, the conflicting jar package maven will not be checked out for us or will be downloaded according to our configuration, and the error will not be reported until compiled, and the error message is very obscure, you need to check it for Baidu for a while to locate the problem.

At this time, we urgently need someone who can tell us in advance that there are traitors in my project, and we need to eliminate them in time, otherwise it will affect the morale of the military.

2. Functional design

1. Maven plug-in, pass parameter instructions to the plug-in

2. Conflict warning of all JAR packages in the project

3. JAR warning can not coexist in the project

4. Custom rule warning

3. Functional technical solutions

3.1 Detailed technical scheme

1. Maven plug-in development, inherit AbstractMojo, the entry method execute, and the command parameters can be passed in by annotation, for example: @Parameter(property = "groupId"), Bian Liang can get the corresponding parameter value of groupId.

2. Get all jar packages and classes in the project: Get the MavenProject through Maven, and after getting the project, you can get all the jars and all their dependency trees, and put all the jar packages into a big map.

3. JAR package conflict algorithm:

A. Conflicting versions of the same JAR package: Configure custom rules into the property file to define which versions of a JAR are incompatible. Rule example: xxxgroupId:xxxArtifactId>5.1.8, then the program thinks that when the jar package of this coordinate is larger than what I want to send (5.1.8), it is a mole and needs to be removed.

B. Conflict between the versions of the same jar package: The number of classes in the two jar packages is different, or the number of classes is the same, but the results are different after MD5, or the size of the same class in the two jar packages is different. The specific code omits 10,000 lines here.

C. Conflicts between different jars and packages: Configure a certain version of a JAR and a certain version of another JAR package to see each other unpleasantly through custom rules configured into the attribute file. Rule example: if Wang Baoqiang groupId: Wang Baoqiang ArtifactId >=2.2.2 then broker groupId: broker ArtifactId>3.3.3, this rule means that Wang Baoqiang jar package version 222 and broker jar package version 333 cannot coexist, otherwise a green hat will be generated, and a green hat alarm will be given.

The following are known to generate green hat alarms:

log4j-over-slf4j and slf4j-log4j12 cannot coexist.

jcl-over-slf4j and slf4j-jcl cannot coexist.

jcl-over-slf4j and commons-logging cannot coexist.

Source: https://www.slf4j.org/codes.html#version_mismatch.

Part of the core code is displayed

private void execute() throws MojoExecutionException {
  this.getLog().info("FindConflicts is working...");
  // preparing before collecting classes & artifacts
  LogConflictsCollector logDepCollector = new LogConflictsCollector();
  VersionConflictCollector versionConflictCollector = new VersionConflictCollector();
  if (versionCheckConfig != null) {
   try {
    versionConflictCollector.init(versionCheckConfig);
   } catch (FileNotFoundException e) {
    this.getLog().info("versionCheckConfig:" + versionCheckConfig + " doesn't exist.");
   } catch (Exception e) {
   }
  }
  Set<String> groupIdsToCheck = null;
  if (groupId != null) {
   String[] a = groupId.split(",");
   if (a.length > 0) {
    groupIdsToCheck = new HashSet<String>();
    for (int i = 0; i < a.length; i++) {
     groupIdsToCheck.add(a[i].trim());
    }
   }
  }
  Set<String> artifactIdsToCheck = null;
  if (artifactId != null) {
   String[] a = artifactId.split(",");
   if (a.length > 0) {
    artifactIdsToCheck = new HashSet<String>();
    for (int i = 0; i < a.length; i++) {
     artifactIdsToCheck.add(a[i].trim());
    }
   }
  }
  int totalJarNum = 0;
  int totalClassNum = 0;
  // key:the id of an artifact, value:the classNum
  Map<String, Integer> totalClassNumMap = new HashMap<String, Integer>();
  // data is used to store the the information of class, key: the className , value is the class information of its.
  Map<String, List<ClzWrapper>> data = new HashMap<String, List<ClzWrapper>>();
  // get the final artifacts
  Set<Artifact> artifacts = this.getProject().getArtifacts();
  for (Iterator<Artifact> iterator = artifacts.iterator(); iterator.hasNext();) {
   Artifact artifact = (Artifact) iterator.next();
   if (!artifact.isOptional()) {
    if ("jar".equals(artifact.getType())) {
     if (groupIdsToCheck != null && !groupIdsToCheck.contains(artifact.getGroupId())) {
      continue;
     }
     if (artifactIdsToCheck != null && !artifactIdsToCheck.contains(artifact.getArtifactId())) {
      continue;
     }
     totalJarNum++;
     ArtifactWrapper artifactWrapper = new ArtifactWrapper();
     artifactWrapper.artifact = artifact;
     artifactWrapper.originFrom = this.getOriginFrom(artifact);
     logDepCollector.collect(artifactWrapper);
     versionConflictCollector.collect(artifactWrapper);
     JarFile jf;
     try {
      jf = new JarFile(artifact.getFile());
      Enumeration<JarEntry> jfs = jf.entries();
      while (jfs.hasMoreElements()) {
       JarEntry jfn = jfs.nextElement();
       String fileName = jfn.getName();
       if (fileName.endsWith(".class")) {
        // ignore inner class 忽略内部类
        if (fileName.indexOf("#34;) == -1) {
         ClzWrapper clzWrapper = new ClzWrapper();
         clzWrapper.className = fileName;
         clzWrapper.artifactWrapper = artifactWrapper;
         clzWrapper.size = jfn.getSize();
         if (data.get(fileName) == null) {
          List<ClzWrapper> clzInfos = new ArrayList<ClzWrapper>();
          clzInfos.add(clzWrapper);
          data.put(fileName, clzInfos);
         } else {
          data.get(fileName).add(clzWrapper);
         }
         logDepCollector.collect(clzWrapper);
         String id = Util.getId(artifact);
         if (totalClassNumMap.get(id) == null) {
          totalClassNumMap.put(id, 1);
         } else {
          totalClassNumMap.put(id, totalClassNumMap.get(id) + 1);
         }
         totalClassNum++;
        }
       }
      }
     } catch (IOException e) {
      e.printStackTrace();
     }
    }
   }
  }
  // iterator each conflicts  迭代器每次冲突
  Set<String> totalConflictJarNum = new HashSet<String>();
  int totalConflictClassNum = 0;
  Set<String> set = data.keySet();
  List<String> list = new ArrayList<String>(set);
  Collections.sort(list, new ClassConflictsComparator());
  Iterator<String> iter = list.iterator();
  List<JarConflictGroup> jarConflictGroups = new ArrayList<JarConflictGroup>();
  Set<String> jarConflictGroupKeys = new HashSet<String>();
  // key:jarConflictsGroupKey, value:conflitsClassNum
  Map<String, Integer> jarConglictGroupConflitsClassNumMap = new HashMap<String, Integer>();
  List<ClassConflict> classConflicts = new ArrayList<ClassConflict>();
  int classConflictNum = 1;
  while (iter.hasNext()) {
   String className = (String) iter.next();
   List<ClzWrapper> clzInfos = data.get(className);
   if (clzInfos.size() == 1) {
    // no conflicts
    continue;
   }
   long clzSize = clzInfos.get(0).size;
   boolean isConflicts = false;
   // only conflicts if the size of class is not equal,
   for (Iterator<ClzWrapper> iterator = clzInfos.iterator(); iterator.hasNext();) {
    ClzWrapper clzInfo = (ClzWrapper) iterator.next();
    if (clzInfo.size != clzSize) {
     isConflicts = true;
     break;
    }
   }
   if (isConflicts) {
    JarConflictGroup jarConflictGroup = new JarConflictGroup();
    for (Iterator<ClzWrapper> iterator = clzInfos.iterator(); iterator.hasNext();) {
     ClzWrapper clzInfo = (ClzWrapper) iterator.next();
     // jar conflicts
     jarConflictGroup.add(clzInfo.artifactWrapper);
     totalConflictJarNum.add(Util.getId(clzInfo.artifactWrapper.artifact));
     // class conflicts
     ClassConflict classConflict = new ClassConflict();
     classConflict.setClassName(clzInfo.className);
     classConflict.setGroupId(clzInfo.artifactWrapper.artifact.getGroupId());
     classConflict.setArtifactId(clzInfo.artifactWrapper.artifact.getArtifactId());
     classConflict.setVersion(clzInfo.artifactWrapper.artifact.getVersion());
     classConflict.setOriginFrom(Util.formatOriginFrom(clzInfo.artifactWrapper.originFrom));
     classConflict.setNumber(classConflictNum);
     classConflicts.add(classConflict);
     totalConflictClassNum++;
    }
    classConflictNum++;
    String jarConflictsGroupKey = jarConflictGroup.getGroupKey();
    if (jarConglictGroupConflitsClassNumMap.get(jarConflictsGroupKey) == null) {
     jarConglictGroupConflitsClassNumMap.put(jarConflictsGroupKey, clzInfos.size());
    } else {
     jarConglictGroupConflitsClassNumMap.put(jarConflictsGroupKey, clzInfos.size() + jarConglictGroupConflitsClassNumMap.get(jarConflictsGroupKey));
    }
    if (!jarConflictGroupKeys.contains(jarConflictsGroupKey)) {
     jarConflictGroupKeys.add(jarConflictsGroupKey);
     jarConflictGroups.add(jarConflictGroup);
    }
   }
  }
  // jarConflicts
  for (Iterator<JarConflictGroup> iterator = jarConflictGroups.iterator(); iterator.hasNext();) {
   JarConflictGroup jarConflictGroup = (JarConflictGroup) iterator.next();
   jarConflictGroup.setConflitsClassNum(jarConglictGroupConflitsClassNumMap.get(jarConflictGroup.getGroupKey()));
   int groupTotalClass = 0;
   List<ArtifactWrapper> artifactWrappers = jarConflictGroup.getArtifactWrappers();
   if (artifactWrappers != null && artifactWrappers.size() > 0) {
    for (Iterator<ArtifactWrapper> iterator_1 = artifactWrappers.iterator(); iterator_1.hasNext();) {
     ArtifactWrapper artifactWrapper = (ArtifactWrapper) iterator_1.next();
     Artifact artifact = artifactWrapper.artifact;
     groupTotalClass += totalClassNumMap.get(Util.getId(artifact));
    }
    jarConflictGroup.setTotalClassNum(groupTotalClass);
   }
  }
  if (jarConflictGroups.size() > 0) {
   Collections.sort(jarConflictGroups, new JarConflictGroupComparator());
   int number = 1;
   List<JarConflict> jarConflicts = new ArrayList<JarConflict>();
   for (Iterator<JarConflictGroup> iterator = jarConflictGroups.iterator(); iterator.hasNext();) {
    JarConflictGroup jarConflictGroup = (JarConflictGroup) iterator.next();
    jarConflictGroup.setNumber(number++);
    jarConflicts.addAll(jarConflictGroup.getJarConflicts());
   }
   this.getLog().warn("*********************************************Jar Conflicts****************************************************");
   this.getLog().warn((new TableGenerator()).generateTable(jarConflicts));
   this.getLog().info("Jar Conflicts Total: jar conflicts ratio:" + totalConflictJarNum.size() + "/" + totalJarNum + "=" + NumberFormat.getPercentInstance().format((float) totalConflictJarNum.size() / totalJarNum));
   this.getLog().info("Jar Conflicts Solution Hint: choose one artifact of the conflicts, and exclude other artifacts at pom.xml  according to originFrom.");
  } else {
   this.getLog().info("No jar conflicts found!");
  }
  if (showClassConflicts) {
   if (classConflicts.size() > 0) {
    this.getLog().warn("*********************************************Class Conflicts****************************************************");
    this.getLog().warn((new TableGenerator()).generateTable(classConflicts));
    this.getLog().info("Class Conflicts Total: class conflicts ratio:" + totalConflictClassNum + "/" + totalClassNum + "=" + NumberFormat.getPercentInstance().format((float) totalConflictClassNum / totalClassNum));
    this.getLog().info("Class Conflicts Solution Hint: choose one artifact of the conflicts, and exclude other artifacts at pom.xml  according to originFrom.");
   } else {
    this.getLog().info("No class conflicts found!");
   }
  }
  List<LogConflict> logConflicts = logDepCollector.getLogConflicts();
  if (logConflicts != null && logConflicts.size() > 0) {
   this.getLog().warn("*********************************************Log Conflicts****************************************************");
   this.getLog().warn((new TableGenerator()).generateTable(logConflicts));
   this.getLog().info("Log Conflicts Solution Hint: choose one artifact of the conflicts, and exclude other artifacts at pom.xml  according to originFrom.");
   this.getLog().info("As for the conflicts of SLF4J, you can refer to this offical article:https://www.slf4j.org/codes.html#version_mismatch");
  } else {
   this.getLog().info("No log conflicts found!");
  }

  List<VersionConflict> versionConflicts = versionConflictCollector.getVersionConflict();
  if (versionConflicts != null && versionConflicts.size() > 0) {
   this.getLog().warn("*********************************************Version Conflicts****************************************************");
   this.getLog().warn((new TableGenerator()).generateTable(versionConflicts));
   this.getLog().info("Version Conflicts Solution Hint: update the version of the artifact according to requiredVersion");
  } else {
   this.getLog().info("No version conflicts found!");
  }
  this.getLog().info("FindConflicts finished!");

 }