date: 2018-04-26

title: "janusgraph源碼分析1-下載下傳編譯啟動"

author: "鄧子明"

tags:

- 源碼
- janusgraph           

categories:

- 源碼分析
           

janusgraph源碼分析1-下載下傳編譯啟動

研究了好久的 neo4j源碼，現在公司要換 janusgraph，隻要半途而廢開始研究 janusgraph 了

https://github.com/JanusGraph/janusgraph

和

http://janusgraph.org/

一、下載下傳編譯

我直接使用github desktop打開了 janusgraph 的源碼，使用IDEA打開，然後編譯：

# 編譯完整的
mvn -settings ~/opt/soft/apache-maven-3.5.0/conf/settings.xml -Dlicense.skip=true -DskipTests clean install
# 隻編譯core部分
mvn -pl janusgraph-core -am clean install -Dlicense.skip=true -DskipTests -P prod

-rf :janusgraph-test
mvn -pl janusgraph-test -am clean install -Dlicense.skip=true -DskipTests -P prod           

更多閱讀：

JanusGraph 技術交流圈 neo4j技術交流圈 JanusGraph中文翻譯文檔

我們在

janusgraph-test

下面編寫一個例子

FirstTest

：

public class FirstTest {

    public static void main(String[] args) {

        /*
         * The example below will open a JanusGraph graph instance and load The Graph of the Gods dataset diagrammed above.
         * JanusGraphFactory provides a set of static open methods,
         * each of which takes a configuration as its argument and returns a graph instance.
         * This tutorial calls one of these open methods on a configuration
         * that uses the BerkeleyDB storage backend and the Elasticsearch index backend,
         * then loads The Graph of the Gods using the helper class GraphOfTheGodsFactory.
         * This section skips over the configuration details, but additional information about storage backends,
         * index backends, and their configuration are available in
         * Part III, “Storage Backends”, Part IV, “Index Backends”, and Chapter 13, Configuration Reference.
         */

        // Loading the Graph of the Gods Into JanusGraph
        JanusGraph graph = JanusGraphFactory
                .open("janusgraph-dist/src/assembly/cfilter/conf/janusgraph-berkeleyje-es.properties");

        GraphOfTheGodsFactory.load(graph);
        GraphTraversalSource g = graph.traversal();

        /*
         * The typical pattern for accessing data in a graph database is to first locate the entry point into the graph
         * using a graph index. That entry point is an element (or set of elements) 
         * — i.e. a vertex or edge. From the entry elements,
         * a Gremlin path description describes how to traverse to other elements in the graph via the explicit graph structure.
         * Given that there is a unique index on name property, the Saturn vertex can be retrieved.
         * The property map (i.e. the key/value pairs of Saturn) can then be examined.
         * As demonstrated, the Saturn vertex has a name of "saturn, " an age of 10000, and a type of "titan."
         * The grandchild of Saturn can be retrieved with a traversal that expresses:
         * "Who is Saturn’s grandchild?" (the inverse of "father" is "child"). The result is Hercules.
         */
        // Global Graph Indices
        Vertex saturn = g.V().has("name", "saturn").next();
        GraphTraversal<Vertex, Map<String, Object>> vertexMapGraphTraversal = g.V(saturn).valueMap();

        GraphTraversal<Vertex, Object> values = g.V(saturn).in("father").in("father").values("name");

        /*
         * The property place is also in a graph index. The property place is an edge property.
         * Therefore, JanusGraph can index edges in a graph index.
         * It is possible to query The Graph of the Gods for all events that have happened within 50 kilometers of Athens
          * (latitude:37.97 and long:23.72).
          * Then, given that information, which vertices were involved in those events.
         */
        System.out.println(g.E().has("place", geoWithin(Geoshape.circle(37.97, 23.72, 50))));
        System.out.println(g.E().has("place", geoWithin(Geoshape.circle(37.97, 23.72, 50)))
                .as("source").inV()
                .as("god2")
                .select("source").outV()
                .as("god1").select("god1", "god2")
                .by("name"));
    }

}           

然後在"janusgraph-dist/src/assembly/cfilter/conf/janusgraph-berkeleyje-es.properties" 檔案中，将注釋掉的内容取消注釋。

運作發現依賴挺麻煩。

首先運作報錯了：

Exception in thread "main" java.lang.IllegalArgumentException: Could not find implementation class: org.janusgraph.diskstorage.berkeleyje.BerkeleyJEStoreManager           

找到報錯處的代碼，我們發現

janusgraph-core

中通過反射建立一個類，但是這個類在

janusgraph-berkeleyje

中，而前者不依賴後者，是以找不到這個類，我們可以将後者加到前者的依賴，

但是我們發現後者依賴前者，如果加了依賴兩個就互相依賴了，這是 Janus 官方設計的問題。我們隻好在 FirstTest 所在的module中把兩個依賴都加進來試試。

（注意，如果我們将所有的都打進一個包，這個問題就不存在了，但是在本地運作是不一樣的，各自子產品的編譯輸出檔案在不同的地方。）在

janusgraph-test

中添加：

<dependency>
            <groupId>org.janusgraph</groupId>
            <artifactId>janusgraph-berkeleyje</artifactId>
            <version>0.3.0-SNAPSHOT</version>
        </dependency>           

發現

janusgraph-berkeleyje

也依賴了

janusgraph-test

,又互相依賴了，好麻煩。我們寫寫代碼一定要注意這個問題。這裡我的解決方法是直接把 代碼放到

janusgraph-berkeleyje

中運作。

Exception in thread "main" java.lang.IllegalArgumentException: Could not find implementation class: org.janusgraph.diskstorage.es.ElasticSearchIndex           

和上面一樣，還依賴了

janusgraph-es

,我隻好吧代碼複制到

janusgraph-es

的test代碼塊中運作（注意一點是test代碼中），順便在

janusgraph-es

中 添加上

janusgraph-berkeleyje

的依賴。

運作成功了，但是報了連接配接失敗，是因為我本地沒有啟動es，我啟動一下es：

elasticsearch

然後在運作：

Exception in thread "main" org.janusgraph.core.SchemaViolationException: Adding this property for key [~T$SchemaName] and value [rtname] violates a uniqueness constraint [SystemIndex#~T$SchemaName]           

經過google查到原因：

https://groups.google.com/forum/#!topic/aureliusgraphs/vZ_nTXlXj4k

This exception is thrown only when you already have added property key to index. So "name" is already added and next time when you run your program somewhere it is again adding "name" property key. So check if that particular code is running twice           

然後我們可以在我們傳入的配置檔案找到：storage.directory=../db/berkeley ，直接删除這個目錄，再重新運作，就成功了：

11:20:17,051  INFO GraphDatabaseConfiguration:1285 - Set default timestamp provider MICRO
11:20:17,296  INFO GraphDatabaseConfiguration:1492 - Generated unique-instance-id=c0a815a789637-dengzimings-MacBook-Pro-local1
11:20:17,547  INFO Backend:462 - Configuring index [search]
11:20:19,279  INFO Backend:177 - Initiated backend operations thread pool of size 8
11:20:19,461  INFO KCVSLog:753 - Loaded unidentified ReadMarker start time 2018-04-26T03:20:19.408Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@73cd37c0
[GraphStep(edge,[]), HasStep([place.geoWithin(BUFFER (POINT (23.72 37.97), 0.44966))])]
[GraphStep(edge,[]), HasStep([place.geoWithin(BUFFER (POINT (23.72 37.97), 0.44966))])@[source], EdgeVertexStep(IN)@[god2], SelectOneStep(last,source), EdgeVertexStep(OUT)@[god1], SelectStep(last,[god1, god2],[value(name)])]
11:20:29,578  INFO ManagementLogger:192 - Received all acknowledgements for eviction [1]           

然後我們可以去 ../db/berkeley 目錄檢視，多了一些檔案，這些檔案的作用我們後續再分析。

然後我們取es檢視：

curl -XGET 'localhost:9200/_cat/indices?v&pretty'

，發現多了兩個index:

yellow open   janusgraph_edges    QT-E7AV6SMWr8Cu_ywKsXg   5   1          6            0     13.7kb         13.7kb
yellow open   janusgraph_vertices gE4TSXFATnSZUWYdAf46Xg   5   1          6            0     10.9kb         10.9kb           

還可以具體檢視内容。例如名字是titan的内容：

curl -XGET 'localhost:9200/janusgraph_vertices/_search?q=name:titan&pretty'

到現在我們第一個案例就結束了。

g.E().has("place", geoWithin(Geoshape.circle(37.97, 23.72, 50)))
                .as("source").inV()
                .as("god2")
                .select("source").outV()
                .as("god1").select("god1", "god2")
                .by("name")           

這種風格的代碼實際上是groovy語言的代碼，大家可以研究一下groovy語言。

注意事項：

上述第一次運作問題的原因是

janusgraph-core

需要用到

janusgraph-berkeleyje

的類，

但是

janusgraph-berkeleyje

是依賴

janusgraph-core

的，是以兩個互相依賴了。

janus的做法是在core中使用反射，是以編譯通過了，打包到了一起就沒問題了。但是本地運作沒法成功。