Storm 字元統計Demo

1、資料源讀取，字元發射spout類

2、第一次對字元串加工處下，切割處理bolt類

3、送出topology的main函數及字元統計處理類

4、處理結果

[quote]

Thread-22-count-executor[3 3]---word:c count:1

Thread-18-count-executor[2 2]---word:b count:1

Thread-32-count-executor[4 4]---word:a count:1

Thread-32-count-executor[4 4]---word:d count:1

Thread-18-count-executor[2 2]---word:b count:2

Thread-32-count-executor[4 4]---word:d count:2

Thread-32-count-executor[4 4]---word:a count:2

Thread-32-count-executor[4 4]---word:d count:3

[/quote]

5、相關總結

[quote]

1、每一個線程bolt擷取處理資料與上一個bolt或spout輸出的資料方式一緻。

declarer.declare(new Fields("firstSpout"));

2、每一個線程bolt在topology運作中，一直處理運作狀态。而聲明的全局變量是針對每個線程的全局變量，每一個線程輸出統計資料是目前線程的變量資料。

3、每個spout或bolt處理資料時，都可以設定對應的線程數。但spout讀取資料時，會重複讀取資料。

4、bolt與bolt資料傳遞，bolt資料輸出格式與下一個bolt資料接收格式扭轉，都是通過對應的”相同字元”扭轉。

[/quote]

6、相關pom.xml檔案

[quote]

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

<modelVersion>4.0.0</modelVersion>

<groupId>com.test</groupId>

<artifactId>StormMavenProject</artifactId>

<packaging>jar</packaging>

<version>0.0.1-SNAPSHOT</version>

<name>StormMavenProject</name>

<url>http://maven.apache.org</url>

<dependencies>

<dependency>

<groupId>org.ow2.asm</groupId>

<artifactId>asm</artifactId>

<version>5.0.3</version>

</dependency>

<dependency>

<groupId>org.clojure</groupId>

<artifactId>clojure</artifactId>

<version>1.7.0</version>

</dependency>

<dependency>

<groupId>com.lmax</groupId>

<artifactId>disruptor</artifactId>

<version>3.3.2</version>

</dependency>

<dependency>

<groupId>com.esotericsoftware</groupId>

<artifactId>kryo</artifactId>

<version>3.0.3</version>

</dependency>

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-api</artifactId>

<version>2.8</version>

</dependency>

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-core</artifactId>

<version>2.8</version>

</dependency>

<dependency>

<groupId>org.slf4j</groupId>

<artifactId>log4j-over-slf4j</artifactId>

<version>1.6.6</version>

</dependency>

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-slf4j-impl</artifactId>

<version>2.8</version>

</dependency>

<dependency>

<groupId>com.esotericsoftware</groupId>

<artifactId>minlog</artifactId>

<version>1.3.0</version>

</dependency>

<dependency>

<groupId>org.objenesis</groupId>

<artifactId>objenesis</artifactId>

<version>2.1</version>

</dependency>

<dependency>

<groupId>com.esotericsoftware</groupId>

<artifactId>reflectasm</artifactId>

<version>1.10.1</version>

</dependency>

<dependency>

<groupId>javax.servlet</groupId>

<artifactId>servlet-api</artifactId>

<version>2.5</version>

</dependency>

<dependency>

<groupId>org.slf4j</groupId>

<artifactId>slf4j-api</artifactId>

<version>1.7.21</version>

</dependency>

<dependency>

<groupId>org.apache.storm</groupId>

<artifactId>storm-core</artifactId>

<version>1.1.0</version>

</dependency>

<dependency>

<groupId>org.apache.storm</groupId>

<artifactId>storm-rename-hack</artifactId>

<version>1.1.0</version>

</dependency>

<dependency>

<groupId>junit</groupId>

<artifactId>junit</artifactId>

<version>3.8.1</version>

<scope>test</scope>

</dependency>

<dependency>

<groupId>ring-cors</groupId>

<artifactId>ring-cors</artifactId>

<version>0.1.5</version>

</dependency>

</dependencies>

<build>

<finalName>StormMavenProject</finalName>

</build>

</project>

[/quote]