Storm 字符统计Demo

1、数据源读取，字符发射spout类

2、第一次对字符串加工处下，切割处理bolt类

3、提交topology的main函数及字符统计处理类

4、处理结果

[quote]

Thread-22-count-executor[3 3]---word:c count:1

Thread-18-count-executor[2 2]---word:b count:1

Thread-32-count-executor[4 4]---word:a count:1

Thread-32-count-executor[4 4]---word:d count:1

Thread-18-count-executor[2 2]---word:b count:2

Thread-32-count-executor[4 4]---word:d count:2

Thread-32-count-executor[4 4]---word:a count:2

Thread-32-count-executor[4 4]---word:d count:3

[/quote]

5、相关总结

[quote]

1、每一个线程bolt获取处理数据与上一个bolt或spout输出的数据方式一致。

declarer.declare(new Fields("firstSpout"));

2、每一个线程bolt在topology运行中，一直处理运行状态。而声明的全局变量是针对每个线程的全局变量，每一个线程输出统计数据是当前线程的变量数据。

3、每个spout或bolt处理数据时，都可以设置对应的线程数。但spout读取数据时，会重复读取数据。

4、bolt与bolt数据传递，bolt数据输出格式与下一个bolt数据接收格式扭转，都是通过对应的”相同字符”扭转。

[/quote]

6、相关pom.xml文件

[quote]

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

<modelVersion>4.0.0</modelVersion>

<groupId>com.test</groupId>

<artifactId>StormMavenProject</artifactId>

<packaging>jar</packaging>

<version>0.0.1-SNAPSHOT</version>

<name>StormMavenProject</name>

<url>http://maven.apache.org</url>

<dependencies>

<dependency>

<groupId>org.ow2.asm</groupId>

<artifactId>asm</artifactId>

<version>5.0.3</version>

</dependency>

<dependency>

<groupId>org.clojure</groupId>

<artifactId>clojure</artifactId>

<version>1.7.0</version>

</dependency>

<dependency>

<groupId>com.lmax</groupId>

<artifactId>disruptor</artifactId>

<version>3.3.2</version>

</dependency>

<dependency>

<groupId>com.esotericsoftware</groupId>

<artifactId>kryo</artifactId>

<version>3.0.3</version>

</dependency>

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-api</artifactId>

<version>2.8</version>

</dependency>

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-core</artifactId>

<version>2.8</version>

</dependency>

<dependency>

<groupId>org.slf4j</groupId>

<artifactId>log4j-over-slf4j</artifactId>

<version>1.6.6</version>

</dependency>

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-slf4j-impl</artifactId>

<version>2.8</version>

</dependency>

<dependency>

<groupId>com.esotericsoftware</groupId>

<artifactId>minlog</artifactId>

<version>1.3.0</version>

</dependency>

<dependency>

<groupId>org.objenesis</groupId>

<artifactId>objenesis</artifactId>

<version>2.1</version>

</dependency>

<dependency>

<groupId>com.esotericsoftware</groupId>

<artifactId>reflectasm</artifactId>

<version>1.10.1</version>

</dependency>

<dependency>

<groupId>javax.servlet</groupId>

<artifactId>servlet-api</artifactId>

<version>2.5</version>

</dependency>

<dependency>

<groupId>org.slf4j</groupId>

<artifactId>slf4j-api</artifactId>

<version>1.7.21</version>

</dependency>

<dependency>

<groupId>org.apache.storm</groupId>

<artifactId>storm-core</artifactId>

<version>1.1.0</version>

</dependency>

<dependency>

<groupId>org.apache.storm</groupId>

<artifactId>storm-rename-hack</artifactId>

<version>1.1.0</version>

</dependency>

<dependency>

<groupId>junit</groupId>

<artifactId>junit</artifactId>

<version>3.8.1</version>

<scope>test</scope>

</dependency>

<dependency>

<groupId>ring-cors</groupId>

<artifactId>ring-cors</artifactId>

<version>0.1.5</version>

</dependency>

</dependencies>

<build>

<finalName>StormMavenProject</finalName>

</build>

</project>

[/quote]