初衷
Protobuf是Google出品的一款很高效的序列化和反序列化庫,但是也有些小缺點:就是需要先定義好資料結構(.proto),然後編譯為對應的java檔案;
我的需求是這樣的:由另一方給我提供資料結構和序列化之後的資料,我根據資料結構動态解析成對應的Schema,然後反序列化資料後根據Schema擷取對應的值。
準備
<dependency>
<groupId>io.protostuff</groupId>
<artifactId>protostuff-parser</artifactId>
<version>2.0.0-alpha32</version>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>3.5.1</version>
</dependency>
資料結構
syntax = "proto3";
enum Type {
TRANSACTION = 0;
EVENT = 1;
}
message MyMessage {
int32 id = 1;
Type type = 2;
int64 timestamp = 3;
map<string,string> tags = 4;
repeated MyMessage children = 5;
}
解析
/**
* 解析proto
* @param messageId Message的ID
* @param input proto檔案内容
* @return 對應的schema
* @throws Exception
*/
public DynamicSchema parseProto(long messageId, String input) throws Exception {
DynamicSchema schema = null;
if (input != null) {
String protoFile = UUID.randomUUID() + ".proto";
Path tempDirectory = Files.createTempDirectory(PB_TEMP_DIRECTORY);
Path file = Files.write(tempDirectory.resolve(protoFile), input.getBytes());
List<String> msgTypeNames = parseAndGetMsgType(input);
try {
DynamicSchema.Builder schemaBuilder = DynamicSchema.newBuilder();
schemaBuilder.setName(SCHEMA_PREFIX + messageId);
if (msgTypeNames != null) {
for (String msgType : msgTypeNames) {
schemaBuilder.addMessageDefinition(defineMessage(tempDirectory, protoFile, msgType));
}
msgTypeOfChannel.put(messageId, msgTypeNames);
}
schema = schemaBuilder.build();
} finally {
Files.delete(file);
Files.delete(tempDirectory);
}
}
return schema;
}
/**
* 根據schema擷取對應的資料結構(Message)
* @param schema
* @param messageId
* @return
* @throws Exception
*/
public Map<String, Map<String, Object>> parseSchema(DynamicSchema schema, long messageId) throws Exception {
Map<String, Map<String, Object>> fieldMap = new LinkedHashMap<>();
List<String> msgTypeNames = msgTypeOfChannel.get(messageId);
if (msgTypeNames != null) {
for (String msgTypeName : msgTypeNames) {
Descriptors.Descriptor descriptor = schema.getMessageDescriptor(msgTypeName);
Map<String, Object> fm = getFieldMap(descriptor);
if (fm != null) {
fieldMap.put(msgTypeName, fm);
}
List<Descriptors.Descriptor> nestedDescriptors = descriptor.getNestedTypes();
if (fm != null) {
for (Descriptors.Descriptor ds : nestedDescriptors) {
//map
fm.put(ds.getName(), getFieldType("map", null));
}
}
}
}
return fieldMap;
}
測試
使用上面的方法進行解析之後的Schema如下:
types: [MyMessage, MyMessage.tags]
enums: [MyMessage.Type]
file {
name: "pb_schema_1"
message_type {
name: "MyMessage"
field {
name: "id"
number: 1
label: LABEL_OPTIONAL
type: TYPE_INT32
}
field {
name: "timestamp"
number: 3
label: LABEL_OPTIONAL
type: TYPE_INT64
}
field {
name: "children"
number: 5
label: LABEL_REPEATED
type_name: "MyMessage"
}
nested_type {
name: "tags"
field {
name: "key"
number: 1
label: LABEL_OPTIONAL
type: TYPE_STRING
}
field {
name: "value"
number: 2
label: LABEL_OPTIONAL
type: TYPE_STRING
}
}
enum_type {
name: "Type"
value {
name: "TRANSACTION"
number: 0
}
value {
name: "EVENT"
number: 1
}
}
}
}
相關代碼:
https://github.com/shuaiweili/ser-des-example/blob/master/src/test/java/com/git/lee/serde/example/pb/DynamicSchemaTest.java