天天看點
傳回

Protobuf介紹及簡單使用(上）Protobuf用例Caffe.proto檔案參考資料

目錄

Protobuf

介紹

優勢

protobuf文法

Specifying Field Rules

Data type

Data name

Number

Protobuf注釋

保留字段與辨別符

syntax關鍵詞

分号

Protocol字段設定

擷取成員值

 Packed 編碼

optional default設定

用例

proto檔案

編譯proto檔案

Family頭檔案

Family源碼

用例

Makfile

運作

Caffe.proto檔案

參考資料

本節主要解決上節遺留的問題，caffe中的資料結構檔案在\src\caffe\proto\caffe.proto是個什麼東東？如何才能看懂？需要先行補充下Protobuf的基礎知識

Protobuf

介紹

Protobuf是由google開發的一套開源序列化協定架構，類似于XML,JSON，采用協定序列化用于資料存儲與讀取，與XML相比，定義了資料格式更加簡單，資料通路接口可以自動化生成，加快了開發者開發速度，最新的版本為proto3已經支援Java,C++, Python, Java Lite, Ruby,JavaScript, Objective-c 以及C#, go，等常用語言．目前很多項目都在使用Protobuf．

官方連結（不需要翻牆）：https://developers.google.cn/protocol-buffers/

官方文檔：https://developers.google.cn/protocol-buffers/docs/overview#whynotxml

優勢

既然xml已經使用非常廣泛，那麼為什麼還需要Protocol Buffer?其優勢主要有以下幾點：

  • 簡單
  • 比xml要小3到10倍，proto檔案被編譯成bin二進制檔案
  • 比xml要快20到100倍
  • 結構更加清晰，不會産生歧義
  • 生成資料操作類更加容易，程式設計更加容易上手，通路接口代碼基本都是自動生産

 例如定義一個person資料結構，裡面包含name和email兩個個人資訊，xml定義如下:

<person>
    <name>John Doe</name>
    <email>[email protected]</email>
  </person>

而使用protobuf定義時，其txt定義格式如下：

# Textual representation of a protocol buffer.
# This is *not* the binary format used on the wire.
person {
  name: "John Doe"
  email: "[email protected]"
}

是不是有點像python中的元組，采用key-value形式．

protobuf文法

上述是采用txt格式進行定義的，但是一般protobuf都是使用.proto檔案，其文法格式使用Message定義

采用Message定義，person定義如下：

message Person {
  required string name = 1;
  required string email= 2;
}

意思是Person中的第一個字段為name，其資料類型為string,比選項;第二個字段為email，其資料類型為string,比選項．資料結構前加message關鍵詞，意思是定義一個名為Person的資料結構，Message結構裡面的每個成員結構格式如下：

［rules］[data_type] [data_name] = [number]

分别解釋其各個字段意思

Specifying Field Rules

該字段主要是定義該變量的規則，主要有三種規則：

  • required:該字段變量為必須的，執行個體中必須包含的字段．如果是在調試模式下編譯 libprotobuf，則序列化一個未初始化的message 将将導緻斷言失敗。在優化的建構中，将跳過檢查并始終寫入消息。但是，解析未初始化的消息将始終失敗（通過從解析方法傳回 false）
  • optional:該字段是可選的，可以設定也可以不設定該字段．如果未設定可選字段值，則使用預設值。對于簡單類型，你可以指定自己的預設值，就像我們在示例中為電話号碼類型所做的那樣。否則，使用系統預設值：數字類型為 0，字元串為空字元串，bools 為 false。對于嵌入 message，預設值始終是消息的 “預設執行個體” 或 “原型”，其中沒有設定任何字段。調用通路器以擷取尚未顯式設定的 optional（或 required）字段的值始終傳回該字段的預設值。
  • repeated:該字段可以重複任意次數(包括零次)．重複值的順序将保留在 protocol buffer 中。可以将 repeated 字段視為動态大小的數組。

官方文檔解釋如下：

Protobuf介紹及簡單使用(上）Protobuf用例Caffe.proto檔案參考資料

Data type

Protobuf支援常用的資料類型，支援的資料類型如下表：

.proto Type Notes C++ Type Java Type Python Type[2] Go Type
double double double float *float64
float float float float *float32
int32 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. int32 int int *int32
int64 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. int64 long int/long[3] *int64
uint32 Uses variable-length encoding. uint32 int[1] int/long[3] *uint32
uint64 Uses variable-length encoding. uint64 long[1] int/long[3] *uint64
sint32 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int *int32
sint64 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long[3] *int64
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 228. uint32 int[1] int/long[3] *uint32
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 256. uint64 long[1] int/long[3] *uint64
sfixed32 Always four bytes. int32 int int *int32
sfixed64 Always eight bytes. int64 long int/long[3] *int64
bool bool boolean bool *bool
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String unicode (Python 2) or str (Python 3) *string
bytes May contain any arbitrary sequence of bytes. string ByteString bytes []byte

基本能夠滿足其使用要求

Data name

data name為資料成員名

Number

Protobuf是采用key-value形式，将成員名映射為number，這樣就能實作資料的序列化．每個資料字段都有唯一的數值型辨別符．這些辨別符用于辨別字段在消息中的二進制格式，使用中的類型不應該随意改動。需要注意的是，[1-15]内的辨別在編碼時隻占用一個位元組，包含辨別符和字段類型。[16-2047]之間的辨別符占用2個位元組。建議為頻繁出現的消息元素使用[1-15]間的辨別符。如果考慮到以後可能或擴充頻繁元素，可以預留一些辨別符。number最小為為１，最大為229 - 1,

Protobuf介紹及簡單使用(上）Protobuf用例Caffe.proto檔案參考資料
Protobuf介紹及簡單使用(上）Protobuf用例Caffe.proto檔案參考資料

or 536,870,911，　其中19000到19999為Protocol内部使用

Protobuf注釋

在proto檔案中經常需要添加注釋對某個字段進行注釋，支援C風格的雙斜線//單行注釋

保留字段與辨別符

可以使用reserved關鍵字指定保留字段和保留辨別符

message Foo {
    reserved 2, 15, 9 to 11;
    reserved "foo", "bar";
}

syntax關鍵詞

sysntax關鍵字經常用到proto檔案開頭用來表明使用的是哪個proto版本

syntax = "proto2"

分号

行與行之間需要加分号，類似C語言．

Protocol字段設定

在對資料操作過程中經常需要對某個成員變量值進行修該，protocol提供了一系列很友善的API, 使用set函數，其格式為

set_［data_name］

例如定一個下面一個message

message Person {
  required string name = 1;
  required int32 id = 2;
  optional string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    required string number = 1;
    optional PhoneType type = 2 [default = HOME];
  }

  repeated PhoneNumber phone = 4;
}

 分别設定其中的name，id以及email,其操作為：

Person person;
person.set_name("John Doe");
person.set_id(1234);
person.set_email("[email protected]");

擷取成員值

擷取成員值同樣很簡單，隻有直接通路其成員即可，如下例子

fstream input("myfile", ios::in | ios::binary);
Person person;
person.ParseFromIstream(&input);
cout << "Name: " << person.name() << endl;
cout << "E-mail: " << person.email() << endl;

 Packed 編碼

在 proto2 中為我們提供了可選的設定 [packed = true]，而這一可選項在 proto3 中已成預設設定。

packed 目前隻能用于 repeated類型。

packed = true 主要使讓 ProtoBuf 為我們把 repeated primitive 的編碼結果打包，進而進一步壓縮空間，進一步提高效率、速度。這裡打包的含義其實就是：原先的 repeated 字段的編碼結構為 Tag-Length-Value-Tag-Length-Value-Tag-Length-Value...，因為這些 Tag 都是相同的（同一字段），是以可以将這些字段的 Value 打包，即将編碼結構變為 Tag-Length-Value-Value-Value... 

optional default設定

optional字段的預設值可以使用default來設定，例如：

optional int32 num = 1 [default = 0];

将預設值設值為0

用例

下面使用一個簡單的例子說明Protobuf的基本使用方法．

proto檔案

首先編寫proto檔案中，資料格式定義，代碼如下：

syntax = "proto2";

message Person {
  required int32 age = 1;
  required string name = 2;
}

message Family {
  repeated Person person = 1;
}

編譯proto檔案

使用proto編譯器編譯proto檔案，Protobuf安裝不在較長的描述，可自行網上查找，安裝成功後，可以檢視其版本

Protobuf介紹及簡單使用(上）Protobuf用例Caffe.proto檔案參考資料

使用protoc指令編譯proto檔案，編譯完成之後會自動生産．h和．cpp的代碼

protoc --cpp_out=./  family.proto

自動生成的family.pb.cc和family.pb.h檔案，以供後面使用

Protobuf介紹及簡單使用(上）Protobuf用例Caffe.proto檔案參考資料

 protoc指令還可以将proto檔案生成python和java檔案接口，以供python和java使用，可以使用--help指令來檢視protoc參數：

Protobuf介紹及簡單使用(上）Protobuf用例Caffe.proto檔案參考資料

其中－－cpp_out是生成c++接口檔案以及源碼，--jave_out是生成java接口，--python_out是生成python接口．

Family頭檔案

生成的Family頭檔案如下：

// Generated by the protocol buffer compiler.  DO NOT EDIT!
// source: family.proto

#ifndef PROTOBUF_family_2eproto__INCLUDED
#define PROTOBUF_family_2eproto__INCLUDED

#include <string>

#include <google/protobuf/stubs/common.h>

#if GOOGLE_PROTOBUF_VERSION < 2006000
#error This file was generated by a newer version of protoc which is
#error incompatible with your Protocol Buffer headers.  Please update
#error your headers.
#endif
#if 2006001 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION
#error This file was generated by an older version of protoc which is
#error incompatible with your Protocol Buffer headers.  Please
#error regenerate this file with a newer version of protoc.
#endif

#include <google/protobuf/generated_message_util.h>
#include <google/protobuf/message.h>
#include <google/protobuf/repeated_field.h>
#include <google/protobuf/extension_set.h>
#include <google/protobuf/unknown_field_set.h>
// @@protoc_insertion_point(includes)

// Internal implementation detail -- do not call these.
void  protobuf_AddDesc_family_2eproto();
void protobuf_AssignDesc_family_2eproto();
void protobuf_ShutdownFile_family_2eproto();

class Person;
class Family;

// ===================================================================

class Person : public ::google::protobuf::Message {
 public:
  Person();
  virtual ~Person();

  Person(const Person& from);

  inline Person& operator=(const Person& from) {
    CopyFrom(from);
    return *this;
  }

  inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const {
    return _unknown_fields_;
  }

  inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() {
    return &_unknown_fields_;
  }

  static const ::google::protobuf::Descriptor* descriptor();
  static const Person& default_instance();

  void Swap(Person* other);

  // implements Message ----------------------------------------------

  Person* New() const;
  void CopyFrom(const ::google::protobuf::Message& from);
  void MergeFrom(const ::google::protobuf::Message& from);
  void CopyFrom(const Person& from);
  void MergeFrom(const Person& from);
  void Clear();
  bool IsInitialized() const;

  int ByteSize() const;
  bool MergePartialFromCodedStream(
      ::google::protobuf::io::CodedInputStream* input);
  void SerializeWithCachedSizes(
      ::google::protobuf::io::CodedOutputStream* output) const;
  ::google::protobuf::uint8* SerializeWithCachedSizesToArray(::google::protobuf::uint8* output) const;
  int GetCachedSize() const { return _cached_size_; }
  private:
  void SharedCtor();
  void SharedDtor();
  void SetCachedSize(int size) const;
  public:
  ::google::protobuf::Metadata GetMetadata() const;

  // nested types ----------------------------------------------------

  // accessors -------------------------------------------------------

  // required int32 age = 1;
  inline bool has_age() const;
  inline void clear_age();
  static const int kAgeFieldNumber = 1;
  inline ::google::protobuf::int32 age() const;
  inline void set_age(::google::protobuf::int32 value);

  // required string name = 2;
  inline bool has_name() const;
  inline void clear_name();
  static const int kNameFieldNumber = 2;
  inline const ::std::string& name() const;
  inline void set_name(const ::std::string& value);
  inline void set_name(const char* value);
  inline void set_name(const char* value, size_t size);
  inline ::std::string* mutable_name();
  inline ::std::string* release_name();
  inline void set_allocated_name(::std::string* name);

  // @@protoc_insertion_point(class_scope:Person)
 private:
  inline void set_has_age();
  inline void clear_has_age();
  inline void set_has_name();
  inline void clear_has_name();

  ::google::protobuf::UnknownFieldSet _unknown_fields_;

  ::google::protobuf::uint32 _has_bits_[1];
 mutable int _cached_size_;
  ::std::string* name_;
  ::google::protobuf::int32 age_;
  friend void  protobuf_AddDesc_family_2eproto();
  friend void protobuf_AssignDesc_family_2eproto();
  friend void protobuf_ShutdownFile_family_2eproto();

  void InitAsDefaultInstance();
  static Person* default_instance_;
};
// -------------------------------------------------------------------

class Family : public ::google::protobuf::Message {
 public:
  Family();
  virtual ~Family();

  Family(const Family& from);

  inline Family& operator=(const Family& from) {
    CopyFrom(from);
    return *this;
  }

  inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const {
    return _unknown_fields_;
  }

  inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() {
    return &_unknown_fields_;
  }

  static const ::google::protobuf::Descriptor* descriptor();
  static const Family& default_instance();

  void Swap(Family* other);

  // implements Message ----------------------------------------------

  Family* New() const;
  void CopyFrom(const ::google::protobuf::Message& from);
  void MergeFrom(const ::google::protobuf::Message& from);
  void CopyFrom(const Family& from);
  void MergeFrom(const Family& from);
  void Clear();
  bool IsInitialized() const;

  int ByteSize() const;
  bool MergePartialFromCodedStream(
      ::google::protobuf::io::CodedInputStream* input);
  void SerializeWithCachedSizes(
      ::google::protobuf::io::CodedOutputStream* output) const;
  ::google::protobuf::uint8* SerializeWithCachedSizesToArray(::google::protobuf::uint8* output) const;
  int GetCachedSize() const { return _cached_size_; }
  private:
  void SharedCtor();
 void SharedDtor();
  void SetCachedSize(int size) const;
  public:
  ::google::protobuf::Metadata GetMetadata() const;

  // nested types ----------------------------------------------------

  // accessors -------------------------------------------------------

  // repeated .Person person = 1;
  inline int person_size() const;
  inline void clear_person();
  static const int kPersonFieldNumber = 1;
  inline const ::Person& person(int index) const;
  inline ::Person* mutable_person(int index);
  inline ::Person* add_person();
  inline const ::google::protobuf::RepeatedPtrField< ::Person >&
      person() const;
  inline ::google::protobuf::RepeatedPtrField< ::Person >*
      mutable_person();

  // @@protoc_insertion_point(class_scope:Family)
 private:

  ::google::protobuf::UnknownFieldSet _unknown_fields_;

  ::google::protobuf::uint32 _has_bits_[1];
  mutable int _cached_size_;
  ::google::protobuf::RepeatedPtrField< ::Person > person_;
  friend void  protobuf_AddDesc_family_2eproto();
  friend void protobuf_AssignDesc_family_2eproto();
  friend void protobuf_ShutdownFile_family_2eproto();

  void InitAsDefaultInstance();
  static Family* default_instance_;
};
// ===================================================================


// ===================================================================

// Person

可以看到其提供set_name(),set_age()接口，以及其他接口以供使用

Family源碼

family.pb.cc為自動生成的源碼檔案，是其接口實作，有興趣可以分析一波

// Generated by the protocol buffer compiler.  DO NOT EDIT!
// source: family.proto

#define INTERNAL_SUPPRESS_PROTOBUF_FIELD_DEPRECATION
#include "family.pb.h"

#include <algorithm>

#include <google/protobuf/stubs/common.h>
#include <google/protobuf/stubs/once.h>
#include <google/protobuf/io/coded_stream.h>
#include <google/protobuf/wire_format_lite_inl.h>
#include <google/protobuf/descriptor.h>
#include <google/protobuf/generated_message_reflection.h>
#include <google/protobuf/reflection_ops.h>
#include <google/protobuf/wire_format.h>
// @@protoc_insertion_point(includes)

namespace {

const ::google::protobuf::Descriptor* Person_descriptor_ = NULL;
const ::google::protobuf::internal::GeneratedMessageReflection*
  Person_reflection_ = NULL;
const ::google::protobuf::Descriptor* Family_descriptor_ = NULL;
const ::google::protobuf::internal::GeneratedMessageReflection*
  Family_reflection_ = NULL;

}  // namespace


void protobuf_AssignDesc_family_2eproto() {
  protobuf_AddDesc_family_2eproto();
  const ::google::protobuf::FileDescriptor* file =
    ::google::protobuf::DescriptorPool::generated_pool()->FindFileByName(
      "family.proto");
  GOOGLE_CHECK(file != NULL);
  Person_descriptor_ = file->message_type(0);
  static const int Person_offsets_[2] = {
    GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(Person, age_),
    GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(Person, name_),
  };
  Person_reflection_ =
    new ::google::protobuf::internal::GeneratedMessageReflection(
      Person_descriptor_,
      Person::default_instance_,
      Person_offsets_,
      GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(Person, _has_bits_[0]),
      GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(Person, _unknown_fields_),
      -1,
      ::google::protobuf::DescriptorPool::generated_pool(),
      ::google::protobuf::MessageFactory::generated_factory(),
      sizeof(Person));
  Family_descriptor_ = file->message_type(1);
  static const int Family_offsets_[1] = {
    GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(Family, person_),
  };
  Family_reflection_ =
    new ::google::protobuf::internal::GeneratedMessageReflection(
      Family_descriptor_,
      Family::default_instance_,
      Family_offsets_,
      GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(Family, _has_bits_[0]),
      GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(Family, _unknown_fields_),
      -1,
      ::google::protobuf::DescriptorPool::generated_pool(),
      ::google::protobuf::MessageFactory::generated_factory(),
      sizeof(Family));
}

namespace {

GOOGLE_PROTOBUF_DECLARE_ONCE(protobuf_AssignDescriptors_once_);
inline void protobuf_AssignDescriptorsOnce() {
  ::google::protobuf::GoogleOnceInit(&protobuf_AssignDescriptors_once_,
                 &protobuf_AssignDesc_family_2eproto);
}

void protobuf_RegisterTypes(const ::std::string&) {
  protobuf_AssignDescriptorsOnce();
  ::google::protobuf::MessageFactory::InternalRegisterGeneratedMessage(
    Person_descriptor_, &Person::default_instance());
  ::google::protobuf::MessageFactory::InternalRegisterGeneratedMessage(
    Family_descriptor_, &Family::default_instance());
}

}  // namespace

void protobuf_ShutdownFile_family_2eproto() {
  delete Person::default_instance_;
  delete Person_reflection_;
  delete Family::default_instance_;
  delete Family_reflection_;
}

void protobuf_AddDesc_family_2eproto() {
  static bool already_here = false;
  if (already_here) return;
  already_here = true;
  GOOGLE_PROTOBUF_VERIFY_VERSION;

  ::google::protobuf::DescriptorPool::InternalAddGeneratedFile(
    "\n\014family.proto\"#\n\006Person\022\013\n\003age\030\001 \002(\005\022\014\n"
    "\004name\030\002 \002(\t\"!\n\006Family\022\027\n\006person\030\001 \003(\0132\007."
    "Person", 86);
  ::google::protobuf::MessageFactory::InternalRegisterGeneratedFile(
    "family.proto", &protobuf_RegisterTypes);
  Person::default_instance_ = new Person();
  Family::default_instance_ = new Family();
  Person::default_instance_->InitAsDefaultInstance();
  Family::default_instance_->InitAsDefaultInstance();
  ::google::protobuf::internal::OnShutdown(&protobuf_ShutdownFile_family_2eproto);
}
// Force AddDescriptors() to be called at static initialization time.
struct StaticDescriptorInitializer_family_2eproto {
  StaticDescriptorInitializer_family_2eproto() {
    protobuf_AddDesc_family_2eproto();
  }
} static_descriptor_initializer_family_2eproto_;

// ===================================================================

#ifndef _MSC_VER
const int Person::kAgeFieldNumber;
const int Person::kNameFieldNumber;
#endif  // !_MSC_VER

Person::Person()
  : ::google::protobuf::Message() {
  SharedCtor();
  // @@protoc_insertion_point(constructor:Person)
}

void Person::InitAsDefaultInstance() {
}

Person::Person(const Person& from)
  : ::google::protobuf::Message() {
  SharedCtor();
  MergeFrom(from);
  // @@protoc_insertion_point(copy_constructor:Person)
}

void Person::SharedCtor() {
  ::google::protobuf::internal::GetEmptyString();
  _cached_size_ = 0;
  age_ = 0;
  name_ = const_cast< ::std::string*>(&::google::protobuf::internal::GetEmptyStringAlreadyInited());
  ::memset(_has_bits_, 0, sizeof(_has_bits_));
}

Person::~Person() {
  // @@protoc_insertion_point(destructor:Person)
  SharedDtor();
}

void Person::SharedDtor() {
  if (name_ != &::google::protobuf::internal::GetEmptyStringAlreadyInited()) {
    delete name_;
  }
  if (this != default_instance_) {
  }
}

void Person::SetCachedSize(int size) const {
  GOOGLE_SAFE_CONCURRENT_WRITES_BEGIN();
  _cached_size_ = size;
  GOOGLE_SAFE_CONCURRENT_WRITES_END();
}
const ::google::protobuf::Descriptor* Person::descriptor() {
  protobuf_AssignDescriptorsOnce();
  return Person_descriptor_;
}

const Person& Person::default_instance() {
  if (default_instance_ == NULL) protobuf_AddDesc_family_2eproto();
  return *default_instance_;
}

Person* Person::default_instance_ = NULL;

Person* Person::New() const {
  return new Person;
}

void Person::Clear() {
  if (_has_bits_[0 / 32] & 3) {
    age_ = 0;
    if (has_name()) {
      if (name_ != &::google::protobuf::internal::GetEmptyStringAlreadyInited()) {
        name_->clear();
      }
    }
  }
  ::memset(_has_bits_, 0, sizeof(_has_bits_));
  mutable_unknown_fields()->Clear();
}

bool Person::MergePartialFromCodedStream(
    ::google::protobuf::io::CodedInputStream* input) {
#define DO_(EXPRESSION) if (!(EXPRESSION)) goto failure
  ::google::protobuf::uint32 tag;
  // @@protoc_insertion_point(parse_start:Person)
  for (;;) {
    ::std::pair< ::google::protobuf::uint32, bool> p = input->ReadTagWithCutoff(127);
    tag = p.first;
    if (!p.second) goto handle_unusual;
    switch (::google::protobuf::internal::WireFormatLite::GetTagFieldNumber(tag)) {
      // required int32 age = 1;
      case 1: {
        if (tag == 8) {
          DO_((::google::protobuf::internal::WireFormatLite::ReadPrimitive<
                   ::google::protobuf::int32, ::google::protobuf::internal::WireFormatLite::TYPE_INT32>(
                 input, &age_)));
          set_has_age();
        } else {
          goto handle_unusual;
        } 
        if (input->ExpectTag(18)) goto parse_name;
        break;
      }

      // required string name = 2;
      case 2: {
        if (tag == 18) {
         parse_name:
          DO_(::google::protobuf::internal::WireFormatLite::ReadString(
                input, this->mutable_name()));
          ::google::protobuf::internal::WireFormat::VerifyUTF8StringNamedField(
            this->name().data(), this->name().length(),
            ::google::protobuf::internal::WireFormat::PARSE,
            "name");
        } else {
          goto handle_unusual;
        } 
        if (input->ExpectAtEnd()) goto success;
        break;
      }

      default: {
      handle_unusual:
        if (tag == 0 ||
            ::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) ==
            ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) {
          goto success;
        }
        DO_(::google::protobuf::internal::WireFormat::SkipField(
              input, tag, mutable_unknown_fields()));
        break;
      }
    }
  }
success:
  // @@protoc_insertion_point(parse_success:Person)
  return true;
failure: 
  // @@protoc_insertion_point(parse_failure:Person)
  return false;
#undef DO_
}

void Person::SerializeWithCachedSizes(
    ::google::protobuf::io::CodedOutputStream* output) const {
  // @@protoc_insertion_point(serialize_start:Person)
  // required int32 age = 1;
  if (has_age()) {
    ::google::protobuf::internal::WireFormatLite::WriteInt32(1, this->age(), output);
  }

  // required string name = 2;
  if (has_name()) {
    ::google::protobuf::internal::WireFormat::VerifyUTF8StringNamedField(
      this->name().data(), this->name().length(),
      ::google::protobuf::internal::WireFormat::SERIALIZE,
     "name");
    ::google::protobuf::internal::WireFormatLite::WriteStringMaybeAliased(
      2, this->name(), output);
  }

  if (!unknown_fields().empty()) {
    ::google::protobuf::internal::WireFormat::SerializeUnknownFields(
        unknown_fields(), output);
  }
  // @@protoc_insertion_point(serialize_end:Person)
}

::google::protobuf::uint8* Person::SerializeWithCachedSizesToArray(
    ::google::protobuf::uint8* target) const {
  // @@protoc_insertion_point(serialize_to_array_start:Person)
  // required int32 age = 1;
  if (has_age()) {
    target = ::google::protobuf::internal::WireFormatLite::WriteInt32ToArray(1, this->age(), target);
  }

  // required string name = 2;
  if (has_name()) {
    ::google::protobuf::internal::WireFormat::VerifyUTF8StringNamedField(
      this->name().data(), this->name().length(),
      ::google::protobuf::internal::WireFormat::SERIALIZE,
      "name");
    target =
      ::google::protobuf::internal::WireFormatLite::WriteStringToArray(
        2, this->name(), target);
  }

  if (!unknown_fields().empty()) {
    target = ::google::protobuf::internal::WireFormat::SerializeUnknownFieldsToArray(
        unknown_fields(), target);
  }
  // @@protoc_insertion_point(serialize_to_array_end:Person)
  return target;
}

int Person::ByteSize() const {
  int total_size = 0;

  if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) {
    // required int32 age = 1;
    if (has_age()) {
      total_size += 1 +
        ::google::protobuf::internal::WireFormatLite::Int32Size(
          this->age());
    }

    // required string name = 2;
    if (has_name()) {
      total_size += 1 +
        ::google::protobuf::internal::WireFormatLite::StringSize(
          this->name());
    }
  
  }
  if (!unknown_fields().empty()) {
    total_size +=
      ::google::protobuf::internal::WireFormat::ComputeUnknownFieldsSize(
        unknown_fields());
  }
  GOOGLE_SAFE_CONCURRENT_WRITES_BEGIN();
  _cached_size_ = total_size;
  GOOGLE_SAFE_CONCURRENT_WRITES_END();
  return total_size;
}

void Person::MergeFrom(const ::google::protobuf::Message& from) {
  GOOGLE_CHECK_NE(&from, this);
  const Person* source =
    ::google::protobuf::internal::dynamic_cast_if_available<const Person*>(
      &from);
  if (source == NULL) {
    ::google::protobuf::internal::ReflectionOps::Merge(from, this);
  } else {
    MergeFrom(*source);
  }
}

void Person::MergeFrom(const Person& from) {
  GOOGLE_CHECK_NE(&from, this);
  if (from._has_bits_[0 / 32] & (0xffu << (0 % 32))) {
    if (from.has_age()) {
      set_age(from.age());
    }
    if (from.has_name()) {
      set_name(from.name());
    }
  }
  mutable_unknown_fields()->MergeFrom(from.unknown_fields());
}

void Person::CopyFrom(const ::google::protobuf::Message& from) {
  if (&from == this) return;
  Clear();
  MergeFrom(from);
}

void Person::CopyFrom(const Person& from) {
  if (&from == this) return;
  Clear();
  MergeFrom(from);
}

bool Person::IsInitialized() const {
  if ((_has_bits_[0] & 0x00000003) != 0x00000003) return false;

  return true;
}
void Person::Swap(Person* other) {
  if (other != this) {
    std::swap(age_, other->age_);
    std::swap(name_, other->name_);
    std::swap(_has_bits_[0], other->_has_bits_[0]);
    _unknown_fields_.Swap(&other->_unknown_fields_);
    std::swap(_cached_size_, other->_cached_size_);
  }
}

::google::protobuf::Metadata Person::GetMetadata() const {
  protobuf_AssignDescriptorsOnce();
  ::google::protobuf::Metadata metadata;
  metadata.descriptor = Person_descriptor_;
  metadata.reflection = Person_reflection_;
  return metadata;
}


// ===================================================================

#ifndef _MSC_VER
const int Family::kPersonFieldNumber;
#endif  // !_MSC_VER

Family::Family()
  : ::google::protobuf::Message() {
  SharedCtor();
  // @@protoc_insertion_point(constructor:Family)
}

void Family::InitAsDefaultInstance() {
}

Family::Family(const Family& from)
  : ::google::protobuf::Message() {
  SharedCtor();
  MergeFrom(from);
  // @@protoc_insertion_point(copy_constructor:Family)
}

void Family::SharedCtor() {
  _cached_size_ = 0;
  ::memset(_has_bits_, 0, sizeof(_has_bits_));
}

Family::~Family() {
  // @@protoc_insertion_point(destructor:Family)
  SharedDtor();
}

void Family::SharedDtor() {
  if (this != default_instance_) {
  }
}

void Family::SetCachedSize(int size) const {
  GOOGLE_SAFE_CONCURRENT_WRITES_BEGIN();
  _cached_size_ = size;
  GOOGLE_SAFE_CONCURRENT_WRITES_END();
}
const ::google::protobuf::Descriptor* Family::descriptor() {
  protobuf_AssignDescriptorsOnce();
  return Family_descriptor_;
}

const Family& Family::default_instance() {
  if (default_instance_ == NULL) protobuf_AddDesc_family_2eproto();
  return *default_instance_;
}

Family* Family::default_instance_ = NULL;

Family* Family::New() const {
  return new Family;
}

void Family::Clear() {
  person_.Clear();
  ::memset(_has_bits_, 0, sizeof(_has_bits_));
  mutable_unknown_fields()->Clear();
}

bool Family::MergePartialFromCodedStream(
    ::google::protobuf::io::CodedInputStream* input) {
#define DO_(EXPRESSION) if (!(EXPRESSION)) goto failure
  ::google::protobuf::uint32 tag;
  // @@protoc_insertion_point(parse_start:Family)
  for (;;) {
    ::std::pair< ::google::protobuf::uint32, bool> p = input->ReadTagWithCutoff(127);
    tag = p.first;
    if (!p.second) goto handle_unusual;
    switch (::google::protobuf::internal::WireFormatLite::GetTagFieldNumber(tag)) {
      // repeated .Person person = 1;
      case 1: {
        if (tag == 10) {
         parse_person:
          DO_(::google::protobuf::internal::WireFormatLite::ReadMessageNoVirtual(
                input, add_person()));
        } else {
          goto handle_unusual;
        }
        if (input->ExpectTag(10)) goto parse_person;
        if (input->ExpectAtEnd()) goto success;
        break;
      }

      default: {
      handle_unusual:
        if (tag == 0 ||
            ::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) ==
            ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) {
          goto success;
        }
        DO_(::google::protobuf::internal::WireFormat::SkipField(
              input, tag, mutable_unknown_fields()));
        break;
      }
    }
  }
success:
  // @@protoc_insertion_point(parse_success:Family)
  return true;
failure:
  // @@protoc_insertion_point(parse_failure:Family)
  return false;
#undef DO_
}

void Family::SerializeWithCachedSizes(
    ::google::protobuf::io::CodedOutputStream* output) const {
  // @@protoc_insertion_point(serialize_start:Family)
  // repeated .Person person = 1;
  for (int i = 0; i < this->person_size(); i++) {
    ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray(
      1, this->person(i), output);
  }

  if (!unknown_fields().empty()) {
    ::google::protobuf::internal::WireFormat::SerializeUnknownFields(
        unknown_fields(), output);
  }
  // @@protoc_insertion_point(serialize_end:Family)
}

::google::protobuf::uint8* Family::SerializeWithCachedSizesToArray(
    ::google::protobuf::uint8* target) const {
  // @@protoc_insertion_point(serialize_to_array_start:Family)
  // repeated .Person person = 1;
  for (int i = 0; i < this->person_size(); i++) {
    target = ::google::protobuf::internal::WireFormatLite::
      WriteMessageNoVirtualToArray(
        1, this->person(i), target);
  }

  if (!unknown_fields().empty()) {
    target = ::google::protobuf::internal::WireFormat::SerializeUnknownFieldsToArray(
        unknown_fields(), target);
  }
  // @@protoc_insertion_point(serialize_to_array_end:Family)
  return target;
}

int Family::ByteSize() const {
  int total_size = 0;
 // repeated .Person person = 1;
  total_size += 1 * this->person_size();
  for (int i = 0; i < this->person_size(); i++) {
    total_size +=
      ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual(
        this->person(i));
  }

  if (!unknown_fields().empty()) {
    total_size +=
      ::google::protobuf::internal::WireFormat::ComputeUnknownFieldsSize(
        unknown_fields());
  }
  GOOGLE_SAFE_CONCURRENT_WRITES_BEGIN();
  _cached_size_ = total_size;
  GOOGLE_SAFE_CONCURRENT_WRITES_END();
  return total_size;
}

void Family::MergeFrom(const ::google::protobuf::Message& from) {
  GOOGLE_CHECK_NE(&from, this);
  const Family* source =
    ::google::protobuf::internal::dynamic_cast_if_available<const Family*>(
      &from);
  if (source == NULL) {
    ::google::protobuf::internal::ReflectionOps::Merge(from, this);
  } else {
    MergeFrom(*source);
  }
}

void Family::MergeFrom(const Family& from) {
  GOOGLE_CHECK_NE(&from, this);
  person_.MergeFrom(from.person_);
  mutable_unknown_fields()->MergeFrom(from.unknown_fields());
}

void Family::CopyFrom(const ::google::protobuf::Message& from) {
  if (&from == this) return;
  Clear();
  MergeFrom(from);
}

void Family::CopyFrom(const Family& from) {
  if (&from == this) return;
  Clear();
  MergeFrom(from);
}

bool Family::IsInitialized() const {

  if (!::google::protobuf::internal::AllAreInitialized(this->person())) return false;
  return true;
}

void Family::Swap(Family* other) {
  if (other != this) {
    person_.Swap(&other->person_);
 mutable_unknown_fields()->MergeFrom(from.unknown_fields());
}

void Family::CopyFrom(const ::google::protobuf::Message& from) {
  if (&from == this) return;
  Clear();
  MergeFrom(from);
}

void Family::CopyFrom(const Family& from) {
  if (&from == this) return;
  Clear();
  MergeFrom(from);
}

bool Family::IsInitialized() const {

  if (!::google::protobuf::internal::AllAreInitialized(this->person())) return false;
  return true;
}

void Family::Swap(Family* other) {
  if (other != this) {
    person_.Swap(&other->person_);
    std::swap(_has_bits_[0], other->_has_bits_[0]);
    _unknown_fields_.Swap(&other->_unknown_fields_);
    std::swap(_cached_size_, other->_cached_size_);
  }
}

::google::protobuf::Metadata Family::GetMetadata() const {
  protobuf_AssignDescriptorsOnce();
  ::google::protobuf::Metadata metadata;
  metadata.descriptor = Family_descriptor_;
  metadata.reflection = Family_reflection_;
  return metadata;
}


// @@protoc_insertion_point(namespace_scope)

// @@protoc_insertion_point(global_scope)

用例

使用用例要求對Famyily添加兩個成員，并分别設定其name和age,  用例先後先要加載生成的頭檔案famiy.pb.h，用例代碼如下較為簡單：

#include "family.pb.h"
#include <stdio.h>
#include <iostream>
#include <fstream>
#include <string>

using namespace std;

int main(){
  GOOGLE_PROTOBUF_VERIFY_VERSION;

  Family  family;
  Person* person;

  person = family.add_person();
  person->set_age(25);
  person->set_name("John");

  person = family.add_person();
  person->set_age(40);
  person->set_name("Tony");

  int size = family.person_size();

  printf("size : %d \r\n", size);
  for(int i = 0; i<size; i++)
  {
    Person psn=family.person(i);
    cout <<"Name :"<<psn.name()<<", age: "<<psn.age()<<endl;
  }

  return 0;
}

Makfile

編寫makefile，不僅要編譯編寫的用例檔案，還要編譯自動生成的family.pb.cc檔案，編譯選項要加上

pkg-config --cflags --libs protobuf`

Makfile源碼如下：

CC=g++
LIB_PATH=
HEAD_PATH=

TARGET = test
CPPFLAGS = -v -std=c++11
LIB_NAME =

CPP_FILE = family.pb.cc main.cpp

$(TARGET):
        $(CC) $(CPPFLAGS) $(CPP_FILE) $(LIB_PATH) $(LIB_NAME) $(HEAD_PATH) `pkg-config --cflags --libs protobuf` -o [email protected] 
clean:
        rm -f $(TARGET)

運作

用例運作結果：

Protobuf介紹及簡單使用(上）Protobuf用例Caffe.proto檔案參考資料

Caffe.proto檔案

到目前為止Protobuf應該有所了解，上述資料足以我們能夠了解src\caffe\proto\caffe.proto檔案，Protobuf這層洋蔥應該也可以剝掉．見caffe.proto檔案開頭

syntax = "proto2";

package caffe;

// Specifies the shape (dimensions) of a Blob.
message BlobShape {
  repeated int64 dim = 1 [packed = true];
}

message BlobProto {
  optional BlobShape shape = 7;
  repeated float data = 5 [packed = true];
  repeated float diff = 6 [packed = true];
  repeated double double_data = 8 [packed = true];
  repeated double double_diff = 9 [packed = true];

  // 4D dimensions -- deprecated.  Use "shape" instead.
  optional int32 num = 1 [default = 0];
  optional int32 channels = 2 [default = 0];
  optional int32 height = 3 [default = 0];
  optional int32 width = 4 [default = 0];
}

檔案開頭 syntax = "proto2";表明使用的是proto2版本

  • 在下行代碼中看到一個關鍵字package，主要一個聲名符，用來防止不同的消息類型有命名沖突。包的聲明符會根據使用語言的不同影響生成的代碼。對于C++，産生的類會被包裝在C++的命名空間。package caffe，表明是在caffe的命名空間
  • 接下來定義一個BlobShape資料結構，資料結構中隻包含一個字段dim，資料類型為int64，該資料類型為repeated，可以包含多個dim, 使用之前可以使用dim_size來計算其資料中有多少個dim，例如：
  • Protobuf介紹及簡單使用(上）Protobuf用例Caffe.proto檔案參考資料
  • BlobProto結構定義，是Blob中比較關鍵的資料結構。在該資料中首先可以看到首先定義一個 BlobShape字段(ProtoBuf字段定義可以包含之前定義的資料結構)，該字段是可選的，但是其number并不是為1，而是放到了後面。由此可以看到message中的字段定義并不是定義的第一個其number就為1，而是按照其經常使用頻率進行調整。在BlobProto中經常使用到的就是4維參數，是以将4維參數放到前面，從1到4依次維num, channels, heigt, width。緊接着存放資料順序維data， diff, shape, double_data, double_diff。在我們實際使用中定義message字段，也可以将經常使用到的頻率較高的放到前面以提高通路效率及速度。
  • 接下來定義BlobProtoVector資料結構，包含字段類型為可選的BlobProto

剩餘其他的不再解釋，應該自己可以足夠看懂，是不是并沒有想象中的那麼複雜，限于篇幅大小，将在下節中介紹protobuf中的檔案讀寫：

Protobuf介紹及簡單使用(下）之檔案讀寫

參考資料

https://developers.google.cn/protocol-buffers/docs/overview#whynotxml

https://www.jianshu.com/p/d2bed3614259

https://blog.csdn.net/u014630623/article/details/88992609

Caffe Protobuf
上一篇: Protobuf簡單例子
下一篇: Git配置及常用指令使用一.Git配置二.基本指令三.Git 與bycon Compare 配合使用

繼續閱讀