8000 Extract ProtobufConvert derive to separate crate by pavel-mukhanov · Pull Request #1501 · exonum/exonum · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Extract ProtobufConvert derive to separate crate #1501

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 37 commits into from
Oct 24, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
6001f9a
Move proto to separate crate.
pavel-mukhanov Oct 9, 2019
27b9006
Fix some comments.
pavel-mukhanov Oct 9, 2019
52cde7a
Update exonum/src/messages/tests.rs
Oct 10, 2019
3a98da2
Use env variables in common and crypto.
pavel-mukhanov Oct 10, 2019
e66eeb8
Add protobuf generator.
pavel-mukhanov Oct 10, 2019
6c24b44
Fixes.
pavel-mukhanov Oct 10, 2019
a9b57dc
Fix build.
pavel-mukhanov Oct 10, 2019
7a33f1c
Polish code.
pavel-mukhanov Oct 10, 2019
f881a1c
Fix lints.
pavel-mukhanov Oct 10, 2019
29d450c
Update components/merkledb/src/macros.rs
Oct 11, 2019
cfc0143
Update components/build/src/lib.rs
Oct 11, 2019
6915489
Fix review comments and add docs.
pavel-mukhanov Oct 11, 2019
79fef04
Merge remote-tracking branch 'origin/extract_proto_sep_crate' into ex…
pavel-mukhanov Oct 11, 2019
097808c
Add changelog.
pavel-mukhanov Oct 11, 2019
dedbb12
Update CHANGELOG.md
Oct 11, 2019
fc9e17b
Add BinaryValue and ObjectHash derive macros.
pavel-mukhanov Oct 15, 2019
8e10f4f
WIP.
pavel-mukhanov Oct 15, 2019
c6f0e6b
WIP.
pavel-mukhanov Oct 15, 2019
442fa16
Polish code.
pavel-mukhanov Oct 15, 2019
397dd63
Remove "crate" attribute from protobuf convert.
pavel-mukhanov Oct 16, 2019
9a172a1
Fix benches.
pavel-mukhanov Oct 16, 2019
8102214
Merge upstream.
pavel-mukhanov Oct 16, 2019
0b85f41
Polish some code.
pavel-mukhanov Oct 16, 2019
cdaf6bd
Revert merge artifact.
pavel-mukhanov Oct 16, 2019
c98dbdf
Add doc and changelog.
pavel-mukhanov Oct 16, 2019
32b7614
Change ProtobufConvert to attribure macro.
pavel-mukhanov Oct 18, 2019
efd8151
Polish code.
pavel-mukhanov Oct 18, 2019
e9fa68c
Polish code 2.
pavel-mukhanov Oct 18, 2019
cb81870
Fix typo.
pavel-mukhanov Oct 18, 2019
4d2a612
Fix comments.
pavel-mukhanov Oct 18, 2019
c874497
Make ProtobufConvert derive macro again.
pavel-mukhanov Oct 18, 2019
6cb5df4
Polish code.
pavel-mukhanov Oct 18, 2019
ee261f0
Fix typo.
pavel-mukhanov Oct 22, 2019
7433538
Bundle proto-derive with proto.
pavel-mukhanov Oct 23, 2019
6707ebe
Fix doc in ProtobufConvert derive macro.
pavel-mukhanov Oct 23, 2019
35bbe31
Merge branch 'dynamic_services' into extract-proto-derive
pavel-mukhanov Oct 23, 2019
f8629b6
Fix doc tests.
pavel-mukhanov Oct 23, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,16 @@ The project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html)

#### exonum-proto

- Introduced new crate `exonum-proto`. Trait `ProtobufConvert` is moved to
`exonum-proto` crate. (#1496)
- Introduced a new crate `exonum-proto`. Trait `ProtobufConvert` is moved
to this crate. (#1496)

#### exonum-proto-derive

- Introduced a new crate `exonum-proto-derive`. Derive macro `ProtobufConvert` is
moved to this crate. (#1501)

- Derive macro `ProtobufConvert` now doesn't derive `BinaryValue` and `ObjectHash`
traits. There are separate derive macros for them in `exonum-derive` crate. (#1501)

#### exonum-build

Expand Down
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,6 @@ members = [
"components/keys",
"components/merkledb",
"components/proto",
"components/proto-derive",
]
exclude = [ "exonum/fuzz" ]
118 changes: 118 additions & 0 deletions components/derive/src/db_traits.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
// Copyright 2019 The Exonum Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

use darling::FromDeriveInput;
use proc_macro::TokenStream;
use proc_macro2::{Ident, Span};
use quote::{quote, ToTokens};
use syn::DeriveInput;

#[derive(Debug, FromDeriveInput)]
struct BinaryValueStruct {
ident: Ident,
}

#[derive(Debug, FromDeriveInput)]
struct ObjectHashStruct {
ident: Ident,
}

impl ObjectHashStruct {
pub fn implement_object_hash(&self) -> impl ToTokens {
let name = &self.ident;

quote! {
impl exonum_merkledb::ObjectHash for #name {
fn object_hash(&self) -> exonum_crypto::Hash {
use exonum_merkledb::BinaryValue;
let v = self.to_bytes();
exonum_crypto::hash(&v)
}
}
}
}
}

impl BinaryValueStruct {
pub fn implement_binary_value(&self) -> impl ToTokens {
let name = &self.ident;

quote! {
// This trait assumes that we work with trusted data so we can unwrap here.
impl exonum_merkledb::BinaryValue for #name {
fn to_bytes(&self) -> Vec<u8> {
self.to_pb().write_to_bytes().expect(
concat!("Failed to serialize in BinaryValue for ", stringify!(#name))
)
}

fn from_bytes(value: std::borrow::Cow<[u8]>) -> Result<Self, failure::Error> {
let mut block = <Self as exonum_proto::ProtobufConvert>::ProtoStruct::new();
block.merge_from_bytes(value.as_ref())?;
exonum_proto::ProtobufConvert::from_pb(block)
}
}
}
}
}

impl ToTokens for BinaryValueStruct {
fn to_tokens(&self, tokens: &mut proc_macro2::TokenStream) {
let mod_name = Ident::new(
&format!("binary_value_impl_{}", self.ident),
Span::call_site(),
);

let binary_value = self.implement_binary_value();
let expanded = quote! {
mod #mod_name {
use super::*;

use protobuf::Message as _ProtobufMessage;
use exonum_proto::ProtobufConvert;

#binary_value
}
};

tokens.extend(expanded);
}
}

impl ToTokens for ObjectHashStruct {
fn to_tokens(&self, tokens: &mut proc_macro2::TokenStream) {
let object_hash = self.implement_object_hash();
let expanded = quote! { #object_hash };

tokens.extend(expanded);
}
}

pub fn binary_value(input: TokenStream) -> TokenStream {
let input: DeriveInput = syn::parse(input).unwrap();

let db_object = BinaryValueStruct::from_derive_input(&input)
.unwrap_or_else(|e| panic!("BinaryValue: {}", e));
let tokens = quote! { #db_object };
tokens.into()
}

pub fn object_hash(input: TokenStream) -> TokenStream {
let input: DeriveInput = syn::parse(input).unwrap();

let db_object =
ObjectHashStruct::from_derive_input(&input).unwrap_or_else(|e| panic!("ObjectHash: {}", e));
let tokens = quote! { #db_object };
tokens.into()
}
74 changes: 38 additions & 36 deletions components/derive/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,56 +20,58 @@

extern crate proc_macro;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needless ?


mod db_traits;
mod execution_error;
mod exonum_service;
mod pb_convert;
mod service_factory;

use darling::FromMeta;
use proc_macro::TokenStream;
use quote::ToTokens;
use syn::{Attribute, NestedMeta};

/// Derive `ProtobufConvert` trait.
/// Derive `BinaryValue` trait.
/// Target type must implement `ProtobufConvert` trait.
///
/// Attributes:
///
/// ## Required
///
/// * `#[exonum(pb = "path")]`
///
/// Path is the name of the corresponding protobuf generated struct.
///
/// ## Optional
///
/// * `#[exonum(crate = "path")]`
/// # Example
/// ```ignore
/// #[derive(Clone, Debug, BinaryValue)]
/// #[protobuf_convert(source = "proto::Wallet")]
/// pub struct Wallet {
/// /// `PublicKey` of the wallet.
/// pub pub_key: PublicKey,
/// /// Current balance of the wallet.
/// pub balance: u64,
/// }
///
/// Prefix of the `exonum` crate(usually "crate" or "exonum").
/// let wallet = Wallet::new();
/// let bytes = wallet.to_bytes();
/// ```
#[proc_macro_derive(BinaryValue)]
pub fn binary_value(input: TokenStream) -> TokenStream {
db_traits::binary_value(input)
}

/// Derive `ObjectHash` trait.
/// Target type must implement `BinaryValue` trait.
///
/// * `#[exonum(serde_pb_convert)]`
///
/// Implement `serde::{Serialize, Deserialize}` using structs that were generated with
/// protobuf.
/// For example, it should be used if you want json representation of your struct
/// to be compatible with protobuf representation (including proper nesting of fields).
/// ```text
/// For example, struct with `exonum::crypto::Hash` with this
/// (de)serializer will be represented as
/// StructName {
/// "hash": {
/// data: [1, 2, ...]
/// },
/// // ...
/// }
/// // With default (de)serializer.
/// StructName {
/// "hash": "12af..." // HEX
/// // ...
/// # Example
/// ```ignore
/// #[protobuf_convert(source = "proto::Wallet")]
/// #[derive(Clone, Debug, ProtobufConvert, BinaryValue, ObjectHash)]
/// pub struct Wallet {
/// /// `PublicKey` of the wallet.
/// pub pub_key: PublicKey,
/// /// Current balance of the wallet.
/// pub balance: u64,
/// }
///
/// let wallet = Wallet::new();
/// let hash = wallet.object_hash();
/// ```
#[proc_macro_derive(ProtobufConvert, attributes(exonum))]
pub fn generate_protobuf_convert(input: TokenStream) -> TokenStream {
pb_convert::implement_protobuf_convert(input)
#[proc_macro_derive(ObjectHash)]
pub fn object_hash(input: TokenStream) -> TokenStream {
db_traits::object_hash(input)
}

/// Derive `ServiceFactory` and `ServiceDispatcher` traits.
Expand Down
23 changes: 23 additions & 0 deletions components/proto-derive/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
[package]
name = "exonum-proto-derive"
version = "0.12.0"
edition = "2018"
authors = ["The Exonum Team <exonum@bitfury.com>"]
homepage = "https://exonum.com/"
repository = "https://github.com/exonum/exonum"
documentation = "https://docs.rs/exonum-proto-derive"
license = "Apache-2.0"
keywords = ["protobuf", "exonum"]
categories = ["development-tools"]
description = "Helper macros for serialization structs in protobuf."

[lib]
proc-macro = true

[dependencies]
darling = "0.10.0"
heck = "0.3.1"
proc-macro2 = "1.0"
quote = "1.0"
syn = "1.0"
semver = "0.9"
124 changes: 124 additions & 0 deletions components/proto-derive/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
// Copyright 2019 The Exonum Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

//! This crate provides macros for deriving some useful methods and traits for the exonum services.

#![recursion_limit = "128"]
#![deny(unsafe_code, bare_trait_objects)]
#![warn(missing_docs, missing_debug_implementations)]

extern crate proc_macro;

mod pb_convert;

use proc_macro::TokenStream;
use syn::{Attribute, NestedMeta};

/// ProtobufConvert derive macro.
///
/// Attributes:
///
/// ## Required
///
/// * `#[protobuf_convert(source = "path")]`
///
/// ```ignore
/// #[derive(Clone, Debug, BinaryValue, ObjectHash, ProtobufConvert)]
/// #[protobuf_convert(source = "proto::Wallet")]
/// pub struct Wallet {
/// /// `PublicKey` of the wallet.
/// pub pub_key: PublicKey,
/// /// Current balance of the wallet.
/// pub balance: u64,
/// }
///
/// let wallet = Wallet::new();
/// let serialized_wallet = wallet.to_pb();
///
/// let deserialized_wallet = ProtobufConvert::from_pb(serialized_wallet).unwrap();
/// assert_eq!(wallet, deserialized_wallet);
/// ```
///
/// Corresponding proto file:
/// ```proto
/// message Wallet {
/// // Public key of the wallet owner.
/// exonum.crypto.PublicKey pub_key = 1;
/// // Current balance.
/// uint64 balance = 2;
/// }
/// ```
///
/// This macro can also be applied to enums. In proto files enums are represented
/// by `oneof` field. You can specify `oneof` field name, default is "message".
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly that:

  • The protobuf message must contain only this oneof field group (additional fields are not allowed)?
  • The Rust enum should have variants with a single unnamed field? (Or are zero-field variants allowed too?)

It could be worth specifying these details.

/// Corresponding proto file must contain only this oneof field. Possible enum
/// variants are zero-field and one-field variants.
/// ```ignore
/// #[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize, ProtobufConvert)]
/// #[protobuf_convert(source = "schema::runtime::ConfigChange", oneof_field = "message")]
/// pub enum ConfigChange {
/// /// New consensus config.
/// Consensus(ConsensusConfig),
/// /// New service instance config.
/// Service(ServiceConfig),
/// }
/// ```
///
/// Corresponding proto file:
/// ```proto
/// message ConfigChange {
/// oneof message {
/// // New consensus config.
/// exonum.Config consensus = 1;
/// // New service instance config.
/// ServiceConfig service = 2;
/// }
/// }
/// ```
///
/// Path is the name of the corresponding protobuf generated struct.
///
/// * `#[protobuf_convert(source = "path", serde_pb_convert)]`
///
/// Implement `serde::{Serialize, Deserialize}` using structs that were generated with
/// protobuf.
/// For example, it should be used if you want json representation of your struct
/// to be compatible with protobuf representation (including proper nesting of fields).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit / off-topic: it's not clear what "compatible" means in this context. E.g., bytes should be base64-encoded according to Protobuf 3 spec, but the example below shows that it is serialized as an array.

/// For example, struct with `exonum::crypto::Hash` with this
/// (de)serializer will be represented as
/// ```text
/// StructName {
/// "hash": {
/// "data": [1, 2, ...]
/// },
/// // ...
/// }
/// // With default (de)serializer.
/// StructName {
/// "hash": "12af..." // HEX
/// // ...
/// }
/// ```
#[proc_macro_derive(ProtobufConvert, attributes(protobuf_convert))]
pub fn protobuf_convert(input: TokenStream) -> TokenStream {
pb_convert::implement_protobuf_convert(input)
}

pub(crate) fn find_protobuf_convert_meta(args: &[Attribute]) -> Option<NestedMeta> {
args.as_ref()
.iter()
.filter_map(|a| a.parse_meta().ok())
.find(|m| m.path().is_ident("protobuf_convert"))
.map(NestedMeta::from)
}
Loading
0