springboot-grpc最大传输上限问题探索

场景：

基于grpc搭建的微服务，在调用的时候一个批次传输数据量太大导致服务器报如下错误：
搭建微服务 https://www.jianshu.com/p/2207011c0164

2019-03-06 12:46:07.544  WARN 2188 --- [-worker-ELG-3-7] io.grpc.netty.NettyServerStream          : Exception processing messageio.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: io.grpc.netty.NettyServerStream$TransportState: Frame size 10311685 exceeds maximum: 4194304.     at io.grpc.Status.asRuntimeException(Status.java:517) ~[grpc-core-1.10.0.jar:1.10.0]    at io.grpc.internal.MessageDeframer.processHeader(MessageDeframer.java:391) ~[grpc-core-1.10.0.jar:1.10.0]    at io.grpc.internal.MessageDeframer.deliver(MessageDeframer.java:271) ~[grpc-core-1.10.0.jar:1.10.0]    at io.grpc.internal.MessageDeframer.request(MessageDeframer.java:165) ~[grpc-core-1.10.0.jar:1.10.0]    at io.grpc.internal.AbstractStream$TransportState.requestMessagesFromDeframer(AbstractStream.java:202) ~[grpc-core-1.10.0.jar:1.10.0]    at io.grpc.netty.NettyServerStream$Sink$1.run(NettyServerStream.java:100) [grpc-netty-1.10.0.jar:1.10.0]    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:163) [netty-common-4.1.29.Final.jar:4.1.29.Final]    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java) [netty-common-4.1.29.Final.jar:4.1.29.Final]    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) [netty-common-4.1.29.Final.jar:4.1.29.Final]    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:446) [netty-transport-4.1.29.Final.jar:4.1.29.Final]    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884) [netty-common-4.1.29.Final.jar:4.1.29.Final]    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.29.Final.jar:4.1.29.Final]    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_05]

代码复现：

HelloController

@RestControllerpublic class HelloController {    @Autowired    private HelloService service;    @GetMapping("/hello")    public String sayHello(String name) {        //此处构造超长消息        StringBuffer sb = new StringBuffer();        for (int i = 0; i < 1024*2014; i++) {            sb.append(name);        }        return service.sendMessage(sb.toString());    }}

HelloService

@Service(value = "helloService")public class HelloService extends HelloServiceGrpc.HelloServiceImplBase {    @GrpcClient("local-grpc-server")    private Channel channel;    public String sendMessage(String name) {        HelloServiceGrpc.HelloServiceBlockingStub stub = HelloServiceGrpc.newBlockingStub(channel);        HelloResponse response = stub.sayHello(HelloRequest.newBuilder().setName(name).build());        return response.getMessage();    }}

服务端

@GrpcService(HelloServiceGrpc.class)public class HelloService extends HelloServiceGrpc.HelloServiceImplBase {    @Override    public void sayHello(HelloRequest request, StreamObserver<HelloResponse> responseObserver) {        String name = request.getName();        System.out.println("received name: "+name);        //此处为了不影响页面响应，直接返回一个字符串，不把全部消息返回        HelloResponse response = HelloResponse.newBuilder().setMessage("welcome to gRPC").build();        responseObserver.onNext(response);        responseObserver.onCompleted();    }}

调用：
http://localhost:8080/hello?name=world

结果：

出现如片段1的报错信息，接下来对问题进行详细探索并且给出解决方案

探索：

step1:阅读原有日志

io.grpc.netty.NettyServerStream$TransportState: Frame size 10311685 exceeds maximum: 4194304.
稍微翻译一下： 帧长度为10311685，超过最大值 4194304

step2：grpc官方状态码解释：

grpc官方状态码解释（grpc官方资源）

HTTP2错误码解释（资源来源于网络）

那么我们的错误就很明显了：
RESOURCE_EXHAUSTED…并且运行时提供有额外的错误详情，表示耗尽资源是带宽
那么通过上面各种巴拉巴拉一大堆，说人话，就是我们的消息字符串超常了，最大4m，我们传过去10m，报错了

找解决方案：

思路整理：

1：通过调整最大传输上限参数
客户端具体参数查看:
net.devh.springboot.autoconfigure.grpc.client.GrpcChannelProperties
类，其中有以下参数：

@Datapublic class GrpcChannelProperties {    public static final String DEFAULT_HOST = "127.0.0.1";    public static final Integer DEFAULT_PORT = 9090;    public static final GrpcChannelProperties DEFAULT = new GrpcChannelProperties();    private List<String> host = new ArrayList<String>() {        private static final long serialVersionUID = -8367871342050560040L;        {            add(DEFAULT_HOST);        }    };    private List<Integer> port = new ArrayList<Integer>() {        private static final long serialVersionUID = 4705083089654936515L;        {            add(DEFAULT_PORT);        }    };    private boolean plaintext = true;    private boolean enableKeepAlive = false;    private boolean keepAliveWithoutCalls = false;    private long keepAliveTime = 180;    private long keepAliveTimeout = 20;    private int maxInboundMessageSize;}

#客户端参数优化grpc.client.local-grpc-server.maxInBoundMessageSize=20971520

服务端参数查看：
net.devh.springboot.autoconfigure.grpc.server.GrpcServerProperties
参数如下：

@Data@ConfigurationProperties("grpc.server")public class GrpcServerProperties {       private int port = 9090;    private String address = "0.0.0.0";        private int maxMessageSize;    private final Security security = new Security();    @Data    public static class Security {        private Boolean enabled = false;        private String certificateChainPath = "";        private String certificatePath = "";    }}

#服务端参数优化grpc.server.max-message-size=20971520

产生结果：

调用结果

`能够正常返回值``但是会导致响应变慢，资源损耗较大``有时候无法确定最大传输量级，不能正确给设置正确的参数，需要反复增加数字`

2：grpc有流式传输，stream消息
进行此方法前，为了验证，请注掉上面的参数优化步骤！

# grpc.server.max-message-size=20971520

消息定义：

service HelloService {    rpc SayHello (stream HelloRequest) returns (HelloResponse) {}}message HelloRequest {    bytes name = 1;}message HelloResponse {    string code = 1;    string message = 2;}

服务端代码：

@GrpcService(HelloServiceGrpc.class)public class HelloService extends HelloServiceGrpc.HelloServiceImplBase {    private static final Logger logger = LoggerFactory.getLogger(HelloService.class);    @Override    public StreamObserver<HelloRequest> sayHello(StreamObserver<HelloResponse> responseObserver) {        return new StreamObserver<HelloRequest>() {            @Override            public void onNext(HelloRequest value) {                String name = value.getName().toStringUtf8();                logger.info("received name :" + name);            }            @Override            public void onError(Throwable t) {                logger.warn("throw an error :", t);            }            @Override            public void onCompleted() {                 responseObserver.onNext(HelloResponse.newBuilder().setMessage("welcome to gRPC").build());                 responseObserver.onCompleted();            }        };    }}

客户端调用：
— controller

@RestControllerpublic class HelloController {    @Autowired    private HelloService service;    @GetMapping("/hello")    public String sayHello(String name) {        StringBuffer sb = new StringBuffer();        for (int i = 0; i < 1024; i++) {            sb.append(name);        }        return service.sendMessage(sb.toString());    }}

— grpcConfig可以配置多个ServiceStub

@Componentpublic class GrpcConfig {    @GrpcClient(value = "local-grpc-server")    private Channel channel;    @Bean("helloServiceStub")    public HelloServiceGrpc.HelloServiceStub getHelloServiceStub() {        return HelloServiceGrpc.newStub(channel);    }}

service调用grpc

@Service(value = "helloService")public class HelloService{    @Autowired    private  HelloServiceGrpc.HelloServiceStub helloServiceStub;    public String sendMessage(String name) {        //构造一个Request观察者对象，需要的参数是Response的观察者对象，需要重写以下方法        StreamObserver<HelloRequest> helloRequestStreamObservers = helloServiceStub.sayHello(                new StreamObserver<HelloResponse>() {            @Override            public void onNext(HelloResponse value) {                System.out.println("onNext : " + value.getMessage());            }            @Override            public void onError(Throwable t) {                System.out.println(this.getClass().getName() + " onError :" + t.getMessage());            }            @Override            public void onCompleted() {            }        });        for (int i = 0; i < 1024; i++) {            //此处循环使用流的方式发送消息，消息长度是controller里面给过来的，但是重复再发送1024次            //和上面发送的消息总长度一样            helloRequestStreamObservers.onNext(                    HelloRequest.newBuilder()                            .setName(ByteString.copyFrom(name,Charset.forName("UTF-8")))                            .build()            );        }        helloRequestStreamObservers.onCompleted()；         //此处好多人可能疑惑怎么把server端返回的message给前台，后续补充这个        return null;    }}

总结：
看到网上有很多go语言和python写成的demo，有关最大传输上限的改动的，也有用数据流的，也有分块传输的，这篇是本渣根据官方demo http://doc.oschina.net/grpc?t=60134，改出来的，希望大家多多交流指导。结果具体大家可以去测试，但是至少这个是安全的，不会因为下次更大的数据量导致的再次崩溃。
原创帖，转载请注明出处！

原著是一个有趣的人,若有侵权,请通知删除