场景:
基于grpc搭建的微服务,在调用的时候一个批次传输数据量太大导致服务器报如下错误:搭建微服务
https://www.jianshu.com/p/2207011c0164
2019-03-06 12:46:07.544 WARN 2188 --- [-worker-ELG-3-7] io.grpc.netty.NettyServerStream : Exception processing messageio.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: io.grpc.netty.NettyServerStream$TransportState: Frame size 10311685 exceeds maximum: 4194304. at io.grpc.Status.asRuntimeException(Status.java:517) ~[grpc-core-1.10.0.jar:1.10.0] at io.grpc.internal.MessageDeframer.processHeader(MessageDeframer.java:391) ~[grpc-core-1.10.0.jar:1.10.0] at io.grpc.internal.MessageDeframer.deliver(MessageDeframer.java:271) ~[grpc-core-1.10.0.jar:1.10.0] at io.grpc.internal.MessageDeframer.request(MessageDeframer.java:165) ~[grpc-core-1.10.0.jar:1.10.0] at io.grpc.internal.AbstractStream$TransportState.requestMessagesFromDeframer(AbstractStream.java:202) ~[grpc-core-1.10.0.jar:1.10.0] at io.grpc.netty.NettyServerStream$Sink$1.run(NettyServerStream.java:100) [grpc-netty-1.10.0.jar:1.10.0] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:163) [netty-common-4.1.29.Final.jar:4.1.29.Final] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java) [netty-common-4.1.29.Final.jar:4.1.29.Final] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) [netty-common-4.1.29.Final.jar:4.1.29.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:446) [netty-transport-4.1.29.Final.jar:4.1.29.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884) [netty-common-4.1.29.Final.jar:4.1.29.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.29.Final.jar:4.1.29.Final] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_05]
代码复现:
HelloController
@RestControllerpublic class HelloController { @Autowired private HelloService service; @GetMapping("/hello") public String sayHello(String name) { //此处构造超长消息 StringBuffer sb = new StringBuffer(); for (int i = 0; i < 1024*2014; i++) { sb.append(name); } return service.sendMessage(sb.toString()); }}
HelloService
@Service(value = "helloService")public class HelloService extends HelloServiceGrpc.HelloServiceImplBase { @GrpcClient("local-grpc-server") private Channel channel; public String sendMessage(String name) { HelloServiceGrpc.HelloServiceBlockingStub stub = HelloServiceGrpc.newBlockingStub(channel); HelloResponse response = stub.sayHello(HelloRequest.newBuilder().setName(name).build()); return response.getMessage(); }}
服务端
@GrpcService(HelloServiceGrpc.class)public class HelloService extends HelloServiceGrpc.HelloServiceImplBase { @Override public void sayHello(HelloRequest request, StreamObserver<HelloResponse> responseObserver) { String name = request.getName(); System.out.println("received name: "+name); //此处为了不影响页面响应,直接返回一个字符串,不把全部消息返回 HelloResponse response = HelloResponse.newBuilder().setMessage("welcome to gRPC").build(); responseObserver.onNext(response); responseObserver.onCompleted(); }}
调用:
http://localhost:8080/hello?name=world
结果:
出现如片段1的报错信息,接下来对问题进行详细探索并且给出解决方案
探索:
step1:阅读原有日志
io.grpc.netty.NettyServerStream$TransportState: Frame size 10311685 exceeds maximum: 4194304.
稍微翻译一下: 帧长度为10311685,超过最大值 4194304
step2:grpc官方状态码解释:


那么我们的错误就很明显了:
RESOURCE_EXHAUSTED...并且运行时提供有额外的错误详情,表示耗尽资源是带宽那么通过上面各种巴拉巴拉一大堆,说人话,就是我们的消息字符串超常了,最大4m,我们传过去10m,报错了
找解决方案:
思路整理:
1:通过调整最大传输上限参数
客户端具体参数查看:
net.devh.springboot.autoconfigure.grpc.client.GrpcChannelProperties
类,其中有以下参数:
@Datapublic class GrpcChannelProperties { public static final String DEFAULT_HOST = "127.0.0.1"; public static final Integer DEFAULT_PORT = 9090; public static final GrpcChannelProperties DEFAULT = new GrpcChannelProperties(); private List<String> host = new ArrayList<String>() { private static final long serialVersionUID = -8367871342050560040L; { add(DEFAULT_HOST); } }; private List<Integer> port = new ArrayList<Integer>() { private static final long serialVersionUID = 4705083089654936515L; { add(DEFAULT_PORT); } }; private boolean plaintext = true; private boolean enableKeepAlive = false; private boolean keepAliveWithoutCalls = false; private long keepAliveTime = 180; private long keepAliveTimeout = 20; private int maxInboundMessageSize;}
#客户端参数优化grpc.client.local-grpc-server.maxInBoundMessageSize=20971520
服务端参数查看:
net.devh.springboot.autoconfigure.grpc.server.GrpcServerProperties
参数如下:
@Data@ConfigurationProperties("grpc.server")public class GrpcServerProperties { private int port = 9090; private String address = "0.0.0.0"; private int maxMessageSize; private final Security security = new Security(); @Data public static class Security { private Boolean enabled = false; private String certificateChainPath = ""; private String certificatePath = ""; }}
#服务端参数优化grpc.server.max-message-size=20971520
产生结果:

`能够正常返回值``但是会导致响应变慢,资源损耗较大``有时候无法确定最大传输量级,不能正确给设置正确的参数,需要反复增加数字`
2:grpc有流式传输,stream消息进行此方法前,为了验证,请注掉上面的参数优化步骤!
# grpc.server.max-message-size=20971520
消息定义:
service HelloService { rpc SayHello (stream HelloRequest) returns (HelloResponse) {}}message HelloRequest { bytes name = 1;}message HelloResponse { string code = 1; string message = 2;}
服务端代码:
@GrpcService(HelloServiceGrpc.class)public class HelloService extends HelloServiceGrpc.HelloServiceImplBase { private static final Logger logger = LoggerFactory.getLogger(HelloService.class); @Override public StreamObserver<HelloRequest> sayHello(StreamObserver<HelloResponse> responseObserver) { return new StreamObserver<HelloRequest>() { @Override public void onNext(HelloRequest value) { String name = value.getName().toStringUtf8(); logger.info("received name :" + name); } @Override public void onError(Throwable t) { logger.warn("throw an error :", t); } @Override public void onCompleted() { responseObserver.onNext(HelloResponse.newBuilder().setMessage("welcome to gRPC").build()); responseObserver.onCompleted(); } }; }}
客户端调用:
-- controller
@RestControllerpublic class HelloController { @Autowired private HelloService service; @GetMapping("/hello") public String sayHello(String name) { StringBuffer sb = new StringBuffer(); for (int i = 0; i < 1024; i++) { sb.append(name); } return service.sendMessage(sb.toString()); }}
-- grpcConfig可以配置多个ServiceStub
@Componentpublic class GrpcConfig { @GrpcClient(value = "local-grpc-server") private Channel channel; @Bean("helloServiceStub") public HelloServiceGrpc.HelloServiceStub getHelloServiceStub() { return HelloServiceGrpc.newStub(channel); }}
service调用grpc
@Service(value = "helloService")public class HelloService{ @Autowired private HelloServiceGrpc.HelloServiceStub helloServiceStub; public String sendMessage(String name) { //构造一个Request观察者对象,需要的参数是Response的观察者对象,需要重写以下方法 StreamObserver<HelloRequest> helloRequestStreamObservers = helloServiceStub.sayHello( new StreamObserver<HelloResponse>() { @Override public void onNext(HelloResponse value) { System.out.println("onNext : " + value.getMessage()); } @Override public void onError(Throwable t) { System.out.println(this.getClass().getName() + " onError :" + t.getMessage()); } @Override public void onCompleted() { } }); for (int i = 0; i < 1024; i++) { //此处循环使用流的方式发送消息,消息长度是controller里面给过来的,但是重复再发送1024次 //和上面发送的消息总长度一样 helloRequestStreamObservers.onNext( HelloRequest.newBuilder() .setName(ByteString.copyFrom(name,Charset.forName("UTF-8"))) .build() ); } helloRequestStreamObservers.onCompleted(); //此处好多人可能疑惑怎么把server端返回的message给前台,后续补充这个 return null; }}
总结:
看到网上有很多go语言和python写成的demo,有关最大传输上限的改动的,也有用数据流的,也有分块传输的,这篇是本渣根据官方demo http://doc.oschina.net/grpc?t=60134,改出来的,希望大家多多交流指导。结果具体大家可以去测试,但是至少这个是安全的,不会因为下次更大的数据量导致的再次崩溃。原创帖,转载请注明出处!
原著是一个有趣的人,若有侵权,请通知删除
还没有人抢沙发呢~