资讯专栏INFORMATION COLUMN

聊聊jdk httpclient的retry参数

ityouknow / 922人阅读

摘要:序本文主要研究一下的参数这里有一个类型的变量,用来记录请求次数另外还有一个,读取的是值,读取不到默认取,为进入该方法的时候,调用,递增请求次数,然后判断有无超出限制,有则返回带有异常的,即通过返回如果没有超出限制,但是执行请求失败,则

本文主要研究一下jdk httpclient的retry参数

DEFAULT_MAX_ATTEMPTS

java.net.http/jdk/internal/net/http/MultiExchange.java

class MultiExchange {

    static final Logger debug =
            Utils.getDebugLogger("MultiExchange"::toString, Utils.DEBUG);

    private final HttpRequest userRequest; // the user request
    private final HttpRequestImpl request; // a copy of the user request
    final AccessControlContext acc;
    final HttpClientImpl client;
    final HttpResponse.BodyHandler responseHandler;
    final HttpClientImpl.DelegatingExecutor executor;
    final AtomicInteger attempts = new AtomicInteger();
    HttpRequestImpl currentreq; // used for retries & redirect
    HttpRequestImpl previousreq; // used for retries & redirect
    Exchange exchange; // the current exchange
    Exchange previous;
    volatile Throwable retryCause;
    volatile boolean expiredOnce;
    volatile HttpResponse response = null;

    // Maximum number of times a request will be retried/redirected
    // for any reason

    static final int DEFAULT_MAX_ATTEMPTS = 5;
    static final int max_attempts = Utils.getIntegerNetProperty(
            "jdk.httpclient.redirects.retrylimit", DEFAULT_MAX_ATTEMPTS
    );

    //......

}

这里有一个AtomicInteger类型的attempts变量,用来记录请求次数

另外还有一个max_attempts,读取的是jdk.httpclient.redirects.retrylimit值,读取不到默认取DEFAULT_MAX_ATTEMPTS,为5

MultiExchange.responseAsyncImpl

java.net.http/jdk/internal/net/http/MultiExchange.java

    private CompletableFuture responseAsyncImpl() {
        CompletableFuture cf;
        if (attempts.incrementAndGet() > max_attempts) {
            cf = failedFuture(new IOException("Too many retries", retryCause));
        } else {
            if (currentreq.timeout().isPresent()) {
                responseTimerEvent = ResponseTimerEvent.of(this);
                client.registerTimer(responseTimerEvent);
            }
            try {
                // 1. apply request filters
                // if currentreq == previousreq the filters have already
                // been applied once. Applying them a second time might
                // cause some headers values to be added twice: for
                // instance, the same cookie might be added again.
                if (currentreq != previousreq) {
                    requestFilters(currentreq);
                }
            } catch (IOException e) {
                return failedFuture(e);
            }
            Exchange exch = getExchange();
            // 2. get response
            cf = exch.responseAsync()
                     .thenCompose((Response response) -> {
                        HttpRequestImpl newrequest;
                        try {
                            // 3. apply response filters
                            newrequest = responseFilters(response);
                        } catch (IOException e) {
                            return failedFuture(e);
                        }
                        // 4. check filter result and repeat or continue
                        if (newrequest == null) {
                            if (attempts.get() > 1) {
                                Log.logError("Succeeded on attempt: " + attempts);
                            }
                            return completedFuture(response);
                        } else {
                            this.response =
                                new HttpResponseImpl<>(currentreq, response, this.response, null, exch);
                            Exchange oldExch = exch;
                            return exch.ignoreBody().handle((r,t) -> {
                                previousreq = currentreq;
                                currentreq = newrequest;
                                expiredOnce = false;
                                setExchange(new Exchange<>(currentreq, this, acc));
                                return responseAsyncImpl();
                            }).thenCompose(Function.identity());
                        } })
                     .handle((response, ex) -> {
                        // 5. handle errors and cancel any timer set
                        cancelTimer();
                        if (ex == null) {
                            assert response != null;
                            return completedFuture(response);
                        }
                        // all exceptions thrown are handled here
                        CompletableFuture errorCF = getExceptionalCF(ex);
                        if (errorCF == null) {
                            return responseAsyncImpl();
                        } else {
                            return errorCF;
                        } })
                     .thenCompose(Function.identity());
        }
        return cf;
    }

进入该方法的时候,调用attempts.incrementAndGet(),递增请求次数,然后判断有无超出限制,有则返回带有new IOException("Too many retries", retryCause)异常的failedFuture,即通过CompletableFuture.completeExceptionally返回

如果没有超出限制,但是执行请求失败,则调用getExceptionalCF来判断是否应该重试,如果返回null,则重试,通过再次调用responseAsyncImpl,通过这种递归调用完成重试逻辑

MultiExchange.getExceptionalCF

java.net.http/jdk/internal/net/http/MultiExchange.java

    /**
     * Takes a Throwable and returns a suitable CompletableFuture that is
     * completed exceptionally, or null.
     */
    private CompletableFuture getExceptionalCF(Throwable t) {
        if ((t instanceof CompletionException) || (t instanceof ExecutionException)) {
            if (t.getCause() != null) {
                t = t.getCause();
            }
        }
        if (cancelled && t instanceof IOException) {
            if (!(t instanceof HttpTimeoutException)) {
                t = toTimeoutException((IOException)t);
            }
        } else if (retryOnFailure(t)) {
            Throwable cause = retryCause(t);

            if (!(t instanceof ConnectException)) {
                if (!canRetryRequest(currentreq)) {
                    return failedFuture(cause); // fails with original cause
                }
            }

            // allow the retry mechanism to do its work
            retryCause = cause;
            if (!expiredOnce) {
                if (debug.on())
                    debug.log(t.getClass().getSimpleName() + " (async): retrying...", t);
                expiredOnce = true;
                // The connection was abruptly closed.
                // We return null to retry the same request a second time.
                // The request filters have already been applied to the
                // currentreq, so we set previousreq = currentreq to
                // prevent them from being applied again.
                previousreq = currentreq;
                return null;
            } else {
                if (debug.on()) {
                    debug.log(t.getClass().getSimpleName()
                            + " (async): already retried once.", t);
                }
                t = cause;
            }
        }
        return failedFuture(t);
    }

    private boolean retryOnFailure(Throwable t) {
        return t instanceof ConnectionExpiredException
                || (RETRY_CONNECT && (t instanceof ConnectException));
    }

    /** Returns true if the given request can be automatically retried. */
    private static boolean canRetryRequest(HttpRequest request) {
        if (RETRY_ALWAYS)
            return true;
        if (isIdempotentRequest(request))
            return true;
        return false;
    }

    /** Returns true is given request has an idempotent method. */
    private static boolean isIdempotentRequest(HttpRequest request) {
        String method = request.method();
        switch (method) {
            case "GET" :
            case "HEAD" :
                return true;
            default :
                return false;
        }
    }

    private Throwable retryCause(Throwable t) {
        Throwable cause = t instanceof ConnectionExpiredException ? t.getCause() : t;
        return cause == null ? t : cause;
    }

    /** True if ALL ( even non-idempotent ) requests can be automatic retried. */
    private static final boolean RETRY_ALWAYS = retryPostValue();
    /** True if ConnectException should cause a retry. Enabled by default */
    private static final boolean RETRY_CONNECT = retryConnect();

    private static boolean retryPostValue() {
        String s = Utils.getNetProperty("jdk.httpclient.enableAllMethodRetry");
        if (s == null)
            return false;
        return s.isEmpty() ? true : Boolean.parseBoolean(s);
    }

    private static boolean retryConnect() {
        String s = Utils.getNetProperty("jdk.httpclient.disableRetryConnect");
        if (s == null)
            return false;
        return s.isEmpty() ? true : Boolean.parseBoolean(s);
    }
如果cancelled为true且是IOException则直接返回,否则先判断retryOnFailure再判断canRetryRequest(如果不是ConnectException才走canRetryRequest这个判断)

retryOnFailure方法判断如果是ConnectionExpiredException或者是ConnectException且开启retryConnect,则返回true

RETRY_CONNECT读取的是jdk.httpclient.disableRetryConnect参数,如果值为null,则方法返回false,即不进行retryConnect

canRetryRequest首先判断RETRY_ALWAYS,在判断isIdempotentRequest(GET、HEAD方法才重试),都不是则返回false

RETRY_ALWAYS读取的是jdk.httpclient.enableAllMethodRetry,如果值为null,则方法返回false,即不进行retryPostValue

如果该重试的话,则返回null,responseAsyncImpl里头在getExceptionalCF返回null的时候,重新调用了一次responseAsyncImpl,通过递归调用来完成重试逻辑

NetProperties

java.base/sun/net/NetProperties.java

public class NetProperties {
    private static Properties props = new Properties();
    static {
        AccessController.doPrivileged(
            new PrivilegedAction() {
                public Void run() {
                    loadDefaultProperties();
                    return null;
                }});
    }

    private NetProperties() { };


    /*
     * Loads the default networking system properties
     * the file is in jre/lib/net.properties
     */
    private static void loadDefaultProperties() {
        String fname = StaticProperty.javaHome();
        if (fname == null) {
            throw new Error("Can"t find java.home ??");
        }
        try {
            File f = new File(fname, "conf");
            f = new File(f, "net.properties");
            fname = f.getCanonicalPath();
            InputStream in = new FileInputStream(fname);
            BufferedInputStream bin = new BufferedInputStream(in);
            props.load(bin);
            bin.close();
        } catch (Exception e) {
            // Do nothing. We couldn"t find or access the file
            // so we won"t have default properties...
        }
    }

    /**
     * Get a networking system property. If no system property was defined
     * returns the default value, if it exists, otherwise returns
     * null.
     * @param      key  the property name.
     * @throws  SecurityException  if a security manager exists and its
     *          checkPropertiesAccess method doesn"t allow access
     *          to the system properties.
     * @return the String value for the property,
     *         or null
     */
    public static String get(String key) {
        String def = props.getProperty(key);
        try {
            return System.getProperty(key, def);
        } catch (IllegalArgumentException e) {
        } catch (NullPointerException e) {
        }
        return null;
    }

    /**
     * Get an Integer networking system property. If no system property was
     * defined returns the default value, if it exists, otherwise returns
     * null.
     * @param   key     the property name.
     * @param   defval  the default value to use if the property is not found
     * @throws  SecurityException  if a security manager exists and its
     *          checkPropertiesAccess method doesn"t allow access
     *          to the system properties.
     * @return the Integer value for the property,
     *         or null
     */
    public static Integer getInteger(String key, int defval) {
        String val = null;

        try {
            val = System.getProperty(key, props.getProperty(key));
        } catch (IllegalArgumentException e) {
        } catch (NullPointerException e) {
        }

        if (val != null) {
            try {
                return Integer.decode(val);
            } catch (NumberFormatException ex) {
            }
        }
        return defval;
    }

    /**
     * Get a Boolean networking system property. If no system property was
     * defined returns the default value, if it exists, otherwise returns
     * null.
     * @param   key     the property name.
     * @throws  SecurityException  if a security manager exists and its
     *          checkPropertiesAccess method doesn"t allow access
     *          to the system properties.
     * @return the Boolean value for the property,
     *         or null
     */
    public static Boolean getBoolean(String key) {
        String val = null;

        try {
            val = System.getProperty(key, props.getProperty(key));
        } catch (IllegalArgumentException e) {
        } catch (NullPointerException e) {
        }

        if (val != null) {
            try {
                return Boolean.valueOf(val);
            } catch (NumberFormatException ex) {
            }
        }
        return null;
    }

}

这里通过loadDefaultProperties先加载默认配置,读取的是JAVA_HOME/conf/net.properties文件

然后getString、getInteger、getBoolean方法采用的是System.getProperty来读取,而net.properties值仅仅作为System.getProperty的defaultValue

因此要设置httpclient相关参数,只需要通过System.setProperty或者-D来设置即可

net.properties

/Library/java/JavaVirtualMachines/jdk-11.jdk/Contents/home/conf/net.properties

java.net.useSystemProxies=false
http.nonProxyHosts=localhost|127.*|[::1]
ftp.nonProxyHosts=localhost|127.*|[::1]
jdk.http.auth.tunneling.disabledSchemes=Basic
net.properties文件默认设置了如上四个参数
相关异常 HttpTimeoutException

java.net.http/java/net/http/HttpTimeoutException.java

/**
 * Thrown when a response is not received within a specified time period.
 *
 * @since 11
 */
public class HttpTimeoutException extends IOException {

    private static final long serialVersionUID = 981344271622632951L;

    /**
     * Constructs an {@code HttpTimeoutException} with the given detail message.
     *
     * @param message
     *        The detail message; can be {@code null}
     */
    public HttpTimeoutException(String message) {
        super(message);
    }
}

属于java.net.http包,继承至IOException

如果设置了request的timeout,则注册ResponseTimerEvent,在超时时抛出HttpTimeoutException: request timed out,同时设置MultiExchange的cancelled为true

这类由于客户端设置超时引起的HttpTimeoutException,不会进行重试,即使开启相关重试参数

如果这个时间设置得太短,则在connect的时候就超时了,这个时候会抛出HttpConnectTimeoutException,而非HttpTimeoutException: request timed out

HttpConnectTimeoutException

java.net.http/java/net/http/HttpConnectTimeoutException.java

/**
 * Thrown when a connection, over which an {@code HttpRequest} is intended to be
 * sent, is not successfully established within a specified time period.
 *
 * @since 11
 */
public class HttpConnectTimeoutException extends HttpTimeoutException {

    private static final long serialVersionUID = 321L + 11L;

    /**
     * Constructs an {@code HttpConnectTimeoutException} with the given detail
     * message.
     *
     * @param message
     *        The detail message; can be {@code null}
     */
    public HttpConnectTimeoutException(String message) {
        super(message);
    }
}

属于java.net.http包,继承至HttpTimeoutException

如果设置了client的connectTimeout,则会注册ConnectTimerEvent,在超时时抛出ConnectException("HTTP connect timed out"),同时设置MultiExchange的cancelled为true,这个在MultiExchange.getExceptionalCF方法里头会被包装为HttpConnectTimeoutException

ConnectionExpiredException

java.net.http/jdk/internal/net/http/common/ConnectionExpiredException.java

/**
 * Signals that an end of file or end of stream has been reached
 * unexpectedly before any protocol specific data has been received.
 */
public final class ConnectionExpiredException extends IOException {
    private static final long serialVersionUID = 0;

    /**
     * Constructs a {@code ConnectionExpiredException} with a detail message of
     * "subscription is finished" and the given cause.
     *
     * @param   cause the throwable cause
     */
    public ConnectionExpiredException(Throwable cause) {
        super("subscription is finished", cause);
    }
}

一般是在read error的时候触发,比如EOFException,IOException("connection reset by peer),或者SSLHandshakeException

小结

jdk httpclient的retry参数涉及到的参数如下:

jdk.httpclient.redirects.retrylimit(默认为5,用来控制重试次数,不过实际上还有expiredOnce参数,看代码貌似顶多重试一次)

jdk.httpclient.disableRetryConnect(默认为null,即RETRY_CONNECT为false,不在ConnectException的时候retry)

jdk.httpclient.enableAllMethodRetry(默认为null,即RETRY_ALWAYS为false,即需要判断请求方法是否幂等来决定是否重试)

是否重试的判断逻辑如下:

如果重试次数超过限制,则返回失败,否则往下

如果cancelled为true(这里如果request设置了timeout,触发时cancelled设置为true)且是IOException(例如设置了连接超时抛出的HttpConnectTimeoutException),则不走重试逻辑;否则往下

如果retryOnFailure(ConnectionExpiredException,或者ConnectException且开启retryConnect),则往下

如果是异常不是ConnectException,则还额外判断canRetryRequest(判断该请求类型是否允许重试),满足则往下

如果expiredOnce为false,则返回null,即满足重试条件,走递归重试

doc

HttpClient javadoc

文章版权归作者所有,未经允许请勿转载,若此文章存在违规行为,您可以联系管理员删除。

转载请注明本文地址:https://www.ucloud.cn/yun/77355.html

相关文章

  • [case39]聊聊jdk httpclientexecutor

    摘要:序本文主要研究一下的这里如果的为,则会创建这里如果是的话,参数传递的是如果是同步的方法,则传的值是这里创建了一个,然后调用,这里使用了可以看到这里使用的是的方法注意这个方法是才有的,也是在这里使用的由于默认是使用创建的, 序 本文主要研究一下jdk httpclient的executor HttpClientImpl java.net.http/jdk/internal/net/htt...

    dabai 评论0 收藏0
  • 聊聊jdk httpclientconnect timeout异常

    摘要:序本文主要研究一下的异常实例代码异常日志如下最后调用这里调用获取连接如果没有连接会新创建一个,走的是这里先是调用了获取连接,然后调用进行连接这里委托给这里如果有设置的话,则会创建一个调用进行连接,如果连接未 序 本文主要研究一下httpclient的connect timeout异常 实例代码 @Test public void testConnectTimeout()...

    张利勇 评论0 收藏0
  • 聊聊jdk httpclientConnectionPool

    摘要:调用计算的时间,这个方法会清理移除并过期的连接除了清理过期的连接外,还通过间接触发,去清理关闭或异常的连接 序 本文主要研究一下jdk httpclient的ConnectionPool HttpConnection HttpConnection.getConnection java.net.http/jdk/internal/net/http/HttpConnection.java ...

    Worktile 评论0 收藏0
  • RestTemplate集成Ribbbon

    摘要:的类图如下主要根据创建扩展了,创建拦截的,这里会设置拦截器,这是集成的核心,当发起请求调用的时候,会先经过拦截器,然后才真正发起请求。和是配合使用的,最大重试次数是针对每一个的,如果设置,这样触发最大重试次数就是次。 上一篇文章我们分析了ribbon的核心原理,接下来我们来看看springcloud是如何集成ribbon的,不同的springcloud的组件(feign,zuul,Re...

    wall2flower 评论0 收藏0
  • spring-cloud-feign源码深度解析

    摘要:内部使用了的动态代理为目标接口生成了一个动态代理类,这里会生成一个动态代理原理统一的方法拦截器,同时为接口的每个方法生成一个拦截器,并解析方法上的元数据,生成一个请求模板。的核心源码解析到此结束了,不知道是否对您有无帮助,可留言跟我交流。 Feign是一个声明式的Web服务客户端。这使得Web服务客户端的写入更加方便 要使用Feign创建一个界面并对其进行注释。它具有可插拔注释支持,包...

    vibiu 评论0 收藏0

发表评论

0条评论

ityouknow

|高级讲师

TA的文章

阅读更多
最新活动
阅读需要支付1元查看
<