1.请求防重
同一请求路径下，短时间内出现相同参数的请求操作。某些重复变更请求如果编码不严谨会导致底层数据和逻辑出现错误。
相同参数的请求直接在最外层做 防重拦截。
解决方案：
- 请求参数唯一标识，前端生成，后端拦截判断一定时间范围内重复拦截
- 重复请求后端拦截，后端解析请求参数，生成唯一码，拦截判断重复
2.业务防重
业务数据重复变更、重复插入，导致业务错误。
解决方案：
- 数据库唯一索引 （数据防重）
- 状态机进行防重（数据防重）
- 数据版本管理 、乐观锁。在变更操作时携带 数据版本号，版本号匹配时才能修改目标数据（数据防重）
- 加锁，同一时间资源访问限制 （逻辑防重）
项目实践：
请求防重和业务防重都需要做。请求防重可以做成通用逻辑or组件抽取出来。这里只介绍请求防重的一个具体实现。
整体大致逻辑 ：
1、后端读取所有请求参数
2、使用hash算法将请求参数和路径 hash后生成 请求唯一码
3、利用redis的 单线程执行命令的机制，使用 setNx 判断 是否存在 相同key，不存在就插入
实现碰到的问题
如何拦截目标请求？
因为项目使用 spring ，直接使用 扩展点 HandlerInterceptor ，直接使用 preHandle 扩展点进行拦截判断 方法上是否标注 @RepeatRequest 注解。
注解
import org.springframework.core.annotation.AliasFor;
import java.lang.annotation.*;
@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
public @interface RepeatRequest {
@AliasFor("interval")
long value() default 1000L;
/**
* 重复请求的判定间隔时间 不能 <= 0 ，否则失效
*/
@AliasFor("value")
long interval() default 1000L;
/**
* 判定 重复请求后的提示信息
*
* @return
*/
String message() default "请勿重复提交,请稍后再试";
}拦截器
public interface HandlerInterceptor {
/**
* 主要拦截逻辑
*
**/
default boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
return true;
}
default void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, @Nullable ModelAndView modelAndView) throws Exception {
}
default void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, @Nullable Exception ex) throws Exception {
}
}具体列子
/**
* @author: yu
* @description: 防重拦截 短时间内 重复提交 重复请求
* @create: 2023-07-18 16:02
**/
public class RepeatSubmitInterceptor implements HandlerInterceptor {
private RedisService redisService;
static final String REPEAT_SUBMIT_KEY = "repeat_submit_key:%s_%s";
private static final Logger logger = LoggerFactory.getLogger(RepeatSubmitInterceptor.class);
public RepeatSubmitInterceptor(RedisService redisService) {
this.redisService = redisService;
}
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
if (!(handler instanceof HandlerMethod)) {
return true;
}
HandlerMethod handlerMethod = (HandlerMethod) handler;
Method method = handlerMethod.getMethod();
RepeatRequest annotation = AnnotationUtils.getAnnotation(method, RepeatRequest.class);
if (annotation == null) {
return true;
}
long interval = annotation.interval();
if (interval <= 0L) {
return true;
}
String message = annotation.message();
String controllerMethodPath = handlerMethod.toString();
// 这里使用了相对路径 ，没有用绝对路径，参数相同的不同调用方也会拦住
String requestURI = request.getRequestURI();
Map<String, String[]> parameterMap = request.getParameterMap();
String messagePayload = getMessagePayload(request);
String paramsMap = JSONUtil.toJsonStr(parameterMap);
String appName = SpringUtil.getApplicationContext().getEnvironment().getProperty("app.name");
String redisKey = String.format(REPEAT_SUBMIT_KEY, appName, controllerMethodPath);
String value = requestURI + paramsMap + messagePayload;
String encodeValue = Hashing.md5().hashString(value, StandardCharsets.UTF_8).toString();
boolean exist = false;
try {
// 后续 节省空间 看以替换成 bitmap
exist = !redisService.setNx(redisKey + encodeValue, encodeValue, interval, TimeUnit.MILLISECONDS);
} catch (Exception e) {
// redis 不可用，不阻塞主流程 ，只打印日志
logger.error("redis 异常", e);
}
if (exist) {
logger.info("重复请求 rediskey:{},value ：{}", redisKey+encodeValue, value);
throw new BaseRunTimeException(BaseExceptionEnum.SYS_ERROR.getCode(), message);
}
logger.info("paramsMap:{},messagePayload:{}", paramsMap, messagePayload);
return true;
}
/**
* 1.处理注解信息
*
* 1.取出所有参数 和 url
* 2.拼接成 唯一码 ，存入redis
* 3.判断是否存在
*/
protected String getMessagePayload(HttpServletRequest request) {
KdmpContentCachingRequestWrapper wrapper = WebUtils.getNativeRequest(request, KdmpContentCachingRequestWrapper.class);
if (wrapper == null) {
return null;
}
byte[] buf = wrapper.getContentAsByteArray();
if (buf.length > 0) {
int length = buf.length;
try {
return new String(buf, 0, length, wrapper.getCharacterEncoding());
} catch (UnsupportedEncodingException ex) {
return "[unknown]";
}
}
return null;
}
}
后端如何读取所有请求参数 ？
request对象中的 body 数据不支持重复读取 ，所以需要首先实现可以重复读取body的 request 对象。
使用 HttpServletRequestWrapper 的功能，同时参考 spring-web utils包中 ContentCachingRequestWrapper 的实现 。主要 缓存了流的内容，实现可重复读取的流。
实现 MyContentCachingRequestWrapper
import org.apache.commons.io.IOUtils;
import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.lang.Nullable;
import org.springframework.web.util.ContentCachingResponseWrapper;
import org.springframework.web.util.WebUtils;
import javax.servlet.ReadListener;
import javax.servlet.ServletInputStream;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletRequestWrapper;
import java.io.*;
import java.net.URLEncoder;
import java.util.*;
/**
* {@link javax.servlet.http.HttpServletRequest} wrapper that caches all content read from
* the {@linkplain #getInputStream() input stream} and {@linkplain #getReader() reader},
* and allows this content to be retrieved via a {@link #getContentAsByteArray() byte array}.
*
* <p>This class acts as an interceptor that only caches content as it is being
* read but otherwise does not cause content to be read. That means if the request
* content is not consumed, then the content is not cached, and cannot be
* retrieved via {@link #getContentAsByteArray()}.
*
* <p>Used e.g. by {@link org.springframework.web.filter.AbstractRequestLoggingFilter}.
* Note: As of Spring Framework 5.0, this wrapper is built on the Servlet 3.1 API.
*
* @author Juergen Hoeller
* @author Brian Clozel
* @see ContentCachingResponseWrapper
* <p>
* 重写下，可以重复读取流
* @since 4.1.3
*/
public class MyContentCachingRequestWrapper extends HttpServletRequestWrapper {
private final ByteArrayOutputStream cachedContent;
@Nullable
private ServletInputStream inputStream;
@Nullable
private BufferedReader reader;
/**
* Create a new ContentCachingRequestWrapper for the given servlet request.
*
* @param request the original servlet request
*/
public MyContentCachingRequestWrapper(HttpServletRequest request) {
super(request);
int contentLength = request.getContentLength();
this.cachedContent = new ByteArrayOutputStream(contentLength >= 0 ? contentLength : 1024);
// 直接缓存流
try {
if (isFormBody()) {
ServletInputStream inputStream = this.getRequest().getInputStream();
IOUtils.copy(inputStream, this.cachedContent);
}
} catch (IOException e) {
throw new RuntimeException(e);
}
}
@Override
public ServletInputStream getInputStream() throws IOException {
// 每次都是一个新流
this.inputStream = new ContentCachingInputStream(cachedContent.toByteArray());
return this.inputStream;
}
@Override
public String getCharacterEncoding() {
String enc = super.getCharacterEncoding();
return (enc != null ? enc : WebUtils.DEFAULT_CHARACTER_ENCODING);
}
@Override
public BufferedReader getReader() throws IOException {
if (this.reader == null) {
this.reader = new BufferedReader(new InputStreamReader(getInputStream(), getCharacterEncoding()));
}
return this.reader;
}
@Override
public String getParameter(String name) {
return super.getParameter(name);
}
@Override
public Map<String, String[]> getParameterMap() {
return super.getParameterMap();
}
@Override
public Enumeration<String> getParameterNames() {
return super.getParameterNames();
}
@Override
public String[] getParameterValues(String name) {
return super.getParameterValues(name);
}
private boolean isFormBody() {
String contentType = getContentType();
return (contentType != null &&
(contentType.contains(MediaType.APPLICATION_JSON_VALUE)
|| contentType.contains(MediaType.APPLICATION_XML_VALUE)
)
);
}
private boolean isFormPost() {
String contentType = getContentType();
return (contentType != null && (contentType.contains(MediaType.APPLICATION_FORM_URLENCODED_VALUE) || contentType.contains(MediaType.MULTIPART_FORM_DATA_VALUE)) &&
(HttpMethod.POST.matches(getMethod())
|| HttpMethod.GET.matches(getMethod())
|| HttpMethod.PUT.matches(getMethod())
|| HttpMethod.DELETE.matches(getMethod())
|| HttpMethod.HEAD.matches(getMethod())
|| HttpMethod.PATCH.matches(getMethod())
)
);
}
/**
* Return the cached request content as a byte array.
* <p>The returned array will never be larger than the content cache limit.
* <p><strong>Note:</strong> The byte array returned from this method
* reflects the amount of content that has has been read at the time when it
* is called. If the application does not read the content, this method
* returns an empty array.
*
* @see #ContentCachingRequestWrapper(HttpServletRequest, int)
*/
public byte[] getContentAsByteArray() {
return this.cachedContent.toByteArray();
}
/**
* Template method for handling a content overflow: specifically, a request
* body being read that exceeds the specified content cache limit.
* <p>The default implementation is empty. Subclasses may override this to
* throw a payload-too-large exception or the like.
*
* @param contentCacheLimit the maximum number of bytes to cache per request
* which has just been exceeded
* @see #ContentCachingRequestWrapper(HttpServletRequest, int)
* @since 4.3.6
*/
protected void handleContentOverflow(int contentCacheLimit) {
}
private class ContentCachingInputStream extends ServletInputStream {
private final ByteArrayInputStream is;
private boolean overflow = false;
public ContentCachingInputStream(byte[] content) {
this.is = new ByteArrayInputStream(content);
}
@Override
public int read() throws IOException {
int ch = this.is.read();
return ch;
}
@Override
public int read(byte[] b) throws IOException {
int count = this.is.read(b);
return count;
}
@Override
public int read(final byte[] b, final int off, final int len) throws IOException {
int count = this.is.read(b, off, len);
return count;
}
@Override
public int readLine(final byte[] b, final int off, final int len) throws IOException {
int count = this.is.read(b, off, len);
return count;
}
@Override
public boolean isFinished() {
return this.is.available() == 0;
}
@Override
public boolean isReady() {
return true;
}
@Override
public void setReadListener(ReadListener readListener) {
}
}
}
注册 filter
public class RequestWrapFilter extends OncePerRequestFilter implements OrderedFilter {
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) throws ServletException, IOException {
HttpServletRequest requestToUse = request;
if (!(request instanceof MyContentCachingRequestWrapper)) {
requestToUse = new MyContentCachingRequestWrapper(request);
}
logger.debug("RequestWrapperFilter success");
filterChain.doFilter(requestToUse, response);
}
@Override
public int getOrder() {
return REQUEST_WRAPPER_FILTER_MAX_ORDER ;
}
}结尾： 上述代码虽然不完整，但是已经包含了 核心逻辑和思想。
迭代想法：组件化可以封装成 springboot -starter 开放调用、代码步骤抽象、判重逻辑抽象支持自定义如 mysql 等其他方式