Bootstrap

企业微信会话内容存档功能实现及常见问题

需求: 应公司监管要求,需要把公司在企业微信的聊天记录下载下来保存到私有服务器.
参考博客:
企业微信官网:https://developer.work.weixin.qq.com/document/path/9136
博客: https://blog.csdn.net/qq_42851002/article/details/119460215
参考demo:https://github.com/hufeiyaya/QyChat

基本思路:
0,登录企业微信管理后台开启企微存档权限,并配置可信任IP和公钥.
1,创建项目并集成企业微信SDK;
2,初始化SDK;
3,调用SDK接口拉取数据并保存加密数据;
4,对数据进行解密并分类保存;
5,拉取媒体文件并保存
//目录结构
在这里插入图片描述

示例代码:

    public void pullWechatMsg() {
        //1,初始化sdk
        //初始化时请填入自己企业的corpid与secrectkey。
        log.info("开始执行初始化sdk操作");
        long ret = 0;
        long sdk = Finance.NewSdk();
        ret = Finance.Init(sdk, corpId, secret);
        if(ret != 0){
            Finance.DestroySdk(sdk);
            log.error("初始化sdk失败,失败消息为: ret = {}" , ret);
            return;
        }
        log.info("初始化sdk成功,开始执行拉取消息操作");

        //2,调用sdk接口拉取消息
        //每次使用GetChatData拉取存档前需要调用NewSlice获取一个slice,在使用完slice中数据后,还需要调用FreeSlice释放。
        long slice = Finance.NewSlice();
        Long latestSeq = encryptMapper.selectLatestSeq();
        if(latestSeq == null) {
            latestSeq = 1L;
        } else {
            latestSeq ++ ;
        }

        proxy = "xxx.xxx.xxx.xxx:8080/qywechat-huihua";
       // proxy = "";    //注意:没有代理时proxy和passwd设置为空即可
        passwd = "";
        ret = Finance.GetChatData(sdk, latestSeq, limit, proxy, passwd, timeout, slice);
        if (ret != 0) {
            log.error("调用sdk拉取消息接口失败,失败消息为 ret = {}" , ret);
            Finance.FreeSlice(slice);
            return;
        }
        String contentResult = Finance.GetContentFromSlice(slice);
        log.info("调用sdk拉取消息接口:seq = {} ,拉取密文结果contentResult = {}" , latestSeq ,contentResult);
        Finance.FreeSlice(slice);
        ChatDataResult chatDataResult = JSON.parseObject(contentResult, ChatDataResult.class);
        List<ChatData> chatDataList = chatDataResult.getChatdata();
        WechatChatdataEncrypt chatdataEncrypt = null;
        for(ChatData chatData : chatDataList) {
            log.info("将加密数据保存到数据库中");
            chatdataEncrypt = new WechatChatdataEncrypt();
            chatdataEncrypt.setId(UUID.randomUUID().toString());
            chatdataEncrypt.setSeq(chatData.getSeq());
            chatdataEncrypt.setMsgid(chatData.getMsgid());
            chatdataEncrypt.setPublickeyVer(chatData.getPublickey_ver());
            chatdataEncrypt.setEncryptRandomKey(chatData.getEncrypt_random_key());
            chatdataEncrypt.setEncryptChatMsg(chatData.getEncrypt_chat_msg());
            encryptMapper.addEncrypeData(chatdataEncrypt);

            log.info("开始执行解密操作");
            //对消息进行解密处理
            String encryptRandomKey = chatData.getEncrypt_random_key();
            String encryptChatMsg = chatData.getEncrypt_chat_msg();

            //每次使用DecryptData解密会话存档前需要调用NewSlice获取一个slice,在使用完slice中数据后,还需要调用FreeSlice释放。
            String encrypt_key = null;
            try {
                encrypt_key = RSAUtil.getEncryptKey(encryptRandomKey);
            } catch (Exception e) {
                e.printStackTrace();
            }
            long msg = Finance.NewSlice();
            ret = Finance.DecryptData(sdk, encrypt_key, encryptChatMsg, msg);
            if (ret != 0) {
              //  System.out.println("调用sdk解密接口失败, 失败消息为: " + ret);
                log.info("调用sdk解密接口失败, 失败消息ret = {}" , ret);
                Finance.FreeSlice(msg);
                return;
            }

            String encryptResult = Finance.GetContentFromSlice(msg);
            log.info("解密后的消息为 encryptResult = {} " , encryptResult);
            JSONObject encryptChatDataJsonObject = JSON.parseObject(encryptResult);
            String msgtype = encryptChatDataJsonObject.getString("msgtype");
            // 将解密后的消息存储到数据库中
            WechatChatdataDecrypt chatdataDecrypt = new WechatChatdataDecrypt();
            if("switch".equals(encryptChatDataJsonObject.getString("action"))) {
                //切换企业日志
                log.info("处理switch 消息"+encryptChatDataJsonObject);
                chatdataDecrypt.setMsgid(encryptChatDataJsonObject.getString("msgid"));
                chatdataDecrypt.setAction(encryptChatDataJsonObject.getString("action"));
                chatdataDecrypt.setFrom(encryptChatDataJsonObject.getString("user"));
                chatdataDecrypt.setMsgtime(encryptChatDataJsonObject.getLong("time"));
                chatdataDecrypt.setMsgtype("switch");
            } else {
                chatdataDecrypt.setId(UUID.randomUUID().toString());
                chatdataDecrypt.setMsgid(encryptChatDataJsonObject.getString("msgid"));
                chatdataDecrypt.setAction(encryptChatDataJsonObject.getString("action"));
                chatdataDecrypt.setFrom(encryptChatDataJsonObject.getString("from"));
                chatdataDecrypt.setToList(encryptChatDataJsonObject.getString("tolist"));
                chatdataDecrypt.setRoomid(encryptChatDataJsonObject.getString("roomid"));
                chatdataDecrypt.setMsgtime(encryptChatDataJsonObject.getLong("msgtime"));
                chatdataDecrypt.setMsgtype(encryptChatDataJsonObject.getString("msgtype"));
                if("text".equals(msgtype) || "revoke".equals(msgtype)
                        || "agree".equals(msgtype) || "disagree".equals(msgtype)
                        || "card".equals(msgtype) || "location".equals(msgtype)
                        || "link".equals(msgtype) || "weapp".equals(msgtype)
                        || "chatrecord".equals(msgtype) || "todo".equals(msgtype)
                        || "vote".equals(msgtype) ||"collect".equals(msgtype)
                        || "redpacket".equals(msgtype) ||"meeting".equals(msgtype)
                        || "calendar".equals(msgtype) || "sphfeed".equals(msgtype)) {
                    //文本,撤回消息,同意会话,名片,位置,链接,小程序,会话记录,待办消息,投票消息,填表消息,红包消息,会议邀请消息,日程消息,视频号消息
                    chatdataDecrypt.setContent(encryptChatDataJsonObject.getString(msgtype));
                } else if("image".equals(msgtype) || "voice".equals(msgtype)
                        || "video".equals(msgtype) || "emotion".equals(msgtype)
                        || "file".equals(msgtype)) {
                    //图片,语音,视频,表情,文件
                    chatdataDecrypt.setContent(encryptChatDataJsonObject.getString(msgtype));

                    log.info("开始执行拉取媒体文件操作");
                    JSONObject fileJsonObject = encryptChatDataJsonObject.getJSONObject(msgtype);

                   // if (saveMediafile(sdk, msgtype, fileJsonObject, sdkfileid, md5sum)) return;
                   saveMediafile(sdk, chatData.getMsgid(), msgtype, fileJsonObject);

                } else if("docmsg".equals(msgtype)){
                    //在线文档消息
                    chatdataDecrypt.setContent(encryptChatDataJsonObject.getString("doc"));
                } else if("markdown".equals(msgtype) || "news".equals(msgtype)
                        || "voiptext".equals(msgtype) || "qydiskfile".equals(msgtype)) {
                    //MarkDown格式消息,图文消息,音视频通话,微盘文件
                    chatdataDecrypt.setContent(encryptChatDataJsonObject.getString("info"));
                } else if("mixed".equals(msgtype)) {
                    chatdataDecrypt.setContent(encryptChatDataJsonObject.getString(msgtype));

                    //混合消息
                    JSONObject mixedObject = encryptChatDataJsonObject.getJSONObject("mixed");
                    JSONArray itemJSONArray = mixedObject.getJSONArray("item");
                    for(int i = 0; i < itemJSONArray.size(); i ++) {
                        JSONObject itemJsonObject = itemJSONArray.getJSONObject(i);
                        String itemType = itemJsonObject.getString("type");
                        if("image".equals(itemType) || "voice".equals(itemType)
                                || "video".equals(itemType) || "emotion".equals(itemType)
                                || "file".equals(itemType)) {
                            //如果混合消息item属于这五种类型,则需要下载对应的媒体文件
                            String contentString = itemJsonObject.getString("content");
                            JSONObject contentJSONObject = JSONObject.parseObject(contentString);
                            String sdkfileid = contentJSONObject.getString("sdkfileid");
                            String md5sum = contentJSONObject.getString("md5sum");
                            // if (saveMediafile(sdk, msgtype, fileJsonObject, sdkfileid, md5sum)) return;
                            saveMediafile(sdk,  chatData.getMsgid(),msgtype, contentJSONObject);

                        }
                    }

                } else if("meeting_voice_call".equals(msgtype)) {
                    // 音频存档消息
                    chatdataDecrypt.setRoomid(encryptChatDataJsonObject.getString("voiceid"));
                    chatdataDecrypt.setContent(encryptChatDataJsonObject.getString(msgtype));
                    JSONObject meetingVoiceCallJSONObject = encryptChatDataJsonObject.getJSONObject("meeting_voice_call");
                    saveMediafile(sdk, chatData.getMsgid(), msgtype, meetingVoiceCallJSONObject);
                } else if("voip_doc_share".equals(msgtype)) {
                    //音频共享文档消息
                    chatdataDecrypt.setRoomid(encryptChatDataJsonObject.getString("voipid"));
                    chatdataDecrypt.setContent(encryptChatDataJsonObject.getString(msgtype));
                    JSONObject voipDocShareJSONObject = encryptChatDataJsonObject.getJSONObject("voip_doc_share");
                    saveMediafile(sdk, chatData.getMsgid(),msgtype, voipDocShareJSONObject);
                } else if ("external_redpacket".equals(msgtype)) {
                    //互通红包消息
                    chatdataDecrypt.setContent(encryptChatDataJsonObject.getString("redpacket"));
                }

            }
            // 将明文消息保存到数据库中
            decrptMapper.addDecrypeData(chatdataDecrypt);
        }

    }

文件存储部分代码:(我们是把文件存储到SDS对象存储桶中)

private boolean saveMediafile(long sdk,String msgId, String msgtype, JSONObject fileJsonObject) {
        String sdkfileid = fileJsonObject.getString("sdkfileid");
        String md5sum = "";
        if("meeting_voice_call".equals(msgtype)) {
            md5sum = Md5Util.getMd5(sdkfileid);
        } else {

            md5sum = fileJsonObject.getString("md5sum");
        }
        long ret;
        String suffix = "";
        switch(msgtype) {
            case "image" :
                suffix = ".jpg";
                break;
            case "voice" :
                suffix = ".amr";
                break;
            case "video" :
                suffix = ".mp4";
                break;
            case "emotion" :
                Integer type = fileJsonObject.getInteger("type");
                if(1 == type) {
                    suffix = ".gif";
                } else if(2 == type) {
                    suffix = ".png";
                }
                break;
            case "file" :
                suffix = "." + fileJsonObject.getString("fileext");
                break;
        }
        String saveFileName = "";
        if("voip_doc_share".equals(msgtype)) {
            saveFileName = fileJsonObject.getString("filename");
        } else {
            saveFileName = md5sum + suffix;
        }
        String savefile = rootPath + saveFileName;
        File savePathFile = new File(savefile);
        if(!savePathFile.getParentFile().exists()) {
            savePathFile.getParentFile().mkdirs();
        }
        String indexbuf = "";
        while(true){
            //每次使用GetMediaData拉取存档前需要调用NewMediaData获取一个media_data,在使用完media_data中数据后,还需要调用FreeMediaData释放。
            long media_data = Finance.NewMediaData();
            ret = Finance.GetMediaData(sdk, indexbuf, sdkfileid, proxy, passwd, timeout, media_data);
            if(ret!=0){
                log.info("调用获取媒体文件接口失败,错误信息为 :" + ret);
                Finance.FreeMediaData(media_data);
                return true;
            }
            log.info("调用获取媒体文件接口返回值信息 outindex len:%d, data_len:%d, is_finis:%d\n",Finance.GetIndexLen(media_data),Finance.GetDataLen(media_data), Finance.IsMediaDataFinish(media_data));
            try {
                //大于512k的文件会分片拉取,此处需要使用追加写,避免后面的分片覆盖之前的数据。
                FileOutputStream outputStream  = new FileOutputStream(new File(savefile), true);
                outputStream.write(Finance.GetData(media_data));
                outputStream.close();
            } catch (Exception e) {
                e.printStackTrace();
            }

            if(Finance.IsMediaDataFinish(media_data) == 1)
            {
                //已经拉取完成最后一个分片
                Finance.FreeMediaData(media_data);
                break;
            }
            else
            {
                //获取下次拉取需要使用的indexbuf
                indexbuf = Finance.GetOutIndexBuf(media_data);
                Finance.FreeMediaData(media_data);
            }
        }

        //将文件存储到EOS,并将信息存储到数据库
        long fileSize = savePathFile.length();
        if(fileSize < fileSingleMaxSize) {
            //小于100M的使用整体上传
            //将文件保存在存储桶中
            XSKYObjectStorageUtils.upladFile(savePathFile,saveFileName);
        } else {
            //大于100M的使用分段上传
            XSKYObjectStorageUtils.multipartUpload(savePathFile,saveFileName);
        }
        //将本地文件删除
        savePathFile.delete();

        //将文件信息保存到数据库中
        WechatMediadata mediadata = new WechatMediadata();
        mediadata.setId(UUID.randomUUID().toString());
        mediadata.setMsgid(msgId);
        mediadata.setSdkfileid(sdkfileid);
        mediadata.setFileName(saveFileName);
        mediadata.setFileType(msgtype);
        mediadata.setFileSize(fileSize);
        mediadata.setAwsKey(saveFileName);
        mediadataMapper.addMediaData(mediadata);

        return true;
    }

SDS工具类:

package cn.com.sinosure.dsp.storage.utils;

import java.io.BufferedInputStream;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;

import javax.servlet.http.HttpServletResponse;

import com.amazonaws.services.s3.model.*;
import com.amazonaws.services.s3.model.lifecycle.LifecycleFilter;
import com.amazonaws.services.s3.model.lifecycle.LifecyclePrefixPredicate;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.util.FileCopyUtils;
import org.springframework.web.multipart.MultipartFile;

import com.amazonaws.AmazonClientException;
import com.amazonaws.AmazonServiceException;
import com.amazonaws.ClientConfiguration;
import com.amazonaws.Protocol;
import com.amazonaws.SdkClientException;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.internal.SSEResultBase;
import com.amazonaws.services.s3.model.DeleteObjectsRequest.KeyVersion;

import cn.com.sinosure.dsp.file.dao.DspFile;
import cn.com.sinosure.dsp.utils.DspException;

/**
 * xsky对象存储API
 */
public class XSKYObjectStroageUtils {

	private static final Logger logger = LoggerFactory.getLogger(XSKYObjectStroageUtils.class);
	private static AmazonS3 client;
	private static String bucketName;

	
	/**
	 * 获取aws实例
	 * 
	 * @return
	 */
	public static void initAmazonS3Client(String akey, String skey, String endpoint, String bucketName) {
		logger.info("获取client实例 bucketName={}", bucketName);
		XSKYObjectStroageUtils.bucketName = bucketName;
		

		// 初始化S3 configure 的实例
		AWSCredentials credentials = new BasicAWSCredentials(akey, skey);
		// 设置连接时的参数
		ClientConfiguration clientConfig = new ClientConfiguration();
		// 设置连接方式为HTTP,可选参数为HTTP和HTTPS
		clientConfig.setProtocol(Protocol.HTTP);
		// 设置use expect continue标志位
		clientConfig.setUseExpectContinue(false);
		clientConfig.setMaxConnections(500);
		try {
			// 设置Endpoint
			AwsClientBuilder.EndpointConfiguration end_point = new AwsClientBuilder.EndpointConfiguration(
					endpoint, "us-east-1");
			// 创建连接,替换原AmazonS3Client接口
			XSKYObjectStroageUtils.client = AmazonS3ClientBuilder.standard()
					.withCredentials(new AWSStaticCredentialsProvider(credentials))
					.withClientConfiguration(clientConfig)
					.withEndpointConfiguration(end_point)
					.withPathStyleAccessEnabled(true).build();

		} catch (AmazonServiceException e) {
			e.printStackTrace();
		} catch (SdkClientException e) {
			e.printStackTrace();
		}
	}
	
	/**
	 * 上传文件
	 * 
	 * @param multipartFile
	 *            文件对象
	 * @param keyName
	 *            新文件名称
	 * @return
	 */
	public static void uploadFile(MultipartFile multipartFile, String keyName) {
		// 上传一个文件作为一个对象
		// Specify server-side encryption.
		InputStream inputStream = null ;
		try {
			inputStream = multipartFile.getInputStream();
			PutObjectRequest putRequest = new PutObjectRequest(bucketName, keyName,
					inputStream, new ObjectMetadata());

			putRequest.withCannedAcl(CannedAccessControlList.BucketOwnerFullControl);

			PutObjectResult putResult = client.putObject(putRequest);

			logger.info("上传文件eTag:" + putResult.getETag() + " contentMD5:" + putResult.getContentMd5());
		} catch (AmazonServiceException ase) {
			logger.error("Caught an AmazonServiceException when put object_acl of \"" + keyName + "\".");
			logger.error("Error Message: " + ase.getMessage() + "  AWS Error Code:   " + ase.getErrorCode()
					+ " Error Type:       " + ase.getErrorType());
			throw new AmazonServiceException("对象存储保存异常!");
		} catch (Exception e) {
			e.printStackTrace();
			throw new DspException("文件上传失败!");
		}finally {
			if(inputStream!=null){
				try {
					inputStream.close();
				} catch (IOException e) {

				}
			}
		}

	}

	/**
	 * 上传文件
	 * 
	 * @param file
	 *            文件对象
	 * @param keyName
	 *            新文件名称
	 * @return
	 */
	public static void uploadFile(File file, String keyName) {
		try {
			
			PutObjectRequest putRequest = new PutObjectRequest(bucketName, keyName, file);
			putRequest.withCannedAcl(CannedAccessControlList.BucketOwnerFullControl);
			PutObjectResult putResult = client.putObject(putRequest);

			logger.info("上传文件eTag:" + putResult.getETag() + " contentMD5:" + putResult.getContentMd5());

		} catch (AmazonServiceException ase) {
			logger.error("Caught an AmazonServiceException when put object_acl of \"" + keyName + "\".");
			logger.error("Error Message: " + ase.getMessage() + "  AWS Error Code:   " + ase.getErrorCode()
					+ " Error Type:       " + ase.getErrorType());
			throw new AmazonServiceException("对象存储保存异常!");
		} catch (Exception e) {
			e.printStackTrace();
			throw new DspException("文件上传失败!");
		}
	}

	/**
	 * 下载文件
	 * 
	 * @param keyName
	 *            filePath
	 * @param fullName
	 *            文件名称
	 * @param response
	 * @throws IOException
	 */
	public static void downloadFile(HttpServletResponse response, String fullName, String keyName) {
		logger.info("Downloading an object");
		
		S3Object fullObject = null;
		OutputStream out = null;
		InputStream in = null;
		// 获取对象
		try {
			fullObject = client.getObject(new GetObjectRequest(bucketName, keyName));
			in = fullObject.getObjectContent();
			
			response.setHeader("Accept-Ranges", "bytes");
			response.setHeader("Access-Control-Allow-Origin", "*");
			response.setContentType("application/octet-stream");
			// 文件名
			response.setHeader("Content-Disposition",
					"attachment; filename=" + new String(fullName.getBytes("GBK"), "ISO-8859-1"));
			out = response.getOutputStream();
			byte[] bt = new byte[10240];
			int length;
			while ((length = in.read(bt)) > 0) {
				out.write(bt, 0, length);
			}
			out.close();
			in.close();
		} catch (IOException e) {
			e.printStackTrace();
			throw new DspException("文件下载异常");
		} finally {
			try {
				if (out != null) {
					out.close();
				}
				if (in != null) {
					in.close();
				}
				// 为了确保网络连接断开,请关闭任何打开的输入流
				if (fullObject != null) {
					fullObject.close();
				}
			} catch (IOException e) {
				e.printStackTrace();
			}

		}
	}
	

	/**
	 * 获取文件流
	 * 
	 * @param keyName
	 * @return
	 */
	public static InputStream getFileInputStream(String keyName) {
		
		// 获取对象
		try {
			return client.getObject(new GetObjectRequest(bucketName, keyName)).getObjectContent();
		} catch (Exception e) {
			e.printStackTrace();
			throw new DspException("从对象存储上获取文件流异常");
		} 
	}

	/**
	 * 下载文件
	 * 
	 * @param dspFiles
	 *            dspFile
	 * @param fullName
	 *            文件名称
	 * @param response
	 * @throws IOException
	 */
	public static void downloadFileZIP(HttpServletResponse response, String fullName, List<DspFile> dspFiles)
			throws IOException {
		logger.info("Downloading an object");
		if (dspFiles == null || dspFiles.size() == 0) {
			logger.debug("需要压缩的文件个数为0");
			return;
		}
		
		S3Object fullObject = null;
		OutputStream sos = null;
		InputStream in = null;
		ZipOutputStream zos = null;
		int i = 1;
		// 文件名
		response.setHeader("Content-Disposition",
				"attachment; filename=" + new String(fullName.getBytes("GBK"), "ISO-8859-1"));
		// 获取对象
		try {
			sos = response.getOutputStream();
			zos = new ZipOutputStream(sos);
			BufferedInputStream bis = null;
			for (DspFile file : dspFiles) {
				try {
					File destFile = new File(file.getFilePath());
					if (!destFile.exists()) {
						throw new DspException("目标文件不存在");
					}

					if (!destFile.canRead()) {
						throw new DspException("目标文件不可读");
					}
					fullObject = client.getObject(new GetObjectRequest(bucketName, file.getFilePath()));
					// 为了防止文件名重复,在文件前面添加序号
					String fileName = file.getDisplayFileName();
					fileName = fileName.replaceAll("[/\\\\:\\*\\?\\<\\>\\|\"]", "_");
					zos.putNextEntry(new ZipEntry("(" + i + ")" + fileName));
					i++;
					in = fullObject.getObjectContent();
					bis = new BufferedInputStream(in);
					byte[] buffer = new byte[2048];
					int length = 0;
					while ((length = bis.read(buffer)) != -1) {
						zos.write(buffer, 0, length);
					}
				} catch (Exception e) {
					logger.error("生成压缩文件失败", e);
					e.printStackTrace();
					throw new DspException("生成压缩文件失败");
				} finally {
					try {
						if (bis != null) {
							bis.close();
						}
						if (in != null) {
							in.close();
						}
						if (sos != null) {
							sos.flush();
						}

					} catch (IOException e) {
						e.printStackTrace();
					}
				}
			}
		} catch (Exception e) {
			if (sos != null) {
				try {
					sos.close();
				} catch (Exception ex2) {
					throw new RuntimeException(ex2.getMessage());
				}
			}
			if (in != null) {
				try {
					in.close();
				} catch (Exception ex3) {
					throw new RuntimeException(ex3.getMessage());
				}
			}
			if (zos != null) {
				try {
					zos.close();
				} catch (Exception ex3) {
					throw new RuntimeException(ex3.getMessage());
				}
			}
		} finally {
			// 为了确保网络连接断开,请关闭任何打开的输入流
			if (fullObject != null) {
				fullObject.close();
			}
			if (zos != null) {
				zos.flush();
			}
		}
	}

	/**
	 * 分段上传
	 * 
	 * @param file
	 * @param keyName
	 *            新文件名称
	 */
	public static void multipartUpload(File file, String keyName) {
		try {
			
			long contentLength = file.length();
			// etag 列表传递给 CompleteMultipartUploadRequest 请求
			List<PartETag> partETags = new ArrayList<>();
			long partSize = 5L * 1024 * 1024; // 设置分片大小为 5 MB.
			InitiateMultipartUploadRequest initRequest = new InitiateMultipartUploadRequest(bucketName,
					keyName);
			InitiateMultipartUploadResult initResponse = client.initiateMultipartUpload(initRequest);
			// 上传分段
			long filePosition = 0;
			for (int i = 1; filePosition < contentLength; i++) {
				// 因为最后的部分可能小于 5 MB,根据需要调整分片大小。
				partSize = Math.min(partSize, (contentLength - filePosition));
				// 创建一个上传分段的请求
				UploadPartRequest uploadRequest = new UploadPartRequest().withBucketName(bucketName)
						.withKey(keyName).withUploadId(initResponse.getUploadId()).withPartNumber(i)
						.withFileOffset(filePosition).withFile(file).withPartSize(partSize);
				// 上传分片,并将 etag 保存到列表中
				UploadPartResult uploadResult = client.uploadPart(uploadRequest);
				partETags.add(uploadResult.getPartETag());
				filePosition += partSize;
			}
			// 分段上传完成
			CompleteMultipartUploadRequest compRequest = new CompleteMultipartUploadRequest(bucketName,
					keyName, initResponse.getUploadId(), partETags);
			client.completeMultipartUpload(compRequest);

		} catch (AmazonServiceException e) {
			// 服务端错误
			e.printStackTrace();
		} catch (AmazonClientException e) {
			// 客户端错误
			e.printStackTrace();
		}
	}

	/**
	 * 分段上传列表
	 * 
	 * @return
	 */
	public static List<MultipartUpload> listMultipartUploads() {
		logger.info("分段上传列表");
		List<MultipartUpload> uploads = null;
		try {
			
			// 检索正在进行的分段上传列表
			ListMultipartUploadsRequest allMultipartUploadsRequest = new ListMultipartUploadsRequest(
					bucketName);
			MultipartUploadListing multipartUploadListing = client.listMultipartUploads(allMultipartUploadsRequest);
			uploads = multipartUploadListing.getMultipartUploads();
			// 显示正在进行的分段上传信息
			logger.info(uploads.size() + " multipart upload(s) in progress.");
			/*
			 * for (MultipartUpload u : uploads) {
			 * logger.info("Upload in progress: Key = \"" + u.getKey() +
			 * "\", id = " + u.getUploadId()); }
			 */

		} catch (AmazonServiceException e) {
			// 服务端错误
			e.printStackTrace();
		} catch (AmazonClientException e) {
			// 客户端错误
			e.printStackTrace();
		}
		return uploads;
	}

	/**
	 * 复制文件
	 * 
	 * @param sourceKey
	 *            源文件
	 * @param destinationKey
	 *            目的地文件
	 */
	public static void copyFile(String sourceKey, String destinationKey) {
		// 在同一个桶中拷贝一个新对象
		CopyObjectRequest copyObjRequest = new CopyObjectRequest(bucketName, sourceKey, bucketName, destinationKey);
		client.copyObject(copyObjRequest);
	}

	/**
	 * 移动文件
	 * 
	 * @param sourceKey
	 *            源文件
	 * @param destinationKey
	 *            目的地文件
	 */
	public static void moveFile(String sourceKey, String destinationKey) {
		// Make a copy of the object and use server-side encryption when
		// storing the copy.
		CopyObjectRequest copyObjRequest = new CopyObjectRequest(bucketName, sourceKey, bucketName, destinationKey);
		CopyObjectResult result = client.copyObject(copyObjRequest);
		logger.info(result.getSSEAlgorithm());
		getEncryptionStatus(result);
		// Delete the original
		client.deleteObject(bucketName, sourceKey);
		logger.info("Unencrypted object \"" + sourceKey + "\" deleted.");
	}
	

	private static void getEncryptionStatus(SSEResultBase response) {
		String encryptionStatus = response.getSSEAlgorithm();
		if (encryptionStatus == null) {
			encryptionStatus = "Not encrypted with SSE";
		}
		logger.info("Object encryption status is: " + encryptionStatus);
	}

	/**
	 * 批量删除
	 * 
	 * @param keyNames
	 *            集合文件keyName
	 * @return
	 */
	public static int batchDeleteFile(List<KeyVersion> keyNames) {
		logger.info("批量删除文件");
		// 删除对象
		DeleteObjectsRequest multiObjectDeleteRequest = new DeleteObjectsRequest(bucketName).withKeys(keyNames)
				.withQuiet(false);
		// 确认对象是否已经删除
		DeleteObjectsResult delObjRes = client.deleteObjects(multiObjectDeleteRequest);

		int successfulDeletes = delObjRes.getDeletedObjects().size();
		logger.info(successfulDeletes + " objects successfully deleted.");

		return successfulDeletes;
	}

	/**
	 * 删除文件
	 * 
	 * @param keyName
	 *            文件名称
	 */
	public static void deleteFile(String keyName) {
		logger.info("删除文件 keyName={}", keyName);
		client.deleteObject(bucketName, keyName);
	}

	/**
	 * 查找prefix目录下的文件
	 * 
	 * @param prefix
	 *            例:text/
	 * @return
	 */
	public static List<S3ObjectSummary> listObjects(String prefix) {
		logger.info("查找前缀" + prefix + "的文件");
		return client.listObjectsV2(bucketName, prefix).getObjectSummaries();
	}

	

	public static void uploadFile(byte[] bytes, String keyName) {
		try {
			ObjectMetadata objectMetadata = new ObjectMetadata();
			objectMetadata.setContentLength(bytes.length);
			PutObjectRequest putRequest = new PutObjectRequest(bucketName, keyName,
					new ByteArrayInputStream(bytes), objectMetadata);
			putRequest.withCannedAcl(CannedAccessControlList.BucketOwnerFullControl);
			PutObjectResult putResult = client.putObject(putRequest);

			logger.info("上传文件eTag:" + putResult.getETag() + " contentMD5:" + putResult.getContentMd5());

		} catch (AmazonServiceException ase) {
			logger.error("Caught an AmazonServiceException when put object_acl of \"" + keyName + "\".");
			logger.error("Error Message: " + ase.getMessage() + "  AWS Error Code:   " + ase.getErrorCode()
					+ " Error Type:       " + ase.getErrorType());
			throw new DspException("文件上传失败!");
		}
	}

	/**
	 * 导出一个文件到本地存储上
	 * 
	 * @param sourceFilePath
	 * @param destFilePath
	 */
	public static void exportFile(String sourceFilePath, String destFilePath) {
		try {
			InputStream in = getFileInputStream(sourceFilePath);
			writeToLocal(in, destFilePath);
		} catch (Exception e) {
			e.printStackTrace();
			throw new DspException("从对象存储导出文件到本地异常");
		}
	}

	/**
	 * 设置生命周期
	 *
	 * @param lifecycle 需要设置的目录
	 */
	public static void setLifecycle(Map<String,Integer> lifecycle) {
		//设置多个
		List<BucketLifecycleConfiguration.Rule> listRules = new ArrayList<>();
		for(String key : lifecycle.keySet()){
			BucketLifecycleConfiguration.Rule rule = new BucketLifecycleConfiguration.Rule();
			rule.setStatus(BucketLifecycleConfiguration.ENABLED);
			//value当做过期时间
			rule.setExpirationInDays(lifecycle.get(key));
			//key为过期目录
			rule.setFilter(new LifecycleFilter().withPredicate(new LifecyclePrefixPredicate(key)));
			listRules.add(rule);
		}
		//BucketLifecycleConfiguration withRules = new BucketLifecycleConfiguration().withRules(rule);
		//设置多个
		BucketLifecycleConfiguration withRules = new BucketLifecycleConfiguration().withRules(listRules);
		SetBucketLifecycleConfigurationRequest request = new SetBucketLifecycleConfigurationRequest(bucketName, withRules);
		client.setBucketLifecycleConfiguration(request);
	}

	/**
	 * 获取设置的生命周期
	 */
	public static List<BucketLifecycleConfiguration.Rule> getLifecycle() {
		logger.info("获取生命周期");
		BucketLifecycleConfiguration configuration = client.getBucketLifecycleConfiguration(bucketName);
        List<BucketLifecycleConfiguration.Rule> rules = configuration.getRules();
        return rules;
	}


	/**
	 * 将文件从对象存储写到本地目录
	 * 
	 * @param in
	 * @param destFilePath
	 * @throws IOException 
	 * @throws FileNotFoundException 
	 */
	private static void writeToLocal(InputStream in, String destFilePath) throws FileNotFoundException, IOException {
		FileCopyUtils.copy(in, new FileOutputStream(destFilePath));
	}

	/**
	 * 读取文件
	 * 
	 * @param filePath
	 * @return
	 */
	public static byte[] getFileContent(String filePath) {
		ByteArrayOutputStream out = new ByteArrayOutputStream();
		InputStream input = null;
		try {
			int index;
			input = getFileInputStream(filePath);
			byte[] buffer = new byte[2048];
			while((index = input.read(buffer))!= -1){
				out.write(buffer, 0, index);;
			}
		} catch (Exception e) {
			e.printStackTrace();
			throw new DspException("从对象存储加载文件异常");
		} finally {
			if (input != null) {
				try {
					input.close();
				} catch (IOException e) {
					e.printStackTrace();
				}
			}
		}
		return out.toByteArray();
	}

	/**
	 * 校验文件是否存在
	 * @param filePath
	 * @return
	 */
	public static boolean checkFileExists(String filePath) {
		return client.doesObjectExist(bucketName, filePath);
	}

}

SDS配置类:

package cn.com.sinosure.dsp.storage.impl;

import javax.annotation.PostConstruct;
import javax.inject.Named;

import org.slf4j.Logger;
import org.springframework.beans.factory.annotation.Value;

import cn.com.sinosure.common.log.LOG;
import cn.com.sinosure.dsp.storage.utils.XSKYObjectStroageUtils;

@Named
public class XSKYObjectStorge2 {
	
	private static final Logger log = LOG.get(XSKYObjectStorge2.class);

	@Value("${dsp.akey}")
	public String akey;
	
	@Value("${dsp.skey}")
	public String skey;

	@Value("${dsp.endpoint}")
	public String endpoint;
	
	@Value("${dsp.bucket.name}")
	public String bucketName;
	

	/**
	 * 获取aws实例
	 * 
	 * @return
	 */
	@PostConstruct
	private void initAmazonClient() {
		XSKYObjectStroageUtils.initAmazonS3Client(akey, skey, endpoint, bucketName);
	}
}

Finance代码

package com.tencent.wework;

/* sdk返回数据
typedef struct Slice_t {
    char* buf;
    int len;
} Slice_t;

typedef struct MediaData {
    char* outindexbuf;
    int out_len;
    char* data;    
    int data_len;
    int is_finish;
} MediaData_t;
*/

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.net.URLDecoder;
import java.nio.file.Files;

public class Finance {
    public native static long NewSdk();

	/**
	 * 初始化函数
	 * Return值=0表示该API调用成功
	 * 
	 * @param [in]  sdk			NewSdk返回的sdk指针
	 * @param [in]  corpid      调用企业的企业id,例如:wwd08c8exxxx5ab44d,可以在企业微信管理端--我的企业--企业信息查看
	 * @param [in]  secret		聊天内容存档的Secret,可以在企业微信管理端--管理工具--聊天内容存档查看
	 *						
	 *
	 * @return 返回是否初始化成功
	 *      0   - 成功
	 *      !=0 - 失败
	 */
    public native static int Init(long sdk, String corpid, String secret);

	/**
	 * 拉取聊天记录函数
	 * Return值=0表示该API调用成功
	 * 
	 *
	 * @param [in]  sdk				NewSdk返回的sdk指针
	 * @param [in]  seq				从指定的seq开始拉取消息,注意的是返回的消息从seq+1开始返回,seq为之前接口返回的最大seq值。首次使用请使用seq:0
	 * @param [in]  limit			一次拉取的消息条数,最大值1000条,超过1000条会返回错误
	 * @param [in]  proxy			使用代理的请求,需要传入代理的链接。如:socks5://10.0.0.1:8081 或者 http://10.0.0.1:8081
	 * @param [in]  passwd			代理账号密码,需要传入代理的账号密码。如 user_name:passwd_123
	 * @param [out] chatDatas		返回本次拉取消息的数据,slice结构体.内容包括errcode/errmsg,以及每条消息内容。


	 *
	 * @return 返回是否调用成功
	 *      0   - 成功
	 *      !=0 - 失败	
	 */		
    public native static int GetChatData(long sdk, long seq, long limit, String proxy, String passwd, long timeout, long chatData);

	/**
	 * 拉取媒体消息函数
	 * Return值=0表示该API调用成功
	 * 
	 *
	 * @param [in]  sdk				NewSdk返回的sdk指针
	 * @param [in]  sdkFileid		从GetChatData返回的聊天消息中,媒体消息包括的sdkfileid
	 * @param [in]  proxy			使用代理的请求,需要传入代理的链接。如:socks5://10.0.0.1:8081 或者 http://10.0.0.1:8081
	 * @param [in]  passwd			代理账号密码,需要传入代理的账号密码。如 user_name:passwd_123
	 * @param [in]  indexbuf		媒体消息分片拉取,需要填入每次拉取的索引信息。首次不需要填写,默认拉取512k,后续每次调用只需要将上次调用返回的outindexbuf填入即可。
	 * @param [out] media_data		返回本次拉取的媒体数据.MediaData结构体.内容包括data(数据内容)/outindexbuf(下次索引)/is_finish(拉取完成标记)
	 
	 *
	 * @return 返回是否调用成功
	 *      0   - 成功
	 *      !=0 - 失败
	 */
    public native static int GetMediaData(long sdk, String indexbuf, String sdkField, String proxy, String passwd, long timeout, long mediaData);

    /**
     * @brief 解析密文
     * @param [in]  encrypt_key, getchatdata返回的encrypt_key
     * @param [in]  encrypt_msg, getchatdata返回的content
     * @param [out] msg, 解密的消息明文
	 * @return 返回是否调用成功
	 *      0   - 成功
	 *      !=0 - 失败
     */
    public native static int DecryptData(long sdk, String encrypt_key, String encrypt_msg, long msg);
	
    public native static void DestroySdk(long sdk);
    public native static long NewSlice();
    /**
     * @brief 释放slice,和NewSlice成对使用
     * @return 
     */
    public native static void FreeSlice(long slice);

    /**
     * @brief 获取slice内容
     * @return 内容
     */
    public native static String GetContentFromSlice(long slice);

    /**
     * @brief 获取slice内容长度
     * @return 内容
     */
    public native static int GetSliceLen(long slice);
    public native static long NewMediaData();
    public native static void FreeMediaData(long mediaData);

    /**
     * @brief 获取mediadata outindex
     * @return outindex
     */
    public native static String GetOutIndexBuf(long mediaData);
    /**
     * @brief 获取mediadata data数据
     * @return data
     */
    public native static byte[] GetData(long mediaData);
    public native static int GetIndexLen(long mediaData);
    public native static int GetDataLen(long mediaData);

    /**
     * @brief 判断mediadata是否结束
     * @return 1完成、0未完成
     */
    public native static int IsMediaDataFinish(long mediaData);

    /*static {
		String OS = System.getProperty("os.name").toUpperCase();

		if(OS.contains("WIN"))
		{
			System.out.println("OS"+OS);
			System.loadLibrary("WeWorkFinanceSdk");
		}
		else
		{
			System.out.println("OS"+OS);
			System.loadLibrary("WeWorkFinanceSdk_Java");
		}
    }*/

	public static boolean isWindows() {
		String osName = System.getProperties().getProperty("os.name");
		System.out.println("current system is : " + osName);
		return osName.toUpperCase().indexOf("WINDOWS") != -1;
	}

	static {
		if(isWindows()) {
			/*String path = Finance.class.getResource("").getPath().replaceAll("%20"," ")
					.replaceFirst("/","").replace("/","\\\\");
			System.out.println(path.concat("libeay32.dll"));
			System.load(path.concat("libeay32.dll"));
			System.load(path.concat("libprotobuf.dll"));
			System.load(path.concat("ssleay32.dll"));
			System.load(path.concat("libcurl.dll"));
			System.load(path.concat("WeWorkFinanceSdk.dll"));*/
			//System.out.println("OS"+OS);
			System.loadLibrary("WeWorkFinanceSdk");

		}else {
//如果使用linux系统,打成jar包后无法使用包内的so文件需要先将.so文件复制到linux目录下,然后再加载
			try {
				String newSoPath = loadlib("/usr/lib/libWeWorkFinanceSdk_Java.so","libWeWorkFinanceSdk_Java.so");
				System.load(newSoPath +"/" + "libWeWorkFinanceSdk_Java.so");
			} catch (IOException e) {
				e.printStackTrace();
			}

		}
	}

	public static String loadlib(String fileName,String sourcePath) throws IOException {
		InputStream in = Finance.class.getResourceAsStream(sourcePath);

		byte[] buffer = new byte[1024];
		File temp = new File(fileName);
		Files.deleteIfExists(temp.toPath());
		FileOutputStream fos = new FileOutputStream(temp);
		int read = -1;
		while((read = in.read(buffer)) != -1) {
			fos.write(buffer, 0, read);
		}
		fos.close();
		in.close();
		String abPath = temp.getAbsolutePath();
		return abPath.substring(0,abPath.lastIndexOf("/"));
	}

}

pom.xml文件

        <!--引入对象存储依赖 开始-->
       <!-- <dependency>
            <groupId>aws-java-sdk-s3</groupId>
            <artifactId>com.amazonaws</artifactId>
            <version>1.11.769</version>
        </dependency>-->
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.5.9</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>fluent-hc</artifactId>
            <version>4.5.9</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpcore</artifactId>
            <version>4.4.11</version>
        </dependency>
        <!--引入对象存储依赖 结束-->
    
 <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>

        <resources>
            <resource>
                <directory>src/main/java</directory>
                <includes>
                    <include>**/*.dll</include>
                    <include>**/*.so</include>
                </includes>
                <filtering>false</filtering>
            </resource>
            <resource>
                <directory>src/main/resources</directory>
                <filtering>true</filtering>
            </resource>
        </resources>
    </build>

常见问题

  1. 问题一: 2023-10-31 15:57:37.694 [task-1] ERROR
    o.s.a.i.SimpleAsyncUncaughtExceptionHandler - Unexpected exception
    occurred invoking async method: public void
    com.vocust.qywx.demo.service.impl.ScheduledTasks.run(java.lang.String[])
    throws java.lang.Exception java.lang.UnsatisfiedLinkError: no
    WeWorkFinanceSdk in java.library.path

解决办法:
windows系统:
方法一:我们只需要在C:\Windows\system32 文件夹里放入我们要使用的dll文件即可。
博客: https://blog.csdn.net/wzp12321/article/details/117744159

方法二:加载代码中的.dll文件
博客: https://blog.csdn.net/weixin_42932323/article/details/118326236

linux系统:
方法一:把libWeWorkFinanceSdk_Java.so放在/usr/lib目录下.
方法二:加载代码中的so文件
方法三(无法加载jar包中的so文件):先把代码中的so文件复制到服务器指定目录下,然后加载服务器目录中的该文件.
参考博客: https://codeleading.com/article/3083329531/
解决办法: 将jar包中的.so文件复制到/usr/lib目录下,然后加载/usr/lib中的.so文件.

  1. 问题二:java.lang.UnsatisfiedLinkError:xxxx: Can`t load AMD 64-bit .dll
    on a IA 32-bit platform

问题描述: 在企业微信存档功能中把代码中的.dll中的文件加载进来后,请求接口报这个错误.
问题原因: 因为本机JDK版本为32位的,而加载的dll为64位版本. 内网的eclipse中jre使用的是eclipse自带的jre,版本为32位的.
解决办法: 将eclipse中的jre配置成本地jre(64位)的.
博客: https://blog.csdn.net/qq_43409973/article/details/130707848

  1. 问题三:错误码:10001

原因1: 后台没有开启企业微信内容存档权限
开启权限后,corpid和secret错误时报41001

调用拉取数据接口成功时,错误码为 0.

原因2:网络问题–代理信息配置错误
解决办法: 使用nginx的正向代理.nginx做正向代理https需要手动给nginx添加ngx_http_proxy_connect_module模块.
参考博客: https://blog.csdn.net/qq_44659804/article/details/128997510
https://zhuanlan.zhihu.com/p/629353778?utm_id=0

server {
        listen 8080;
        server_name localhost;
        resolver 114.114.114.114 ipv6=off;
        proxy_connect;
        proxy_connect_allow 443 80;
        proxy_connect_connect_timeout  10s;
        proxy_connect_data_timeout     10s;
        access_log logs/access_proxy.log main;
        location /qywechat-huihuai/ {
            proxy_pass $scheme://$host$request_uri/;
        }

原因3:网络策略问题
问题描述: 生产之前拉取数据100%成功,后来突然大量报错,报10001错误,尤其是媒体文件大量报错。
原因:接口向外请求时会转发到两台服务器,有一台的请求不通。
解决办法:排查网络转发问题。

  1. 问题四:错误码:301042

问题原因:该IP地址不在企业微信白名单内.
解决办法: 1,将服务器的公网IP配置到企业微信管理后台的可信任IP地址.即错误日志中出现的IP地址(from ip: 106.120.215.130)
2,本地电脑登陆VPN,然后将VPN在公网的IP地址配置到企业微信管理后台.

  1. 问题五: org.apache.ibatis.binding.BindingException Invalid boud
    statement (not found):

com.sinosure.huihua.mapper.WechatChatdataEncryptMapper.selectLatestSeq
问题原因: xml文件没有打进去
解决办法: 在storage-core的xml中添加如下代码


src/main/resources
true


问题六:错误码10003

问题描述:拉取媒体文件时,报出10003错误,多次重试仍报这个错误。
问题原因: 官网回复: 这类问题一般都是客户端版本太低,消息记录上传数据不全导致的
在这里插入图片描述

;