读取oss中10万条数据,结果读取一半就返回结果了,没有报任何错误,一直以为是网络原因所以加入了各种超时时间,结果也不好用~求大佬帮忙解答
public List getOssData(String path) {
int HttpResult; // 服务器返回的状态
List<String> list = new ArrayList<>();
try
{
ClientBuilderConfiguration conf = new ClientBuilderConfiguration();
conf.setConnectionTimeout(20*60 * 1000);
conf.setSocketTimeout(20*60 * 1000);
conf.setConnectionRequestTimeout(20*60 * 1000);
conf.setMaxConnections(3);
OSS ossClient = new OSSClientBuilder().build(aliyunConfig.getEndpoint(), aliyunConfig.getAccessKeyId(), aliyunConfig.getAccessKeySecret(),conf);
Date expiration = new Date(System.currentTimeMillis() + 3600 * 1000);
URL url = ossClient.generatePresignedUrl(aliyunConfig.getBucketName(), path, expiration);
// 试图连接并取得返回状态码
URLConnection urlconn = url.openConnection();
// 设置连接主机超时时间
urlconn.setConnectTimeout(10*60 * 1000);
// 设置从主机读取数据超时
urlconn.setReadTimeout(10*60 * 1000);
urlconn.connect();
HttpURLConnection httpconn =(HttpURLConnection)urlconn;
HttpResult = httpconn.getResponseCode();
if(HttpResult != HttpURLConnection.HTTP_OK) {
logger.error("无法连接到");
} else {
CsvReader cr = new CsvReader(urlconn.getInputStream(), ',', Charset.forName("UTF-8"));
cr.readHeaders();
String last="";
while(cr.readRecord()){
if (cr.getRawRecord().length()!=0){
//logger.info(cr.getRawRecord());
last=cr.getRawRecord();
if(!StringUtils.isEmpty(cr.getRawRecord().trim())) {
list.add(cr.getRawRecord().trim());
}
}else{
logger.error("下载出错!");
break;
}
}
logger.info("解析最后一条数据:{}",last);
cr.close();
urlconn.getInputStream().close();
}
ossClient.shutdown();
} catch (Exception e) {
logger.error("读数据异常", e);
}
logger.info("oss解析内容:{} 条",list.size());
return list;
}
版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。