poi大数据导出excel

2018-09-24 21:17:41 lichunericli 阅读数 5053

  1. 大数据量的导入

当Excel中的数据量超过10万行时,在用POI读取文件流时很容易引起失败,需要引入xlsx-streamer来进行资源的打开,剩下的处理同POI处理上百行数据量类似:filePath=>FileInputStream=>Workbook=>Sheet=>Row=>Cell=>Java反射获取值。

 <dependency>
       <groupId>com.monitorjbl</groupId>
       <artifactId>xlsx-streamer</artifactId>
       <version>1.2.1</version>
</dependency>  

使用StreamingReader来进行对Workbook的获取。需要注意主要针对超过10万行要解析的Excel的Workbook获取,缓存到内存中的行数默认是10行,读取资源时缓存到内存的字节大小默认是1024,资源必须打开,File或者InputStream都可以,但是只能打开XLSX格式的文件。
public static Workbook obtainWorkbookByStream(String filePath) throws Exception {
      // StreamingReader用于读取Excel的内容,不能写入,不能随机读取Excel的内容
      FileInputStream in = new FileInputStream(filePath);
      Workbook workbook = StreamingReader.builder().rowCacheSize(100).bufferSize(4096).open(in);
      return workbook;
}

2. 大数据量的导出

对于大数据量的导出通常采用分而治之的思想,将大数据量分批次的导出到多个Excel文件或者单个Excel文件的多个sheet,也可导出到多个Excel文件后合并从单独的文件,其实现方式可参考:https://github.com/chunericli/xpt-excel-extension

总结:大数据量的处理需要注意对内存使用的影响和对业务的影响。除非必要,要不然最好使用异步的方式进行处理,即首先保存文件元数据信息,然后分批次保存数据和分批次读取数据。

2018-09-25 20:18:05 qq_35206261 阅读数 30676

一. 简介

          excel导出,如果数据量在百万级,会出现俩点内存溢出的问题:

          1. 查询数据量过大,导致内存溢出。 该问题可以通过分批查询来解决;

          2. 最后下载的时候大EXCEL转换的输出流内存溢出;该方式可以通过新版的SXSSFWorkbook来解决,可通过其构造函数执指定在内存中缓存的行数,剩余的会自动缓存在硬盘的临时目录上,同时,并不会存在页面卡顿的情况;

          3. 为了能够使用不同的mapper并分批写数据, 采用了外观模式和模板方法模式,大体分三步:

              a. 根据总数量生成excel,确定sheet的数量和写标题;

              b. 写数据,在可变的匿名内部类中实现写入逻辑;
              c. 转换输出流进行下载;

          4. 使用案例在3.3 ServiceImpl中,可自行书写。

          5. 最近太忙,写的太仓促,性能我感觉还有一倍的优化空间,比如循环次数,现在存在无意义空循环(这个耗时比较多)的情况,缓冲流,mybatis的浮标等,待优化........   

          6. 优化空间很大  刚试了下 把空循环去掉  4个字段 90W条数据  40s  180W  75s  300W 122s

          7. 转战阿里开源的EasyExcel了, 那个内存控制在kb级别,绝对不会内存溢出。使用说明请参见博主的另一篇文章:

阿里开源(EasyExcel)---导出EXCEL

          8. 导入的也有,请参见另一篇文章: 阿里开源(EasyExcel)---导入EXCEL

二. 工具代码

2.1 配置

2.1.1 pom.xml

      <!-- poi -->
      <dependency>
          <groupId>org.apache.poi</groupId>
          <artifactId>poi</artifactId>
          <version>3.17</version>
      </dependency>
      <dependency>
          <groupId>org.apache.poi</groupId>
          <artifactId>poi-ooxml</artifactId>
          <version>3.17</version>
      </dependency>

注: 如果是springboot2.0,则不需要poi依赖,如果是1.0,则需要poi依赖,并且poi和poi-ooxml的版本要保持一致。

          别的依赖我就不加了。

2.1.2 application.yml

# pagehelper
pagehelper:
    helperDialect: mysql
    reasonable: false # 如果没有数据  返回空 而非最后一页的数据
    supportMethodsArguments: true
    params: count=countSql
    returnPageInfo: check

注:  reasonable一定要为false, 其他的我就不粘了。

2.2 ExcelConstant

package com.yzx.caasscs.constant;

/**
 * @author qjwyss
 * @date 2018/9/19
 * @description EXCEL常量类
 */
public class ExcelConstant {

    /**
     * 每个sheet存储的记录数 100W
     */
    public static final Integer PER_SHEET_ROW_COUNT = 1000000;

    /**
     * 每次向EXCEL写入的记录数(查询每页数据大小) 20W
     */
    public static final Integer PER_WRITE_ROW_COUNT = 200000;


    /**
     * 每个sheet的写入次数 5
     */
    public static final Integer PER_SHEET_WRITE_COUNT = PER_SHEET_ROW_COUNT / PER_WRITE_ROW_COUNT;


}

注: xlsx模式的excel每个sheet最多存储104W,此处我就每个sheet存储了 100W数据;每次查询20W数据; 自己根据内存来调合适的大小,写入次数待优化。

2.3 写数据委托类

package com.yzx.caasscs.util;

import org.apache.poi.xssf.streaming.SXSSFSheet;

/**
 * @author qjwyss
 * @date 2018/9/20
 * @description EXCEL写数据委托类
 */
public interface WriteExcelDataDelegated {

    /**
     * EXCEL写数据委托类  针对不同的情况自行实现
     *
     * @param eachSheet     指定SHEET
     * @param startRowCount 开始行
     * @param endRowCount   结束行
     * @param currentPage   分批查询开始页
     * @param pageSize      分批查询数据量
     * @throws Exception
     */
    public abstract void writeExcelData(SXSSFSheet eachSheet, Integer startRowCount, Integer endRowCount, Integer currentPage, Integer pageSize) throws Exception;


}

2.4 DateUtil工具类(非必须)

package com.yzx.caasscs.util;

import java.text.SimpleDateFormat;
import java.util.Date;

/**
 * @author qjwyss
 * @date 2018/9/20
 * @description 日期工具类
 */
public class DateUtil {

    public static final String YYYY_MM_DD_HH_MM_SS = "yyyy-MM-dd HH:mm:ss";


    /**
     * 将日期转换为字符串
     *
     * @param date   DATE日期
     * @param format 转换格式
     * @return 字符串日期
     */
    public static String formatDate(Date date, String format) {
        SimpleDateFormat simpleDateFormat = new SimpleDateFormat(format);
        return simpleDateFormat.format(date);
    }


}

2.5 POI工具类

package com.yzx.caasscs.util;


import com.yzx.caasscs.constant.ExcelConstant;
import org.apache.poi.xssf.streaming.SXSSFCell;
import org.apache.poi.xssf.streaming.SXSSFRow;
import org.apache.poi.xssf.streaming.SXSSFSheet;
import org.apache.poi.xssf.streaming.SXSSFWorkbook;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import javax.servlet.http.HttpServletResponse;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.util.Date;

/**
 * @author qjwyss
 * @date 2018/9/18
 * @description POI导出工具类
 */
public class PoiUtil {

    private final static Logger logger = LoggerFactory.getLogger(PoiUtil.class);

    /**
     * 初始化EXCEL(sheet个数和标题)
     *
     * @param totalRowCount 总记录数
     * @param titles        标题集合
     * @return XSSFWorkbook对象
     */
    public static SXSSFWorkbook initExcel(Integer totalRowCount, String[] titles) {

        // 在内存当中保持 100 行 , 超过的数据放到硬盘中在内存当中保持 100 行 , 超过的数据放到硬盘中
        SXSSFWorkbook wb = new SXSSFWorkbook(100);

        Integer sheetCount = ((totalRowCount % ExcelConstant.PER_SHEET_ROW_COUNT == 0) ?
                (totalRowCount / ExcelConstant.PER_SHEET_ROW_COUNT) : (totalRowCount / ExcelConstant.PER_SHEET_ROW_COUNT + 1));

        // 根据总记录数创建sheet并分配标题
        for (int i = 0; i < sheetCount; i++) {
            SXSSFSheet sheet = wb.createSheet("sheet" + (i + 1));
            SXSSFRow headRow = sheet.createRow(0);

            for (int j = 0; j < titles.length; j++) {
                SXSSFCell headRowCell = headRow.createCell(j);
                headRowCell.setCellValue(titles[j]);
            }
        }

        return wb;
    }


    /**
     * 下载EXCEL到本地指定的文件夹
     *
     * @param wb         EXCEL对象SXSSFWorkbook
     * @param exportPath 导出路径
     */
    public static void downLoadExcelToLocalPath(SXSSFWorkbook wb, String exportPath) {
        FileOutputStream fops = null;
        try {
            fops = new FileOutputStream(exportPath);
            wb.write(fops);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (null != wb) {
                try {
                    wb.dispose();
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
            if (null != fops) {
                try {
                    fops.close();
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        }
    }


    /**
     * 下载EXCEL到浏览器
     *
     * @param wb       EXCEL对象XSSFWorkbook
     * @param response
     * @param fileName 文件名称
     * @throws IOException
     */
    public static void downLoadExcelToWebsite(SXSSFWorkbook wb, HttpServletResponse response, String fileName) throws IOException {

        response.setHeader("Content-disposition", "attachment; filename="
                + new String((fileName + ".xlsx").getBytes("utf-8"), "ISO8859-1"));//设置下载的文件名

        OutputStream outputStream = null;
        try {
            outputStream = response.getOutputStream();
            wb.write(outputStream);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (null != wb) {
                try {
                    wb.dispose();
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
            if (null != outputStream) {
                try {
                    outputStream.close();
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        }
    }


    /**
     * 导出Excel到本地指定路径
     *
     * @param totalRowCount           总记录数
     * @param titles                  标题
     * @param exportPath              导出路径
     * @param writeExcelDataDelegated 向EXCEL写数据/处理格式的委托类 自行实现
     * @throws Exception
     */
    public static final void exportExcelToLocalPath(Integer totalRowCount, String[] titles, String exportPath, WriteExcelDataDelegated writeExcelDataDelegated) throws Exception {

        logger.info("开始导出:" + DateUtil.formatDate(new Date(), DateUtil.YYYY_MM_DD_HH_MM_SS));

        // 初始化EXCEL
        SXSSFWorkbook wb = PoiUtil.initExcel(totalRowCount, titles);

        // 调用委托类分批写数据
        int sheetCount = wb.getNumberOfSheets();
        for (int i = 0; i < sheetCount; i++) {
            SXSSFSheet eachSheet = wb.getSheetAt(i);

            for (int j = 1; j <= ExcelConstant.PER_SHEET_WRITE_COUNT; j++) {

                int currentPage = i * ExcelConstant.PER_SHEET_WRITE_COUNT + j;
                int pageSize = ExcelConstant.PER_WRITE_ROW_COUNT;
                int startRowCount = (j - 1) * ExcelConstant.PER_WRITE_ROW_COUNT + 1;
                int endRowCount = startRowCount + pageSize - 1;


                writeExcelDataDelegated.writeExcelData(eachSheet, startRowCount, endRowCount, currentPage, pageSize);

            }
        }


        // 下载EXCEL
        PoiUtil.downLoadExcelToLocalPath(wb, exportPath);

        logger.info("导出完成:" + DateUtil.formatDate(new Date(), DateUtil.YYYY_MM_DD_HH_MM_SS));
    }


    /**
     * 导出Excel到浏览器
     *
     * @param response
     * @param totalRowCount           总记录数
     * @param fileName                文件名称
     * @param titles                  标题
     * @param writeExcelDataDelegated 向EXCEL写数据/处理格式的委托类 自行实现
     * @throws Exception
     */
    public static final void exportExcelToWebsite(HttpServletResponse response, Integer totalRowCount, String fileName, String[] titles, WriteExcelDataDelegated writeExcelDataDelegated) throws Exception {

        logger.info("开始导出:" + DateUtil.formatDate(new Date(), DateUtil.YYYY_MM_DD_HH_MM_SS));

        // 初始化EXCEL
        SXSSFWorkbook wb = PoiUtil.initExcel(totalRowCount, titles);


        // 调用委托类分批写数据
        int sheetCount = wb.getNumberOfSheets();
        for (int i = 0; i < sheetCount; i++) {
            SXSSFSheet eachSheet = wb.getSheetAt(i);

            for (int j = 1; j <= ExcelConstant.PER_SHEET_WRITE_COUNT; j++) {

                int currentPage = i * ExcelConstant.PER_SHEET_WRITE_COUNT + j;
                int pageSize = ExcelConstant.PER_WRITE_ROW_COUNT;
                int startRowCount = (j - 1) * ExcelConstant.PER_WRITE_ROW_COUNT + 1;
                int endRowCount = startRowCount + pageSize - 1;

                writeExcelDataDelegated.writeExcelData(eachSheet, startRowCount, endRowCount, currentPage, pageSize);

            }
        }


        // 下载EXCEL
        PoiUtil.downLoadExcelToWebsite(wb, response, fileName);

        logger.info("导出完成:" + DateUtil.formatDate(new Date(), DateUtil.YYYY_MM_DD_HH_MM_SS));
    }


}

三. 使用DEMO

3.1 Controller

package com.yzx.caasscs.controller.organiza;

import com.yzx.caasscs.controller.BaseController;
import com.yzx.caasscs.service.organiza.UserService;
import com.yzx.caasscs.vo.PageVO;
import com.yzx.caasscs.vo.ResultVO;
import com.yzx.caasscs.vo.organiza.UserVO;
import io.swagger.annotations.Api;
import io.swagger.annotations.ApiImplicitParam;
import io.swagger.annotations.ApiImplicitParams;
import io.swagger.annotations.ApiOperation;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import springfox.documentation.annotations.ApiIgnore;

import javax.servlet.http.HttpServletResponse;

/**
 * @author qjwyss
 * @date 2018/8/30
 * @description 用户控制类
 */
@Api(tags = {"UserController"}, description = "用户Controller")
@RestController
@RequestMapping(value = "/user")
public class UserController extends BaseController {

    @Autowired
    private UserService userService;


    @ApiOperation("导出用户EXCEL")
    @ApiImplicitParams({
            @ApiImplicitParam(paramType = "query", dataType = "Long", name = "loginUid", value = "登录用户UID", required = true),
            @ApiImplicitParam(paramType = "query", dataType = "Long", name = "uid", value = "用户UID", required = true)
    })
    @GetMapping("/export")
    public ResultVO<Void> exportUser(@ApiIgnore UserVO userVO, HttpServletResponse response) throws Exception {
        return this.userService.export(userVO, response);
    }

}

3.2 Service

package com.yzx.caasscs.service.organiza;

import com.yzx.caasscs.vo.PageVO;
import com.yzx.caasscs.vo.ResultVO;
import com.yzx.caasscs.vo.organiza.UserVO;

import javax.servlet.http.HttpServletResponse;

/**
 * @author qjwyss
 * @date 2018/8/30
 * @description 用户SERVICE
 */
public interface UserService {


    /**
     * 导出用户EXCEL
     *
     * @param userVO
     * @return VOID
     * @throws Exception
     */
    ResultVO<Void> export(UserVO userVO, HttpServletResponse response) throws Exception;


}

3.3 ServiceImpl

package com.yzx.caasscs.service.impl.organiza;

import com.github.pagehelper.PageHelper;
import com.github.pagehelper.PageInfo;
import com.yzx.caasscs.constant.middleware.RedisConstant;
import com.yzx.caasscs.constant.organiza.UserApartmentConstant;
import com.yzx.caasscs.constant.organiza.UserConstant;
import com.yzx.caasscs.constant.organiza.UserRoleConstant;
import com.yzx.caasscs.constant.sys.SysLogConstant;
import com.yzx.caasscs.entity.dscaasscs.organiza.User;
import com.yzx.caasscs.entity.dscaasscs.organiza.UserApartment;
import com.yzx.caasscs.entity.dscaasscs.organiza.UserRole;
import com.yzx.caasscs.mapper.dscaasscs.organiza.UserApartmentMapper;
import com.yzx.caasscs.mapper.dscaasscs.organiza.UserMapper;
import com.yzx.caasscs.mapper.dscaasscs.organiza.UserRoleMapper;
import com.yzx.caasscs.mapper.dscaasscs.sys.RoleMenuMapper;
import com.yzx.caasscs.service.organiza.UserService;
import com.yzx.caasscs.service.sys.SysLogService;
import com.yzx.caasscs.util.CommonUtil;
import com.yzx.caasscs.util.DateUtil;
import com.yzx.caasscs.util.PoiUtil;
import com.yzx.caasscs.util.WriteExcelDataDelegated;
import com.yzx.caasscs.vo.PageVO;
import com.yzx.caasscs.vo.ResultVO;
import com.yzx.caasscs.vo.organiza.UserApartmentVO;
import com.yzx.caasscs.vo.organiza.UserRoleVO;
import com.yzx.caasscs.vo.organiza.UserVO;
import com.yzx.caasscs.vo.sys.MenuVO;
import com.yzx.caasscs.vo.sys.RoleMenuVO;
import com.yzx.caasscs.vo.sys.SysLogVO;
import net.sf.json.JSONObject;
import org.apache.poi.xssf.streaming.SXSSFRow;
import org.apache.poi.xssf.streaming.SXSSFSheet;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.util.CollectionUtils;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;

/**
 * @author qjwyss
 * @date 2018/8/30
 * @description 用户SERVICEIMPL
 */
@Service
public class UserServiceImpl implements UserService {

    @Autowired
    private UserMapper userMapper;

    @Autowired
    private HttpServletRequest request;


    @Override
    public ResultVO<Void> export(UserVO userVO, HttpServletResponse response) throws Exception {

        // 总记录数
        Integer totalRowCount = this.userMapper.selectUserVOCount(userVO);

        // 导出EXCEL文件名称
        String filaName = "用户EXCEL";

        // 标题
        String[] titles = {"账号", "密码", "状态", "昵称", "职位", "手机号", "邮箱", "创建人ID", "创建时间", "修改人ID", "修改时间"};

        // 开始导入
        PoiUtil.exportExcelToWebsite(response, totalRowCount, filaName, titles, new WriteExcelDataDelegated() {
            @Override
            public void writeExcelData(SXSSFSheet eachSheet, Integer startRowCount, Integer endRowCount, Integer currentPage, Integer pageSize) throws Exception {

                PageHelper.startPage(currentPage, pageSize);
                List<UserVO> userVOList = userMapper.selectUserVOList(userVO);

                if (!CollectionUtils.isEmpty(userVOList)) {

                    // --------------   这一块变量照着抄就行  强迫症 后期也封装起来     ----------------------
                    for (int i = startRowCount; i <= endRowCount; i++) {
                        SXSSFRow eachDataRow = eachSheet.createRow(i);
                        if ((i - startRowCount) < userVOList.size()) {

                            UserVO eachUserVO = userVOList.get(i - startRowCount);
                            // ---------   这一块变量照着抄就行  强迫症 后期也封装起来     -----------------------

                            eachDataRow.createCell(0).setCellValue(eachUserVO.getUsername() == null ? "" : eachUserVO.getUsername());
                            eachDataRow.createCell(1).setCellValue(eachUserVO.getPassword() == null ? "" : eachUserVO.getPassword());
                            eachDataRow.createCell(2).setCellValue(eachUserVO.getUserState() == null ? "" : (eachUserVO.getUserState() == 1 ? "启用" : "停用"));
                            eachDataRow.createCell(3).setCellValue(eachUserVO.getNickname() == null ? "" : eachUserVO.getNickname());
                            eachDataRow.createCell(4).setCellValue(eachUserVO.getPosition() == null ? "" : eachUserVO.getPosition());
                            eachDataRow.createCell(5).setCellValue(eachUserVO.getMobile() == null ? "" : eachUserVO.getMobile());
                            eachDataRow.createCell(6).setCellValue(eachUserVO.getEmail() == null ? "" : eachUserVO.getEmail());
                            if (null != eachUserVO.getCreateUid()) {
                                eachDataRow.createCell(7).setCellValue(eachUserVO.getCreateUid());
                            }
                            if (null != eachUserVO.getCreateTime()) {
                                eachDataRow.createCell(8).setCellValue(DateUtil.formatDate(eachUserVO.getCreateTime(), DateUtil.YYYY_MM_DD_HH_MM_SS));
                            }
                            if (null != eachUserVO.getUpdateUid()) {
                                eachDataRow.createCell(9).setCellValue(eachUserVO.getUpdateUid());
                            }
                            if (null != eachUserVO.getUpdateTime()) {
                                eachDataRow.createCell(10).setCellValue(DateUtil.formatDate(eachUserVO.getUpdateTime(), DateUtil.YYYY_MM_DD_HH_MM_SS));
                            }
                        }
                    }
                }

            }
        });

        return ResultVO.getSuccess("导出用户EXCEL成功");
    }


}

3.4 mapper

package com.yzx.caasscs.mapper.dscaasscs.organiza;

import com.yzx.caasscs.entity.dscaasscs.organiza.User;
import com.yzx.caasscs.vo.organiza.UserVO;

import java.util.List;

/**
 * @author qjwyss
 * @date 2018/8/29
 * @description 用户MAPPER
 */
public interface UserMapper {


    /**
     * 查询用户列表
     *
     * @param userVO
     * @return UserVO集合
     * @throws Exception
     */
    List<UserVO> selectUserVOList(UserVO userVO) throws Exception;


    /**
     * 查询用户数量
     *
     * @param userVO
     * @return 用户数量
     * @throws Exception
     */
    Integer selectUserVOCount(UserVO userVO) throws Exception;

}

3.5 mapper.xml

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
<mapper namespace="com.yzx.caasscs.mapper.dscaasscs.organiza.UserMapper" >

  <resultMap id="BaseResultMap" type="com.yzx.caasscs.entity.dscaasscs.organiza.User" >
    <id column="uid" property="uid" jdbcType="BIGINT" />
    <result column="username" property="username" jdbcType="VARCHAR" />
    <result column="password" property="password" jdbcType="VARCHAR" />
    <result column="state" property="state" jdbcType="INTEGER" />
    <result column="nickname" property="nickname" jdbcType="VARCHAR" />
    <result column="position" property="position" jdbcType="VARCHAR" />
    <result column="mobile" property="mobile" jdbcType="VARCHAR" />
    <result column="email" property="email" jdbcType="VARCHAR" />
    <result column="create_uid" property="createUid" jdbcType="BIGINT" />
    <result column="create_time" property="createTime" jdbcType="TIMESTAMP" />
    <result column="update_uid" property="updateUid" jdbcType="BIGINT" />
    <result column="update_time" property="updateTime" jdbcType="TIMESTAMP" />
  </resultMap>


  <!-- 查询用户分页列表返回MAP -->
  <resultMap id="SelectUserVOListMap" type="com.yzx.caasscs.vo.organiza.UserVO">
    <id column="uid" property="uid" jdbcType="BIGINT" />
    <result column="username" property="username" jdbcType="VARCHAR" />
    <result column="user_state" property="userState" jdbcType="INTEGER" />
    <result column="nickname" property="nickname" jdbcType="VARCHAR" />
    <result column="position" property="position" jdbcType="VARCHAR" />
    <result column="mobile" property="mobile" jdbcType="VARCHAR" />
    <result column="email" property="email" jdbcType="VARCHAR" />
    <result column="create_uid" property="createUid" jdbcType="BIGINT" />
    <result column="create_time" property="createTime" jdbcType="TIMESTAMP" />
    <result column="update_uid" property="updateUid" jdbcType="BIGINT" />
    <result column="update_time" property="updateTime" jdbcType="TIMESTAMP" />
    <result column="apartment_name" property="apartmentName" jdbcType="VARCHAR" />
    <result column="role_names" property="roleNames" jdbcType="VARCHAR" />
  </resultMap>


  <sql id="Base_Column_List" >
    uid, username, password, state, nickname, position, mobile, email, create_uid, create_time,
    update_uid, update_time
  </sql>


  <!-- 查询用户列表 -->
  <select id="selectUserVOList" parameterType="com.yzx.caasscs.vo.organiza.UserVO" resultMap="SelectUserVOListMap">
    SELECT
		U.uid, U.username, U.state AS user_state, U.nickname, U.position, U.mobile, U.email, U.create_uid, U.create_time,
		U.update_uid, U.update_time,
		A.apartment_name,
		(
			SELECT
				GROUP_CONCAT( R.role_name ) AS role_name_list
			FROM
				user_role AS UR
				LEFT JOIN role AS R ON R.rid = UR.rid
			WHERE
				UR.state > 0
				AND UR.uid = U.uid
			GROUP BY UR.uid
		) role_names
    FROM
        `user` AS U
        LEFT JOIN user_apartment AS UA ON UA.uid = U.uid
        LEFT JOIN apartment AS A ON A.aid = UA.aid
    WHERE
        U.state > 0
        <if test="userState != null">
          AND U.state = #{userState,jdbcType=INTEGER}
        </if>
        <if test="nickname != null and nickname != ''">
          AND U.nickname LIKE CONCAT("%", #{nickname,jdbcType=VARCHAR}, "%")
        </if>
        <if test="apartmentName != null and apartmentName != ''">
          AND A.apartment_name LIKE CONCAT("%", #{apartmentName,jdbcType=VARCHAR}, "%")
        </if>
    ORDER BY U.create_time DESC

  </select>


  <!-- 查询用户列表数量 -->
  <select id="selectUserVOCount" parameterType="com.yzx.caasscs.vo.organiza.UserVO" resultType="java.lang.Integer" >
    SELECT
        COUNT(1)
    FROM
        `user` AS U
        LEFT JOIN user_apartment AS UA ON UA.uid = U.uid
        LEFT JOIN apartment AS A ON A.aid = UA.aid
    WHERE
        U.state > 0
        <if test="userState != null">
          AND U.state = #{userState,jdbcType=INTEGER}
        </if>
        <if test="nickname != null and nickname != ''">
          AND U.nickname LIKE CONCAT("%", #{nickname,jdbcType=VARCHAR}, "%")
        </if>
        <if test="apartmentName != null and apartmentName != ''">
          AND A.apartment_name LIKE CONCAT("%", #{apartmentName,jdbcType=VARCHAR}, "%")
        </if>
  </select>
  

</mapper>

4. 测试

每个sheet存100W,每次查询写20W,每条记录10个字段,时间如下:

数据量 时间
100W 3.5分钟
150W 7分钟

 

 

2017-12-20 19:14:07 sanpangouba 阅读数 3665

POI导出大数据量excel

注:项目源码及后续更新请点击

1、ExcelUtils类:

package Utils;

import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import org.apache.poi.common.usermodel.Hyperlink;
import org.apache.poi.hssf.usermodel.HSSFCellStyle;
import org.apache.poi.hssf.usermodel.HSSFFont;
import org.apache.poi.hssf.util.HSSFColor;
import org.apache.poi.ss.usermodel.CellStyle;
import org.apache.poi.ss.usermodel.CreationHelper;
import org.apache.poi.ss.usermodel.Font;
import org.apache.poi.ss.util.CellRangeAddress;
import org.apache.poi.xssf.streaming.SXSSFCell;
import org.apache.poi.xssf.streaming.SXSSFRow;
import org.apache.poi.xssf.streaming.SXSSFSheet;
import org.apache.poi.xssf.streaming.SXSSFWorkbook;
import org.apache.poi.xssf.usermodel.XSSFHyperlink;

import java.io.IOException;
import java.io.OutputStream;
import java.math.BigDecimal;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.LinkedHashMap;
import java.util.Map;

/**
 * Created by Cheung on 2017/12/19.
 *
 * Apache POI操作Excel对象
 * HSSF:操作Excel 2007之前版本(.xls)格式,生成的EXCEL不经过压缩直接导出
 * XSSF:操作Excel 2007及之后版本(.xlsx)格式,内存占用高于HSSF
 * SXSSF:从POI3.8 beta3开始支持,基于XSSF,低内存占用,专门处理大数据量(建议)。
 *
 * 注意:
 * 		值得注意的是SXSSFWorkbook只能写(导出)不能读(导入)
 *
 * 说明:
 * 		.xls格式的excel(最大行数65536行,最大列数256列)
 * 	   	.xlsx格式的excel(最大行数1048576行,最大列数16384列)
 */
public class ExcelUtil {

	public static final String DEFAULT_DATE_PATTERN = "yyyy年MM月dd日";// 默认日期格式
	public static final int DEFAULT_COLUMN_WIDTH = 17;// 默认列宽


	/**
	 * 导出Excel(.xlsx)格式
	 * @param titleList 表格头信息集合
	 * @param dataArray 数据数组
	 * @param os 文件输出流
	 */
	public static void exportExcel(ArrayList<LinkedHashMap> titleList, JSONArray dataArray, OutputStream os) {
		String datePattern = DEFAULT_DATE_PATTERN;
		int minBytes = DEFAULT_COLUMN_WIDTH;

		/**
		 * 声明一个工作薄
 		 */
		SXSSFWorkbook workbook = new SXSSFWorkbook(1000);// 大于1000行时会把之前的行写入硬盘
		workbook.setCompressTempFiles(true);

		// 表头1样式
		CellStyle title1Style = workbook.createCellStyle();
		title1Style.setAlignment(HSSFCellStyle.ALIGN_CENTER);// 水平居中
		title1Style.setVerticalAlignment(HSSFCellStyle.VERTICAL_CENTER);// 垂直居中
		Font titleFont = workbook.createFont();// 字体
		titleFont.setFontHeightInPoints((short) 20);
		titleFont.setBoldweight((short) 700);
		title1Style.setFont(titleFont);

		// 表头2样式
		CellStyle title2Style = workbook.createCellStyle();
		title2Style.setAlignment(HSSFCellStyle.ALIGN_CENTER);
		title2Style.setVerticalAlignment(HSSFCellStyle.VERTICAL_CENTER);
		title2Style.setBorderTop(HSSFCellStyle.BORDER_THIN);// 上边框
		title2Style.setBorderRight(HSSFCellStyle.BORDER_THIN);// 右
		title2Style.setBorderBottom(HSSFCellStyle.BORDER_THIN);// 下
		title2Style.setBorderLeft(HSSFCellStyle.BORDER_THIN);// 左
		Font title2Font = workbook.createFont();
		title2Font.setUnderline((byte) 1);
		title2Font.setColor(HSSFColor.BLUE.index);
		title2Style.setFont(title2Font);

		// head样式
		CellStyle headerStyle = workbook.createCellStyle();
		headerStyle.setAlignment(HSSFCellStyle.ALIGN_CENTER);
		headerStyle.setVerticalAlignment(HSSFCellStyle.VERTICAL_CENTER);
		headerStyle.setFillForegroundColor(HSSFColor.LIGHT_GREEN.index);// 设置颜色
		headerStyle.setFillPattern(HSSFCellStyle.SOLID_FOREGROUND);// 前景色纯色填充
		headerStyle.setBorderTop(HSSFCellStyle.BORDER_THIN);
		headerStyle.setBorderRight(HSSFCellStyle.BORDER_THIN);
		headerStyle.setBorderBottom(HSSFCellStyle.BORDER_THIN);
		headerStyle.setBorderLeft(HSSFCellStyle.BORDER_THIN);
		Font headerFont = workbook.createFont();
		headerFont.setFontHeightInPoints((short) 12);
		headerFont.setBoldweight(HSSFFont.BOLDWEIGHT_BOLD);
		headerStyle.setFont(headerFont);

		// 单元格样式
		CellStyle cellStyle = workbook.createCellStyle();
		cellStyle.setAlignment(HSSFCellStyle.ALIGN_CENTER);
		cellStyle.setVerticalAlignment(HSSFCellStyle.VERTICAL_CENTER);
		cellStyle.setBorderTop(HSSFCellStyle.BORDER_THIN);
		cellStyle.setBorderRight(HSSFCellStyle.BORDER_THIN);
		cellStyle.setBorderBottom(HSSFCellStyle.BORDER_THIN);
		cellStyle.setBorderLeft(HSSFCellStyle.BORDER_THIN);
		Font cellFont = workbook.createFont();
		cellFont.setBoldweight(HSSFFont.BOLDWEIGHT_NORMAL);
		cellStyle.setFont(cellFont);


		String title1 = (String) titleList.get(0).get("title1");
		String title2 = (String) titleList.get(0).get("title2");
		LinkedHashMap<String, String> headMap = titleList.get(1);

		/**
		 * 生成一个(带名称)表格
		 */
		SXSSFSheet sheet = (SXSSFSheet) workbook.createSheet(title1);
		sheet.createFreezePane(0, 3, 0, 3);// (单独)冻结前三行

		/**
		 * 生成head相关信息+设置每列宽度
		 */
		int[] colWidthArr = new int[headMap.size()];// 列宽数组
		String[] headKeyArr = new String[headMap.size()];// headKey数组
		String[] headValArr = new String[headMap.size()];// headVal数组
		int i = 0;
		for (Map.Entry<String, String> entry : headMap.entrySet()) {
			headKeyArr[i] = entry.getKey();
			headValArr[i] = entry.getValue();

			int bytes = headKeyArr[i].getBytes().length;
			colWidthArr[i] = bytes < minBytes ? minBytes : bytes;
			sheet.setColumnWidth(i, colWidthArr[i] * 256);// 设置列宽
			i++;
		}


		/**
		 * 遍历数据集合,产生Excel行数据
 		 */
		int rowIndex = 0;
		for (Object obj : dataArray) {
			// 生成title+head信息
			if (rowIndex == 0) {
				SXSSFRow title1Row = (SXSSFRow) sheet.createRow(0);// title1行
				title1Row.createCell(0).setCellValue(title1);
				title1Row.getCell(0).setCellStyle(title1Style);
				sheet.addMergedRegion(new CellRangeAddress(0, 0, 0, headMap.size() - 1));// 合并单元格

				SXSSFRow title2Row = (SXSSFRow) sheet.createRow(1);// title2行
				title2Row.createCell(0).setCellValue(title2);

				CreationHelper createHelper = workbook.getCreationHelper();
				XSSFHyperlink  hyperLink = (XSSFHyperlink) createHelper.createHyperlink(Hyperlink.LINK_URL);
				hyperLink.setAddress(title2);
				title2Row.getCell(0).setHyperlink(hyperLink);// 添加超链接

				title2Row.getCell(0).setCellStyle(title2Style);
				sheet.addMergedRegion(new CellRangeAddress(1, 1, 0, headMap.size() - 1));// 合并单元格

				SXSSFRow headerRow = (SXSSFRow) sheet.createRow(2);// head行
				for (int j = 0; j < headValArr.length; j++) {
					headerRow.createCell(j).setCellValue(headValArr[j]);
					headerRow.getCell(j).setCellStyle(headerStyle);
				}
				rowIndex = 3;
			}

			JSONObject jo = (JSONObject) JSONObject.toJSON(obj);
			// 生成数据
			SXSSFRow dataRow = (SXSSFRow) sheet.createRow(rowIndex);// 创建行
			for (int k = 0; k < headKeyArr.length; k++) {
				SXSSFCell cell = (SXSSFCell) dataRow.createCell(k);// 创建单元格
				Object o = jo.get(headKeyArr[k]);
				String cellValue = "";

				if (o == null) {
					cellValue = "";
				} else if (o instanceof Date) {
					cellValue = new SimpleDateFormat(datePattern).format(o);
				} else if (o instanceof Float || o instanceof Double) {
					cellValue = new BigDecimal(o.toString()).setScale(2, BigDecimal.ROUND_HALF_UP).toString();
				} else {
					cellValue = o.toString();
				}

				if (cellValue.equals("true")) {
					cellValue = "男";
				} else if (cellValue.equals("false")) {
					cellValue = "女";
				}

				cell.setCellValue(cellValue);
				cell.setCellStyle(cellStyle);
			}
			rowIndex++;
		}

		try {
			workbook.write(os);
			os.flush();// 刷新此输出流并强制将所有缓冲的输出字节写出
			os.close();// 关闭流
			workbook.dispose();// 释放workbook所占用的所有windows资源
		} catch (IOException e) {
			e.printStackTrace();
		}
	}

}


2、Student类:

package domain;

import java.util.Date;

/**
 * Created by Cheung on 2017/12/19.
 */
public class Student {

	private String name;
	private int age;
	private Date birthday;
	private float height;
	private double weight;
	private boolean sex;

	public String getName() {
		return name;
	}

	public void setName(String name) {
		this.name = name;
	}

	public int getAge() {
		return age;
	}

	public void setAge(int age) {
		this.age = age;
	}

	public Date getBirthday() {
		return birthday;
	}

	public void setBirthday(Date birthday) {
		this.birthday = birthday;
	}

	public float getHeight() {
		return height;
	}

	public void setHeight(float height) {
		this.height = height;
	}

	public double getWeight() {
		return weight;
	}

	public void setWeight(double weight) {
		this.weight = weight;
	}

	public boolean isSex() {
		return sex;
	}

	public void setSex(boolean sex) {
		this.sex = sex;
	}
}

3、ExcelExportTest类:

package controller;

import Utils.ExcelUtil;
import com.alibaba.fastjson.JSONArray;
import domain.Student;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.util.ArrayList;
import java.util.Date;
import java.util.LinkedHashMap;

/**
 * Created by Cheung on 2017/12/19.
 */
public class ExcelExportTest {

	public static void main(String[] args) throws IOException {
		// 模拟10W条数据
		int count = 100000;
	 	JSONArray studentArray = new JSONArray();
		for (int i = 0; i < count; i++) {
			Student s = new Student();
			s.setName("POI" + i);
			s.setAge(i);
			s.setBirthday(new Date());
			s.setHeight(i);
			s.setWeight(i);
			s.setSex(i % 2 == 0 ? false : true);
			studentArray.add(s);
		}

		ArrayList<LinkedHashMap> titleList = new ArrayList<LinkedHashMap>();
		LinkedHashMap<String, String> titleMap = new LinkedHashMap<String, String>();
		titleMap.put("title1","POI导出大数据量Excel Demo");
		titleMap.put("title2","https://github.com/550690513");
		LinkedHashMap<String, String> headMap = new LinkedHashMap<String, String>();
		headMap.put("name", "姓名");
		headMap.put("age", "年龄");
		headMap.put("birthday", "生日");
		headMap.put("height", "身高");
		headMap.put("weight", "体重");
		headMap.put("sex", "性别");
		titleList.add(titleMap);
		titleList.add(headMap);



		File file = new File("D://ExcelExportDemo/");
		if (!file.exists()) file.mkdir();// 创建该文件夹目录
		OutputStream os = null;
		Date date = new Date();
		try {
			// .xlsx格式
			os = new FileOutputStream(file.getAbsolutePath() + "/" + date.getTime() + ".xlsx");
			System.out.println("正在导出xlsx...");
			ExcelUtil.exportExcel(titleList, studentArray, os);
			System.out.println("导出完成...共" + count + "条数据,用时" + (new Date().getTime() - date.getTime()) + "ms");
			System.out.println("文件路径:" + file.getAbsolutePath() + "/" + date.getTime() + ".xlsx");
		} catch (Exception e) {
			e.printStackTrace();
		} finally {
			os.close();
		}

	}
}

4、测试结果:10W条数据导出,用时3s多。






项目后续更新请移步至:https://github.com/550690513

Fork me on Github:基于Apache POI导出大数据量Excel的实现

2018-11-13 20:40:46 qq_16855077 阅读数 765

前提:

由于本人采取的是利用反射的方式,这样的好处,可以兼容,导出所有类型的数据,例如,订单,充值记录等等。

既然是公有的接口,肯定有一定的约束规范。

(1)导出的表头需要自己设置一个动态数组

(2) 头部的宽度需要自己设置,也可以自己写一个方法,根据字体的长度,形成一个头部的动态数组长度。

(3)返回的实体类,一定要遵循,跟头部一一对应的原则,例如:

导出的实体类,name要显示在excel的第一列,名字就得放在number这个字段的前面

1.pom.xml配置

<!-- poi -->
		<dependency>
			<groupId>org.apache.poi</groupId>
			<artifactId>poi</artifactId>
			<version>3.8</version>
		</dependency>
		<dependency>
			<groupId>org.apache.poi</groupId>
			<artifactId>poi-ooxml</artifactId>
			<version>3.8</version>
		</dependency>
		<dependency>
			<groupId>org.apache.poi</groupId>
			<artifactId>poi-scratchpad</artifactId>
			<version>3.8</version>
		</dependency>
		<dependency>
			<groupId>org.apache.poi</groupId>
			<artifactId>poi-ooxml-schemas</artifactId>
			<version>3.8</version>
		</dependency>

2.前端代码

2.1jsp页面代码

 <div class="float-r">
		<a class="red_btn_big addOrEditFoodCategory" type="add" onclick="list.exportReCharge()">+ 导出充值记录</a>
</div>

2.2 js代码

	/**
		 * 导出充值交易信息
		 */
		exportReCharge : function() {
			var methodOfPayment = $('#methodOfPayment option:selected').val();
			var paymentStatus = $('#paymentStatus option:selected').val();
			var serviceType = $('#serviceType option:selected').val();
			var startTime = $('#date0').val();
			var endTime = $('#date1').val();
			var keywords = $('#keywords').val();
			layer.confirm('确定导出充值数据吗?', {
				icon : 3,
				title : '提示'
			}, function(index) {
				window.location.href = "export?search=" + keywords
						+ "&channels=" + methodOfPayment + "&state="
						+ paymentStatus + "&serviceType=" + serviceType
						+ "&startTime=" + startTime + "&endTime=" + endTime;
				layer.close(index);
			});
		}

2.3后台controller代码

@RequestMapping("export")
	public void exportCharge(HttpServletRequest request, HttpServletResponse response, OrderExportVo exportVo) {
		Principal principal = getLoginAdminInfo();
		if (null == principal) {
			logger.info("--------------获取登录用户身份信息为空!");
			return;
		}
		HashMap<String, Object> conditionMap = new HashMap<>();

		try {
			boolean superAdmin = false; // 是否为超级管理员
			if(principal.getOperatorId() != null && StringUtils.isBlank(principal.getTreeIds())){// 非超级管理员
				superAdmin = true;
			} 

			PayOrderInfo payOrderInfo = new PayOrderInfo();
			payOrderInfo.setState(exportVo.getState());
			payOrderInfo.setServiceType(exportVo.getServiceType());
			payOrderInfo.setChannels(exportVo.getChannels());
			
			conditionMap.put("payOrderInfo", payOrderInfo);
			conditionMap.put("superAdmin", superAdmin);
			conditionMap.put("operatorId", principal.getOperatorId());
			conditionMap.put("adminId", principal.getTreeIds());
			conditionMap.put("search", exportVo.getSearch());
			conditionMap.put("startTime", exportVo.getStartTime());
			conditionMap.put("endTime", exportVo.getEndTime());
			Collection<OrderVo> dataset = payOrderInfoMapper.getPayOrderList(conditionMap);
			
			
			for (OrderVo orderVo : dataset) {
				String channelsName = "";
				if(orderVo.getChannels() != null){
					channelsName = ChannelsType.getType(orderVo.getChannels()).getDesc();
				}
				orderVo.setChannelsName(channelsName);
				orderVo.setStateName(OrderState.getType(orderVo.getState()).getDesc());
				String serviceTypeName="";
				if(orderVo.getServiceType() != null){
					serviceTypeName = ServiceType.getType(orderVo.getServiceType()).getDesc();
				}
				orderVo.setServiceTypeName(serviceTypeName);
			}
			
			ExportExcelUtils<OrderVo> excelUtils = new ExportExcelUtils<>();
			SXSSFWorkbook exportExcel = null;
			String[] headers = { "用户名","手机号","充值金额","退款金额", "余额", "充值平台", "订单号", "支付时间", "支付状态", "支付类型", "操作者" };
			int[] widths = {68,90,80,80,80,85,189,140,68,68,68};
			exportExcel = excelUtils.exportExcel("充值", headers,widths, dataset, response.getOutputStream());
			String currentDate = TimeUtils.getCurrentTimeStr("yyyyMMddHHmmss");
			excelUtils.createSXSSFWorkbook(exportExcel, response, "充值记录" + currentDate + ".xlsx");
		} catch (IOException e) {
			logger.info("导出充值交易记录失败!", e.getMessage());
		}
	}

2.4 excel辅助类

package com.parwa.web.util;

import java.io.IOException;
import java.io.OutputStream;
import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.sql.Timestamp;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.Collection;
import java.util.Date;
import java.util.Iterator;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServletResponse;

import org.apache.poi.hssf.usermodel.HSSFCell;
import org.apache.poi.hssf.usermodel.HSSFCellStyle;
import org.apache.poi.hssf.usermodel.HSSFClientAnchor;
import org.apache.poi.hssf.usermodel.HSSFFont;
import org.apache.poi.hssf.usermodel.HSSFPatriarch;
import org.apache.poi.hssf.usermodel.HSSFRichTextString;
import org.apache.poi.hssf.usermodel.HSSFRow;
import org.apache.poi.hssf.usermodel.HSSFSheet;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
import org.apache.poi.hssf.util.HSSFColor;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.CellStyle;
import org.apache.poi.ss.usermodel.Drawing;
import org.apache.poi.ss.usermodel.Font;
import org.apache.poi.ss.usermodel.IndexedColors;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.ss.usermodel.Workbook;
import org.apache.poi.xssf.streaming.SXSSFWorkbook;
import org.apache.poi.xssf.usermodel.XSSFClientAnchor;
import org.apache.poi.xssf.usermodel.XSSFRichTextString;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
 * 导出Excel工具 
 * @author wude
 *
 * @param <T>
 */
public class ExportExcelUtils<T> {
	private static final Logger logger = LoggerFactory.getLogger(ExportExcelUtils.class);

	public void exportExcel(String title, Collection<T> dataset, OutputStream out) {
		exportExcel(title, null, dataset, out, "yyyy-MM-dd HH:mm:ss");
	}
	
	public void exportExcel(String title, String[] headers, Collection<T> dataset, OutputStream out) {
		exportExcel(title, headers, dataset, out, "yyyy-MM-dd HH:mm:ss");
	}
	
	/**
	 * 生成excel
	 * @param title     sheet名称
	 * @param headers   表头
	 * @param width     宽
	 * @param dataset   当前list对象
	 * @param out 
	 * @return
	 */
	public SXSSFWorkbook exportExcel(String title, String[] headers,int[] width,Collection<T> dataset, OutputStream out) {
		SXSSFWorkbook exportExcel = exportBigDataExcel(title, headers,width,dataset, out, "yyyy-MM-dd HH:mm:ss");
		return exportExcel;
	}
	
	/**
	 * 创建excel文档
	 * @param wb
	 * @param response
	 * @param fileName   文件名称
	 * @throws IOException
	 */
    public  void createSXSSFWorkbook(Workbook wb,HttpServletResponse response,String fileName) throws IOException {
        ServletOutputStream out = null;
        try {
            // 设置响应头
            response.addHeader("Content-Disposition", "attachment;filename="
                    + new String(fileName.getBytes("GBK"), "ISO8859_1"));
            out = response.getOutputStream();
            wb.write(out);
        } catch (Exception e) {
        	logger.info("导出excel报错!",e.getMessage());
        } finally {
            if (out != null) {
                    out.close();
            }
        }
    }
	
	
	/**
	 * 利用了JAVA的反射机制,可以将放置在JAVA集合中并且符号一定条件的数据以EXCEL 的形式输出到指定IO设备上
	 * @param title 表格标题名
	 * @param title 列宽度
	 * @param headers 表格属性列名数组
	 * @param dataset 需要显示的数据集合,集合中一定要放置符合javabean风格的类的对象。此方法支持的javabean属性的数据类型有基本数据类型及String,Date,byte[](图片数据)
	 * @param out 与输出设备关联的流对象,可以将EXCEL文档导出到本地文件或者网络中
	 * @param pattern 如果有时间数据,设定输出格式。默认为"yyy-MM-dd HH:mm:ss"
	 */
	@SuppressWarnings({ "rawtypes", "unchecked" })
	public SXSSFWorkbook exportBigDataExcel(String title, String[] headers,int[] width,Collection<T> dataset, OutputStream out, String pattern) {
		/** 声明一个工作簿 **/
		SXSSFWorkbook workbook = new SXSSFWorkbook(100);   //100为内存缓存数据
		if (dataset == null || dataset.size() == 0) {
	            // 防止数据为空的情况下,excel无法打开
			workbook.createSheet();
	        return workbook;
	    }
		/** 声明一个表格 **/
		Sheet sheet = workbook.createSheet(title);
		/** 设置表格默认列宽度为15个字节 **/
		sheet.setDefaultColumnWidth((int) 16);
		sheet.autoSizeColumn(1);
		/** 生成样式,用于表格标题行 **/
		CellStyle style = workbook.createCellStyle();
		/** 设置样式 **/
		style.setFillForegroundColor(IndexedColors.LEMON_CHIFFON.index);
	    style.setFillPattern(CellStyle.SOLID_FOREGROUND);
	    style.setBorderBottom(CellStyle.BORDER_THIN);
	    style.setBorderLeft(CellStyle.BORDER_THIN);
	    style.setBorderRight(CellStyle.BORDER_THIN);
	    style.setBorderTop(CellStyle.BORDER_THIN);
	    style.setAlignment(CellStyle.ALIGN_CENTER);
	    /** 生成字体 **/
	    Font font = workbook.createFont();
	    font.setColor(IndexedColors.VIOLET.index);
	    font.setFontHeightInPoints((short) 13);
	    font.setBoldweight(Font.BOLDWEIGHT_BOLD);
	    /** 将字体应用到样式中 **/
	    style.setFont(font);
	    
	    /** 样式二,用于表格内容行 **/
	    CellStyle style2 = workbook.createCellStyle();
	    style2.setFillForegroundColor(IndexedColors.WHITE.index);
	    style2.setFillPattern(CellStyle.SOLID_FOREGROUND);
	    style2.setBorderBottom(CellStyle.BORDER_THIN);
	    style2.setBorderLeft(CellStyle.BORDER_THIN);
	    style2.setBorderRight(CellStyle.BORDER_THIN);
	    style2.setBorderTop(CellStyle.BORDER_THIN);
	    style2.setAlignment(CellStyle.ALIGN_CENTER); //水平布局:居中;
	    style2.setVerticalAlignment(CellStyle.VERTICAL_CENTER);
      	/** 字体二 **/
	    Font font2 = workbook.createFont();
	    font.setFontHeightInPoints((short) 12);
	    font2.setBoldweight(Font.BOLDWEIGHT_NORMAL);
	    font2.setColor(IndexedColors.BLACK.index);
	    style2.setFont(font2);
	    /** 声明画图顶级管理器 **/
	    Drawing patriarch = sheet.createDrawingPatriarch();
	    
	    /** 设置表格标题行 **/
	    Row row = sheet.createRow(0);
	    for (int i = 0; i < headers.length; i++) {
	       Cell cell = row.createCell(i);
	       cell.setCellStyle(style);
	       XSSFRichTextString text = new XSSFRichTextString(headers[i]);
	       cell.setCellValue(text);
	       sheet.setColumnWidth(i, (int) (width[i] * 35.7));
	    }
	    
	    /** 以下是数据内容 **/
	    Iterator<T> it = dataset.iterator();
	    int index = 0;
	    while(it.hasNext() && it != null){
	    	index++;
	    	row = sheet.createRow(index);
	    	T t = (T) it.next();
	    	/** 利用反射,根据javabean属性的先后顺序,动态调用getXxx()方法得到属性值 **/
	    	Field[] fields = t.getClass().getDeclaredFields();
	    	for(int i = 0;i < fields.length;i++){
	    		if(i == headers.length){
	    			break;
	    		}
	    		Cell cell = row.createCell(i);
	            
	            Field field = fields[i];
	            String fieldName = field.getName();
	            String getMethodName = "get" + fieldName.substring(0, 1).toUpperCase() + fieldName.substring(1);
	            
	            try {
	                Class tCls = t.getClass();
	                Method getMethod = tCls.getMethod(getMethodName, new Class[] {});
	                Object value = getMethod.invoke(t, new Object[] {});
	                /** 判断值的类型后进行强制类型转换 **/
	                String textValue = null;
	                if (value instanceof Boolean) {
	                	boolean bValue = (Boolean) value;
	                	textValue = "是";
	                	if (!bValue) {
	                		textValue ="否";
	                	}
	                } else if (value instanceof Date) {
	                	Date date = (Date) value;
	                	SimpleDateFormat sdf = new SimpleDateFormat(pattern);
	                    textValue = sdf.format(date);
	                } else if (value instanceof byte[]) {
	                	/** 有图片时,设置行高为60px **/
	                	row.setHeightInPoints(60);
	                	/** 设置图片所在列宽度为80px,注意这里单位的一个换算 **/
	                	sheet.setColumnWidth(i, (short) (35.7 * 80));
	                	byte[] bsValue = (byte[]) value;
	                	XSSFClientAnchor anchor = new XSSFClientAnchor(0, 0, 1023, 255, (short) 6, index, (short) 6, index);
	                 	anchor.setAnchorType(2);
	                	patriarch.createPicture(anchor, workbook.addPicture(bsValue, Workbook.PICTURE_TYPE_JPEG));
	                } else if(value instanceof Calendar){
	                	Calendar cale = Calendar.getInstance();  
	                	Date tasktime = cale.getTime();  
	                	SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
	                	textValue = df.format(tasktime);
	                } else{
	                	/** 其它数据类型都当作字符串简单处理 **/
	                	if(value != null){
	                		textValue = value.toString();
	                	}
	                }
	                /** 如果不是图片数据,就利用正则表达式判断textValue是否全部由数字组成 **/
	                if(textValue!=null){
	                	Pattern p = Pattern.compile("^//d+(//.//d+)?$");  
	                	Matcher matcher = p.matcher(textValue);
	                	if(matcher.matches()){
	                		/** 是数字当作double处理 **/
	                		cell.setCellValue(Double.parseDouble(textValue));
	                   }else{
	                	   XSSFRichTextString richString = new XSSFRichTextString(textValue);
	                	   Font font3 = workbook.createFont();
	                	   font3.setColor(IndexedColors.BLACK.index);
	                	   richString.applyFont(font3);
	                	   cell.setCellValue(richString);
	                   }
	                }else{
	                	cell.setCellValue("");
	                }
	            } catch (Exception e) {
	            	e.printStackTrace();
	            	logger.info("导出excel报错!",e.getMessage());
	            } finally {
	                //清理资源
	            }
	            cell.setCellStyle(style2);
	    	}
	    }
		return workbook;
	}
	
	/**
	 * 利用了JAVA的反射机制,可以将放置在JAVA集合中并且符号一定条件的数据以EXCEL 的形式输出到指定IO设备上
	 * @param title 表格标题名
	 * @param headers 表格属性列名数组
	 * @param dataset 需要显示的数据集合,集合中一定要放置符合javabean风格的类的对象。此方法支持的javabean属性的数据类型有基本数据类型及String,Date,byte[](图片数据)
	 * @param out 与输出设备关联的流对象,可以将EXCEL文档导出到本地文件或者网络中
	 * @param pattern 如果有时间数据,设定输出格式。默认为"yyy-MM-dd HH:mm:ss"
	 */
	@SuppressWarnings({ "rawtypes", "unchecked" })
	public HSSFWorkbook exportExcel(String title, String[] headers, Collection<T> dataset, OutputStream out, String pattern) {
		/** 声明一个工作簿 **/
		HSSFWorkbook workbook = new HSSFWorkbook();
		/** 声明一个表格 **/
		HSSFSheet sheet = workbook.createSheet(title);
		/** 设置表格默认列宽度为15个字节 **/
		sheet.setDefaultColumnWidth((int) 16);
		sheet.autoSizeColumn(1);
		/** 生成样式,用于表格标题行 **/
		HSSFCellStyle style = workbook.createCellStyle();
		/** 设置样式 **/
		style.setFillForegroundColor(HSSFColor.LEMON_CHIFFON.index);
	    style.setFillPattern(HSSFCellStyle.SOLID_FOREGROUND);
	    style.setBorderBottom(HSSFCellStyle.BORDER_THIN);
	    style.setBorderLeft(HSSFCellStyle.BORDER_THIN);
	    style.setBorderRight(HSSFCellStyle.BORDER_THIN);
	    style.setBorderTop(HSSFCellStyle.BORDER_THIN);
	    style.setAlignment(HSSFCellStyle.ALIGN_CENTER);
	    /** 生成字体 **/
	    HSSFFont font = workbook.createFont();
	    font.setColor(HSSFColor.VIOLET.index);
	    font.setFontHeightInPoints((short) 12);
	    font.setBoldweight(HSSFFont.BOLDWEIGHT_BOLD);
	    /** 将字体应用到样式中 **/
	    style.setFont(font);
	    
	    /** 样式二,用于表格内容行 **/
	    HSSFCellStyle style2 = workbook.createCellStyle();
	    style2.setFillForegroundColor(HSSFColor.WHITE.index);
	    style2.setFillPattern(HSSFCellStyle.SOLID_FOREGROUND);
	    style2.setBorderBottom(HSSFCellStyle.BORDER_THIN);
	    style2.setBorderLeft(HSSFCellStyle.BORDER_THIN);
	    style2.setBorderRight(HSSFCellStyle.BORDER_THIN);
	    style2.setBorderTop(HSSFCellStyle.BORDER_THIN);
	    style2.setAlignment(HSSFCellStyle.ALIGN_LEFT);
	    style2.setVerticalAlignment(HSSFCellStyle.VERTICAL_CENTER);
      	/** 字体二 **/
	    HSSFFont font2 = workbook.createFont();
	    font2.setBoldweight(HSSFFont.BOLDWEIGHT_NORMAL);
	    font2.setColor(HSSFColor.BLACK.index);
	    style2.setFont(font2);
	    /** 声明画图顶级管理器 **/
	    HSSFPatriarch patriarch = sheet.createDrawingPatriarch();
	    
	    /** 设置表格标题行 **/
	    HSSFRow row = sheet.createRow(0);
	    for (int i = 0; i < headers.length; i++) {
	       HSSFCell cell = row.createCell(i);
	       cell.setCellStyle(style);
	       HSSFRichTextString text = new HSSFRichTextString(headers[i]);
	       cell.setCellValue(text);
	    }
	    
	    /** 以下是数据内容 **/
	    Iterator<T> it = dataset.iterator();
	    int index = 0;
	    while(it.hasNext() && it != null){
	    	index++;
	    	row = sheet.createRow(index);
	    	T t = (T) it.next();
	    	/** 利用反射,根据javabean属性的先后顺序,动态调用getXxx()方法得到属性值 **/
	    	Field[] fields = t.getClass().getDeclaredFields();
	    	for(int i = 0;i < fields.length;i++){
	    		HSSFCell cell = row.createCell(i);
	            
	            Field field = fields[i];
	            String fieldName = field.getName();
	            String getMethodName = "get" + fieldName.substring(0, 1).toUpperCase() + fieldName.substring(1);
	            
	            try {
	                Class tCls = t.getClass();
	                Method getMethod = tCls.getMethod(getMethodName, new Class[] {});
	                Object value = getMethod.invoke(t, new Object[] {});
	                /** 判断值的类型后进行强制类型转换 **/
	                String textValue = null;
	                if (value instanceof Boolean) {
	                	boolean bValue = (Boolean) value;
	                	textValue = "是";
	                	if (!bValue) {
	                		textValue ="否";
	                	}
	                } else if (value instanceof Date) {
	                	Date date = (Date) value;
	                	SimpleDateFormat sdf = new SimpleDateFormat(pattern);
	                    textValue = sdf.format(date);
	                } else if (value instanceof byte[]) {
	                	/** 有图片时,设置行高为60px **/
	                	row.setHeightInPoints(60);
	                	/** 设置图片所在列宽度为80px,注意这里单位的一个换算 **/
	                	sheet.setColumnWidth(i, (short) (35.7 * 80));
	                	byte[] bsValue = (byte[]) value;
	                	HSSFClientAnchor anchor = new HSSFClientAnchor(0, 0, 1023, 255, (short) 6, index, (short) 6, index);
	                 	anchor.setAnchorType(2);
	                	patriarch.createPicture(anchor, workbook.addPicture(bsValue, HSSFWorkbook.PICTURE_TYPE_JPEG));
	                } else if(value instanceof Calendar){
	                	Calendar cale = Calendar.getInstance();  
	                	Date tasktime = cale.getTime();  
	                	SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
	                	textValue = df.format(tasktime);
	                } else{
	                	/** 其它数据类型都当作字符串简单处理 **/
	                	if(value != null){
	                		textValue = value.toString();
	                	}
	                }
	                /** 如果不是图片数据,就利用正则表达式判断textValue是否全部由数字组成 **/
	                if(textValue!=null){
	                	Pattern p = Pattern.compile("^//d+(//.//d+)?$");  
	                	Matcher matcher = p.matcher(textValue);
	                	if(matcher.matches()){
	                		/** 是数字当作double处理 **/
	                		cell.setCellValue(Double.parseDouble(textValue));
	                   }else{
	                	   HSSFRichTextString richString = new HSSFRichTextString(textValue);
	                	   HSSFFont font3 = workbook.createFont();
	                	   font3.setColor(HSSFColor.BLACK.index);
	                	   richString.applyFont(font3);
	                	   cell.setCellValue(richString);
	                   }
	                }else{
	                	cell.setCellValue("");
	                }
	            } catch (SecurityException e) {
	                e.printStackTrace();
	            } catch (NoSuchMethodException e) {
	                e.printStackTrace();
	            } catch (IllegalArgumentException e) {
	                e.printStackTrace();
	            } catch (IllegalAccessException e) {
	                e.printStackTrace();
	            } catch (InvocationTargetException e) {
	                e.printStackTrace();
	            } finally {
	                //清理资源
	            }
	    	}
	    }
		return workbook;
	}
	
	
	public static void main(String[] args) {
		Timestamp time = TimeUtils.getCurrentTime();
		if(time instanceof Date){
			System.out.println("时间类型");
		}
	}
}

可能存在问题 

实体类

 因为set get是遵守驼峰规则的,按道理生成的get方法应该是getHMaxTemp,但是这里却不是,就是因为h后面接的是大写字母,如果是thMaxTemp,生成的get方法,就是getThMaxTemp,所以编码的时候,保持一个良好的习惯很重要,但是,项目用了一段时间,如果只是这个大小写的问题,就大改动,这样改动量也太大勒,这里我分享另外一种方法。

找到这个工具类的String getMethodName = "get" + fieldName.substring(0, 1).toUpperCase() + fieldName.substring(1);,如上图所示换成下面的图片效果

需要增加一个辅助反射类

ReflectUtil.java


package com.cloudtech.web.util;
 
import java.lang.reflect.Field;
import java.util.ArrayList;

import com.cloudtech.web.entity.RuleCode;
 
public class ReflectUtil {
 
    /**
     * 使用反射设置变量值
     *
     * @param target 被调用对象
     * @param fieldName 被调用对象的字段,一般是成员变量或静态变量,不可是常量!
     * @param value 值
     * @param <T> value类型,泛型
     */
    public static <T> void setValue(Object target,String fieldName,T value) {
        try {
            Class c = target.getClass();
            Field f = c.getDeclaredField(fieldName);
            f.setAccessible(true);
            f.set(target, value);
        } catch (NoSuchFieldException e) {
            e.printStackTrace();
        } catch (IllegalAccessException e) {
            e.printStackTrace();
        }
    }
 
    /**
     * 使用反射获取变量值
     *
     * @param target 被调用对象
     * @param fieldName 被调用对象的字段,一般是成员变量或静态变量,不可以是常量
     * @param <T> 返回类型,泛型
     * @return 值
     */
    public static <T> T getValue(Object target,String fieldName) {
        T value = null;
        try {
            Class c = target.getClass();
            Field f = c.getDeclaredField(fieldName);
            f.setAccessible(true);
            value = (T) f.get(target);
        } catch (NoSuchFieldException e) {
            e.printStackTrace();
        } catch (IllegalAccessException e) {
            e.printStackTrace();
        }
        return value;
    }
    
    public static void main(String[] args) {
		RuleCode code = new RuleCode();
		code.setId(1234);
		//Object value = ReflectUtil.getValue(code, "id");
		System.out.println(code.getId());
    /*	String s ="avgSpd gt hThreeMaxSpd";
    	String replace = s.replace("gt", "");
    	String[] split = replace.split(" ");
    	System.out.println();*/
	}
}

 3.对比

建议用后面这种,毕竟可以兼容实体类字段命名不规范的问题

4.优化后excel辅助类

注意:使用的时候,使用反射方式,去掉多余的代码,使用SXSSFWorkbook,性能更优

package com.cloudtech.web.util;

import java.io.IOException;
import java.io.OutputStream;
import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.math.BigDecimal;
import java.sql.Timestamp;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.Collection;
import java.util.Date;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.Map.Entry;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServletResponse;

import org.apache.poi.hssf.usermodel.HSSFCell;
import org.apache.poi.hssf.usermodel.HSSFCellStyle;
import org.apache.poi.hssf.usermodel.HSSFClientAnchor;
import org.apache.poi.hssf.usermodel.HSSFFont;
import org.apache.poi.hssf.usermodel.HSSFPatriarch;
import org.apache.poi.hssf.usermodel.HSSFRichTextString;
import org.apache.poi.hssf.usermodel.HSSFRow;
import org.apache.poi.hssf.usermodel.HSSFSheet;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
import org.apache.poi.hssf.util.HSSFColor;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.CellStyle;
import org.apache.poi.ss.usermodel.Drawing;
import org.apache.poi.ss.usermodel.Font;
import org.apache.poi.ss.usermodel.IndexedColors;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.ss.usermodel.Workbook;
import org.apache.poi.xssf.streaming.SXSSFWorkbook;
import org.apache.poi.xssf.usermodel.XSSFRichTextString;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * 导出Excel工具
 * 
 * @author wude
 *
 * @param <T>
 */
public class ExportExcelUtils<T> {
	private static final Logger logger = LoggerFactory.getLogger(ExportExcelUtils.class);

	public void exportExcel(String title, Collection<T> dataset, OutputStream out) {
		exportExcel(title, null, dataset, out, "yyyy-MM-dd HH:mm:ss");
	}

	/**
	 * 生成excel
	 * 
	 * @param title
	 *            sheet名称
	 * @param headers
	 *            表头
	 * @param width
	 *            宽
	 * @param dataset
	 *            当前list对象
	 * @param out
	 * @return
	 */
	public SXSSFWorkbook exportExcel(String title, String[] headers, int[] width, Collection<T> dataset,
			OutputStream out) {
		SXSSFWorkbook exportExcel = exportBigDataExcel(title, headers, width, dataset, out, "yyyy-MM-dd HH:mm:ss");
		return exportExcel;
	}

	public SXSSFWorkbook exportMapExcel(String title, String[] headers, int[] width,
			LinkedHashMap<String, LinkedHashMap<String, Object>> dataset, OutputStream out) {
		SXSSFWorkbook exportExcel = exportBigMapDataExcel(title, headers, width, dataset, out, "yyyy-MM-dd HH:mm:ss");
		return exportExcel;
	}

	/**
	 * 创建excel文档
	 * 
	 * @param wb
	 * @param response
	 * @param fileName
	 *            文件名称
	 * @throws IOException
	 */
	public void createSXSSFWorkbook(Workbook wb, HttpServletResponse response, String fileName) throws IOException {
		ServletOutputStream out = null;
		try {
			// 设置响应头
			response.addHeader("Content-Disposition",
					"attachment;filename=" + new String(fileName.getBytes("GBK"), "ISO8859_1"));
			out = response.getOutputStream();
			wb.write(out);
		} catch (Exception e) {
			logger.info("导出excel报错!", e.getMessage());
		} finally {
			if (out != null) {
				out.close();
			}
		}
	}

	/**
	 * 利用了JAVA的反射机制,可以将放置在JAVA集合中并且符号一定条件的数据以EXCEL 的形式输出到指定IO设备上
	 * 
	 * @param title
	 *            表格标题名
	 * @param title
	 *            列宽度
	 * @param headers
	 *            表格属性列名数组
	 * @param dataset
	 *            需要显示的数据集合,集合中一定要放置符合javabean风格的类的对象。
	 *            此方法支持的javabean属性的数据类型有基本数据类型及String,Date,byte[](图片数据)
	 * @param out
	 *            与输出设备关联的流对象,可以将EXCEL文档导出到本地文件或者网络中
	 * @param pattern
	 *            如果有时间数据,设定输出格式。默认为"yyy-MM-dd HH:mm:ss"
	 */
	@SuppressWarnings({ "rawtypes", "unchecked" })
	public SXSSFWorkbook exportBigDataExcel(String title, String[] headers, int[] width, Collection<T> dataset,
			OutputStream out, String pattern) {
		/** 声明一个工作簿 **/
		SXSSFWorkbook workbook = new SXSSFWorkbook(100); // 100为内存缓存数据
		if (dataset == null || dataset.size() == 0) {
			// 防止数据为空的情况下,excel无法打开
			workbook.createSheet();
			return workbook;
		}
		/** 声明一个表格 **/
		Sheet sheet = workbook.createSheet(title);
		/** 设置表格默认列宽度为15个字节 **/
		sheet.setDefaultColumnWidth((int) 16);
		sheet.autoSizeColumn(1);
		/** 生成样式,用于表格标题行 **/
		CellStyle style = workbook.createCellStyle();
		/** 设置样式 **/
		style.setFillForegroundColor(IndexedColors.LEMON_CHIFFON.index);
		style.setFillPattern(CellStyle.SOLID_FOREGROUND);
		style.setBorderBottom(CellStyle.BORDER_THIN);
		style.setBorderLeft(CellStyle.BORDER_THIN);
		style.setBorderRight(CellStyle.BORDER_THIN);
		style.setBorderTop(CellStyle.BORDER_THIN);
		style.setAlignment(CellStyle.ALIGN_CENTER);
		/** 生成字体 **/
		Font font = workbook.createFont();
		font.setColor(IndexedColors.VIOLET.index);
		font.setFontHeightInPoints((short) 13);
		font.setBoldweight(Font.BOLDWEIGHT_BOLD);
		/** 将字体应用到样式中 **/
		style.setFont(font);

		/** 样式二,用于表格内容行 **/
		CellStyle style2 = workbook.createCellStyle();
		style2.setFillForegroundColor(IndexedColors.WHITE.index);
		style2.setFillPattern(CellStyle.SOLID_FOREGROUND);
		style2.setBorderBottom(CellStyle.BORDER_THIN);
		style2.setBorderLeft(CellStyle.BORDER_THIN);
		style2.setBorderRight(CellStyle.BORDER_THIN);
		style2.setBorderTop(CellStyle.BORDER_THIN);
		style2.setAlignment(CellStyle.ALIGN_CENTER); // 水平布局:居中;
		style2.setVerticalAlignment(CellStyle.VERTICAL_CENTER);
		/** 字体二 **/
		Font font2 = workbook.createFont();
		font.setFontHeightInPoints((short) 12);
		font2.setBoldweight(Font.BOLDWEIGHT_NORMAL);
		font2.setColor(IndexedColors.BLACK.index);
		style2.setFont(font2);
		/** 声明画图顶级管理器 **/
		Drawing patriarch = sheet.createDrawingPatriarch();

		/** 设置表格标题行 **/
		Row row = sheet.createRow(0);
		for (int i = 0; i < headers.length; i++) {
			Cell cell = row.createCell(i);
			cell.setCellStyle(style);
			XSSFRichTextString text = new XSSFRichTextString(headers[i]);
			cell.setCellValue(text);
			sheet.setColumnWidth(i, (int) (width[i] * 35.7));
		}

		/** 以下是数据内容 **/
		Iterator<T> it = dataset.iterator();
		int index = 0;
		while (it.hasNext() && it != null) {
			index++;
			row = sheet.createRow(index);
			T t = (T) it.next();
			/** 利用反射,根据javabean属性的先后顺序,动态调用getXxx()方法得到属性值 **/
			Field[] fields = t.getClass().getDeclaredFields();
			for (int i = 0; i < fields.length; i++) {
				if (i == headers.length) {
					break;
				}
				Cell cell = row.createCell(i);

				Field field = fields[i];
				String fieldName = field.getName();

				try {
					Object value = ReflectUtil.getValue(t, fieldName);
					/** 判断值的类型后进行强制类型转换 **/
					String textValue = null;
					if (value == null) {
						cell.setCellValue("");
					} else {
						if (value instanceof Boolean) {
							boolean bValue = (Boolean) value;
							textValue = "是";
							if (!bValue) {
								textValue = "否";
							}
							cell.setCellValue(textValue);
						} else if (value instanceof Date) {
							Date date = (Date) value;
							SimpleDateFormat sdf = new SimpleDateFormat(pattern);
							textValue = sdf.format(date);
							cell.setCellValue(textValue);
						} else if (value instanceof Integer) {
							if (value != null) {
								int intValue = new BigDecimal(value.toString()).intValue();
								cell.setCellValue(intValue);
							}
						} else {
							/** 其它数据类型都当作字符串简单处理 **/
							if (value != null) {
								textValue = value.toString();
								cell.setCellValue(textValue);
							}
						}
					}
				} catch (Exception e) {
					e.printStackTrace();
					logger.info("导出excel报错!", e.getMessage());
				} finally {
					// 清理资源
				}
				cell.setCellStyle(style2);
			}
		}
		return workbook;
	}

	@SuppressWarnings({ "rawtypes", "unchecked" })
	public SXSSFWorkbook exportBigMapDataExcel(String title, String[] headers, int[] width,
			LinkedHashMap<String, LinkedHashMap<String, Object>> dataset, OutputStream out, String pattern) {
		/** 声明一个工作簿 **/
		SXSSFWorkbook workbook = new SXSSFWorkbook(100); // 100为内存缓存数据
		if (dataset == null || dataset.size() == 0) {
			// 防止数据为空的情况下,excel无法打开
			workbook.createSheet();
			return workbook;
		}
		/** 声明一个表格 **/
		Sheet sheet = workbook.createSheet(title);
		/** 设置表格默认列宽度为15个字节 **/
		sheet.setDefaultColumnWidth((int) 16);
		sheet.autoSizeColumn(1);
		/** 生成样式,用于表格标题行 **/
		CellStyle style = workbook.createCellStyle();
		/** 设置样式 **/
		style.setFillForegroundColor(IndexedColors.LEMON_CHIFFON.index);
		style.setFillPattern(CellStyle.SOLID_FOREGROUND);
		style.setBorderBottom(CellStyle.BORDER_THIN);
		style.setBorderLeft(CellStyle.BORDER_THIN);
		style.setBorderRight(CellStyle.BORDER_THIN);
		style.setBorderTop(CellStyle.BORDER_THIN);
		style.setAlignment(CellStyle.ALIGN_CENTER);
		/** 生成字体 **/
		Font font = workbook.createFont();
		font.setColor(IndexedColors.VIOLET.index);
		font.setFontHeightInPoints((short) 13);
		font.setBoldweight(Font.BOLDWEIGHT_BOLD);
		/** 将字体应用到样式中 **/
		style.setFont(font);

		/** 样式二,用于表格内容行 **/
		CellStyle style2 = workbook.createCellStyle();
		style2.setFillForegroundColor(IndexedColors.WHITE.index);
		style2.setFillPattern(CellStyle.SOLID_FOREGROUND);
		style2.setBorderBottom(CellStyle.BORDER_THIN);
		style2.setBorderLeft(CellStyle.BORDER_THIN);
		style2.setBorderRight(CellStyle.BORDER_THIN);
		style2.setBorderTop(CellStyle.BORDER_THIN);
		style2.setAlignment(CellStyle.ALIGN_CENTER); // 水平布局:居中;
		style2.setVerticalAlignment(CellStyle.VERTICAL_CENTER);
		/** 字体二 **/
		Font font2 = workbook.createFont();
		font.setFontHeightInPoints((short) 12);
		font2.setBoldweight(Font.BOLDWEIGHT_NORMAL);
		font2.setColor(IndexedColors.BLACK.index);
		style2.setFont(font2);
		/** 声明画图顶级管理器 **/
		Drawing patriarch = sheet.createDrawingPatriarch();

		/** 设置表格标题行 **/
		Row row = sheet.createRow(0);
		for (int i = 0; i < headers.length; i++) {
			Cell cell = row.createCell(i);
			cell.setCellStyle(style);
			XSSFRichTextString text = new XSSFRichTextString(headers[i]);
			cell.setCellValue(text);
			sheet.setColumnWidth(i, (int) (width[i] * 35.7));
		}

		/** 以下是数据内容 **/
		int index = 0;
		for (Entry<String, LinkedHashMap<String, Object>> key : dataset.entrySet()) {
			index++;
			row = sheet.createRow(index);

			Cell cell = row.createCell(0);
			cell.setCellValue(key.getKey());
			cell.setCellStyle(style2);

			LinkedHashMap<String, Object> hourValues = key.getValue();
			int i = 1;
			for (Entry<String, Object> hourKey : hourValues.entrySet()) {
				cell = row.createCell(i);
				try {
					/** 判断值的类型后进行强制类型转换 **/
					String textValue = hourKey.getValue().toString();
					cell.setCellValue(textValue);
				} catch (Exception e) {
					e.printStackTrace();
					logger.info("导出excel报错!", e.getMessage());
				} finally {
					// 清理资源
				}
				cell.setCellStyle(style2);
				i++;
			}
		}
		/*
		 * Iterator<T> it = (Iterator<T>) dataset.keySet().iterator(); int index
		 * = 0; while (it.hasNext() && it != null) { index++; row =
		 * sheet.createRow(index); T t = (T) it.next();
		 *//** 利用反射,根据javabean属性的先后顺序,动态调用getXxx()方法得到属性值 **/
		/*
		 * Field[] fields = t.getClass().getDeclaredFields(); for (int i = 0; i
		 * < fields.length; i++) { if (i == headers.length) { break; } Cell cell
		 * = row.createCell(i);
		 * 
		 * Field field = fields[i]; String fieldName = field.getName(); //
		 * String getMethodName = "get" + fieldName.substring(0, //
		 * 1).toUpperCase() + fieldName.substring(1);
		 * 
		 * try { Class tCls = t.getClass(); Object value =
		 * ReflectUtil.getValue(t, fieldName); // Method getMethod =
		 * tCls.getMethod(getMethodName, new // Class[] {}); // Object value =
		 * getMethod.invoke(t, new Object[] {});
		 *//** 判断值的类型后进行强制类型转换 **/
		/*
		 * String textValue = null; if (value instanceof Boolean) { boolean
		 * bValue = (Boolean) value; textValue = "是"; if (!bValue) { textValue =
		 * "否"; } } else if (value instanceof Date) { Date date = (Date) value;
		 * SimpleDateFormat sdf = new SimpleDateFormat(pattern); textValue =
		 * sdf.format(date); } else if (value instanceof byte[]) {
		 *//** 有图片时,设置行高为60px **/
		/*
		 * row.setHeightInPoints(60);
		 *//** 设置图片所在列宽度为80px,注意这里单位的一个换算 **/
		/*
		 * sheet.setColumnWidth(i, (short) (35.7 * 80)); byte[] bsValue =
		 * (byte[]) value; XSSFClientAnchor anchor = new XSSFClientAnchor(0, 0,
		 * 1023, 255, (short) 6, index, (short) 6, index);
		 * anchor.setAnchorType(2); patriarch.createPicture(anchor,
		 * workbook.addPicture(bsValue, Workbook.PICTURE_TYPE_JPEG)); } else if
		 * (value instanceof Calendar) { Calendar cale = Calendar.getInstance();
		 * Date tasktime = cale.getTime(); SimpleDateFormat df = new
		 * SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); textValue =
		 * df.format(tasktime); } else {
		 *//** 其它数据类型都当作字符串简单处理 **/
		/*
		 * if (value != null) { textValue = value.toString(); } }
		 *//** 如果不是图片数据,就利用正则表达式判断textValue是否全部由数字组成 **/
		/*
		 * if (textValue != null) { Pattern p =
		 * Pattern.compile("^//d+(//.//d+)?$"); Matcher matcher =
		 * p.matcher(textValue); if (matcher.matches()) {
		 *//** 是数字当作double处理 **//*
								 * cell.setCellValue(Double.parseDouble(
								 * textValue)); } else { XSSFRichTextString
								 * richString = new
								 * XSSFRichTextString(textValue); Font font3 =
								 * workbook.createFont();
								 * font3.setColor(IndexedColors.BLACK.index);
								 * richString.applyFont(font3);
								 * cell.setCellValue(richString); } } else {
								 * cell.setCellValue(""); } } catch (Exception
								 * e) { e.printStackTrace();
								 * logger.info("导出excel报错!", e.getMessage()); }
								 * finally { // 清理资源 }
								 * cell.setCellStyle(style2); } }
								 */
		return workbook;
	}

	/**
	 * 利用了JAVA的反射机制,可以将放置在JAVA集合中并且符号一定条件的数据以EXCEL 的形式输出到指定IO设备上
	 * 
	 * @param title
	 *            表格标题名
	 * @param headers
	 *            表格属性列名数组
	 * @param dataset
	 *            需要显示的数据集合,集合中一定要放置符合javabean风格的类的对象。
	 *            此方法支持的javabean属性的数据类型有基本数据类型及String,Date,byte[](图片数据)
	 * @param out
	 *            与输出设备关联的流对象,可以将EXCEL文档导出到本地文件或者网络中
	 * @param pattern
	 *            如果有时间数据,设定输出格式。默认为"yyy-MM-dd HH:mm:ss"
	 */
	@SuppressWarnings({ "rawtypes", "unchecked" })
	public HSSFWorkbook exportExcel(String title, String[] headers, Collection<T> dataset, OutputStream out,
			String pattern) {
		/** 声明一个工作簿 **/
		HSSFWorkbook workbook = new HSSFWorkbook();
		/** 声明一个表格 **/
		HSSFSheet sheet = workbook.createSheet(title);
		/** 设置表格默认列宽度为15个字节 **/
		sheet.setDefaultColumnWidth((int) 16);
		sheet.autoSizeColumn(1);
		/** 生成样式,用于表格标题行 **/
		HSSFCellStyle style = workbook.createCellStyle();
		/** 设置样式 **/
		style.setFillForegroundColor(HSSFColor.LEMON_CHIFFON.index);
		style.setFillPattern(HSSFCellStyle.SOLID_FOREGROUND);
		style.setBorderBottom(HSSFCellStyle.BORDER_THIN);
		style.setBorderLeft(HSSFCellStyle.BORDER_THIN);
		style.setBorderRight(HSSFCellStyle.BORDER_THIN);
		style.setBorderTop(HSSFCellStyle.BORDER_THIN);
		style.setAlignment(HSSFCellStyle.ALIGN_CENTER);
		/** 生成字体 **/
		HSSFFont font = workbook.createFont();
		font.setColor(HSSFColor.VIOLET.index);
		font.setFontHeightInPoints((short) 12);
		font.setBoldweight(HSSFFont.BOLDWEIGHT_BOLD);
		/** 将字体应用到样式中 **/
		style.setFont(font);

		/** 样式二,用于表格内容行 **/
		HSSFCellStyle style2 = workbook.createCellStyle();
		style2.setFillForegroundColor(HSSFColor.WHITE.index);
		style2.setFillPattern(HSSFCellStyle.SOLID_FOREGROUND);
		style2.setBorderBottom(HSSFCellStyle.BORDER_THIN);
		style2.setBorderLeft(HSSFCellStyle.BORDER_THIN);
		style2.setBorderRight(HSSFCellStyle.BORDER_THIN);
		style2.setBorderTop(HSSFCellStyle.BORDER_THIN);
		style2.setAlignment(HSSFCellStyle.ALIGN_LEFT);
		style2.setVerticalAlignment(HSSFCellStyle.VERTICAL_CENTER);
		/** 字体二 **/
		HSSFFont font2 = workbook.createFont();
		font2.setBoldweight(HSSFFont.BOLDWEIGHT_NORMAL);
		font2.setColor(HSSFColor.BLACK.index);
		style2.setFont(font2);
		/** 声明画图顶级管理器 **/
		HSSFPatriarch patriarch = sheet.createDrawingPatriarch();

		/** 设置表格标题行 **/
		HSSFRow row = sheet.createRow(0);
		for (int i = 0; i < headers.length; i++) {
			HSSFCell cell = row.createCell(i);
			cell.setCellStyle(style);
			HSSFRichTextString text = new HSSFRichTextString(headers[i]);
			cell.setCellValue(text);
		}

		/** 以下是数据内容 **/
		Iterator<T> it = dataset.iterator();
		int index = 0;
		while (it.hasNext() && it != null) {
			index++;
			row = sheet.createRow(index);
			T t = (T) it.next();
			/** 利用反射,根据javabean属性的先后顺序,动态调用getXxx()方法得到属性值 **/
			Field[] fields = t.getClass().getDeclaredFields();
			for (int i = 0; i < fields.length; i++) {
				HSSFCell cell = row.createCell(i);

				Field field = fields[i];
				String fieldName = field.getName();
				String getMethodName = "get" + fieldName.substring(0, 1).toUpperCase() + fieldName.substring(1);

				try {
					Class tCls = t.getClass();
					Method getMethod = tCls.getMethod(getMethodName, new Class[] {});
					Object value = getMethod.invoke(t, new Object[] {});
					/** 判断值的类型后进行强制类型转换 **/
					String textValue = null;
					if (value instanceof Boolean) {
						boolean bValue = (Boolean) value;
						textValue = "是";
						if (!bValue) {
							textValue = "否";
						}
					} else if (value instanceof Date) {
						Date date = (Date) value;
						SimpleDateFormat sdf = new SimpleDateFormat(pattern);
						textValue = sdf.format(date);
					} else if (value instanceof byte[]) {
						/** 有图片时,设置行高为60px **/
						row.setHeightInPoints(60);
						/** 设置图片所在列宽度为80px,注意这里单位的一个换算 **/
						sheet.setColumnWidth(i, (short) (35.7 * 80));
						byte[] bsValue = (byte[]) value;
						HSSFClientAnchor anchor = new HSSFClientAnchor(0, 0, 1023, 255, (short) 6, index, (short) 6,
								index);
						anchor.setAnchorType(2);
						patriarch.createPicture(anchor, workbook.addPicture(bsValue, HSSFWorkbook.PICTURE_TYPE_JPEG));
					} else if (value instanceof Calendar) {
						Calendar cale = Calendar.getInstance();
						Date tasktime = cale.getTime();
						SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
						textValue = df.format(tasktime);
					} else {
						/** 其它数据类型都当作字符串简单处理 **/
						if (value != null) {
							textValue = value.toString();
						}
					}
					/** 如果不是图片数据,就利用正则表达式判断textValue是否全部由数字组成 **/
					if (textValue != null) {
						Pattern p = Pattern.compile("^//d+(//.//d+)?$");
						Matcher matcher = p.matcher(textValue);
						if (matcher.matches()) {
							/** 是数字当作double处理 **/
							cell.setCellValue(Double.parseDouble(textValue));
						} else {
							HSSFRichTextString richString = new HSSFRichTextString(textValue);
							HSSFFont font3 = workbook.createFont();
							font3.setColor(HSSFColor.BLACK.index);
							richString.applyFont(font3);
							cell.setCellValue(richString);
						}
					} else {
						cell.setCellValue("");
					}
				} catch (SecurityException e) {
					e.printStackTrace();
				} catch (NoSuchMethodException e) {
					e.printStackTrace();
				} catch (IllegalArgumentException e) {
					e.printStackTrace();
				} catch (IllegalAccessException e) {
					e.printStackTrace();
				} catch (InvocationTargetException e) {
					e.printStackTrace();
				} finally {
					// 清理资源
				}
			}
		}
		return workbook;
	}

	public static void main(String[] args) {
		Timestamp time = TimeUtils.getCurrentTime();
		if (time instanceof Date) {
			System.out.println("时间类型");
		}
	}

}

4.性能分析

列有69列,导出excel分析

优化前:

通过图片,我们可以发现sql的时间很长,当然跟我的列有关,69列

5w数据需要31.42s,而1w数据是6.5s,我们是不是可以做一些优化。列入拿1w作为切割,5w数据分为5次查询出来,再组装数据,是不是可以提升查询数据。所以,这里需要使用多线程。这样整个导出的时长就短了不少。

 

优化后:

通过数据分析,发现,性能方面有质的提高,而且缓存数大小,性能没有什么明显的提示,所以建议缓存数还是设置为100把 

 

缓存数:

设置为100后,超过100的部分会写入硬盘,而不是所有的都存放在内存中

性能优化部分: 

/** 如果不是图片数据,就利用正则表达式判断textValue是否全部由数字组成 **/
					if (textValue != null) {
						Pattern p = Pattern.compile("^//d+(//.//d+)?$");
						Matcher matcher = p.matcher(textValue);
						if (matcher.matches()) {
							/** 是数字当作double处理 **/
							cell.setCellValue(Double.parseDouble(textValue));
						} else {
							XSSFRichTextString richString = new XSSFRichTextString(textValue);
							Font font3 = workbook.createFont();
							font3.setColor(IndexedColors.BLACK.index);
							richString.applyFont(font3);
							cell.setCellValue(richString);
						}
					} else {
						cell.setCellValue("");
					}

主要是正则方面的判断,消耗勒大量的数据,导致时间很长

2016-07-02 10:59:43 qiaoshuai0920 阅读数 29412

Java实现导出excel表格功能,大部分都会使用apache poi,apache poi API 地址
POI之前的版本不支持大数据量处理,如果数据过多则经常报OOM错误,有时候调整JVM大小效果也不是太好。3.8版本的POI新出来了SXSSFWorkbook,可以支持大数据量的操作,只是SXSSFWorkbook只支持.xlsx格式,不支持.xls格式。
3.8版本的POI对excel的导出操作,一般只使用HSSFWorkbook以及SXSSFWorkbook,HSSFWorkbook用来处理较少的数据量,SXSSFWorkbook用来处理大数据量以及超大数据量的导出。
代码:
git地址 有3.9jar包

package qs.test;

import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;

import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.ss.util.CellReference;
import org.apache.poi.xssf.streaming.SXSSFWorkbook;

/**
 * ClassName: SXSSFTest
 * @Description: TODO
 * @author qiaoshuai
 */
public class SXSSFTest {

    public static void main(String[] args) throws IOException {
        // 创建基于stream的工作薄对象的
        SXSSFWorkbook wb = new SXSSFWorkbook(100); // keep 100 rows in memory,
                                                    // exceeding rows will be
                                                    // flushed to disk
        // SXSSFWorkbook wb = new SXSSFWorkbook();
        // wb.setCompressTempFiles(true); // temp files will be gzipped
        Sheet sh = wb.createSheet();
        // 使用createRow将信息写在内存中。
        for (int rownum = 0; rownum < 1000; rownum++) {
            Row row = sh.createRow(rownum);
            for (int cellnum = 0; cellnum < 10; cellnum++) {
                Cell cell = row.createCell(cellnum);
                String address = new CellReference(cell).formatAsString();
                cell.setCellValue(address);
            }

        }

        // Rows with rownum < 900 are flushed and not accessible
        // 当使用getRow方法访问的时候,将内存中的信息刷新到硬盘中去。
        for (int rownum = 0; rownum < 900; rownum++) {
            System.out.println(sh.getRow(rownum));
        }

        // ther last 100 rows are still in memory
        for (int rownum = 900; rownum < 1000; rownum++) {
            System.out.println(sh.getRow(rownum));
        }
        // 写入文件中
        FileOutputStream out = new FileOutputStream("G://sxssf.xlsx");
        wb.write(out);
        // 关闭文件流对象
        out.close();
        System.out.println("基于流写入执行完毕!");
    }

}

在此基础上再优化的方案是导出的Excel表格生成多个工作表即生成多个sheet。
代码:

import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.util.ArrayList;
import java.util.Date;
import java.util.LinkedHashMap;
import java.util.List;

import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;

import com.common.DateFormatUtil;

public class ExlUtil2 {

    /**
     * @param excelHeader
     *            表头信息
     * @param list
     *            要导出到excel的数据源,List类型
     * @param sheetName
     *            表名
     * @return
     */
    public static ResponseEntity<byte[]> getDataStream(ExcelHeader excelHeader,
            List list, String sheetName) {
        LinkedHashMap<String, List> map = new LinkedHashMap<String, List>();
        List<String[]> headNames = new ArrayList<String[]>();
        List<String[]> fieldNames = new ArrayList<String[]>();
        String[] sheetNames = new String[100];
        //处理Excel生成多个工作表 
        //定义为每个工作表数据为50000条
        if (list.size() > 50000) {
            int k = (list.size() + 50000) / 50000;
            for (int i = 1; i <= k; i++) {
                if (i < k) {
                    map.put(sheetName + i,
                            list.subList((i - 1) * 50000, i * 50000));
                } else {
                    map.put(sheetName + i,
                            list.subList((i - 1) * 50000, list.size()));
                }

                headNames.add(excelHeader.getHeadNames().get(0));
                fieldNames.add(excelHeader.getFieldNames().get(0));
                sheetNames[i - 1] = sheetName;
            }

        } else {
            map.put(sheetName, list);
            headNames.add(excelHeader.getHeadNames().get(0));
            fieldNames.add(excelHeader.getFieldNames().get(0));
            sheetNames[0] = sheetName;
        }

        byte[] buffer = null;

        try {
            buffer = ExcelUtil2.output(headNames, fieldNames, sheetNames, map);

        } catch (IllegalArgumentException | IllegalAccessException
                | IOException e) {
            e.printStackTrace();
        }
        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_OCTET_STREAM);

        /*
         * try { sheetName=URLEncoder.encode(sheetName,"UTF8"); } catch
         * (UnsupportedEncodingException e) { e.printStackTrace(); }
         */
        try {
            sheetName = new String(sheetName.getBytes("gbk"), "iso-8859-1");
        } catch (UnsupportedEncodingException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
        String fileGenerateTime = DateFormatUtil.toStr(new Date());
        headers.setContentDispositionFormData("attachment", sheetName
                + fileGenerateTime + ".xlsx");
        return new ResponseEntity<byte[]>(buffer, headers, HttpStatus.CREATED);
    };

}

代码