Fast export to excel

• Download source code (VB.NET and C#) — 23.4 KB

Introduction

Exporting data from a .NET application to Excel is a very common requirement. A simple search on the Web results in several examples that show us the method to copy data and put it into the Excel cells. However, there is a payload with this method: each interaction to put a value into an Excel cell requires an InterOp invocation. If the amount of data to transfer is huge, we have a problem with the performance of the method. Is there a better way to accomplish this?

Traditional «COPY CELL-BY-CELL» Method

Searching the Web for a method to transfer data to Excel, the most commonly used method consists of copying the values cell by cell into Excel. The following C# code shows how to transfer data from a DataTable to an Excel sheet, copying each value cell by cell:

for (int col = 0; col < dataTable.Columns.Count; col++)
{
    for (int row = 0; row < dataTable.Rows.Count; row++)
    {
        excelSheet.Cells[row + 1, col + 1] = 
                dataTable.Rows[row].ItemArray[col];
    }
}

Each InterOp invocation has an associated payload in performance, so a large amount of data can degenerate our application.

A «Fast Bulk-Copy» Method

Our method consists of using the Value2 property for the Range class provided by the Microsoft Excel Object Library. We can select a range of cells, and assign a value for all of them, with just one InterOp invocation. To correctly assign a value to a range of cells, we can use a bi-dimensional object array. The following C# code shows how to transfer data from a bi-dimensional object array to a range of cells:

excelSheet.get_Range("A1:H25", Type.Missing).Value2 = 
    bidimensionalObjectArray;

Measuring the Performance

The source code included with this article shows a small Windows application which uses the two described methods to export the same data to Excel. It shows the time that it takes for each method to finish. This DEMO uses the Northwind database to create an SQL Server local connection. It generates a DataSet with the content of the Customers table. To make the amount of data more significant, we duplicate the DataTable to obtain 24 copies from it. Then we apply the two methods to generate two Excel books, one for each method.

The source code includes a C# and a VB.NET version for the DEMO application. My own testing shows me that this method is about 35 times faster. Test it and arrive at your own conclusions.

History

• November 28, 2007: First publication

Export data to Excel in ASP.NET C#

It is not so rare that you have to provide some sort of export of your data stored in database. I have a feeling that clients really like to have that functionality, probably because they are more comfortable with Excel than with any other tool.

First thing that might pop to your mind is to add a reference to Office library, but that would require that you have Office on the hosting machine which might not be the case as most of hosts do not have it.

Second thing is that if you add reference to Office library you would probably end up with using of unmanaged code which, sometimes is not so easy to handle regarding releasing the memory and processes that support it.

Luckily you do not need to have Office installed on your hosting machine to produce an Excel sheet. So far, I’ve been doing this two ways:

Exporting data to CSV (Comma Separated Values) file

CSV is legacy, but still often used

DataTable result = new DataTable();
string connectionString = System.Configuration.ConfigurationManager.ConnectionStrings["AdventureWorks2008R2"].ConnectionString;
DateTime startDate = SqlDateTime.MinValue.Value;
DateTime endDate = SqlDateTime.MaxValue.Value;
if (!DateTime.TryParse(this.dpStartDate.Text, out startDate))
{
startDate = SqlDateTime.MinValue.Value;
}
if (!DateTime.TryParse(this.dpEndDate.Text, out endDate))
{
endDate = SqlDateTime.MaxValue.Value;
}
using (SqlCommand command = new SqlCommand())
{
using (SqlConnection connection = new SqlConnection(connectionString))
{
command.CommandText = "SELECT * FROM Employee WHERE HireDate BETWEEN @StartDate AND @EndDate";
command.Connection = connection;
command.CommandType = CommandType.Text;
command.Parameters.AddWithValue("@StartDate", startDate);
command.Parameters.AddWithValue("@EndDate", endDate);
using (SqlDataAdapter adapter = new SqlDataAdapter(command))
{
                 adapter.Fill(result);
                    if (result != null)
                    {
                        using (result)
                        {
                            Response.ContentEncoding = new UTF8Encoding(true);
                            Response.Charset = Encoding.UTF8.WebName;
                            Response.ContentType = "application/vnd.ms-excel; charset=utf-8";
                            Response.AddHeader("content-disposition", string.Format("attachment;filename=dataexport_{0}.csv", DateTime.Now.ToString("yyyyMMddHHmmss")));
                            StringBuilder fileContent = new StringBuilder();
                            foreach (var col in result.Columns)
                            {
                                fileContent.Append(col.ToString()   ",");
                            }
                            fileContent.Replace(",", System.Environment.NewLine, fileContent.Length - 1, 1);
                            foreach (DataRow dr in result.Rows)
                            {
                                foreach (var column in dr.ItemArray)
                                {
                                    fileContent.Append("""   column.ToString()   "",");
                                }
                                fileContent.Replace(",", System.Environment.NewLine, fileContent.Length - 1, 1);
                            }

                            //Write encoding characters first
                            Response.Write('uFEFF');

                            //Write the content
                            Response.Write(fileContent.ToString());

                            Response.End();
                        }
                    }
                }
            }
        }
    

at for exporting tabular data. Excel is pretty good in dealing with CSV data, so all you have to do in your ASP.NET application is to return CSV data with headers that will offer user to open file using Excel application.

For example I’m going to export filtered employee records from MS sample AdventureWorks database which can be found on Microsoft’s website http://msftdbprodsamples.codeplex.com/releases/view/93587

 DataTable result = new DataTable();
        string connectionString = System.Configuration.ConfigurationManager.ConnectionStrings["AdventureWorks2008R2"].ConnectionString;
        DateTime startDate = SqlDateTime.MinValue.Value;
        DateTime endDate = SqlDateTime.MaxValue.Value;

        if (!DateTime.TryParse(this.dpStartDate.Text, out startDate))
        {
            startDate = SqlDateTime.MinValue.Value;
        }
        if (!DateTime.TryParse(this.dpEndDate.Text, out endDate))
        {
            endDate = SqlDateTime.MaxValue.Value;
        }
        using (SqlCommand command = new SqlCommand())
        {
            using (SqlConnection connection = new SqlConnection(connectionString))
            {
                command.CommandText = "SELECT * FROM Employee WHERE HireDate BETWEEN @StartDate AND @EndDate";
                command.Connection = connection;
                command.CommandType = CommandType.Text;
                command.Parameters.AddWithValue("@StartDate", startDate);
                command.Parameters.AddWithValue("@EndDate", endDate);
                using (SqlDataAdapter adapter = new SqlDataAdapter(command))
                {
                    adapter.Fill(result);
                    if (result != null)
                    {
                        using (result)
                        {
                            Response.ContentEncoding = new UTF8Encoding(true);
                            Response.Charset = Encoding.UTF8.WebName;
                            Response.ContentType = "application/vnd.ms-excel; charset=utf-8";
                            Response.AddHeader("content-disposition", string.Format("attachment;filename=dataexport_{1}.csv", DateTime.Now.ToString("yyyyMMddHHmmss")));
                            StringBuilder fileContent = new StringBuilder();
                            foreach (var col in result.Columns)
                            {
                                fileContent.Append(col.ToString() + ",");
                            }
                            fileContent.Replace(",", System.Environment.NewLine, fileContent.Length - 1, 1);
                            foreach (DataRow dr in result.Rows)
                            {
                                foreach (var column in dr.ItemArray)
                                {
                                    fileContent.Append(""" + column.ToString() + "",");
                                }
                                fileContent.Replace(",", System.Environment.NewLine, fileContent.Length - 1, 1);
                            }

                            //Write encoding characters first
                            Response.Write('uFEFF');

                            //Write the content
                            Response.Write(fileContent.ToString());

                            Response.End();
                        }
                    }
                }
            }
        }
    

Exporting data as HTML table

This is another way to push non native Excel file to Excel and make it displayed as tabular data. The idea is to load DataTable or any collection data source and bind it to a GridView.

When it’s binded, render out control layout which will be an HTML table and push it to excel by attaching headers, same as in previous example.

 DataTable result = new DataTable();
        string connectionString = System.Configuration.ConfigurationManager.ConnectionStrings["AdventureWorks2008R2"].ConnectionString;
        DateTime startDate = SqlDateTime.MinValue.Value;
        DateTime endDate = SqlDateTime.MaxValue.Value;

        if (!DateTime.TryParse(this.dpStartDate.Text, out startDate))
        {
            startDate = SqlDateTime.MinValue.Value;
        }
        if (!DateTime.TryParse(this.dpEndDate.Text, out endDate))
        {
            endDate = SqlDateTime.MaxValue.Value;
        }
        using (SqlCommand command = new SqlCommand())
        {
            using (SqlConnection connection = new SqlConnection(connectionString))
            {
                command.CommandText = "SELECT * FROM Employee WHERE HireDate BETWEEN @StartDate AND @EndDate";
                command.Connection = connection;
                command.CommandType = CommandType.Text;
                command.Parameters.AddWithValue("@StartDate", startDate);
                command.Parameters.AddWithValue("@EndDate", endDate);
                using (SqlDataAdapter adapter = new SqlDataAdapter(command))
                {
                    adapter.Fill(result);
                    if (result != null)
                    {
                        using (result)
                        {
                            Response.ContentEncoding = new UTF8Encoding(true);
                            Response.Charset = Encoding.UTF8.WebName;
                            Response.ContentType = "application/vnd.ms-excel; charset=utf-8";
                            Response.AddHeader("content-disposition", string.Format("attachment;filename=dataexport_{0}.xls", DateTime.Now.ToString("yyyyMMddHHmmss")));

                            StringWriter sw = new StringWriter();
                            HtmlTextWriter hw = new HtmlTextWriter(sw);
                            var grid = new GridView();
                            grid.DataSource = result;
                            grid.DataBind();
                            grid.RenderControl(hw);


                            //Write encoding characters first
                            Response.Write('uFEFF');

                            //Write the content
                            Response.Write(sw);

                            Response.End();
                        }
                    }
                }
            }
        }
    

References

Disclaimer

Purpose of the code contained in snippets or available for download in this article is solely for learning and demo purposes. Author will not be held responsible for any failure or damages caused due to any other usage.

I have an interesting conundrum here, how do I quickly (under 1 minute) export a large datatable (filled from SQL, 35,000 rows) into an Excel spreadsheet for users. I have code in place that can handle the export, and while nothing is «wrong» with the code per se, it is infuriatingly slow taking 4 minutes to export the entire file (sometimes longer if a user has less RAM or is running more on their system). Sadly, this is an improvement over the 10+ minutes it used to take using our old method. Simply put, can this be made any faster, without using 3rd party components? If so, how? My code is as follows, the slow down occurs between messageboxes 6 and 7 where each row is written. Thank you all for taking the time to take a look at this:

    Private Sub btnTest_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnJeffTest.Click
           Test(MySPtoExport)
    End Sub

Private Sub Test(ByVal SQL As String)
    'Declare variables used to execute the VUE Export stored procedure
    MsgBox("start stop watch")
    Dim ConnectionString As New SqlConnection(CType(ConfigurationManager.AppSettings("ConnString"), String))
    Dim cmdSP As New SqlClient.SqlCommand
    Dim MyParam As New SqlClient.SqlParameter
    Dim MyDataAdapter As New SqlClient.SqlDataAdapter
    Dim ExportDataSet As New DataTable
    Dim FilePath As String

    MsgBox("stop 1 - end of declare")

    Try

        ' open the connection
        ConnectionString.Open()

        ' Use the connection for this sql command
        cmdSP.Connection = ConnectionString

        'set this command as a stored procedure command
        cmdSP.CommandType = CommandType.StoredProcedure

        'get the stored procedure name and plug it in
        cmdSP.CommandText = SQL

        'Add the Start Date parameter if required
        Select Case StDt
            Case Nothing
                ' there's no parameter to add
            Case Is = 0
                ' there's no parameter to add
            Case Else
                'add the parameter name, it's direction and its value
                MyParam = cmdSP.Parameters.Add("@StartDate", SqlDbType.VarChar)
                MyParam.Direction = ParameterDirection.Input
                MyParam.Value = Me.txtStartDate.Text
        End Select
        MsgBox("stop 2 - sql ready")
        'Add the End Date parameter if required
        Select Case EdDt
            Case Nothing
                ' there's no parameter to add
            Case Is = 0
                ' there's no parameter to add
            Case Else
                'add the parameter name, it's direction and its value

                MyParam = cmdSP.Parameters.Add("@EndDate", SqlDbType.VarChar)
                MyParam.Direction = ParameterDirection.Input
                MyParam.Value = Me.txtEndDate.Text
        End Select

        'Add the single parameter 1 parameter if required
        Select Case SPar1
            Case Is = Nothing
                ' there's no parameter to add
            Case Is = ""
                ' there's no parameter to add
            Case Else
                'add the parameter name, it's direction and its value
                MyParam = cmdSP.Parameters.Add(SPar1, SqlDbType.VarChar)
                MyParam.Direction = ParameterDirection.Input
                MyParam.Value = Me.txtSingleReportCrt1.Text
        End Select

        'Add the single parameter 2 parameter if required
        Select Case Spar2
            Case Is = Nothing
                ' there's no parameter to add
            Case Is = ""
                ' there's no parameter to add
            Case Else
                'add the parameter name, it's direction and its value
                MyParam = cmdSP.Parameters.Add(Spar2, SqlDbType.VarChar)
                MyParam.Direction = ParameterDirection.Input
                MyParam.Value = Me.txtSingleReportCrt2.Text
        End Select

        MsgBox("stop 3 - params ready")

        'Prepare the data adapter with the selected command 
        MyDataAdapter.SelectCommand = cmdSP

        ' Set the accept changes during fill to false for the NYPDA export
        MyDataAdapter.AcceptChangesDuringFill = False

        'Fill the Dataset tables (Table 0 = Exam Eligibilities, Table 1  = Candidates Demographics)
        MyDataAdapter.Fill(ExportDataSet)

        'Close the connection
        ConnectionString.Close()

        'refresh the destination path in case they changed it
        SPDestination = txtPDFDestination.Text

        MsgBox("stop 4 - procedure ran, datatable filled")

        Select Case ExcelFile
            Case True

                FilePath = SPDestination & lblReportName.Text & ".xls"

                Dim _excel As New Microsoft.Office.Interop.Excel.Application
                Dim wBook As Microsoft.Office.Interop.Excel.Workbook
                Dim wSheet As Microsoft.Office.Interop.Excel.Worksheet

                wBook = _excel.Workbooks.Add()
                wSheet = wBook.ActiveSheet()

                Dim dt As System.Data.DataTable = ExportDataSet
                Dim dc As System.Data.DataColumn
                Dim dr As System.Data.DataRow
                Dim colIndex As Integer = 0
                Dim rowIndex As Integer = 0

                MsgBox("stop 5 - excel stuff declared")

                For Each dc In dt.Columns
                    colIndex = colIndex + 1
                    _excel.Cells(1, colIndex) = dc.ColumnName
                Next

                MsgBox("stop 6 - Header written")

                For Each dr In dt.Rows
                    rowIndex = rowIndex + 1
                    colIndex = 0
                    For Each dc In dt.Columns
                        colIndex = colIndex + 1
                        _excel.Cells(rowIndex + 1, colIndex) = dr(dc.ColumnName)
                    Next
                Next

                MsgBox("stop 7 - rows written")

                wSheet.Columns.AutoFit()

                MsgBox("stop 8 - autofit complete")

                Dim strFileName = SPDestination & lblReportName.Text & ".xls"

                If System.IO.File.Exists(strFileName) Then
                    System.IO.File.Delete(strFileName)
                End If

                MsgBox("stop 9 - file checked")

                wBook.SaveAs(strFileName)
                wBook.Close()
                _excel.Quit()
        End Select

        MsgBox("File " & lblReportName.Text & " Exported Successfully!")


        'Dispose of unneeded objects
        MyDataAdapter.Dispose()
        ExportDataSet.Dispose()
        StDt = Nothing
        EdDt = Nothing
        SPar1 = Nothing
        Spar2 = Nothing
        MyParam = Nothing
        cmdSP.Dispose()
        cmdSP = Nothing
        MyDataAdapter = Nothing
        ExportDataSet = Nothing

    Catch ex As Exception
        '  Something went terribly wrong.  Warn user.
        MessageBox.Show("Error: " & ex.Message, "Stored Procedure Running Process ", _
       MessageBoxButtons.OK, MessageBoxIcon.Error)

    Finally
        'close the connection in case is still open
        If Not ConnectionString.State = ConnectionState. Then
            ConnectionString.Close()
            ConnectionString = Nothing
        End If

        ' reset the fields
        ResetFields()

    End Try
End Sub