Open source.NET Fast and easy to avoid OOM’s Excel processing tools

Open source.NET Fast and easy to avoid OOM’s Excel processing tools

2022-12-07 0 561
Resource Number 47871 Last Updated 2025-02-21
¥ 0USD Upgrade VIP
Download Now Matters needing attention
Can't download? Please contact customer service to submit a link error!
Value-added Service: Installation Guide Environment Configuration Secondary Development Template Modification Source Code Installation

This issue recommends one. NET processing Excel search, write, fill data tool – MiniExcel.

Open source.NET Fast and easy to avoid OOM’s Excel processing tools插图

At present, most mainstream frameworks need to load all data into memory for easy operation, but this will lead to memory consumption problems, MiniExcel tries to write the underlying algorithm logic from the Stream Angle, which can reduce the original 1000 MB occupancy to a few MB, to avoid insufficient memory.

Open source.NET Fast and easy to avoid OOM’s Excel processing tools插图1

peculiarity

  • Low memory consumption, avoiding OOM and frequent Full GC
  • Supports real-time operation of each row of data
  • Combined with LINQ delayed query feature, it can achieve low consumption, fast paging and other complex queries
  • Lightweight, do not need to install Microsoft Office, COM+, DLL less than 150KB
  • asy to use API style

Performance comparison

  • Import and query Excel comparison

Logic: Test1,000,000×10.xlsx benchmark and mainstream framework for performance testing, a total of 1,000,000 rows * 10 columns pen “HelloWorld”, file size 23 MB

Open source.NET Fast and easy to avoid OOM’s Excel processing tools插图2

  • Export and create Excel comparison

Logic: Create 10 million “HelloWorld”

Open source.NET Fast and easy to avoid OOM’s Excel processing tools插图3

Code example

  • Query The query Excel returns the IEnumerable data
public class UserAccount
{
    public Guid ID { get; set; }
    public string Name { get; set; }
    public DateTime BoD { get; set; }
    public int Age { get; set; }
    public bool VIP { get; set; }
    public decimal Points { get; set; }
}

var rows = MiniExcel.Query<UserAccount>(path);

// or

using (var stream = File.OpenRead(path))
    var rows = stream.Query<UserAccount>();

a45db2b796cc424390ca85f8379e29a0noop.image_

  • Query supports Deferred Execution, which can work with LINQ First/Take/Skip to make complex queries with low consumption and high efficiency

For example, query the first batch of data

var row = MiniExcel.Query(path).First();
Assert.Equal("HelloWorld", row.A);

// or

using (var stream = File.OpenRead(path))
{
    var row = stream.Query().First();
    Assert.Equal("HelloWorld", row.A);
}

Efficiency comparison with other frameworks:

Open source.NET Fast and easy to avoid OOM’s Excel processing tools插图5

  • Query reads Excel and returns the DataTable

Note: Not recommended because DataTable loads all data into memory and loses MiniExcel’s low memory consumption.

Note: Not recommended because DataTable loads all data into memory and loses MiniExcel's low memory consumption.提醒 : 不建议使用,因为DataTable会将数据全载入内存,失去MiniExcel低内存消耗功能。

Open source.NET Fast and easy to avoid OOM’s Excel processing tools插图6

  • Specifies the cell to start reading data.
MiniExcel.Query(path,useHeaderRow:true,startCell:"B3")

Open source.NET Fast and easy to avoid OOM’s Excel processing tools插图7

  • Create multiple sheets
// 1. Dictionary<string,object>
var users = new[] { new { Name = "Jack", Age = 25 }, new { Name = "Mike", Age = 44 } };
var department = new[] { new { ID = "01", Name = "HR" }, new { ID = "02", Name = "IT" } };
var sheets = new Dictionary<string, object>
{
    ["users"] = users,
    ["department"] = department
};
MiniExcel.SaveAs(path, sheets);

// 2. DataSet
var sheets = new DataSet();
sheets.Add(UsersDataTable);
sheets.Add(DepartmentDataTable);
//..
MiniExcel.SaveAs(path, sheets);

6

  • IEnumerable Data fill

Note1: Peer from left to right with the first IEnumerableUse as the list source (multiple sets in the same column are not supported)

//1. By POCO
var value = new
{
    employees = new[] {
        new {name="Jack",department="HR"},
        new {name="Lisa",department="HR"},
        new {name="John",department="HR"},
        new {name="Mike",department="IT"},
        new {name="Neo",department="IT"},
        new {name="Loan",department="IT"}
    }
};
MiniExcel.SaveAsByTemplate(path, templatePath, value);

//2. By Dictionary
var value = new Dictionary<string, object>()
{
    ["employees"] = new[] {
        new {name="Jack",department="HR"},
        new {name="Lisa",department="HR"},
        new {name="John",department="HR"},
        new {name="Mike",department="IT"},
        new {name="Neo",department="IT"},
        new {name="Loan",department="IT"}
    }
};
MiniExcel.SaveAsByTemplate(path, templatePath, value);

Template:

Open source.NET Fast and easy to avoid OOM’s Excel processing tools插图8

The end result:

Open source.NET Fast and easy to avoid OOM’s Excel processing tools插图9

—END—

Open source protocol: Apache2.0

资源下载此资源为免费资源立即下载
Telegram:@John_Software

Disclaimer: This article is published by a third party and represents the views of the author only and has nothing to do with this website. This site does not make any guarantee or commitment to the authenticity, completeness and timeliness of this article and all or part of its content, please readers for reference only, and please verify the relevant content. The publication or republication of articles by this website for the purpose of conveying more information does not mean that it endorses its views or confirms its description, nor does it mean that this website is responsible for its authenticity.

Ictcoder Free source code Open source.NET Fast and easy to avoid OOM’s Excel processing tools https://ictcoder.com/kyym/open-source-net-fast-and-easy-to-avoid-ooms-excel-processing-tools.html

Share free open-source source code

Q&A
  • 1, automatic: after taking the photo, click the (download) link to download; 2. Manual: After taking the photo, contact the seller to issue it or contact the official to find the developer to ship.
View details
  • 1, the default transaction cycle of the source code: manual delivery of goods for 1-3 days, and the user payment amount will enter the platform guarantee until the completion of the transaction or 3-7 days can be issued, in case of disputes indefinitely extend the collection amount until the dispute is resolved or refunded!
View details
  • 1. Heptalon will permanently archive the process of trading between the two parties and the snapshots of the traded goods to ensure that the transaction is true, effective and safe! 2, Seven PAWS can not guarantee such as "permanent package update", "permanent technical support" and other similar transactions after the merchant commitment, please identify the buyer; 3, in the source code at the same time there is a website demonstration and picture demonstration, and the site is inconsistent with the diagram, the default according to the diagram as the dispute evaluation basis (except for special statements or agreement); 4, in the absence of "no legitimate basis for refund", the commodity written "once sold, no support for refund" and other similar statements, shall be deemed invalid; 5, before the shooting, the transaction content agreed by the two parties on QQ can also be the basis for dispute judgment (agreement and description of the conflict, the agreement shall prevail); 6, because the chat record can be used as the basis for dispute judgment, so when the two sides contact, only communicate with the other party on the QQ and mobile phone number left on the systemhere, in case the other party does not recognize self-commitment. 7, although the probability of disputes is very small, but be sure to retain such important information as chat records, mobile phone messages, etc., in case of disputes, it is convenient for seven PAWS to intervene in rapid processing.
View details
  • 1. As a third-party intermediary platform, Qichou protects the security of the transaction and the rights and interests of both buyers and sellers according to the transaction contract (commodity description, content agreed before the transaction); 2, non-platform online trading projects, any consequences have nothing to do with mutual site; No matter the seller for any reason to require offline transactions, please contact the management report.
View details

Related Article

make a comment
No comments available at the moment
Official customer service team

To solve your worries - 24 hours online professional service