Skip to content

Add pivot table with limitations #2551

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 31, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .prettier
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@
"bracketSpacing": false,
"printWidth": 100,
"trailingComma": "all",
"bracketSpacing": false,
"arrowParens": "avoid"
"arrowParens": "avoid",
"singleQuote": true,
}
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ npm install exceljs

# New Features!

* Merged [Add pivot table with limitations #2551](https://github.com/exceljs/exceljs/pull/2551). <br/> Many thanks to Protobi and <a href="https://github.com/mikez">Michael</a> for this contribution!
* Merged [fix: styles rendering in case when "numFmt" is present in conditional formatting rules (resolves #1814) #1815](https://github.com/exceljs/exceljs/pull/1815). <br/> Many thanks to [@andreykrupskii](https://github.com/andreykrupskii) for this contribution!
* Merged [inlineStr cell type support #1575 #1576](https://github.com/exceljs/exceljs/pull/1576). <br/> Many thanks to [@drdmitry](https://github.com/drdmitry) for this contribution!
* Merged [Fix parsing of boolean attributes #1849](https://github.com/exceljs/exceljs/pull/1849). <br/> Many thanks to [@bno1](https://github.com/bno1) for this contribution!
Expand Down
132 changes: 132 additions & 0 deletions lib/doc/pivot-table.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
const {objectFromProps, range, toSortedArray} = require('../utils/utils');

// TK(2023-10-10): turn this into a class constructor.

function makePivotTable(worksheet, model) {
// Example `model`:
// {
// // Source of data: the entire sheet range is taken,
// // akin to `worksheet1.getSheetValues()`.
// sourceSheet: worksheet1,
//
// // Pivot table fields: values indicate field names;
// // they come from the first row in `worksheet1`.
// rows: ['A', 'B'],
// columns: ['C'],
// values: ['E'], // only 1 item possible for now
// metric: 'sum', // only 'sum' possible for now
// }

validate(worksheet, model);

const {sourceSheet} = model;
let {rows, columns, values} = model;

const cacheFields = makeCacheFields(sourceSheet, [...rows, ...columns]);

// let {rows, columns, values} use indices instead of names;
// names can then be accessed via `pivotTable.cacheFields[index].name`.
// *Note*: Using `reduce` as `Object.fromEntries` requires Node 12+;
// ExcelJS is >=8.3.0 (as of 2023-10-08).
const nameToIndex = cacheFields.reduce((result, cacheField, index) => {
result[cacheField.name] = index;
return result;
}, {});
rows = rows.map(row => nameToIndex[row]);
columns = columns.map(column => nameToIndex[column]);
values = values.map(value => nameToIndex[value]);

// form pivot table object
return {
sourceSheet,
rows,
columns,
values,
metric: 'sum',
cacheFields,
// defined in <pivotTableDefinition> of xl/pivotTables/pivotTable1.xml;
// also used in xl/workbook.xml
cacheId: '10',
};
}

function validate(worksheet, model) {
if (worksheet.workbook.pivotTables.length === 1) {
throw new Error(
'A pivot table was already added. At this time, ExcelJS supports at most one pivot table per file.'
);
}

if (model.metric && model.metric !== 'sum') {
throw new Error('Only the "sum" metric is supported at this time.');
}

const headerNames = model.sourceSheet.getRow(1).values.slice(1);
const isInHeaderNames = objectFromProps(headerNames, true);
for (const name of [...model.rows, ...model.columns, ...model.values]) {
if (!isInHeaderNames[name]) {
throw new Error(`The header name "${name}" was not found in ${model.sourceSheet.name}.`);
}
}

if (!model.rows.length) {
throw new Error('No pivot table rows specified.');
}

if (!model.columns.length) {
throw new Error('No pivot table columns specified.');
}

if (model.values.length !== 1) {
throw new Error('Exactly 1 value needs to be specified at this time.');
}
Comment on lines +72 to +82
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would It be enabled to add partially configured pivot table without for instance rows set?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this time, the code doesn't support that. You need at least one item in each of {rows, columns, values}.
However, this could be a potential feature down the road if it is requested.

}

function makeCacheFields(worksheet, fieldNamesWithSharedItems) {
// Cache fields are used in pivot tables to reference source data.
//
// Example
// -------
// Turn
//
// `worksheet` sheet values [
// ['A', 'B', 'C', 'D', 'E'],
// ['a1', 'b1', 'c1', 4, 5],
// ['a1', 'b2', 'c1', 4, 5],
// ['a2', 'b1', 'c2', 14, 24],
// ['a2', 'b2', 'c2', 24, 35],
// ['a3', 'b1', 'c3', 34, 45],
// ['a3', 'b2', 'c3', 44, 45]
// ];
// fieldNamesWithSharedItems = ['A', 'B', 'C'];
//
// into
//
// [
// { name: 'A', sharedItems: ['a1', 'a2', 'a3'] },
// { name: 'B', sharedItems: ['b1', 'b2'] },
// { name: 'C', sharedItems: ['c1', 'c2', 'c3'] },
// { name: 'D', sharedItems: null },
// { name: 'E', sharedItems: null }
// ]

const names = worksheet.getRow(1).values;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it always start from row 1?

Copy link
Contributor Author

@mikez mikez Oct 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made the simplifying assumption that the source data is represented by the entire sheet, with the first row representing the header. Do you think this assumption is too simplistic and won't capture 95% of the cases?

const nameToHasSharedItems = objectFromProps(fieldNamesWithSharedItems, true);

const aggregate = columnIndex => {
const columnValues = worksheet.getColumn(columnIndex).values.splice(2);
const columnValuesAsSet = new Set(columnValues);
return toSortedArray(columnValuesAsSet);
};

// make result
const result = [];
for (const columnIndex of range(1, names.length)) {
const name = names[columnIndex];
const sharedItems = nameToHasSharedItems[name] ? aggregate(columnIndex) : null;
result.push({name, sharedItems});
}
return result;
}

module.exports = {makePivotTable};
3 changes: 3 additions & 0 deletions lib/doc/workbook.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ class Workbook {
this.title = '';
this.views = [];
this.media = [];
this.pivotTables = [];
this._definedNames = new DefinedNames();
}

Expand Down Expand Up @@ -174,6 +175,7 @@ class Workbook {
contentStatus: this.contentStatus,
themes: this._themes,
media: this.media,
pivotTables: this.pivotTables,
calcProperties: this.calcProperties,
};
}
Expand Down Expand Up @@ -215,6 +217,7 @@ class Workbook {
this.views = value.views;
this._themes = value.themes;
this.media = value.media || [];
this.pivotTables = value.pivotTables || [];
}
}

Expand Down
22 changes: 22 additions & 0 deletions lib/doc/worksheet.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ const Enums = require('./enums');
const Image = require('./image');
const Table = require('./table');
const DataValidations = require('./data-validations');
const {makePivotTable} = require('./pivot-table');
const Encryptor = require('../utils/encryptor');
const {copyStyle} = require('../utils/copy-style');

Expand Down Expand Up @@ -124,6 +125,8 @@ class Worksheet {
// for tables
this.tables = {};

this.pivotTables = [];

this.conditionalFormattings = [];
}

Expand Down Expand Up @@ -806,6 +809,23 @@ class Worksheet {
return Object.values(this.tables);
}

// =========================================================================
// Pivot Tables
addPivotTable(model) {
// eslint-disable-next-line no-console
console.warn(
`Warning: Pivot Table support is experimental.
Please leave feedback at https://github.com/exceljs/exceljs/discussions/2575`
);

const pivotTable = makePivotTable(this, model);

this.pivotTables.push(pivotTable);
this.workbook.pivotTables.push(pivotTable);
Comment on lines +823 to +824
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What will happen if someone remove the pivot table from one of these places, or overwrite one of them?
What do you think about making wb.workbook.pivotTables a getter returning pivottables from all worksheets? Or .. a function?
Can it attach one pivotTable to multiple sheets?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What will happen if someone remove the pivot table from one of these places, or overwrite one of them?

@Siemienik It seems you're pointing at the mutability of worksheet.pivotTables and worksheet.workbook.pivotTables and concerns for that clients might mutate this. If you think this type of safety is needed and consistent with elsewhere in ExcelJS, we can definitely make it a getter. I'll take your advice here.

Can it attach one pivotTable to multiple sheets?

At this time, I intend for there to be at most one pivotTable in at most one sheet across the entire workbook. The user should not have to mutate worksheet.pivotTables or worksheet.workbook.pivotTables and only access worksheet.addPivotTable.


return pivotTable;
}

// ===========================================================================
// Conditional Formatting
addConditionalFormatting(cf) {
Expand Down Expand Up @@ -854,6 +874,7 @@ class Worksheet {
media: this._media.map(medium => medium.model),
sheetProtection: this.sheetProtection,
tables: Object.values(this.tables).map(table => table.model),
pivotTables: this.pivotTables,
conditionalFormattings: this.conditionalFormattings,
};

Expand Down Expand Up @@ -920,6 +941,7 @@ class Worksheet {
tables[table.name] = t;
return tables;
}, {});
this.pivotTables = value.pivotTables;
this.conditionalFormattings = value.conditionalFormattings;
}
}
Expand Down
35 changes: 34 additions & 1 deletion lib/utils/utils.js
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,11 @@ const utils = {
},
inherits,
dateToExcel(d, date1904) {
return 25569 + ( d.getTime() / (24 * 3600 * 1000) ) - (date1904 ? 1462 : 0);
// eslint-disable-next-line no-mixed-operators
return 25569 + d.getTime() / (24 * 3600 * 1000) - (date1904 ? 1462 : 0);
},
excelToDate(v, date1904) {
// eslint-disable-next-line no-mixed-operators
const millisecondSinceEpoch = Math.round((v - 25569 + (date1904 ? 1462 : 0)) * 24 * 3600 * 1000);
return new Date(millisecondSinceEpoch);
},
Expand Down Expand Up @@ -167,6 +169,37 @@ const utils = {
parseBoolean(value) {
return value === true || value === 'true' || value === 1 || value === '1';
},

*range(start, stop, step = 1) {
const compareOrder = step > 0 ? (a, b) => a < b : (a, b) => a > b;
for (let value = start; compareOrder(value, stop); value += step) {
yield value;
}
},

toSortedArray(values) {
const result = Array.from(values);

// Note: per default, `Array.prototype.sort()` converts values
// to strings when comparing. Here, if we have numbers, we use
// numeric sort.
if (result.every(item => Number.isFinite(item))) {
const compareNumbers = (a, b) => a - b;
return result.sort(compareNumbers);
}

return result.sort();
},

objectFromProps(props, value = null) {
// *Note*: Using `reduce` as `Object.fromEntries` requires Node 12+;
// ExcelJs is >=8.3.0 (as of 2023-10-08).
// return Object.fromEntries(props.map(property => [property, value]));
return props.reduce((result, property) => {
result[property] = value;
return result;
}, {});
},
};

module.exports = utils;
15 changes: 7 additions & 8 deletions lib/xlsx/rel-type.js
Original file line number Diff line number Diff line change
@@ -1,21 +1,20 @@
'use strict';

module.exports = {
OfficeDocument:
'http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument',
OfficeDocument: 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument',
Worksheet: 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/worksheet',
CalcChain: 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/calcChain',
SharedStrings:
'http://schemas.openxmlformats.org/officeDocument/2006/relationships/sharedStrings',
SharedStrings: 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/sharedStrings',
Styles: 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/styles',
Theme: 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/theme',
Hyperlink: 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/hyperlink',
Image: 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/image',
CoreProperties:
'http://schemas.openxmlformats.org/package/2006/relationships/metadata/core-properties',
ExtenderProperties:
'http://schemas.openxmlformats.org/officeDocument/2006/relationships/extended-properties',
CoreProperties: 'http://schemas.openxmlformats.org/package/2006/relationships/metadata/core-properties',
ExtenderProperties: 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/extended-properties',
Comments: 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/comments',
VmlDrawing: 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/vmlDrawing',
Table: 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/table',
PivotCacheDefinition: 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/pivotCacheDefinition',
PivotCacheRecords: 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/pivotCacheRecords',
PivotTable: 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/pivotTable',
};
29 changes: 29 additions & 0 deletions lib/xlsx/xform/book/workbook-pivot-cache-xform.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
const BaseXform = require('../base-xform');

class WorkbookPivotCacheXform extends BaseXform {
render(xmlStream, model) {
xmlStream.leafNode('pivotCache', {
cacheId: model.cacheId,
'r:id': model.rId,
});
}

parseOpen(node) {
if (node.name === 'pivotCache') {
this.model = {
cacheId: node.attributes.cacheId,
rId: node.attributes['r:id'],
};
return true;
}
return false;
}

parseText() {}

parseClose() {
return false;
}
}

module.exports = WorkbookPivotCacheXform;
12 changes: 8 additions & 4 deletions lib/xlsx/xform/book/workbook-xform.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ const SheetXform = require('./sheet-xform');
const WorkbookViewXform = require('./workbook-view-xform');
const WorkbookPropertiesXform = require('./workbook-properties-xform');
const WorkbookCalcPropertiesXform = require('./workbook-calc-properties-xform');
const WorkbookPivotCacheXform = require('./workbook-pivot-cache-xform');

class WorkbookXform extends BaseXform {
constructor() {
Expand All @@ -31,6 +32,11 @@ class WorkbookXform extends BaseXform {
childXform: new DefinedNameXform(),
}),
calcPr: new WorkbookCalcPropertiesXform(),
pivotCaches: new ListXform({
tag: 'pivotCaches',
count: false,
childXform: new WorkbookPivotCacheXform(),
}),
};
}

Expand All @@ -53,10 +59,7 @@ class WorkbookXform extends BaseXform {
});
}

if (
sheet.pageSetup &&
(sheet.pageSetup.printTitlesRow || sheet.pageSetup.printTitlesColumn)
) {
if (sheet.pageSetup && (sheet.pageSetup.printTitlesRow || sheet.pageSetup.printTitlesColumn)) {
const ranges = [];

if (sheet.pageSetup.printTitlesColumn) {
Expand Down Expand Up @@ -99,6 +102,7 @@ class WorkbookXform extends BaseXform {
this.map.sheets.render(xmlStream, model.sheets);
this.map.definedNames.render(xmlStream, model.definedNames);
this.map.calcPr.render(xmlStream, model.calcProperties);
this.map.pivotCaches.render(xmlStream, model.pivotTables);

xmlStream.closeNode();
}
Expand Down
Loading