Skip to main content

groupBy API

To work with nested data in tidy, you use the groupBy, which runs a subflow of tidy functions on nested groups of the data and can be exported into different forms.

groupBy​

Restructures the data to be nested by the specified group keys then runs a tidy flow on each of the leaf sets. Grouped data can be exported into different shapes via group export helpers, or if not specified, will be ungrouped back to a flat list of items.

Parameters​

groupKeys​

| string /* key of item */
| (item: object) => any
| Array<string | (item: object) => any>

Either a key in the item or an accessor function that returns the grouping value, or an array combining these two options.

Note that the grouping logic uses strict equality === to see if keys are the same, so if you're using non-primitive values that are equal, but not identical (e.g. ["foo", "bar"] !== ["foo", "bar"]), they won't be grouped together. In those cases, the current recommendation is specifying groupKeys as a function that returns a primitive value (e.g. (item) => item.arrKey.join('---')).

fns​

Array<(item: object[]) => object[]>

A tidy subflow: an array of tidy functions to run on the leaf sets of the grouped data.

options?​

| {
    addGroupKeys?: boolean;
  }
// or export functions (see below)
| groupBy.grouped()
| groupBy.entries()
| groupBy.entriesObject()
| groupBy.object()
| groupBy.map()
| groupBy.keys()
| groupBy.values()
| groupBy.levels()

Options to configure how groupBy operates:

  • addGroupKeys = true: Whether to merge group keys back into the objects.

Usage​

const data = [
  { str: 'a', ing: 'x', foo: 'G', value: 1 },
  { str: 'b', ing: 'x', foo: 'H', value: 100 },
  { str: 'b', ing: 'x', foo: 'K', value: 200 },
  { str: 'a', ing: 'y', foo: 'G', value: 2 },
  { str: 'a', ing: 'y', foo: 'H', value: 3 },
  { str: 'a', ing: 'y', foo: 'K', value: 4 },
  { str: 'b', ing: 'y', foo: 'G', value: 300 },
  { str: 'b', ing: 'z', foo: 'H', value: 400 },
  { str: 'a', ing: 'z', foo: 'K', value: 5 },
  { str: 'a', ing: 'z', foo: 'G', value: 6 },
]

tidy(
  data,
  groupBy('str', [
    summarize({ total: sum('value') })
  ])
)
// output:
[{ str: 'a', total: 21 },
 { str: 'b', total: 1000 }]

tidy(
  data,
  groupBy(['str', 'ing'], [
    summarize({ total: sum('value') })
  ])
)
// output:
[
  { str: 'a', ing: 'x', total: 1 },
  { str: 'a', ing: 'y', total: 9 },
  { str: 'a', ing: 'z', total: 11 },
  { str: 'b', ing: 'x', total: 300 },
  { str: 'b', ing: 'y', total: 300 },
  { str: 'b', ing: 'z', total: 400 },
]

Group Exports​

The final argument to groupBy can also be an export function:

  • groupBy.entries(options?)
  • groupBy.entriesObject(options?)
  • groupBy.grouped(options?)
  • groupBy.keys(options?)
  • groupBy.map(options?)
  • groupBy.object(options?)
  • groupBy.values(options?)

The levels export allows combining any of the above at different depths of the tree:

  • groupBy.levels(options?)

Group Export Options​

These functions take options as their argument, which includes the options mentioned above plus some extras specific to exporting:

{
  addGroupKeys?: boolean; // from groupBy options

  flat?: boolean;
  compositeKey?: (keys: any[]) => string;
  single?: boolean;
  mapLeaf?: (value: any) => any;
  mapLeaves?: (values: any[]) => any;
  mapEntry?: (entry: [any, any], level: number) => any;
  levels?: (
    | 'entries'
    | 'entries-object'
    | 'object'
    | 'map'
    | 'keys'
    | 'values'
    | LevelSpec
  )[];
}
  • export = 'ungrouped': Specifies how the data should be exported from the groupBy function. If anything besides nully or "ungrouped", the remaining options will be interpreted to inform the output.
  • flat?: if all nested levels should be brought to a single top level
  • compositeKey?: when flat is true, how to flatten nested keys (default joins with "/")
  • single?: whether the leaf sets consist of just one item (typical after summarize). if true, uses the first element in the leaf set instead of an array
  • mapLeaf?: operation called on each leaf during export to map it to a different value (default: identity)
  • mapLeaves?: operation called on each leaf set to map the array of values to a different value. Similar to rollup from d3-collection nest or d3-array (default: identity)
  • mapEntry?: when export = "entries" only operation called on entries to map from [key, values] to whatever the output of this is (e.g. { key, values }) (default: identity)
  • levels?: required when export = "levels" only specifies the export operation for each level of the grouping

groupBy.entries()​

Exports the data as entries arrays where the keys are the value of the key for that group level and the values are either nested entries or a flat list of items if it is a leaf set.

const data = [
  { str: 'a', ing: 'x', foo: 'G', value: 1 },
  { str: 'b', ing: 'x', foo: 'H', value: 100 },
  { str: 'b', ing: 'x', foo: 'K', value: 200 },
  { str: 'a', ing: 'y', foo: 'G', value: 2 },
  { str: 'a', ing: 'y', foo: 'H', value: 3 },
  { str: 'a', ing: 'y', foo: 'K', value: 4 },
  { str: 'b', ing: 'y', foo: 'G', value: 300 },
  { str: 'b', ing: 'z', foo: 'H', value: 400 },
  { str: 'a', ing: 'z', foo: 'K', value: 5 },
  { str: 'a', ing: 'z', foo: 'G', value: 6 },
]

tidy(
  data,
  groupBy('str', [
    summarize({ total: sum('value') })
  ], groupBy.entries())
)
// output:
[
  ["a", [{"str": "a", "total": 21}]], 
  ["b", [{"str": "b", "total": 1000}]]
]

tidy(
  data,
  groupBy(['str', 'ing'], [
    summarize({ total: sum('value') })
  ], groupBy.entries({ single: true }))
)
// output:
[
  [
    "a",
    [
      ["x", {"str": "a", "ing": "x", "total": 1}],
      ["y", {"str": "a", "ing": "y", "total": 9}],
      ["z", {"str": "a", "ing": "z", "total": 11}]
    ]
  ],
  [
    "b",
    [
      ["x", {"str": "b", "ing": "x", "total": 300}],
      ["y", {"str": "b", "ing": "y", "total": 300}],
      ["z", {"str": "b", "ing": "z", "total": 400}]
    ]
  ]
]

groupBy.entriesObject()​

Exports the data as entries objects { key: string, values: any[] } where the keys are the value of the key for that group level and the values are either nested entries or a flat list of items if it is a leaf set.

const data = [
  { str: 'a', ing: 'x', foo: 'G', value: 1 },
  { str: 'b', ing: 'x', foo: 'H', value: 100 },
  { str: 'b', ing: 'x', foo: 'K', value: 200 },
  { str: 'a', ing: 'y', foo: 'G', value: 2 },
  { str: 'a', ing: 'y', foo: 'H', value: 3 },
  { str: 'a', ing: 'y', foo: 'K', value: 4 },
  { str: 'b', ing: 'y', foo: 'G', value: 300 },
  { str: 'b', ing: 'z', foo: 'H', value: 400 },
  { str: 'a', ing: 'z', foo: 'K', value: 5 },
  { str: 'a', ing: 'z', foo: 'G', value: 6 },
]

tidy(
  data,
  groupBy('str', [
    summarize({ total: sum('value') })
  ], groupBy.entriesObject())
)
// output:
[
  {"key": "a", "values": [{"str": "a", "total": 21}]},
  {"key": "b", "values": [{"str": "b", "total": 1000}]}
]

tidy(
  data,
  groupBy(['str', 'ing'], [
    summarize({ total: sum('value') })
  ], groupBy.entriesObject({ single: true }))
)
// output:
[
  {
    "key": "a",
    "values": [
      {"key": "x", "values": {"str": "a", "ing": "x", "total": 1}},
      {"key": "y", "values": {"str": "a", "ing": "y", "total": 9}},
      {"key": "z", "values": {"str": "a", "ing": "z", "total": 11}}
    ]
  },
  {
    "key": "b",
    "values": [
      {"key": "x", "values": {"str": "b", "ing": "x", "total": 300}},
      {"key": "y", "values": {"str": "b", "ing": "y", "total": 300}},
      {"key": "z", "values": {"str": "b", "ing": "z", "total": 400}}
    ]
  }
]

groupBy.grouped()​

Exports the data as a Grouped Map where the keys are tuples [keyName, keyValue] and the values are either nested Grouped Maps or a flat list of items if it is a leaf set. Note this is similar to groupBy.map, but it uses a tuple for the keys.

const data = [
  { str: 'a', ing: 'x', foo: 'G', value: 1 },
  { str: 'b', ing: 'x', foo: 'H', value: 100 },
  { str: 'b', ing: 'x', foo: 'K', value: 200 },
  { str: 'a', ing: 'y', foo: 'G', value: 2 },
  { str: 'a', ing: 'y', foo: 'H', value: 3 },
  { str: 'a', ing: 'y', foo: 'K', value: 4 },
  { str: 'b', ing: 'y', foo: 'G', value: 300 },
  { str: 'b', ing: 'z', foo: 'H', value: 400 },
  { str: 'a', ing: 'z', foo: 'K', value: 5 },
  { str: 'a', ing: 'z', foo: 'G', value: 6 },
]

tidy(
  data,
  groupBy('str', [
    summarize({ total: sum('value') })
  ], groupBy.grouped())
)
// output:
new Map([
  [['str', 'a'], [{ str: 'a', total: 21 }]],
  [['str', 'b'], [{ str: 'b', total: 1000 }]],
]) 

tidy(
  data,
  groupBy(['str', 'ing'], [
    summarize({ total: sum('value') })
  ], groupBy.grouped({ single: true }))
)
// output:
new Map([
  [['str', 'a'], 
   [new Map([
     [['ing', 'x'], { str: 'a', ing: 'x', total: 1 }],
     [['ing', 'y'], { str: 'a', ing: 'y', total: 9 }],
     [['ing', 'z'], { str: 'a', ing: 'z', total: 11 }],
   ])]],

  [['str', 'b'],
   [new Map([
     [['ing', 'x'], { str: 'b', ing: 'x', total: 300 }],
     [['ing', 'y'], { str: 'b', ing: 'y', total: 300 }],
     [['ing', 'z'], { str: 'b', ing: 'z', total: 400 }],
   ])]]
])

groupBy.keys()​

Exports the data as keys arrays, which are similar to entries except they do not include any of the values. Is this useful? Hard to know.

const data = [
  { str: 'a', ing: 'x', foo: 'G', value: 1 },
  { str: 'b', ing: 'x', foo: 'H', value: 100 },
  { str: 'b', ing: 'x', foo: 'K', value: 200 },
  { str: 'a', ing: 'y', foo: 'G', value: 2 },
  { str: 'a', ing: 'y', foo: 'H', value: 3 },
  { str: 'a', ing: 'y', foo: 'K', value: 4 },
  { str: 'b', ing: 'y', foo: 'G', value: 300 },
  { str: 'b', ing: 'z', foo: 'H', value: 400 },
  { str: 'a', ing: 'z', foo: 'K', value: 5 },
  { str: 'a', ing: 'z', foo: 'G', value: 6 },
]

tidy(
  data,
  groupBy('str', [
    summarize({ total: sum('value') })
  ], groupBy.keys())
)
// output:
["a", "b"]

tidy(
  data,
  groupBy(['str', 'ing'], [
    summarize({ total: sum('value') })
  ], groupBy.keys({ single: true }))
)
// output:
[["a", ["x", "y", "z"]], 
 ["b", ["x", "y", "z"]]]

groupBy.map()​

Exports the data as Map objects where the keys are the value of the key for that group level and the values are either nested Map objects or a flat list of items if it is a leaf set. Note this is similar to groupBy.grouped, but it doesn't use a tuple for keys.

const data = [
  { str: 'a', ing: 'x', foo: 'G', value: 1 },
  { str: 'b', ing: 'x', foo: 'H', value: 100 },
  { str: 'b', ing: 'x', foo: 'K', value: 200 },
  { str: 'a', ing: 'y', foo: 'G', value: 2 },
  { str: 'a', ing: 'y', foo: 'H', value: 3 },
  { str: 'a', ing: 'y', foo: 'K', value: 4 },
  { str: 'b', ing: 'y', foo: 'G', value: 300 },
  { str: 'b', ing: 'z', foo: 'H', value: 400 },
  { str: 'a', ing: 'z', foo: 'K', value: 5 },
  { str: 'a', ing: 'z', foo: 'G', value: 6 },
]

tidy(
  data,
  groupBy('str', [
    summarize({ total: sum('value') })
  ], groupBy.grouped())
)
// output:
new Map([
  ['a', [{ str: 'a', total: 21 }]],
  ['b', [{ str: 'b', total: 1000 }]],
]) 

tidy(
  data,
  groupBy(['str', 'ing'], [
    summarize({ total: sum('value') })
  ], groupBy.grouped({ single: true }))
)
// output:
new Map([
  ['a', 
   [new Map([
     ['x', { str: 'a', ing: 'x', total: 1 }],
     ['y', { str: 'a', ing: 'y', total: 9 }],
     ['z', { str: 'a', ing: 'z', total: 11 }],
   ])]],

  ['b',
   [new Map([
     ['x', { str: 'b', ing: 'x', total: 300 }],
     ['y', { str: 'b', ing: 'y', total: 300 }],
     ['z', { str: 'b', ing: 'z', total: 400 }],
   ])]]
])

groupBy.object()​

Exports the data as objects where the keys are the value of the key for that group level and the values are either nested objects or a flat list of items if it is a leaf set.

const data = [
  { str: 'a', ing: 'x', foo: 'G', value: 1 },
  { str: 'b', ing: 'x', foo: 'H', value: 100 },
  { str: 'b', ing: 'x', foo: 'K', value: 200 },
  { str: 'a', ing: 'y', foo: 'G', value: 2 },
  { str: 'a', ing: 'y', foo: 'H', value: 3 },
  { str: 'a', ing: 'y', foo: 'K', value: 4 },
  { str: 'b', ing: 'y', foo: 'G', value: 300 },
  { str: 'b', ing: 'z', foo: 'H', value: 400 },
  { str: 'a', ing: 'z', foo: 'K', value: 5 },
  { str: 'a', ing: 'z', foo: 'G', value: 6 },
]

tidy(
  data,
  groupBy('str', [
    summarize({ total: sum('value') })
  ], groupBy.object())
)
// output:
{
  "a": [{"str": "a", "total": 21}], 
  "b": [{"str": "b", "total": 1000}]
}

tidy(
  data,
  groupBy(['str', 'ing'], [
    summarize({ total: sum('value') })
  ], groupBy.object({ single: true }))
)
// output:
{
  "a": {
    "x": {"str": "a", "ing": "x", "total": 1},
    "y": {"str": "a", "ing": "y", "total": 9},
    "z": {"str": "a", "ing": "z", "total": 11}
  },
  "b": {
    "x": {"str": "b", "ing": "x", "total": 300},
    "y": {"str": "b", "ing": "y", "total": 300},
    "z": {"str": "b", "ing": "z", "total": 400}
  }
}

groupBy.values()​

Exports the data as values arrays which are similar to entries arrays except they contain no keys. Note if you are just trying to get the values back as a single flat list, you do no need to use any export method as that is the default behavior ("ungrouped").

const data = [
  { str: 'a', ing: 'x', foo: 'G', value: 1 },
  { str: 'b', ing: 'x', foo: 'H', value: 100 },
  { str: 'b', ing: 'x', foo: 'K', value: 200 },
  { str: 'a', ing: 'y', foo: 'G', value: 2 },
  { str: 'a', ing: 'y', foo: 'H', value: 3 },
  { str: 'a', ing: 'y', foo: 'K', value: 4 },
  { str: 'b', ing: 'y', foo: 'G', value: 300 },
  { str: 'b', ing: 'z', foo: 'H', value: 400 },
  { str: 'a', ing: 'z', foo: 'K', value: 5 },
  { str: 'a', ing: 'z', foo: 'G', value: 6 },
]

tidy(
  data,
  groupBy('str', [
    summarize({ total: sum('value') })
  ], groupBy.values())
)
// output:
[[{"str": "a", "total": 21}], 
 [{"str": "b", "total": 1000}]]

tidy(
  data,
  groupBy(['str', 'ing'], [
    summarize({ total: sum('value') })
  ], groupBy.values({ single: true }))
)
// output:
[
  [
    {"str": "a", "ing": "x", "total": 1},
    {"str": "a", "ing": "y", "total": 9},
    {"str": "a", "ing": "z", "total": 11}
  ],
  [
    {"str": "b", "ing": "x", "total": 300},
    {"str": "b", "ing": "y", "total": 300},
    {"str": "b", "ing": "z", "total": 400}
  ]
]

groupBy.levels()​

Exports the data in a different way for each level. The last level specified is used for all remaining levels. You must supply a value for levels in the options argument.

const data = [
  { str: 'a', ing: 'x', foo: 'G', value: 1 },
  { str: 'b', ing: 'x', foo: 'H', value: 100 },
  { str: 'b', ing: 'x', foo: 'K', value: 200 },
  { str: 'a', ing: 'y', foo: 'G', value: 2 },
  { str: 'a', ing: 'y', foo: 'H', value: 3 },
  { str: 'a', ing: 'y', foo: 'K', value: 4 },
  { str: 'b', ing: 'y', foo: 'G', value: 300 },
  { str: 'b', ing: 'z', foo: 'H', value: 400 },
  { str: 'a', ing: 'z', foo: 'K', value: 5 },
  { str: 'a', ing: 'z', foo: 'G', value: 6 },
]

tidy(
  data,
  groupBy(['str', 'ing'], [
    summarize({ total: sum('value') })
  ], groupBy.levels({ 
    levels: ['entries-object', 'object'], 
    single: true 
  }))
)
// output:
[
  { // <-- this level is an "entries object"
    "key": "a",
    "values": { // <-- this level is an "object"
      "x": {"str": "a", "ing": "x", "total": 1},
      "y": {"str": "a", "ing": "y", "total": 9},
      "z": {"str": "a", "ing": "z", "total": 11}
    }
  },
  {
    "key": "b",
    "values": {
      "x": {"str": "b", "ing": "x", "total": 300},
      "y": {"str": "b", "ing": "y", "total": 300},
      "z": {"str": "b", "ing": "z", "total": 400}
    }
  }
]

tidy(
  data,
  groupBy(['str', 'ing'], [
    summarize({ total: sum('value') })
  ], groupBy.levels({ 
    levels: ['object', 'entries-object'], // swapped order
    single: true 
  }))
)
// output:
{ // <-- this level is an "object"
  "a": [ // <-- this level is "entries object"s
    {"key": "x", "values": {"str": "a", "ing": "x", "total": 1}},
    {"key": "y", "values": {"str": "a", "ing": "y", "total": 9}},
    {"key": "z", "values": {"str": "a", "ing": "z", "total": 11}}
  ],
  "b": [
    {"key": "x", "values": {"str": "b", "ing": "x", "total": 300}},
    {"key": "y", "values": {"str": "b", "ing": "y", "total": 300}},
    {"key": "z", "values": {"str": "b", "ing": "z", "total": 400}}
  ]
}

Custom Levels Export​

For a more advanced export, a custom levels export can be specified by providing a LevelSpec for the value of levels:

// LevelSpec:
{
  createEmptySubgroup: () => any;
  addSubgroup: (
    parentGrouped: any,
    newSubgroup: any,
    key: any,
    level: number
  ) => void;
  addLeaf: (
    parentGrouped: any, 
    key: any, 
    values: any[], 
    level: number
  ) => void;
}

Probably best to just look at the source code of the existing groupBy methods to get an idea of how to use this.