Cumsum

[jameswex]

Implements cumsum operation

---

This change is 

[dsmilkov]

Thanks. Left a few comments - mostly how to do some transposing and rehsaping before you call the webgl shader. This will simplify the work a lot and allow summation across arbitrary axis.

---

Reviewed 5 of 8 files at r1.
Review status: 5 of 8 files reviewed at latest revision, all discussions resolved, some commit checks failed.

---

src/tensor.ts, line 420 at r1 (raw file):

  /**
   * Returns a `Tensor` that has expanded rank, by inserting a dimension

this jsdoc doesn't seem right. Is this taken from expandDim? Did you mean Compute the cumulative sum of the tensor x along axis.

---

src/tensor.ts, line 423 at r1 (raw file):

   * into the tensor's shape. See `expandDims` for details.
   *
   * @param axis The dimension index at which to insert shape of 1. Defaults to

the axis along which to sum.

---

src/kernels/webgl/cumsum_gpu.ts, line 49 at r1 (raw file):

        ${dtype} adjustableCoords = ${dtype}(${outputCoords});
        int finalCoord = int(${finalCoord});
        

remove whitespace

---

src/kernels/webgl/cumsum_gpu.ts, line 68 at r1 (raw file):

}

function getCoords(rank: number, name: string): string {

you can simplify this shader a bit if you do the following:

• transpose to make axis be the inner-most axis
• reshape to 2D where all other-most shapes are flattened
• shader operates always on a 2d tensor and sums along it's inner-most axis
• after shader runs, undo the reshape, and undo the transpose

---

src/ops/array_ops.ts, line 1147 at r1 (raw file):

   * x.cumsum().print();
   * ```
   *

add one more code snippet with 2d and sum along axis 0

---

src/ops/array_ops.ts, line 1151 at r1 (raw file):

   * @param axis The axis on which to perform the sum. Optional. Defaults to 0.
   * @param exclusive Whether to perform exclusive cumulative sum. Optional.
   *     Defaults to false.

Expand a bit on what exclusive means.

---

src/ops/array_ops.ts, line 1164 at r1 (raw file):

    return ENV.engine.runKernel(
               backend => backend.cumsum(x, axis, exclusive, reverse), {x},

this won't work for axis that are not inner most unless you transpose to make the axis be inner-most, do the computation and then transpose back again. See https://github.com/tensorflow/tfjs-core/blob/master/src/ops/reduction_ops.ts#L118 for how to transpose.

---

src/ops/array_ops_test.ts, line 2295 at r1 (raw file):

  });

  it('2D reverse exclusive', () => {

add test for 2d with axis=0

---

Comments from Reviewable

[jameswex]

added the transpose so that it works on any axis.

---

Review status: 5 of 8 files reviewed at latest revision, 8 unresolved discussions, some commit checks failed.

---

src/tensor.ts, line 420 at r1 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

this jsdoc doesn't seem right. Is this taken from expandDim? Did you mean Compute the cumulative sum of the tensor x along axis.

Done.

---

src/tensor.ts, line 423 at r1 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

the axis along which to sum.

Done.

---

src/kernels/webgl/cumsum_gpu.ts, line 49 at r1 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

remove whitespace

Done.

---

src/kernels/webgl/cumsum_gpu.ts, line 68 at r1 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

you can simplify this shader a bit if you do the following:

• transpose to make axis be the inner-most axis
• reshape to 2D where all other-most shapes are flattened
• shader operates always on a 2d tensor and sums along it's inner-most axis
• after shader runs, undo the reshape, and undo the transpose

added transpose/undo transpose in array_ops so it applies for both cpu and gpu.

---

src/ops/array_ops.ts, line 1147 at r1 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

add one more code snippet with 2d and sum along axis 0

Done.

---

src/ops/array_ops.ts, line 1151 at r1 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

Expand a bit on what exclusive means.

done

---

src/ops/array_ops.ts, line 1164 at r1 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

this won't work for axis that are not inner most unless you transpose to make the axis be inner-most, do the computation and then transpose back again. See https://github.com/tensorflow/tfjs-core/blob/master/src/ops/reduction_ops.ts#L118 for how to transpose.

Done.

---

src/ops/array_ops_test.ts, line 2295 at r1 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

add test for 2d with axis=0

Done.

---

Comments from Reviewable

[nsthorat]

---

Reviewed 1 of 4 files at r3.
Review status: 2 of 8 files reviewed at latest revision, 8 unresolved discussions.

---

src/kernels/webgl/cumsum_gpu.ts, line 3 at r4 (raw file):

/**
 * @license
 * Copyright 2017 Google Inc. All Rights Reserved.

2018

---

src/ops/array_ops.ts, line 1188 at r4 (raw file):

    };
    let value =  ENV.engine.runKernel(
      backend => backend.cumsum(permutedX, axis, exclusive, reverse),

indent 2 more

---

src/ops/array_ops_test.ts, line 2349 at r4 (raw file):

  it('1D reverse', () => {
    const res = tf.tensor1d([1, 2, 3, 4]).cumsum(0, false, true);

pull these false / true to variables for readability

---

src/ops/array_ops_test.ts, line 2394 at r4 (raw file):

  it('3D standard', () => {
    const res =tf.tensor3d([[[0, 1], [2, 3]], [[4, 5], [6, 7]]]).cumsum(2);

spacing here (btw if you use vscode this will auto format for you)

---

Comments from Reviewable

[jameswex]

Review status: 2 of 8 files reviewed at latest revision, 12 unresolved discussions, all commit checks successful.

---

src/kernels/webgl/cumsum_gpu.ts, line 3 at r4 (raw file):

Previously, nsthorat (Nikhil Thorat) wrote…

2018

Done.

---

src/ops/array_ops.ts, line 1188 at r4 (raw file):

Previously, nsthorat (Nikhil Thorat) wrote…

indent 2 more

Done.

---

src/ops/array_ops_test.ts, line 2349 at r4 (raw file):

Previously, nsthorat (Nikhil Thorat) wrote…

pull these false / true to variables for readability

Done.

---

src/ops/array_ops_test.ts, line 2394 at r4 (raw file):

Previously, nsthorat (Nikhil Thorat) wrote…

spacing here (btw if you use vscode this will auto format for you)

Done.

---

Comments from Reviewable