Add tf.linalg.gramSchmidt and tf.eye

[caisq]

• tf.linalg.gramSchmidt performs Gram-Schmidt orthogonalization on
  Matrices or arrays of vectors. It will be used to speed up
  the orthogonal initialization of weights (e.g., in RNNs).
• tf.eye generates identity matrices.

Towards: tensorflow/tfjs#245

---

This change is 

[bileschi]

LGTM
One nit, one food for thought re. random tests.

[bileschi]

xs was already cast to Tensor1D in line 62

[caisq]

The linter complains if I remove this explicit casting.

[bileschi]

This might be a good candidate for a random array as input

[caisq]

Done. Using random vectors and matrics in this test file, wherever it makes sense.

[bileschi]

---

Review status: 0 of 5 files reviewed at latest revision, 2 unresolved discussions.

---

Comments from Reviewable

[davidsoergel]

---

Reviewed 3 of 5 files at r1, 2 of 2 files at r3.
Review status: all files reviewed at latest revision, 2 unresolved discussions, all commit checks successful.

---

src/ops/array_ops.ts, line 376 at r3 (raw file):

Nnumber

typo

---

src/ops/linalg_ops.ts, line 71 at r1 (raw file):

Previously, caisq (Shanqing Cai) wrote…

The linter complains if I remove this explicit casting.

Could avoid this by creating a local xs1d instead of overwriting the xs parameter

---

Comments from Reviewable

[caisq]

Review status: 3 of 5 files reviewed at latest revision, 3 unresolved discussions, some commit checks failed.

---

src/ops/linalg_ops.ts, line 71 at r1 (raw file):

Previously, davidsoergel (David Soergel) wrote…

Could avoid this by creating a local xs1d instead of overwriting the xs parameter

Done.

---

Comments from Reviewable

[caisq]

Review status: 3 of 5 files reviewed at latest revision, 3 unresolved discussions, all commit checks successful.

---

src/ops/array_ops.ts, line 376 at r3 (raw file):

Previously, davidsoergel (David Soergel) wrote…

Nnumber

typo

Done.

---

Comments from Reviewable

[dsmilkov]

Review status: 3 of 5 files reviewed at latest revision, 3 unresolved discussions, some commit checks failed.

---

src/ops/array_ops.ts, line 381 at r4 (raw file):

   */
  @operation
  static eye(n: number, dtype: DataType = 'float32'): Tensor2D {

align with tf.eye by adding numRows, numColumns and batchShape

---

src/ops/linalg_ops.ts, line 46 at r5 (raw file):

   */
  @operation
  static gramSchmidt(xs: Tensor1D[]|Tensor2D): Tensor1D[]|Tensor2D {

what's the tensorflow's equivalent op and can we align that API?

---

Comments from Reviewable

[dsmilkov]

Nice work! Left couple more comments, but nothing major.

---

Review status: 3 of 5 files reviewed at latest revision, 5 unresolved discussions, some commit checks failed.

---

src/ops/linalg_ops.ts, line 35 at r5 (raw file):

   * @param xs The vectors to be orthogonalized, in one of the two following
   *   formats:
   *   - A non-empty `Array` of `Tensor1D`.

Just say: - an array of Tensor1D (no need to mark Array in quotes and no need to mention non-empty since its implicit)

---

src/ops/linalg_ops.ts, line 46 at r5 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

what's the tensorflow's equivalent op and can we align that API?

Talked offline. Resolving

---

src/ops/linalg_ops_test.ts, line 24 at r5 (raw file):

describeWithFlags('gramSchmidt-tiny', ALL_ENVS, () => {
  // TODO(cais): Small things should get tested on ALL_ENVS.

is this TODO outdated?

---

Comments from Reviewable

[caisq]

Review status: 1 of 5 files reviewed at latest revision, 6 unresolved discussions, some commit checks failed.

---

src/ops/array_ops.ts, line 381 at r4 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

align with tf.eye by adding numRows, numColumns and batchShape

Done.

---

src/ops/linalg_ops.ts, line 35 at r5 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

Just say: - an array of Tensor1D (no need to mark Array in quotes and no need to mention non-empty since its implicit)

Done.

---

src/ops/linalg_ops_test.ts, line 24 at r5 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

is this TODO outdated?

Yes. Removed.

---

Comments from Reviewable