Speed up orthogonal initializer by using tf.linalg.gramSchmidt

[caisq]

instead of QR decomposition. This resutls in about 2x speed up
on CPU and 18x speed up on WebGL.

Also in this CL:

• Add test for relatively large-sized orthogonal matrix initialization
  to prevent speed regression.
• In Orthogonal.apply(), print a console warning about possible
  slowness if the # of elements in the matrix is > 2000.
• Remove tfjs_backend's eye function and use the tf.eye from added to
  tfjs-core recently.

BUG Fix slowness in Orthgonal initializer for some RNN layers.

Fixes: tensorflow/tfjs#245

---

This change is 

[bileschi]

---

Review status: 0 of 5 files reviewed at latest revision, all discussions resolved.

---

src/initializers.ts, line 570 at r1 (raw file):

    // TODO(cais): Add seed support.
    const normalizedShape = shape[0] >= shape[1] ? [shape[1], shape[0]] : shape;

>= can probably just be > here

---

src/initializers.ts, line 573 at r1 (raw file):

    if (shape[0] > shape[1]) {
 

Can this happen? Skimming gramSchmidt implementation it seems like it's not possible. That would mean there are more orthogonal vectors than dimensions?

---

src/initializers_test.ts, line 496 at r1 (raw file):

 expectTensorsClose(w.matMul(w.transpose()), eye(n));

Very cool!

---

Comments from Reviewable

[caisq]

Review status: 0 of 5 files reviewed at latest revision, 3 unresolved discussions, all commit checks successful.

---

src/initializers.ts, line 570 at r1 (raw file):

Previously, bileschi (Stanley Bileschi) wrote…

>= can probably just be > here

Right. Done.

---

src/initializers.ts, line 573 at r1 (raw file):

Previously, bileschi (Stanley Bileschi) wrote…

    if (shape[0] > shape[1]) {
 

Can this happen? Skimming gramSchmidt implementation it seems like it's not possible. That would mean there are more orthogonal vectors than dimensions?

The Orthogonal initializer works for both cases: shape[0] >= shape[1] and shape[0] < shape[1]. In either case, it'll do the correct transposition (if any) to make sure that the number of vectors is less than the number of dimensions. This is what this line, together with the const normalizedShape = shape[0] > shape[1] ? [shape[1], shape[0]] : shape; line above, aims to achieve.

---

Comments from Reviewable