Implement public API InferenceMode and its error handling

[ailzhang]

Stack from ghstack:

• Relax some limitations of InferenceMode. #54403 Relax some limitations of InferenceMode.
• Implement public API InferenceMode and its error handling #53343 Implement public API InferenceMode and its error handling

RFC: pytorch/rfcs#17

Differential Revision: D26973911

[facebook-github-bot]

💊 CI failures summary and remediations

As of commit 09f5448 (more details on the Dr. CI page):

---

• 1/1 failures possibly* introduced in this PR
  • 1/1 non-scanned failure(s)

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

[ezyang]

Was reviewing the spec some more and I think we can probably do some more safe relaxations:

• Taking a view on an inference mode tensor outside of inference mode is safe, you just need to produce an inference tensor in this case as well (currently this is an error)
• Inplace modifying an inference mode tensor outside of inference mode is safe and you can skip the VC bump

(Don't block this PR on it)

[bhosmer]

This LGTM per Ed's analysis here and offline conversation 😁 one note about setting requires_grad, but if we want to do anything about that it can definitely wait for a followup.

[bhosmer]

Did we decide there was a reason we wanted to silently ignore, like code reuse? Seems clear it would be saner devex to actually error if you set requires_grad true on an inference tensor.

[ezyang]

The code got edited above, but just to be super explicit: it's not silently ignored; we DO set requires_grad=True (just like in no grad mode); the idea to not error is so that code that initializes parameters can run in inference mode without triggering errors

[ailzhang]

Yea code reuse is the main reason. And another reason is inference tensor can actually have requires_grad=true (and gradients) as shown in the second PR. :D

[albanD]

One case that I don't see mentioned here is:

a = torch.rand(10, requires_grad=True)
with inference_mode():
    b = a.view_as(a)
with torch.no_grad():
    c = b.view_as(b)
c += 1

I guess this is fine, but just want to make sure

pytorch/torch/csrc/autograd/variable.h

Lines 508 to 515 in dfc7fa0

	/// Handles correctly propagating CreationMeta when a new view is created from a previous view.
	/// In general, we don't want the new view to be _less_ restrictive than the previous view
	/// (it's okay to be _more_ restrictive). A CreationMeta value of DEFAULT is currently the least
	/// restrictive, as the behavior for all other CreationMeta values is to error out for in-place ops.
	/// If this changes, the logic here will need to be updated to properly handle the new semantics.
	inline CreationMeta propagate_creation_meta(CreationMeta prev_view_creation_meta, CreationMeta new_view_creation_meta) {
	return (new_view_creation_meta == CreationMeta::DEFAULT) ? prev_view_creation_meta : new_view_creation_meta;
	}

handles that case properly.

Also after looking at all the examples, I am now convinced that we should present inference mode as no grad but more restrictive. As most of the behavior is actually the same.

[albanD]

LGTM!
Thanks for all the updates!

[facebook-github-bot]

@ailzhang merged this pull request in 7caa464.

[facebook-github-bot]

This pull request has been reverted by 263180d.