feat: allow functions decorated with @bpd.remote_function to execute locally

[tswast]

BEGIN_COMMIT_OVERRIDE
feat: allow functions decorated with bpd.remote_function() to execute locally (#704)
END_COMMIT_OVERRIDE

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

• Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
• Ensure the tests and linter pass
• Code coverage does not decrease (if any source code was changed)
• Appropriate docs were updated (if necessary)

🦕

[tswast]

e2e failure tests/system/large/test_remote_function.py::test_remote_function_stringify_with_ibis is a real one. We don't have the functions return ibis objects anymore, so that test will need to be updated.

I'm not sure what's going on with the Could not create or update Cloud Run service bigframes-9c95c91485a237f4d58dba298ce3b103, Container Healthcheck failed. failures. Will investigate further.

[tswast]

Regarding the other failures, looks like the functions were failing to start due to:

ERROR 2024-05-17T22:49:25.369026Z Traceback (most recent call last): File "/layers/google.python.pip/pip/bin/functions-framework", line 8, in <module> sys.exit(_cli())
DEFAULT 2024-05-17T22:49:25.369047Z ^^^^^^
DEFAULT 2024-05-17T22:49:25.369055Z File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
DEFAULT 2024-05-17T22:49:25.370216Z return self.main(*args, **kwargs)
DEFAULT 2024-05-17T22:49:25.370455Z ^^^^^^^^^^^^^^^^^^^^^^^^^^
DEFAULT 2024-05-17T22:49:25.370469Z File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 1078, in main
DEFAULT 2024-05-17T22:49:25.370797Z rv = self.invoke(ctx)
DEFAULT 2024-05-17T22:49:25.370941Z ^^^^^^^^^^^^^^^^
DEFAULT 2024-05-17T22:49:25.370949Z File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
DEFAULT 2024-05-17T22:49:25.371290Z return ctx.invoke(self.callback, **ctx.params)
DEFAULT 2024-05-17T22:49:25.371504Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
DEFAULT 2024-05-17T22:49:25.371513Z File "/layers/google.python.pip/pip/lib/python3.12/site-packages/click/core.py", line 783, in invoke
DEFAULT 2024-05-17T22:49:25.371739Z return __callback(*args, **kwargs)
DEFAULT 2024-05-17T22:49:25.372052Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^
DEFAULT 2024-05-17T22:49:25.372065Z File "/layers/google.python.pip/pip/lib/python3.12/site-packages/functions_framework/_cli.py", line 37, in _cli
DEFAULT 2024-05-17T22:49:25.374076Z app = create_app(target, source, signature_type)
DEFAULT 2024-05-17T22:49:25.374381Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
DEFAULT 2024-05-17T22:49:25.374391Z File "/layers/google.python.pip/pip/lib/python3.12/site-packages/functions_framework/__init__.py", line 288, in create_app
DEFAULT 2024-05-17T22:49:25.375787Z spec.loader.exec_module(source_module)
DEFAULT 2024-05-17T22:49:25.375854Z File "<frozen importlib._bootstrap_external>", line 995, in exec_module
DEFAULT 2024-05-17T22:49:25.375865Z File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
DEFAULT 2024-05-17T22:49:25.375872Z File "/workspace/main.py", line 9, in <module>
DEFAULT 2024-05-17T22:49:25.375963Z udf = cloudpickle.load(f)
DEFAULT 2024-05-17T22:49:25.376108Z ^^^^^^^^^^^^^^^^^^^
WARNING 2024-05-17T22:49:26.383786824Z Container called exit(1).
DEFAULT 2024-05-17T22:49:26.383934Z ModuleNotFoundError: No module named 'pandas'

I wonder if it's because we're pickling more than we used to because the function body is now intact? Will look a bit deeper.

[tswast]

I wonder if it's because we're pickling more than we used to because the function body is now intact? Will look a bit deeper.

Looks like this only affected some specific tests that basically decorated the same function twice. Should be fixed by cc11de5

[tswast]

Remaining failures:

FAILED tests/system/large/test_remote_function.py::test_remote_function_via_session_vpc_invalid
FAILED tests/system/large/ml/test_ensemble.py::test_xgbclassifier_dart_booster_multiple_params
FAILED tests/system/large/ml/test_ensemble.py::test_randomforestclassifier_multiple_params

I don't think test_ensemble.py tests are related, but I will need to investigate all of these more.

[ashleyxuu]

test_remote_function_via_session_vpc_invalid Is this related to your change?

[tswast]

Re: test_remote_function_via_session_vpc_invalid

This is also failing in other PRs unrelated to this change. Filed internal issue 341772893 to investigate.