Skip to content

Add PyTorch to users.rst #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 24, 2023
Merged

Add PyTorch to users.rst #65

merged 1 commit into from
Jul 24, 2023

Conversation

albanD
Copy link
Contributor

@albanD albanD commented Jul 24, 2023

I've seen PRs updating this file so adding PyTorch to the list as well as we're relying on this pretty heavily now!

Thanks for the hard work in maintaining this! It helps a lot to have reliable compat code!

@albanD
Add PyTorch to users.rst
27b9db2 
I've seen PRs updating this file so adding PyTorch to the list as well as we're relying on this pretty heavily now!

Thanks for the hard work in maintaining this! It helps a lot to have reliable compat code!
@vstinner vstinner merged commit 59b1b84 into python:main Jul 24, 2023
@vstinner
Copy link
Member

Merged, thanks.

Oh, that's cool! I'm curious, why kinds of C API functions does PyTorch use/need on old Python versions?

@albanD
Copy link
Contributor Author

albanD commented Jul 24, 2023

We only support 3.8+ as of today. So nothing too crazy on that side.
But we moved from our custom functions to this when adding the 3.11 support!
In particular since we use the frame hook API, we manipulate a lot of Frame/Code objects which heavily changed.

Generally, we also have quite a bit of C API calls, so we're transitioning to use this more now that we have it!

I was curious so pulled our current usage with for word in $(cat foo.txt); do grep -rnwI ../ -e "$word" | grep -v "pythoncapi_compat.h" | grep -v "foo.txt"; done (after manually extracting all the APIs exposed from here

Here is the result:

../torch/csrc/dynamo/cpython_defs.c:304:            f->f_back = (PyFrameObject *)Py_NewRef(back);
../torch/csrc/dynamo/eval_frame.c:258:  PyObject* frame = Py_NewRef(_frame);
../torch/csrc/dynamo/eval_frame.c:835:    Py_XSETREF(profiler_start_hook, Py_NewRef(start));
../torch/csrc/dynamo/eval_frame.c:836:    Py_XSETREF(profiler_end_hook, Py_NewRef(end));
../torch/csrc/dynamo/eval_frame.c:808:  Py_XSETREF(guard_fail_hook, Py_XNewRef(obj));
../torch/csrc/dynamo/eval_frame.c:816:  Py_XSETREF(guard_error_hook, Py_XNewRef(obj));
../torch/csrc/Stream.cpp:156:  Py_SET_TYPE(&THPStreamType, &PyType_Type);
../torch/csrc/autograd/profiler_python.cpp:53:    auto code = THPCodeObjectPtr(PyFrame_GetCode(frame));
../torch/csrc/autograd/profiler_python.cpp:326:    auto code = THPCodeObjectPtr(PyFrame_GetCode(frame));
../torch/csrc/autograd/profiler_python.cpp:833:    auto code = THPCodeObjectPtr(PyFrame_GetCode(frame));
../torch/csrc/lazy/python/python_util.cpp:24:  auto code = THPCodeObjectPtr(PyFrame_GetCode(frame));
../torch/csrc/lazy/python/python_util.cpp:41:      auto code = THPCodeObjectPtr(PyFrame_GetCode(frame));
../torch/csrc/jit/python/python_tracer.cpp:32:    auto code = THPCodeObjectPtr(PyFrame_GetCode(frame));
../torch/csrc/profiler/python/combined_traceback.cpp:41:          CapturedTraceback::PyFrame{PyFrame_GetCode(f), PyFrame_GetLasti(f)});
../torch/include/pybind11/pybind11.h:2693:        PyCodeObject *f_code = PyFrame_GetCode(frame);
../torch/include/pybind11/pytypes.h:583:                PyCodeObject *f_code = PyFrame_GetCode(frame);
../functorch/csrc/dim/dim.cpp:1572:    auto c = mpy::obj<PyCodeObject>::steal(PyFrame_GetCode(f.ptr()));
../third_party/onnx/third_party/pybind11/include/pybind11/pytypes.h:598:                auto *b_frame = PyFrame_GetBack(frame);
../third_party/pybind11/include/pybind11/pytypes.h:598:                auto *b_frame = PyFrame_GetBack(frame);
../torch/csrc/autograd/profiler_python.cpp:784:      // NB: `PyFrame_GetBack` returns a strong reference.
../torch/csrc/autograd/profiler_python.cpp:785:      frame = PyFrame_GetBack(frame);
../torch/csrc/autograd/profiler_python.cpp:847:      auto back = THPFrameObjectPtr(PyFrame_GetBack(frame));
../torch/csrc/autograd/profiler_python.cpp:855:      auto back = THPFrameObjectPtr(PyFrame_GetBack(frame));
../torch/csrc/autograd/profiler_python.cpp:860:      auto back = THPFrameObjectPtr(PyFrame_GetBack(frame));
../torch/csrc/lazy/python/python_util.cpp:46:      auto new_frame = PyFrame_GetBack(frame);
../torch/csrc/jit/python/python_tracer.cpp:39:    auto new_frame = PyFrame_GetBack(frame);
../torch/csrc/profiler/python/combined_traceback.cpp:42:      auto f_back = PyFrame_GetBack(f);
../torch/include/pybind11/pytypes.h:598:                auto *b_frame = PyFrame_GetBack(frame);
../torch/csrc/autograd/profiler_python.cpp:844:      auto locals = THPObjectPtr(PyFrame_GetLocals(frame));
../torch/csrc/autograd/profiler_python.cpp:852:      auto locals = THPObjectPtr(PyFrame_GetLocals(frame));
../torch/csrc/jit/python/python_tracer.cpp:33:    size_t line = PyCode_Addr2Line(code.get(), PyFrame_GetLasti(frame));
../torch/csrc/profiler/python/combined_traceback.cpp:41:          CapturedTraceback::PyFrame{PyFrame_GetCode(f), PyFrame_GetLasti(f)});
../functorch/csrc/dim/dim.cpp:1573:    auto lasti = PyFrame_GetLasti(f.ptr());
../third_party/onnx/third_party/pybind11/include/pybind11/pybind11.h:2691:    PyFrameObject *frame = PyThreadState_GetFrame(PyThreadState_Get());
../third_party/pybind11/include/pybind11/pybind11.h:2691:    PyFrameObject *frame = PyThreadState_GetFrame(PyThreadState_Get());
../torch/include/pybind11/pybind11.h:2691:    PyFrameObject *frame = PyThreadState_GetFrame(PyThreadState_Get());
../functorch/csrc/dim/dim.cpp:1571:    auto f = mpy::obj<PyFrameObject>::steal(PyThreadState_GetFrame(state));
../torch/csrc/autograd/profiler_python.cpp:755:  interpreter_ = PyInterpreterState_Get();
../torch/csrc/dynamo/eval_frame.c:342:  return PyObject_CallOneArg(profiler_start_hook, name_str);
../torch/csrc/dynamo/eval_frame.c:348:  PyObject* res = PyObject_CallOneArg(profiler_end_hook, record);
../torch/csrc/dynamo/eval_frame.c:364:  PyObject* valid = PyObject_CallOneArg(e->check_fn, f_locals);
../torch/csrc/dynamo/cpython_defs.c:309:    if (!PyObject_GC_IsTracked((PyObject *) f)) {
../functorch/csrc/dim/dim.cpp:1530:                names = mpy::object::steal(PyCode_GetVarnames(code_object_));
../functorch/csrc/dim/dim.cpp:1533:                names = mpy::object::steal(PyCode_GetCellvars(code_object_));

@albanD albanD deleted the patch-3 branch July 24, 2023 18:49
@vstinner
Copy link
Member

Oh, I see, many PyCode and PyFrame functions. Good to know, thanks a lot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@albanD @vstinner