Open
Description
Environment
- Pythonnet version: 2.4 master
- Python version: 3.7
- Operating System: Win10 x64
Details
-
Describe what you were trying to get done.
Hello, we are using PythonNet to execute custom python scripts in our C# application. We pass a set of large .net objects (such as DataTable with a big bunch of data) as a script locals. When we execute such sctripts multiple times we got an OutOfMemoryException
-
What commands did you run to trigger this issue? If you can provide a
Minimal, Complete, and Verifiable example
this will help us understand the issue.
public void Execute(string script, IDictionary<string, object> locals, List<string> searchPaths)
{
var modifiedScript = $"{script}{Environment.NewLine}{string.Format(PythonSnippets.SystemEventHandler, pySystemHandlerGuid)}";
_scriptLocals = new PyDict();
using (Py.GIL())
{
if (locals != null)
foreach (var local in locals)
_scriptLocals[local.Key] = local.Value.ToPython();
_pythonScope = Py.CreateScope();
_pythonScope.Exec(modifiedScript, _scriptLocals);
}
}Here we cache PyScope and PyDict. Pydict contains our large .net objects, translated to Python
public void ClearCache()
{
using (Py.GIL())
{
_pythonScope.Exec($"vars().clear()");
foreach (PyObject pObject in _scriptLocals.Items())
{
pObject.Dispose();
}
dynamic gc = Py.Import("gc");
gc.collect();
_scriptLocals.Dispose();
_pythonScope.Dispose();
gc.collect();
}
}This method we execute each time, after we finished our work with script. As you can see, I tried different way to clean up resources, but memory leak still occures, after multiple executions.
I did a snapshot of memory with .net profiler, and found that there a references to my DataTables from the CLRObject and I did not find a way how to release it.
StackTrace
Exception of type 'System.OutOfMemoryException' was thrown.
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.Data.RBTree`1.TreePage..ctor(Int32 size)
at System.Data.RBTree`1.AllocPage(Int32 size)
at System.Data.RBTree`1.GetNewNode(K key)
at System.Data.DataTable.SetNewRecordWorker(DataRow row, Int32 proposedRecord, DataRowAction action, Boolean isInMerge, Boolean suppressEnsurePropertyChanged, Int32 position, Boolean fireEvent, Exception& deferredException)
at System.Data.DataTable.InsertRow(DataRow row, Int64 proposedID, Int32 pos, Boolean fireEvent)
at System.Data.DataTable.LoadDataRow(Object[] values, Boolean fAcceptChanges)
at System.Data.ProviderBase.SchemaMapping.LoadDataRow()
at System.Data.Common.DataAdapter.FillLoadDataRow(SchemaMapping mapping)
at System.Data.Common.DataAdapter.FillFromReader(DataSet dataset, DataTable datatable, String srcTable, DataReaderContainer dataReader, Int32 startRecord, Int32 maxRecords, DataColumn parentChapterColumn, Object parentChapterValue)
at System.Data.Common.DataAdapter.Fill(DataTable[] dataTables, IDataReader dataReader, Int32 startRecord, Int32 maxRecords)
at System.Data.Common.DbDataAdapter.FillInternal(DataSet dataset, DataTable[] datatables, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior)
at System.Data.Common.DbDataAdapter.Fill(DataTable[] dataTables, Int32 startRecord, Int32 maxRecords, IDbCommand command, CommandBehavior behavior)
at System.Data.Common.DbDataAdapter.Fill(DataTable dataTable)
at Geobank.Shared.Data.Server.GetTableFillSchema(IConnection connection, DbConnection cn, DbTransaction tr, String sql, SqlParameter[] parameters)
at Geobank.Shared.Data.Server.GetTableFillSchema(IConnection connection, String sql, SqlParameter[] parameters)
at Geobank.Module.Forms.Services.FormDataViewCacheRepository.GenerateDataView(Guid dataSetGuid)
at Geobank.Module.Forms.Services.FormDataViewCacheRepository.AddDataSetComponent(Guid guid)
at Geobank.Module.Forms.Services.FormDataViewCacheRepository.Add(Guid DataSetGuid)
at Geobank.Module.Forms.Services.FormDesignerExecuteManager.InitializeDataViewsRepository(IDataSource dataSource, IEnumerable`1 dataBindings)
at Geobank.Module.Forms.Services.FormDesignerExecuteManager.CreateExecutableForm(IFormDesignerComponent formDesignerComponent, IDataSource dataSource, IDictionary`2 locals)
at Geobank.Module.Forms.ViewModels.FormDesignerEditViewModel.ExecuteForm(Object sender)
at Geobank.Shared.Helpers.RelayCommand.Execute(Object parameter)
at DevExpress.Xpf.Bars.Native.CommandSourceHelper.Execute(ICommand command, Object commandParameter, IInputElement commandTarget)
at DevExpress.Xpf.Bars.BarItem.ExecuteCommand(ICommand commandToExecute, Object commandParameterToExecute, IInputElement actualCommandTarget)
at DevExpress.Xpf.Bars.BarItem.<>c__DisplayClass308_0.<OnItemClick>b__0()
at DevExpress.Xpf.Bars.BarItem.OnItemClick(BarItemLink link, IBarItemLinkControl linkControl)
at DevExpress.Xpf.Bars.BarItemLinkBase.OnClick(IBarItemLinkControl linkControl)
at DevExpress.Xpf.Bars.BarItemLinkControlStrategy.OnClick()
at DevExpress.Xpf.Bars.BarButtonItemLinkControlStrategy.OnMouseLeftButtonUp(MouseButtonEventArgs args)
at DevExpress.Xpf.Bars.BarItemLinkControlBase.OnMouseLeftButtonUp(MouseButtonEventArgs e)
at System.Windows.UIElement.OnMouseLeftButtonUpThunk(Object sender, MouseButtonEventArgs e)
at System.Windows.Input.MouseButtonEventArgs.InvokeEventHandler(Delegate genericHandler, Object genericTarget)
at System.Windows.RoutedEventArgs.InvokeHandler(Delegate handler, Object target)
at System.Windows.RoutedEventHandlerInfo.InvokeHandler(Object target, RoutedEventArgs routedEventArgs)
at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
at System.Windows.UIElement.ReRaiseEventAs(DependencyObject sender, RoutedEventArgs args, RoutedEvent newEvent)
at System.Windows.UIElement.OnMouseUpThunk(Object sender, MouseButtonEventArgs e)
at System.Windows.Input.MouseButtonEventArgs.InvokeEventHandler(Delegate genericHandler, Object genericTarget)
at System.Windows.RoutedEventArgs.InvokeHandler(Delegate handler, Object target)
at System.Windows.RoutedEventHandlerInfo.InvokeHandler(Object target, RoutedEventArgs routedEventArgs)
at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
at System.Windows.UIElement.RaiseEventImpl(DependencyObject sender, RoutedEventArgs args)
at System.Windows.UIElement.RaiseTrustedEvent(RoutedEventArgs args)
at System.Windows.UIElement.RaiseEvent(RoutedEventArgs args, Boolean trusted)
at System.Windows.Input.InputManager.ProcessStagingArea()
at System.Windows.Input.InputManager.ProcessInput(InputEventArgs input)
at System.Windows.Input.InputProviderSite.ReportInput(InputReport inputReport)
at System.Windows.Interop.HwndMouseInputProvider.ReportInput(IntPtr hwnd, InputMode mode, Int32 timestamp, RawMouseActions actions, Int32 x, Int32 y, Int32 wheel)
at System.Windows.Interop.HwndMouseInputProvider.FilterMessage(IntPtr hwnd, WindowMessage msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at System.Windows.Interop.HwndSource.InputFilterMessage(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
Metadata
Metadata
Assignees
Labels
No labels