Skip to content

Use the BinaryFormatter only on .NET FW and Mono #2470

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions src/embed_tests/StateSerialization/MethodSerialization.cs
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,13 @@
{
using var buf = new MemoryStream();
var formatter = RuntimeData.CreateFormatter();

if (formatter == null)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be a good idea to make the whole test class [Obsolete] to prevent warnings

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, for the time being, we still by default support this functionality on .NET FW and Mono. Which warning are you referring to?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obsolete code warning that comes from using BinaryFormatter. You can see it on GitHub review page.

Assert.Inconclusive("Failed to create formatter for state serialization");

formatter.Serialize(buf, item);

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, x64, ubuntu-22.04, 3.8)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, x64, ubuntu-22.04, 3.10)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, x64, ubuntu-22.04, 3.9)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, x64, ubuntu-22.04, 3.7)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, x64, ubuntu-22.04, 3.11)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, x64, ubuntu-22.04, 3.12)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x64, windows-latest, 3.10)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (macos, x64, macos-13, 3.10)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x64, windows-latest, 3.9)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x64, windows-latest, 3.11)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (macos, x64, macos-13, 3.11)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x64, windows-latest, 3.12)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x64, windows-latest, 3.8)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (macos, x64, macos-13, 3.12)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test ARM64

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x64, windows-latest, 3.7)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x86, windows-latest, 3.8)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x86, windows-latest, 3.11)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x86, windows-latest, 3.9)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x86, windows-latest, 3.12)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x86, windows-latest, 3.10)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (macos, x64, macos-13, 3.7)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (macos, x64, macos-13, 3.8)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (macos, x64, macos-13, 3.9)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 40 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x86, windows-latest, 3.7)

'IFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'
buf.Position = 0;
return (T)formatter.Deserialize(buf);

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, x64, ubuntu-22.04, 3.8)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, x64, ubuntu-22.04, 3.10)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, x64, ubuntu-22.04, 3.9)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, x64, ubuntu-22.04, 3.7)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, x64, ubuntu-22.04, 3.11)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, x64, ubuntu-22.04, 3.12)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x64, windows-latest, 3.10)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (macos, x64, macos-13, 3.10)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x64, windows-latest, 3.9)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x64, windows-latest, 3.11)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (macos, x64, macos-13, 3.11)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x64, windows-latest, 3.12)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x64, windows-latest, 3.8)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (macos, x64, macos-13, 3.12)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test ARM64

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x64, windows-latest, 3.7)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x86, windows-latest, 3.8)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x86, windows-latest, 3.11)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x86, windows-latest, 3.9)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x86, windows-latest, 3.12)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x86, windows-latest, 3.10)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (macos, x64, macos-13, 3.7)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (macos, x64, macos-13, 3.8)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (macos, x64, macos-13, 3.9)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'

Check warning on line 42 in src/embed_tests/StateSerialization/MethodSerialization.cs

View workflow job for this annotation

GitHub Actions / Build and Test (windows, x86, windows-latest, 3.7)

'IFormatter.Deserialize(Stream)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'
}
}

Expand Down
14 changes: 0 additions & 14 deletions src/runtime/StateSerialization/NoopFormatter.cs

This file was deleted.

40 changes: 26 additions & 14 deletions src/runtime/StateSerialization/RuntimeData.cs
Original file line number Diff line number Diff line change
Expand Up @@ -16,21 +16,23 @@ namespace Python.Runtime
public static class RuntimeData
{

public readonly static Func<IFormatter> DefaultFormatterFactory = () =>
public readonly static Func<IFormatter?> DefaultFormatterFactory = () =>
{
try
{
return new BinaryFormatter();
}
catch
{
return new NoopFormatter();
var fw = RuntimeInformation.FrameworkDescription;
if (fw.StartsWith(".NET Framework") || fw.StartsWith("Mono"))
{
return new BinaryFormatter();
}
}
catch {}
return null;
};

private static Func<IFormatter> _formatterFactory { get; set; } = DefaultFormatterFactory;
private static Func<IFormatter?> _formatterFactory { get; set; } = DefaultFormatterFactory;

public static Func<IFormatter> FormatterFactory
public static Func<IFormatter?> FormatterFactory
{
get => _formatterFactory;
set
Expand Down Expand Up @@ -82,6 +84,14 @@ static void ClearCLRData ()

internal static void Stash()
{
ClearCLRData();

IFormatter? formatter = CreateFormatter();

if (formatter == null)
// No formatter defined, exit early
return;

var runtimeStorage = new PythonNetState
{
Metatype = MetaType.SaveRuntimeData(),
Expand All @@ -91,7 +101,6 @@ internal static void Stash()
SharedObjects = SaveRuntimeDataObjects(),
};

IFormatter formatter = CreateFormatter();
var ms = new MemoryStream();
formatter.Serialize(ms, runtimeStorage);

Expand All @@ -102,11 +111,10 @@ internal static void Stash()
Marshal.WriteIntPtr(mem, (IntPtr)ms.Length);
Marshal.Copy(data, 0, mem + IntPtr.Size, (int)ms.Length);

ClearCLRData();

using NewReference capsule = PyCapsule_New(mem, IntPtr.Zero, IntPtr.Zero);
int res = PySys_SetObject("clr_data", capsule.BorrowOrThrow());
PythonException.ThrowIfIsNotZero(res);

PostStashHook?.Invoke();
}

Expand All @@ -124,6 +132,12 @@ internal static void RestoreRuntimeData()

private static void RestoreRuntimeDataImpl()
{
IFormatter? formatter = CreateFormatter();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think coming here should be an error.


if (formatter == null)
// No formatter defined, exit early
return;

PreRestoreHook?.Invoke();
BorrowedReference capsule = PySys_GetObject("clr_data");
if (capsule.IsNull)
Expand All @@ -135,7 +149,6 @@ private static void RestoreRuntimeDataImpl()
byte[] data = new byte[length];
Marshal.Copy(mem + IntPtr.Size, data, 0, length);
var ms = new MemoryStream(data);
var formatter = CreateFormatter();
var storage = (PythonNetState)formatter.Deserialize(ms);

PyCLRMetaType = MetaType.RestoreRuntimeData(storage.Metatype);
Expand Down Expand Up @@ -373,9 +386,8 @@ public static MemoryStream GetSerializationData(string key)
return new MemoryStream(buffer, writable:false);
}

internal static IFormatter CreateFormatter()
internal static IFormatter? CreateFormatter()
{

if (FormatterType != null)
{
return (IFormatter)Activator.CreateInstance(FormatterType);
Expand Down
Loading