Environment

• Pythonnet version: 2.4.0 and ealier
• DotNet version: Tested on Core 2.1 and Core 2.2 but most likely all
• Python version: Tested 3.7.2 & 3.7.4 but I think it could be all
• Operating System:
  Linux version 4.15.0-1049-azure (buildd@lcy01-amd64-027) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10)) System.IO.FileLoadException when calling "import clr" in x86 build #54-Ubuntu SMP Mon Jun 24 20:59:24 UTC 2019

Details

I have spent two days trying to under the following. From C# I call PythonEngine.ImportModule(name) to load a module where the module is a .py file in a folder that is in sys.path.

There are no issues and it works consistently well in native Windows (Server 2016 Datacenter), native Mac and Docker Desktop running Linux containers (Linux version 4.14.131-linuxkit (root@6d384074ad24) (gcc version 8.3.0 (Alpine 8.3.0)) #1 SMP Fri Jul 19 12:31:17 UTC 2019).

Problem happens when I run EXACTLY the same code and Exact same docker image in a linux machine. I have an azure VM running Ubuntu 5.4.0-6ubuntu1~16.04.10 as stated above and then PythonEngine.ImportModule randomly fails. Sometimes it loads the modules, other times when I rerun the container it says the module as not found.

I was not expecting this to happen in a docker container as the environment is supposed to be identical. I then figured out that Docker containers leverage off the host's Kernel which can differ.

For some reason (I don't know why) PythonEngine.ImportModule is unstable in Ubuntu 5.4.0-6ubuntu1~16.04.10.

The work around is editing the Pythonnet library and adding a new load function which is a total secret and will keep to my grave.... bad joke...

   public static PyObject CompileToModule(string name, string code, string filename = "", RunFlagType mode = RunFlagType.File)
           {
               var flag = (IntPtr)mode;
               IntPtr ptr = Runtime.Py_CompileString(code, filename, flag);
               Runtime.CheckExceptionOccurred();

               IntPtr m = Runtime.PyImport_ExecCodeModule(name, ptr);
               Runtime.CheckExceptionOccurred();
               PyObject res = new PyObject(m);

               if(!objs.ContainsKey(name))
                   objs.TryAdd(name, res);
               return res;
           }

If you compile the source code into a Code Object and the transform that into a Module Object the problem disappears and works consistently in Ubuntu 5.4.0-6ubuntu1~16.04.10 also.

Note: Some will ask why I didn't just use the ModuleFromString function. Reason is that I need the module to be linked to a file in order to mimic the result of a module loaded with PythonEngine.ImportModule... am building a really cool PySpark enabled botnet application which needs this...