Passing an arbitrary .NET object as the value of an attribute of PyObject #373

yagweb · 2017-02-10T06:41:51Z

This implement the function of passing an arbitrary .NET object as attribute of PyObject by dynamic type.

Four modifications are located at file pyobject.cs, the methods for dynamic type TryGetMember and TrySetMember were modified, a new method GetAttrDynamic was added for TryGetMember, and a new method SetAttrDynamic was added for TrySetMember.

Unit tests were given in a new file "src/embed_tests/dynamic.cs".

An Example was given in the discussion of issue #370 , the Solution 4.

/Information below is outdated and can be removed**/

However this doesn't fix the issue why sys.stdout returned null #370
...
...

I don't know where to put the test, so I put it here. If needed, it can be added to Embedded tests with some work.

    class dynamicTest
    {
        public void Test()
        {
            var output = new Output();
            using (Py.GIL())
            {
                dynamic sys = Py.Import("sys");
                sys.stdout = output; //Now it's working

                //function test, print in python
                PythonEngine.RunSimpleString("print('Hello!')");

                //Check whether there are the same object
                var result1 = sys.stdout == output;
                Console.WriteLine($"Compare in .NET: {result1}");  //Output True   

                //Check whether we can get the attr of a python object 
                //when the value of attr is not a .NET object
                PythonEngine.RunSimpleString("import sys\n" +
                "from io import StringIO\n" +
                "sys.stderr = StringIO()\n");
                var bb = sys.stderr;
                bb.write("Hello!\n");
                Console.Write(bb.getvalue());

                /********Pass the .NET object in Python side*********/
                PythonEngine.RunSimpleString("import sys\n" +
                "from io import StringIO\n" +
                "sys.stderr = sys.stdout\n");
                //Compare in Python
                PythonEngine.RunSimpleString("import sys\n" +
                "print('Compare in Python: %s' % (sys.stderr is sys.stdout))\n");  //Output False

                //compare in .NET
                var result2 = sys.stdout == sys.stderr;
                Console.WriteLine($"Compare in .NET: {result2}");  //Output True

                /*********Pass the .NET object in .NET side********/
                sys.stdout = sys.stderr;
                //Compare in Python
                PythonEngine.RunSimpleString("import sys\n" +
                "print('Compare in Python: %s' % (sys.stderr is sys.stdout))\n");  //Output False

                //compare in .NET
                var result3 = sys.stdout == sys.stderr;
                Console.WriteLine($"Compare in .NET: {result3}");  //Output True    

                /*********When the .NET object was created in Python side*****************/
                PythonEngine.RunSimpleString(@"
import sys
from Test import Output
sys.stderr = Output()
");
                var aa = sys.stderr; //Return the .NET object, not a PyObject with an IntPtr
                aa.write("Hello!\n");            
            }
        }
    }
    public class Output
    {
        public void write(String str)
        {
            //str = str.Replace("\n", Environment.NewLine);
            Console.Write(str);
        }

        public void writelines(String[] str)
        {
            foreach (String line in str)
            {
                Console.Write(str);
            }
        }
        public void flush() { }
        public void close() { }
    }

den-run-ai · 2017-02-12T05:39:49Z

@yagweb i like this PR, the test looks good and can be added to embedded tests. Let us know if you need any help?

codecov · 2017-02-13T15:58:59Z

Codecov Report

Merging #373 into master will increase coverage by 0.23%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##           master     #373      +/-   ##
==========================================
+ Coverage   63.08%   63.31%   +0.23%     
==========================================
  Files          61       61              
  Lines        5225     5226       +1     
  Branches      860      860              
==========================================
+ Hits         3296     3309      +13     
+ Misses       1709     1696      -13     
- Partials      220      221       +1

Flag	Coverage Δ
#Embedded_Tests	`32.63% <66.66%> (+4.55%)`	✅
#Python_Tests	`60.56% <66.66%> (-0.02%)`	❌
#Setup_Linux	`65% <66.66%> (ø)`	✅
#Setup_Windows	`70.5% <66.66%> (ø)`	✅

Impacted Files	Coverage Δ
src/runtime/pyobject.cs	`25.15% <66.66%> (+2.38%)`	✅
src/runtime/pythonengine.cs	`65.28% <ø> (+0.51%)`	✅
src/runtime/converter.cs	`76.84% <ø> (+1.05%)`	✅

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 94c266e...51d229f. Read the comment docs.

vmuriart · 2017-02-13T15:59:11Z

Reviewing now. I just pushed some whitespace changes to the pr to avoid dealing with those later.
@yagweb I'm adding the unittest you included above as well.

yagweb · 2017-02-13T16:19:35Z

@denfromufa @vmuriart Thank you for your Response. I have added the unittest and fixed a bug of assigning null to the attribute of a pyobject. I'll commit it later.

vmuriart · 2017-02-13T16:33:57Z

@yagweb no problem. I'll stash my changes then.

yagweb · 2017-02-14T04:40:35Z

@denfromufa @vmuriart I am sorry the commits are a little mess, because in the start I did not create a branch for this modification and directly worked on my master branch. I will merge them later.

The AssignObjectInPython has a very strange error, but it works well when I call it in my C# code directly, so I just ignore it.

vmuriart · 2017-02-14T04:47:26Z

@yagweb whats the error you are seeing?

vmuriart · 2017-02-14T04:49:31Z

irt the commit history, you can squash and force-push to cleanup your history.

yagweb · 2017-02-14T06:46:13Z

@vmuriart Thank you for your help. The commits have been squashed.

The error occurs when creating a .NET object in python, and It disappeared when I try to debug with calling it in my C# code.

The error information is,

Python.EmbeddingTest.dynamicTest.AssignObjectInPython:
System.OverflowException : Arithmetic operation resulted in an overflow.

and the traceback is,

at System.IntPtr.op_Explicit(IntPtr value)
at Python.Runtime.ManagedType.GetManagedObject(IntPtr ob) in D:\git\pythonnet\src\runtime\managedtype.cs:line 31
at Python.Runtime.ClassObject.tp_new(IntPtr tp, IntPtr args, IntPtr kw) in D:\git\pythonnet\src\runtime\classobject.cs:line 53
at e__NativeCall.Call_3(IntPtr , IntPtr , IntPtr , IntPtr )
at Python.Runtime.NativeCall.Call_3(IntPtr fp, IntPtr a1, IntPtr a2, IntPtr a3) in D:\git\pythonnet\src\runtime\nativecall.cs:line 128
at Python.Runtime.MetaType.tp_call(IntPtr tp, IntPtr args, IntPtr kw) in D:\git\pythonnet\src\runtime\metatype.cs:line 154
at Python.Runtime.Runtime.PyRun_SimpleString(String code)
at Python.Runtime.PythonEngine.RunSimpleString(String code) in D:\git\pythonnet\src\runtime\pythonengine.cs:line 120
at Python.EmbeddingTest.dynamicTest.AssignObjectInPython() in D:\git\pythonnet\src\embed_tests\dynamic.cs:line 68

vmuriart · 2017-02-14T09:38:25Z

Looks like you found a bug for us! https://www.codeproject.com/Questions/528923/Howplustoplusfix-aplus-OverflowExceptionplusocc

Most likely the link above is the issue/fix. I'll take a look tomorrow to make sure and fix it on master.

yagweb · 2017-02-14T14:31:01Z

@vmuriart Wow, proud of myself^^

I found a way to reproduce the bug. Just call that unit test function twice, at the second call, this exception will occur. The following code can do this. Hope this can help you to fix it.

namespace Test
{
    class Program
    {
        static void Main(string[] args)
        {            
            PythonEngine.Initialize();
            using (Py.GIL())
            {
                PythonEngine.RunSimpleString(@"
import sys
from Test import outstream
sys.stdout = outstream()
");
            }
            PythonEngine.Shutdown();

            PythonEngine.Initialize();
            using (Py.GIL())
            {
                PythonEngine.RunSimpleString(@"
import sys
from Test import outstream
sys.stdout = outstream()
");    //**exception will occor here！**
            }
            PythonEngine.Shutdown();
        }
    }
    
    public class outstream { }
}

vmuriart · 2017-02-14T15:28:48Z

Just pushed a fixup! to the branch to merge with the first commit. Just wanted to clarify in case you hadn't seen one of those before and thought I was being rude 😄 .

vmuriart · 2017-02-14T17:21:33Z

@yagweb got an update. I formatted/clean-up the code, and fixed the overflow bug on #376. Once its merged, I recommend squash all commits here and rebase ontop of master. I can do this for you and force-push the changes here if you want.

The test is still failing though, #376 just allowed the exception underlying exception to bubble up. This is the error i'm seeing now.

The error you are seeing above when doing Initialize, Finalize, Initialize, is the vain of my existence ☹️ . #343 and #365 are examples of similar issues that gave similar errors. They occur because there is an underlying exception that we didn't handle and it causes the 2nd initialize to fail.

yagweb · 2017-02-15T03:49:59Z

@vmuriart another bug of this PR has been fixed. I will try to squash all commits and rebase ontop of master by myself first.

yagweb · 2017-02-15T04:05:32Z

@denfromufa @vmuriart During developing the unittests, I found it's hard to evaluate a python expression and obtain the results. So I modified the PythonEngine.RunString method, add a new inner class in Py called PySession. Now I can do this:

using (var ps = Py.Session())
{
    var result = ps.Eval("1+2"); //return 3

    dynamic sys = ps.Import("sys");  //import, it will also be added in globals
    ps.SetLocal("stdout", new Output()); //declare a local variable
    ps.Exec("sys.stdout = stdout");  //it's working!
    ps.Exec("print('Hello!')");

    sys.stderr = 12;
    ps.Exec("print(sys.stderr+1)");

    ps.SetGlobal("bb", 100);//declare a global variable
    ps.SetLocal("cc", 10);//declare a local variable
    ps.Exec("aa=bb+cc+3");   
    var aa = ps.Get("aa"); //return 113
    Console.WriteLine(aa);
}

what do you think about? I'm not very familiar with git, I'll try to create a new branch and create a new PR for this later.

den-run-ai · 2017-02-15T05:53:38Z

@yagweb regarding PySession can you please open an issue first, so that we can quickly iterate on purpose, design and API. You can open a pull request, but it may get affected by decisions in the issue. I think this closely resembles the Scopes in IronPython.

vmuriart · 2017-02-15T05:57:06Z

@yagweb I think you are missing a reference to <Reference Include="Microsoft.CSharp" /> on EmbeddingTest.csproj. Probably got lost in the merge conflict resolution.

yagweb · 2017-02-15T16:47:32Z

@denfromufa Thank you for your suggestion, I opened the issue here.

yagweb · 2017-02-15T17:12:30Z

@vmuriart Thank you, I have squashed this commit.

vmuriart · 2017-02-15T17:23:19Z

pushed a fixup to minimize whitespace/encoding changes.
From your other pull_request it sounds like AssignObjectInPython test is no longer needed and consequentially neither would outstream be needed?

yagweb · 2017-02-16T17:18:36Z

@denfromufa @vmuriart I readed some source code, and found pythonnet already had some powerful methods. So, this branch undergos a major revision. Please check it again.

the GetAttrDynamic and SetAttrDynamic method is deleted.
the TrySetMember should call set attribute method of CPython no matter it has the attribute or not. Because python object is dynamic. If not, a new attribute cannot be found in CPython. The Refcount has been checked.
Corresponding to the TrySetMember method, the TryGetMember method only get attribute from CPython.
A new Attribute Refcount is added for Pyobject, because I found it's useful when debugging. If you donnot like it, please just remove it.

The unit tests file also changed. The AssignObjectInPython has been removed and the outstream class has been replace with the StringBuilder class.

yagweb · 2017-02-16T17:45:29Z

@vmuriart The method AssignObjectInPython can be removed. but taking the bug it found, I think a new unit test method below could be to the file pyinitialize.cs.

[Test]
[Ignore("System.ArgumentException : Cannot pass a GCHandle across AppDomains")]
public void Main()
{
    PythonEngine.Initialize();
    using (Py.GIL())
    {
        PythonEngine.RunSimpleString(
            "from System.Text import StringBuilder\n"+
            "StringBuilder()\n");
    }
    PythonEngine.Shutdown();

    PythonEngine.Initialize();
    using (Py.GIL())
    {
        PythonEngine.RunSimpleString(
            "from System.Text import StringBuilder\n" +
            "StringBuilder()\n");
    }
    PythonEngine.Shutdown();
}

The code below can achieve the same functionality but with no error. It reveals that,

We should avoid new .NET object in Python when the host is a .NET program.
This may cause by CPython new .NET object in a new .NET environment, not in the current one.(unprecise). Maybe it can help to resolve this bug.

[Test]
public void Main2()
{
    PythonEngine.Initialize();
    using (Py.GIL())
    {
        dynamic sys = PythonEngine.ImportModule("sys");
        sys.attr1 = new StringBuilder();
    }
    PythonEngine.Shutdown();

    PythonEngine.Initialize();
    using (Py.GIL())
    {
        dynamic sys = PythonEngine.ImportModule("sys");
        sys.attr1 = new StringBuilder();
    }
    PythonEngine.Shutdown();
}

vmuriart · 2017-02-16T18:16:55Z

@yagweb thanks for taking the time to digging more into that bug. It looks to be an extension of #343. Your test is very similar to the one in pyinitialize so I'll probably add the test there.

Is the issue only present when using StringBuilder() or can it be any arbitrary object? (ie. a string?).

yagweb · 2017-02-17T01:51:08Z

@vmuriart I tried several RunSimpleString combinations in the two Initialize. If keep the two RunSimpleString statement the same, System.Int32 and System.String can also trigger this exception. If the .NET object created in the two RunSimpleString statement are different, some can trigger the exception, some cannot.
So I think it can be any arbitrary object. I updated my last comment to make the test simpler.

attribute of PyObject by dynamic type

vmuriart

Good to merge once all CI checks are completed.

vmuriart · 2017-02-17T05:10:46Z

@yagweb sorry it took me a while to review, I kept getting distracted while reviewing this.
Looks good. Just rebased it and will merge once all CI checks complete.

irt the other issue. We should open a new issue to summarize the findings.

yagweb · 2017-02-17T06:11:48Z

@vmuriart I got the merged message. I will rebase my branches. Thank you!

vmuriart · 2017-02-17T06:17:44Z

No worries. I had to rebase it because I updated the CHANGELOG and introduced a merge conflict. Also gave me a chance to clean up some whitespacing 🤓

@dmitriyse

* Sanitize how to run tests * Remove unsupported entry points * Adds reference to Python.Test by default for all tests * Remove redundant add_reference * Avoid some implicit AddReferences that were being done Not all tests added reference to Python.Test consistently. Solve this by making `run_test` the only supported method. * Remove dependency on six Remove six.u(...), six.b(...) Not needed since dropped older python versions * Clean-up imports - Fix py2/py3 range/zip behavior - Ensure same functions being used - Fix Exception/System.Exception name clashes * Fix nameclash object * Refactor utility functions * Rename tests * Refactor assertRaises test_exceptions: Add exc_info test & fix Exception nameclash * Format for pep8 and linters Fix max min nameclash * Fix tests comparison to None, True, False, isinstance - Replace type(()) with tuple to clarify intent * Rename test fixtures Can mess with test discovery * Remove unused `using` from testing and format * Use unittest.skip(...) * Cleanup testing format * XML Docs on test fixtures * Update CHANGELOG * Implement Py.SetArgv. * Add a `Py.Throw` function. Checks whether an error occurred and in case it has throws a PythonException. * Add overload for Initialize to pass specific args. * Allow the engine to be initialized as a Disposable. * Use Py.Throw in a few more places. In particular in Py.Import. * Add PY3.7 to travis with allow_failure * Format & Whitespace *.cs linelength 120 Allman * Update CHANGELOG * Change applicable comments to xml-docs * Update API website on docs & fix section headers Runtime.cs section headers got changed to docs. * Alias root.pyHandle to py_clr_module * Syntax cleanup * Simple classes cleanup (pythonnet#348) * Standarize properties used across projects Console had it owns set of properties going on * Add pyproj from @denfromufa Corrected the configs, won't interfere with users without PTVS * Consolidate output paths http://stackoverflow.com/questions/30191387/what-are-the-consequences-of-setting-same-output-path-for-all-configurations * Fix clrmodule config mismapping * Add PY3 project/sln configs * Update test PyImport path Changed when consolidate output paths * Update runtime.csproj * Documentation clean-up * Downgrade to nunit2 * Syntax cleanup * Update CHANGELOG * Clarify pynetinit.c, runtime build directives pynetinit.c: Clarifiy its intent is for PY3 runtime.cs Remove unnecessary check for System.Text Simplifies import section * Whitespace converter.cs * Propercase AssemblyInfo * Refactor nativemethods * Rename `is32bit` to `Is32Bit` for naming consistency * Replace UCS2/UCS4 build directive with Runtime.UCS * Add IsPython2 and IsPython3 * Cleanup runtime config * Remove extra properties/Organize properties * Refactor property conditions * Order Runtime.csproj configuration To make it easier to verify everything is setup correctly Simplified the Mono.Unix include condition * Add pylong lost unit test Wasn't being referenced in the project * Replace #if DEBUG with conditional attribute * Fix and disable StrongNameSigning It works better if they key is in the right folder and its being referenced when signing. It breaks InternalsVisibleTo though, disabling till have time to look into it. (not that it was actually enabled) * Fix debug statement & rename debugprint * Document nPython/Console usage Closes pythonnet#358 * Quiet Test Fixture build warnings about unused events. They will be used (or attempted to be used) from Python. * Mono dependency removed from Linux build. * Remove mono dependency from PY2 as well. * Replace bufLength with size * Remove arch .il files * Update CHANGELOG * Remove Mono.Unix reference from project Remove duplicate property in Python.Test * Clean-up deprWarning * Clean-up attribute names and paranthesis * Use explicit accessors * Clean-up code-style Remove redundant parenthesis, enforce brackets, use keywork types. * Remove obsoleted & Unenforced PermissionSet See: http://stackoverflow.com/questions/11625447/securityaction-requestminimum-is-obsolete-in-net-4-0 * Add expire to license shield Github Cache issue github/markup#224 * Remove usage of #region It hides away the code in Visual Studio * Turn on NUNIT exit code * Fix GC on PyObject during Finalizing Similar to how `PythonDerivedType` finalizes. Closes pythonnet#364 Possibly related to pythonnet#245 * Update CHANGELOG * Add sys.args tests (pythonnet#301) * Use NUnit from Nuget to ensure right version. * Simplify Embedded tests & Add new tests (pythonnet#369) * Add Tuple tests * Update AppVeyor comments and show when tests start. * Rename InitializeTest to pyinitialize Keep consistent with other test classes. * Simplify embed_tests Since the previous bug involving initialize/finalize have been solved, we can confidently upgrade these test to use simpler format. Also Remove Assert fail messages NUnit already says the name of which test failed. * Add e-mail to remove setup.py warnings * Update NUnit syntax Use xUnit2 and nUnit3 friendly syntax * Use string interpolation Removed extra newline at beginning from deprecation warning message * Replace PyString_FromStringAndSize usage to PyString_FromString * Fix xml-docs build warnings * Refactor AppVeyor/Travis embedded test_runner Make it easier to diff and upgrade versions by breaking into multiple lines and using path completion Powershell continuation: http://stackoverflow.com/a/2608186/5208670 Powershell options splatting: http://stackoverflow.com/a/24313253/5208670 * Add pytest to CI pytest on travis initially required `sudo=required` otherwise failed. Without the sudo flag pytest wasn't being upgraded. Needed to force upgrade. pytest-dev/pytest#2240 * Convert unittest to pytest Needs local imports to work. Conversion done with unittests2pytests and a couple regex * Add setup.cfg Control settings for some add-ons like pytest * Simpler tox * Upgrade NUnit to 3.6 * Fix coverage w NUnit3; add OpenCover filter NUnit3/OpenCover behavior changed. Filter removes coverage from Embedded test files by focusing only on types of Python.Runtime.* * Remove check-manifest * Add overload tests from pythonnet#131 * Refactor converter.cs & methodbind*.cs (pythonnet#375) * Cleanup/Comment Python.Runtime.dll.config & travis We don't need the `*.dll` variant since we don't call for those in `runtime.cs` Enable embedded tests on travis * Add LD_LIBRARY_PATH to travis env Find dlls for embedded tests on PY3 * Find dll for PY2 travis embedded tests Copy Python.Runtime.dll.config to embedded tests bin output * Generalize LD_LIBRARY_PATH Avoid having to manually update python/travis versions * Add GetPrecedence reference * Use builtin miniconda * Update version (pythonnet#379) * Update setup.py * Update clr.py * StackOverflow/Slack shields (pythonnet#384) Add StackExchange and Slack shields Shields got too long for a single line, had to break them up into two lines. One for build/quality status, the other for social/info. Add regression number to disabled code. * Slack integration for Travis and Appveyor (pythonnet#386) * Add Slack integration for Travis * Use encrypted string for Travis. * Add hook for Appveyor. * Simplify LD_LIBRARY_PATH on Travis Based from: http://stackoverflow.com/a/24115039/5208670 Moved to own section to separate as a prep-enviroment step * Rename NUnit vars to generic names Makes it easier to switch test-runners in the future. Note the if section is being skipped for NUnit3, but leaving it in case needed to rollback to NUnit2, or upgrade to XUnit * Remove python test STDERR redirect Pytest doesn't need stderr redirect. It behaves * Clean-up AppVeyor build recipe Conda automatically updates when doing install. To disable its autoupdate add conda config --set auto_update_conda False * Add requirements.txt and pytest options Pytest options provide summary for skipped/xfailed tests and adds color on appveyor * Quiet sdist output & update comments on AppVeyor * Add pytest header info & remove unused fixtures Fixtures came from a different project and I forgot to delete them. Add custom header to gather environment info with test results. Remove testdir from pythonpath. pytest handles this on conftest.py * Move Conda env to PowerShell recipe script PY_VER removes the `.` it is used on appveyor_build_recipe.ps1 * Use Codecov report flags Upgrade to Codecov v2.0.6, didn't get released to PyPi. * Clean-up CI configs * Update CHANGELOG * Pass arbitrary .NET object as value of an attr of PyObject by dyn type(pythonnet#373) * Add Coverity badge * Clean-up CHANGELOG * Standardize Python.Test fixture location Ensures that Python.Test is output to same location on both Linux and Windows as part of build. .gitkeep to be ensure folder is part of version control. Can be removed if any file is added to it. * Add unittest for Overflow/across AppDomains Exceptions reproduction (pythonnet#393) First exception shows issue from pythonnet#376 * Re-order Initialize tests * Add documentation and ref to ReInitialize test * Split and re-order TestPyTupleIsTupleType test * Skip PyTuple test with AppDomain issue This skip removes the issue from all PY3 on Travis/AppVeyor. PY27 still has issue randomly on both Travis/AppVeyor x86, x64,. * Clean-up embedded tests Clean-up embedded tests comments and variable types Remove TestFixture attribute. Optional since NUnit 2.5 https://nunit.org/index.php?p=testFixture&r=2.6.4 * Fix conda build log stderr->stdout Recent conda-build update added a new log output to log:info. Powershell interprets this as an ERROR since its on STDERR. Prevent accidental auto-update & display INFO before building. Would have made debugging this psudo-error easier. * Quiet AppVeyor pip/nuget installs Reduce verbosity to make relevant information easier to find. Errors and Warnings are still displayed. Travis doesn't need this since they have log folding. * Allow private env:var to force conda build To trigger conda build, just add/update private env:vars FORCE_CONDA_BUILD on link below. Easier to debug and no need to edit appveyor yml/ps1 https://ci.appveyor.com/project/USER_NAME/pythonnet/settings/environment * Update AUTHORS Add names to a few more contributors * Rename TearDown to Dispose Easier for future possible migration to other frameworks * Relocate Embedded tests fixtures Each test type should contain its own fixtures. Reduces weird dependency of each testing framework * Update LICENSE year & include in recipe * Update conda-recipe version Get it from recipe file instead of git-tag. * Add SharedAssemblyInfo & Update Assembly version Reset version back down to v2.x from v4.0.0.2 See link below for Semanctic Versioning & .NET https://codingforsmarties.wordpress.com/2016/01/21/how-to-version-assemblies-destined-for-nuget/ * Add .bumpversion Configuration based from gh:bumpversion:issues:77#issuecomment-130696156 Usage: bumpversion major -> increases major and adds `dev` if not present bumpversion minor -> increases minor and adds `dev` if not present bumpversion release -> drop the `dev` portion * Add object overload test * Add object type to methodbind * Add more object overload method tests * Update CHANGELOG * Enable embedded_tests to Travis w. conditional filters Add conditional class skip to pytuple for Travis/PY27 Add individual filters to other tests as needed https://www.amido.com/code/conditional-ignore-nunit-and-the-ability-to-conditionally-ignore-a-test/ http://stackoverflow.com/a/16075029/5208670 * Split blank DefineConstants on `*.csproj` Visual Studio by default wants to split these. Any time the csproj is update (new test for example) it splits them. Splitting them now to not worry about them when reviewing pull_requests. * Update CHANGELOG * Fix PythonException GC (pythonnet#400) Since PythonException doesn't inherit from PyObject, need to reapply the same fix as from gh:365. Seen mostly on PY27/Travis on PyTuple tests. Enable PyTuple tests for PY27/Travis * Add Eval(...) and Exec(...) * Keep RunString Public. Fix Assert.AreEqual order - Deprecation/Removal should be a separate issue/pr - In NUnit/XUnit, expected is the first argument, actual is second. Opposite to how Python does it * Update CHANGELOG * Clarify MashalAs on runtime.cs - Shorten MarshalAsAttribute name - Put MashalAs'ed argument on own-line - Flip IF comparison * Fix runtime Initialize dll nameclash Looks like dllLocal never changes from IntPtr.Zero * Update test_sysargv Previous version didn't test correctly * Fix the issue of sys.argv being cleared on import clr. Closes pythonnet#404. * improve tests.pyproj for intellisense and running tests (pythonnet#395) * Update tests.pyproj * Update folder structure After pytest folders were flatten. Add code in fixture Upstream PyImportTest was moved to within Embedded Tests Add pyproj to editorconfig * Add case-sensitivity tests closes #81 * Added branching for ldd command in OSX (pythonnet#406) * Minor style clean-up runtime/pythonengine Add missing brackets Organize using/remove unused Align arguments * Fix Py_Main/PySys_SetArgvEx(...) UCS4/PY3 no mem Based on @dmitriyse work on: dmitriyse@8a70f09 * Update CHANGELOG, remove extra MarshalAs * Move PythonBuildDir when not defined Keep root of solution clean and out of the way from being imported because its on the `cwd`. * Update ARCH check Use sys for x86/x64 check * Enable TestPyTupleInvalidAppend test `AppDomain unload` was solved by pythonnet#400. Closes pythonnet#397. Possibly also solved pythonnet#245. * Add tests/refactor existing * Set language version to 6 Prevent accidental introduction of csharp 7 features. * Add ICustomMarshaler StrMarshaler Useful resources https://msdn.microsoft.com/en-us/library/system.runtime.interopservices.icustommarshaler(v=vs.110).aspx https://limbioliong.wordpress.com/2013/11/03/understanding-custom-marshaling-part-1/ https://github.com/mono/mono/blob/master/mcs/class/Mono.Posix/Mono.Unix/UnixMarshal.cs http://stackoverflow.com/a/33514037/5208670 * Add ICustomMarshaler StrArrayMarshaler * Refactor Marshals * Add ICustomMarshaler Utf8Marshaler Refactor PyString_FromStringAndSize Link explains why `MarshalAs(UnmanagedType.LPWStr)` or `CharSet.Unicode` don't work http://stackoverflow.com/a/25128147/5208670 * Match PyUnicode_AsUnicode signature in UCS2/UCS4 * Refactor GetManagedString * Remove internal PyUnicode_AS_UNICODE Its redundant with PyUnicode_AsUnicode now that the signature is fixed between UCS2/UCS4. Apply char conversion that work on both UCS2/UCS4 * Refactor Encoding check This won't change during runtime. * Remove ExactSpelling Not needed as Python doesn't define character specific functions https://msdn.microsoft.com/en-us/library/system.runtime.interopservices.dllimportattribute.exactspelling(v=vs.110).aspx * Remove CharSet.Ansi Its default charset used https://msdn.microsoft.com/en-us/library/system.runtime.interopservices.dllimportattribute.charset(v=vs.110).aspx#Anchor_1 * Remove CharSet.Unicode where not needed CharSet.Unicode is only needed when args are string type. https://msdn.microsoft.com/en-us/library/system.runtime.interopservices.dllimportattribute.charset(v=vs.110).aspx * Remove CallingConvention * Rename Runtime field `dll` to `PythonDll` Add remove full qualification * Remove unneeded unsafe keyword * Update CHANGELOG * Add Runtime.CheckExceptionOccurred(...) Helps refactor exceptions checks * Replace Py.Throw with Runtime.CheckExceptionOccurred * Whitespace clean-up Runtime * Apply consistent name qualification to runtime.cs * Unify PY3 UCS2/UCS4 unicode methods CustomMarshal implementation deals w UCS2/4 differences * Unify PY2 UCS2/UCS4 unicode methods PyUnicodeEntryPoint is not used for PY3 since it was unified by PEP393. It could be defined as "PyUnicode_" for PY3 and further unify the code between PY2/PY3. Not implementing since not all PY3 methods exist in PY2 * Rename internal methods to proper Python API name - PyUnicode_FromKindAndString to PyUnicode_FromKindAndData - PyString_AS_STRING to PyString_AsString * Fix PY27 dll mapping in Linux/macOS On 5062377 this was fixed for PY3 but left unchanged for PY2. On linux/macOS the library is aliased `python2.7` while in windows its `python 27`. Since internally it wasn't mapped to the correct library in Linux/macOS, we had to remap it again using the dll.config file. Closes pythonnet#120 * Refactor runtime's dllBase As long as the API doesn't change on new python minor releases, all changes needed for new minor versions is isolated to a small section of code. * Style clean-up runtime.cs * Temporary disable Codecov flags Codecov added a limit of 20 uploads due to current on-going bug https://docs.codecov.io/blog/week-8-2017 Also fixed flags casing, documentations says has to be lowercase. * Fix test_multiple_calls_to_initialize Exception check didn't upgrade syntax from unittests. Didn't cause issues because test was passing. * Fix PythonEngine.Version Add TestPythonEngineProperties. Ensure PythonEngine properties are working correctly. Currently only work on 32bit machines. Closes pythonnet#413 * Add PYTHONPATH/PYTHONHOME default value tests Current tests crash on 64bit python on windows, and results get truncated on Linux. When working, PYTHONHOME should match ENV VAR if set. AppVeyor has been updated to test against not blank * Fix get PYTHONHOME/PYTHONPATH marshal Unlike pythonnet#413, the return type changes between PY2/PY3. Extended Custom Marshaler to convert IntPtr to Unicode String * Rename internal StrMarshaler to UcsMarshaler To clarify that this is meant to be applied on Unicode type IntPtr and not strings like ones. Made Marshalers internal, don't see a reason to expose it. * Unset PYTHONHOME in AppVeyor Its messing with `conda` causing it to fail to start. Tests in the `pr` pythonnet#415 make this unnecessary. * Update conda Update to using conda based on PY36. Remove reference to deleted `dll.config` file * Fix PythonEngine PYTHONHOME setter Keep memory reference & fix PY3 marshal * Fix set PythonPath, set ProgramName Note on PythonPath. Its actually mapping to `Py_SetPath` which is very different from PYTHONPATH env var. There is no test on it because it should be set to real paths with libraries. Otherwise it crashes. 2nd Note. `Py_SetPath` doesn't exist on PY27. * Deprecate public RunString Had to remove defaults to disambiguate call on `internal RunString`. Can re-add after removing `public RunString` Closes pythonnet#401 * Combine Py_DEBUG and PYTHON_WITH_PYDEBUG flags They both refer to the PyDebug builds but were added at different times. Closes pythonnet#362 * Remove PYTHON_WITH_WIDE_UNICODE flag ABIFlags were introduced in PY32, and --with_wide_unicode was removed in PY33. https://docs.python.org/3/whatsnew/3.3.html#functionality Closes pythonnet#417 * Refactor PY2/PY3 Marshal in/out String/Unicode * Refactor runtime.cs * Fix Py_SetPath not available in PY2 Closes pythonnet#418 * Calculate Runtime fields before Initialization * Add timing to detect slow tests on pytest * Rename test classes/files Make it easier to distinguish if editor tab refers to class or test. * Add PyTuple Ctor tests * Define and document Py_IncRef/Py_DecRef * Add Tests * Refactor Exception checking on tested classes * Clean-up Embedded tests * Add PyList tests * Add test for pyscript global variable casting Test for pythonnet#420 * Clean-up README.md example * Fix typo in README * Update CHANGELOG.md * Bump version: 2.3.0.dev1 → 2.3.0 release * Bump version: 2.3.0.→ 2.4.0.dev0 Can't start at dev1 as bumpversion breaks if we add a minimum start version for dev. * Build conda recipe on new tags Ensure that new binaries are generated for releases which are usually not from pull_requests. Using APPVEYOR_REPO_TAG_NAME because yields shorter syntax because it's undefined (ie false) when not a tag and APPVEYOR_REPO_TAG has to be compared to lowercase `true` instead of `True` like everything else. Definitions: APPVEYOR_REPO_TAG_NAME - contains tag name for builds started by tag; otherwise this variable is undefined; APPVEYOR_REPO_TAG - true if build has started by pushed tag; otherwise false; https://www.appveyor.com/docs/environment-variables/ * Fix numpy array and README example (pythonnet#427) * Fix numpy array and README example Generic Lists were falling through and being classified as `typecode.Object` To solve this, adding a specific processing branch for `Generic Lists` only to avoid breaking the changes from 471673a Closes pythonnet#249 * Update syntax * Msbuild15 patch (pythonnet#435) * Update AUTHORS.md * Update CHANGELOG.md * Update setup.py * Update bld.bat * Create .mention-bot * WPF DynamicGrid python and XAML layout files (pythonnet#280) * Support clr.GetClrType() - as in IronPython (pythonnet#433) * Support clr.GetClrType() - as in IronPython Implements pythonnet#432 * Tests for clr.GetClrType() * clr.GetClrType test: ensure bad type raises ArgumentException * clr.GetClrType - added xml doc comment, updated AUTHORS.md and CHANGELOG.md * Simplified implementation of clr.GetClrType (taken from IronPython) * Update .mention-bot * Update .mention-bot * Re-add CallingConvention Closes pythonnet#448 * Allow passing None for nullable args (pythonnet#460) * Updated CHANGELOG and AUTHORS (pythonnet#462) * Fixing Travis CI for mono 5.0 (pythonnet#471) * Fixing setup.py with mono 5.0 * Go back to xbuild * Trying to figure out the error cause. Apparently xbuild cannot process the Copy section of the pdb file in the projects. I have commented them out for now, but this is not a real soltion since they are needed in debug versions. Just to se if it fixes the CI. * Implement named arguments and With semantics in C# embedding side (pythonnet#461) * Added python "with" construction * Added some unit tests for new With method * Renamed With tests for easier grouping * Support for named arguments to invoke python methods * Named argument tests cosmetic changed * Fixed failing test in python 2.7 * Reset line endings in csproj to LF * Implements Decimal and Datetime support

yagweb mentioned this pull request Feb 10, 2017

why sys.stdout returned null #370

Closed

den-run-ai requested review from filmor, tonyroberts, den-run-ai and vmuriart February 12, 2017 05:40

vmuriart force-pushed the master branch from eec8893 to d370896 Compare February 13, 2017 15:56

yagweb force-pushed the master branch 3 times, most recently from 7a41ef8 to 21ca6e2 Compare February 14, 2017 06:23

vmuriart mentioned this pull request Feb 14, 2017

Fix OverflowException on 64bit due to bad cast #376

Closed

yagweb force-pushed the master branch from a93bbbc to d914a89 Compare February 15, 2017 05:49

yagweb force-pushed the master branch from 2ec9966 to 6c34d20 Compare February 15, 2017 06:28

yagweb mentioned this pull request Feb 15, 2017

Add a class to manage the context of a interaction with Python and simplify the variable exchanging #380

Closed

den-run-ai approved these changes Feb 15, 2017

View reviewed changes

yagweb force-pushed the master branch 2 times, most recently from 2bceb13 to 32f5878 Compare February 16, 2017 17:08

Add function of passing an arbitrary .NET object as the value of an

51d229f

attribute of PyObject by dynamic type

vmuriart force-pushed the master branch from 32f5878 to 51d229f Compare February 17, 2017 05:04

vmuriart approved these changes Feb 17, 2017

View reviewed changes

vmuriart merged commit 1d583c7 into pythonnet:master Feb 17, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Passing an arbitrary .NET object as the value of an attribute of PyObject #373

Passing an arbitrary .NET object as the value of an attribute of PyObject #373

yagweb commented Feb 10, 2017 •

edited

Loading

den-run-ai commented Feb 12, 2017

codecov bot commented Feb 13, 2017 •

edited

Loading

vmuriart commented Feb 13, 2017

yagweb commented Feb 13, 2017

vmuriart commented Feb 13, 2017

yagweb commented Feb 14, 2017 •

edited

Loading

vmuriart commented Feb 14, 2017

vmuriart commented Feb 14, 2017

yagweb commented Feb 14, 2017

vmuriart commented Feb 14, 2017

yagweb commented Feb 14, 2017 •

edited

Loading

vmuriart commented Feb 14, 2017

vmuriart commented Feb 14, 2017 •

edited

Loading

yagweb commented Feb 15, 2017

yagweb commented Feb 15, 2017 •

edited

Loading

den-run-ai commented Feb 15, 2017

vmuriart commented Feb 15, 2017

yagweb commented Feb 15, 2017

yagweb commented Feb 15, 2017

vmuriart commented Feb 15, 2017

yagweb commented Feb 16, 2017 •

edited

Loading

yagweb commented Feb 16, 2017 •

edited

Loading

vmuriart commented Feb 16, 2017

yagweb commented Feb 17, 2017

vmuriart left a comment

vmuriart commented Feb 17, 2017

yagweb commented Feb 17, 2017

vmuriart commented Feb 17, 2017

Passing an arbitrary .NET object as the value of an attribute of PyObject #373

Passing an arbitrary .NET object as the value of an attribute of PyObject #373

Conversation

yagweb commented Feb 10, 2017 • edited Loading

den-run-ai commented Feb 12, 2017

codecov bot commented Feb 13, 2017 • edited Loading

Codecov Report

vmuriart commented Feb 13, 2017

yagweb commented Feb 13, 2017

vmuriart commented Feb 13, 2017

yagweb commented Feb 14, 2017 • edited Loading

vmuriart commented Feb 14, 2017

vmuriart commented Feb 14, 2017

yagweb commented Feb 14, 2017

vmuriart commented Feb 14, 2017

yagweb commented Feb 14, 2017 • edited Loading

vmuriart commented Feb 14, 2017

vmuriart commented Feb 14, 2017 • edited Loading

yagweb commented Feb 15, 2017

yagweb commented Feb 15, 2017 • edited Loading

den-run-ai commented Feb 15, 2017

vmuriart commented Feb 15, 2017

yagweb commented Feb 15, 2017

yagweb commented Feb 15, 2017

vmuriart commented Feb 15, 2017

yagweb commented Feb 16, 2017 • edited Loading

yagweb commented Feb 16, 2017 • edited Loading

vmuriart commented Feb 16, 2017

yagweb commented Feb 17, 2017

vmuriart left a comment

Choose a reason for hiding this comment

vmuriart commented Feb 17, 2017

yagweb commented Feb 17, 2017

vmuriart commented Feb 17, 2017

yagweb commented Feb 10, 2017 •

edited

Loading

codecov bot commented Feb 13, 2017 •

edited

Loading

yagweb commented Feb 14, 2017 •

edited

Loading

yagweb commented Feb 14, 2017 •

edited

Loading

vmuriart commented Feb 14, 2017 •

edited

Loading

yagweb commented Feb 15, 2017 •

edited

Loading

yagweb commented Feb 16, 2017 •

edited

Loading

yagweb commented Feb 16, 2017 •

edited

Loading