Two separate packages. This is not an "official" answer, and I have no affiliation with MicroPython, I've just been thinking about this a lot (for MicroPython specifically recently, and in general for a long time):

1. If we expect that we might need to do two separate packages in the future, we ought do two separate packages now.

   Because we don't want to set our users up for unexpected breaking/bugs in their workflow later, which will happen if they get used to upip install our-package and then an update to our-package breaks compatibility with MicroPython.

   I have what I like to call "the undo principle": when in doubt, do what is easier to undo. In the case of public interfaces (and the name of a package is a public interface) "easier" includes "for our users", not just for ourselves.

   If we have two separate packages, and we realize we only really need one, it is trivial to make one into a dummy alias (empty package with dependency) of the other, or to take a workflow that generates one package and wrap it to just rerun the build with a different package name.

   But if we have one package, and we later realize we need two, the only ways to "undo" that will effect at least some users. It takes worst-case failure mode from "additional packaging/maintenance burden on our end that can be mostly automated with some one-off effort" to "unexpectedly breaks user code or workflows when they're urgently trying to do something else, an indefinite amount of time, indefinitely into the future".

   Also if we have, or can foresee having, dependencies: what happens if in the future, one of our dependencies decides to split off a separate MicroPython package? If we have two packages already, we can adjust dependencies in the MicroPython version of our package and move on - what solutions are available if we have just one package?

2. Does our code (including dependencies) have high enough odds of not relying on Python features that MicroPython doesn't support?

   My experience with portability (and I spend an unhealthy, neurotic amount of time thinking about and testing portability) is that the answer is "no", because subtle incompatibilities tend to exist all over the place at the "edges".

   I'm talking edges like correct behavior of the else clause in the try statement, or the .throw method on generators - one of those misbehaves in MicroPython, one of those misbehaves inside another Python implementation that recommends you put packages on PyPI - do you know which? Would you have thought to check, if you noticed a dependency added or has some obscure code path which uses one of them?

   Are the portability workarounds you have to make in your own code always going to be simple enough to outweigh the complexity of maintaining separate packages? (Let me put it this way: I found out that else misbehaves in one Python implementation because I was working around finally misbehaving in another more obscure Python implementation. Just knowing about compatibility issues is a huge, deep tarpit - coding around them is even worse.)

   Are you going to be able to bring up and hold in your head all of the possible incompatibilities to properly assess that risk?

   If you have, or can foresee having, dependencies: are the people who maintain your dependencies going to know to work around them correctly?

   Maybe. But when in doubt, "undo principle": do what is easier to undo.

MicroPython is not yet at the level of feature-correctness polish and maintenance manpower that I would consider it a safe bet that your code, and any dependencies that your code has, would remain compatible with MicroPython going forward.

So for myself, unless I'm really confident that I know exactly what my package will do now and in the future will be easy to keep compatible, and I'm really confident that I'm not missing anything in that forecast, I will be making separate packages for anything I'm intentionally packaging for use on MicroPython.

I'm talking edges like correct behavior of the else clause in the try statement, or the .throw method on generators - one of those misbehaves in MicroPython

Any reference to the ticket number for that?

@pfalcon Nope, sorry. Haven't taken the time to figure out the details of how you want that done, or what exactly you want for verification. So far I usually just use the web browser Unicorn emulator for quick compatibility testing:

>>> def g(): yield
...
>>> g().throw(Exception('koz'), None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in g
TypeError: exceptions must derive from BaseException

The expected (what CPython does) behavior is for it to just raise Exception('koz'), same as it would if you didn't pass the second argument at all. (Same with when all three arguments are passed, of course: the logic for determining when the first argument is allowed to be an exception instance in CPython seems to be based on the second argument either being not passed in or None, not just on not being passed in.)

Re the original question, upip uses different compression attributes compared to pip and pip3. Hence separate PyPi repositories are necessary.

@peterhinch Wouldn't the lower compression settings that upip wants get handled transparently by pip? I have never tried because in this case I solved the "ought to" question to my satisfaction before solving the "can" question, but I would've presumed that fully-featured pip just detects the compression settings when decompressing?

@pfalcon I created #4527 , feel free to let me know if there are issues with it or close it as you deem fit.

@mentalisttraceur Alas it does not. The MicroPython format was designed to enable upip to be hosted on RAM-constrained devices such as the ESP8266.

@peterhinch Right, I'm asking about the other way around: I know upip cannot open pip packages, but can pip open upip packages?

Thanks everyone for the detailed responses --- I'm perfectly fine with your recommendations to use separate packages.

This issue may be closed as far as the original question is concerned, but it sounds like there might be more to it.

upip format is effectively a "wheel" (i.e. binary package) using (and breaking) conventions for source packages. So no, pip can't install upip packages, for starters it'll fail on missing setup.py within such a package.

@pfalcon Right, but say we violated the upip spec in that one specific way and shoved a setup.py in there anyway (instead of filtering it out as recommended and implemented in the reference in micropython-lib packaging code), would upip error out on install or would it just ignore the setup.py?

Hopefully you don't mind the maybe obvious questions - I've never had the opportunity to deeply look into the workings of either pip or upip packages.

I'm just trying to think through the possible ways to abuse each package spec step-by-step to see if there is some combination of technically-not-how-it-is-supposed-to-be-done abuses that would cause the code of each to succeed.

Hopefully you don't mind the maybe obvious questions - I've never had the opportunity to deeply look into the workings of either pip or upip packages.

I don't mind, but I don't remember such details either. So, we're in the same position - we both would need to reach for the source to answer the question.

@pfalcon Alright, thank you for taking the time to answer so far.

Okay, @laurensvalk said this can be closed, and I'm done with the pip vs. upip compatibility hacks topic for now, so I'm good with this being closed too.