and more efficient to map to the underlying hardware implementation

Ultimately, with one-to-one (no-op) mapping.

But note that on the STM, each port is actually a pointer and so there is going to be some level of mapping no matter what.

Ok, let's consider how to implement it without any "extra" mapping:

import ffi

Bit0 = 1
Bit1 = 2
Bit2 = 4
...
PortA = 0x40020000 
PortA = 0x40020400 
...
GPIO_BSRR = 0x18 / 4


class GPIO:
    def __init__(self, port, pin):
        self.port = ffi.Pointer(int32_t, port)
        self.pin = pin

    def high(self):
        self.port[GPIO_BSRR] = self.pin

    def low(self):
        self.port[GPIO_BSRR] = self.pin << 16

    def set(self, val):
        if val:
            self.high()
        else:
            self.low()

As you see, there's no "mapping" - all port/pin values used as is, with minimum possible transformations (it would be possible to get rid of << 16 by using uint16_t pointers). Viper would turn that into the most efficient asm code (if it could lower class abstraction, but that's different matter).

Implementation which would use scalar number for pin, would be less efficient, even if by 2-3 instructions. But that's the point - if we talk the ultimately efficient implementation, that's it.

It would be a bit perfectionist to just consider performance gain of 2-3 instructions (even though 2-3 python instructions is considerable performance hit when we talk MHz GPIO flip performance).

But as mentioned above, using ports provides also more faithful representation of the hardware. And note that that may be either physical hardware level of particular MCU, or "virtual" level of particular board.

Pyboard example with ports X and Y was given, but I don't know if pins in these virtual ports are truly equivalent, so instead may present more vivid case where that holds. Jeenode board is designed to provide 4 inasmuch identical ports as possible: http://jeelabs.net/projects/hardware/wiki/JeeNode#Portpin-mapping . So, daughterboards can be plugged into any such port, and replugged freely. And when user replugs a daughterboard, it would be natural to expect to update just a single port number in the app somewhere, not hunt thru the app to do search and replace of "PA1" => "PB1", etc.

In reality, I don't see how GPIO is going to be implemented as you suggest. That code is going to need to be a module, so use of PortA, etc, will need to be something like pyb.PortA. Optimisation of constants from an imported module needs a bit/lot of thought (see #266).

For the pin read/write, yes, it'll be implemented efficiently like your example (but in C). Optimising the initialisation of a port is much less important, and will anyway also include assertions to check that you pass the correct parameters.

The Jeenode board and it's 4 ports are very similar in spirit to the skin X and Y positions. In these 2 cases, the ports are all logically equivalent, but still require a mapping to get to the underlying hardware pins.

In practice it's not common to be able to simply change the port for a peripheral from A to B. For proper configurability, each logical pin used for an attached peripheral has the physical port and pin (and sometimes other things) specified. Eg, the CC3000 driver uses PA14, PC6, PC7, PB13, PB14 and PB15. It's generally like this, a mish-mash of pins from different ports.

Optimisation of constants from an imported module needs a bit/lot of thought (see #266).

Yes, but let's try not to mix 2 different issues, like "as we talk dynamic language which takes hundreds of cycles to schedule a new VM instruction, then it makes no sense to optimize for 2-3 cycles". Dynamic language gets JIT as a next step, and 2-3 cycles count for 50% slowdown.

For the pin read/write, yes, it'll be implemented efficiently like your example (but in C).

I hope you meant "it can be implemented ...", because example I gave is not some theoretical example, but pretty legitimate and practical case how it can be implemented either ;-).

Optimising the initialisation of a port is much less important

That's why I mentioned that perfectionist optimization is not the only usecase for using explicit ports ;-).

In these 2 cases, the ports are all logically equivalent, but still require a mapping

Again, the keyword here is "these 2 cases". There are cases when no mapping required.

In practice it's not common to be able to simply change the port for a peripheral from A to B.

I object. USB, PCI, PCI-X - it's very common (billion cases) for being able to swap ports. Cannot argue for devices like JNode and Freesoc, which probably have just few hundred cases ;-). Arduino? Well, even there exist SoftwareSerial and SoftwareServo libs to put related peripherals on any pins.

Eg, the CC3000 driver uses PA14, PC6, PC7, PB13, PB14 and PB15. It's generally like this, a mish-mash of pins from different ports.

It's just a specific driver which uses these pins. CC3000 itself requires just just an SPI interface (which can be easily bitbanged) and couple of GPIOs (one optionally with irq capability). It's all flexible (well, in my list ;-) ).

Anyway, we digressed again. For me, following differences are clear:

GPIO(scalar_pin) - forces to do pin/port mapping
GPIO(port, pin) - allows to do pin/port mapping, but does not require it

Let's better discuss why GPIO(port, pin) could be worse. One thing is obvious - it's of course easier (especially for beginners) to work with scalar numbers. You could just define board pin to a symbolic name for your current project, voila. Let's take OneWire implementation as an example:

from board import *

MY_DIO = X4

class OneWire:
    def __init__(self, pin_no):
...

With GPIO(port, pin), it seemed that one would need to define both in-port pin and port:

MY_DIO_PORT = PortX
MY_DIO_PIN = Bit4

class OneWire:
    def __init__(self, port, pin):
...

But that's actually not the best way to do it. Instead of passing pin "parameters", one should use dependency injection pattern and pass pin itself (i.e. object):

MY_DIO = some_module.GPIO(PortX, Bit4)

class OneWire:
    def __init__(self, pin):
...

That will allow to configure hardware in one place (and app "config" or initialization routine), and to freely use and mix different GPIO class implementations (e.g. MCU pins and I2C extender).

End result? Beginners learnt something new and useful (you don't have to tell them it's dependency injection, but when they learn they did it all the time thanks to uPy, they will be grateful ;-) ).

Still too complex for beginners? Well:

=== easygpio.py ===
X1 = 1
X2 = 2

class GPIO:
    MAP = {X1:(PortX, Bit0), ...}
    def __init__(self, easypin):
...
=======

I hope you meant "it can be implemented ...", because example I gave is not some theoretical example, but pretty legitimate and practical case how it can be implemented either ;-).

Well, your example still needs proper pin initialisation (direction, mode, speed). And that requires mapping the pin number to bits in the mode selection register.

I meant that pin read/write will be a simple bit set/read in from an appropriate memory address. And it'll be done in C.

I object. USB, PCI, PCI-X - it's very common (billion cases) for being able to swap ports.

What I meant is that it's not common to change PortA to PortB without then also having to also change some pins on those ports. See the CC3000 example. You wouldn't change PA14 to PD14 (say). You'd most likely change also the pin number, eg PD9. With the CC3000, you are spread over 3 ports, so you don't just change 1 definition to change where you plug the CC3000 in to.

My point here is that it's easier to specify the port together with the pin, and consider that as 1 logical entity. The CC3000 is on (PA14, PC6, PC7, PB13, PB14, PB15) and not PORTA(14), PORTC(6, 7), PORTB(13, 14, 14).

GPIO(scalar_pin) - forces to do pin/port mapping

Not true. Depending on the MCU, you could used the high bits to represent the port, and low bits to represent the pin. Eg on the STM, the port address is the base address 0x40020000 plus the high 16 bits of the scalar pin value, and the pin is the low 16 bits. Eg, PA4 = 0x00000004, PB9 = 0x04000009.

GPIO(port, pin) - allows to do pin/port mapping, but does not require it

Not true. Eg on 8-bit AVR you need to know the PORT, DDR and PIN registers for a given port (yes they are sequential). And it depends on whether you use memory access instructions or IO instruction as to their address. On the STM you also need to enable the clock for the GPIO port, which requires a mapping from the GPIO port number to the appropriate RCC register, as well as a mapping to the appropriate bit in that RCC register.

My point is that for a given piece of hardware there is in general going to be a lot more low-level information needed than just (port, pin), and so some mapping/lookup/translation is needed.

Hello,

On Sun, 09 Feb 2014 08:48:23 -0800
Damien George notifications@github.com wrote:

I hope you meant "it can be implemented ...", because example I
gave is not some theoretical example, but pretty legitimate and
practical case how it can be implemented either ;-).

Well, your example still needs proper pin initialisation (direction,
mode, speed). And that requires mapping the pin number to bits in
the mode selection register.

Yes, you're right with this.

My point here is that it's easier to specify the port together with
the pin, and consider that as 1 logical entity. The CC3000 is on
(PA14, PC6, PC7, PB13, PB14, PB15) and not PORTA(14), PORTC(6, 7),
PORTB(13, 14, 14).

Well, per my dependency injection example, that would rather be
Pin(PortA, Bit14), Pin(PortB, Bit13), Pin(PortB, Bit14), etc. -
sometimes the world is just not flat (representable by a single number),
but is made of objects with internal hierarchy and semantics. But of
course, there's a drive to use the simplest representation if it's more
or less possible, that's understandable too.

GPIO(scalar_pin) - forces to do pin/port mapping

Not true. Depending on the MCU, you could used the high bits to
represent the port, and low bits to represent the pin. Eg on the
STM, the port address is the base address 0x40020000 plus the high 16
bits of the scalar pin value, and the pin is the low 16 bits. Eg,
PA4 = 0x00000004, PB9 = 0x04000009.

Well, that doesn't work in general case ;-).

GPIO(port, pin) - allows to do pin/port mapping, but does not
require it

Not true. Eg on 8-bit AVR you need to know the PORT, DDR and PIN
registers

That's true, as I agreed.

My point is that for a given piece of hardware there is in general
going to be a lot more low-level information needed than just (port,
pin), and so some mapping/lookup/translation is needed.

And as a software guy, I can't give up an idea that there's a
possibility for an interface which is at the same time efficient,
flexible, extensible, and general.

So, dropping "no mapping" argument, there're still few left:

1. It's more efficient to enumerate ports and pins (N+M values) than all
   possible combinations (N*M values).
2. Some usages are more naturally fit with ports, and that's consistent
   with how more structured ports (SPI, I2C) would be used (regarding your
   USART1, etc. examples, you'd highly unlikely have class USART1 &
   class USART2 fully not sharing a code, it likely would be USART1 =
   USART(0), or subclassing for the same effect).
3. There can be cases when ports/pins are not (easily) bounded. For
   example, for GPIO emul for linux, I consider having port = 0
   for /dev/console, then other designator, up to "/dev/input/*" string
   for potentially unlimited number of attached USB keyboards. Of course,
   you can say that number of attached USB devices is limited to 127, and
   even if not, a tuple can be used as "pin number" for such extreme cases.

All in all, I even wanted to give up on this discussion already - I
tried to share my ideas yet in @dhylands pull requests to see if other
folks share them, but I obviously can't pledge it's the only right way.

But as you opened a dedicated ticket for that, I decided to sum up
arguments given there, and as a final touch, I'd like to share why I
went to the length of arguing my vision.

So, circa 2005, when I contributed to handhelds.org (Linux
hacking for PocketPCs), there was similar discussion on how to do GPIO
in Linux kernel. The patchset I submitted followed hierarchical approach
of <device, device-pin>, but I was cut by more experienced devel who
said he has some code to do pins as ints. My arguments that using
<device, device-pin> will allow to sqeeze few more percents from
bitbanging, etc. was lost into "ints are KISS" stuff. Ok, what we
needed is a way to have standard GPIO API, so whatever. Now fast
forward to 2011 year, and "pinctrl" subsys was added to kernel, and
here's how it works: "This numberspace is local to each PIN CONTROLLER,
so there may be several such number spaces in a system."

For me, it's a failure to design good GPIO API, and someone should
learn from it, and I'm trying. Then when designing
https://github.com/pfalcon/PeripheralTemplateLibrary I revisited this
question. There, I have freedom to use scalar numbers because I can
split it to port/pin at compile time, but mere look at any datasheet
makes it clear that there're ports and pins, so right representation
for that lib is not try to hide it.

Best regards,
Paul mailto:pmiscml@gmail.com

I've been thinking about this a bit more, and would like to throw out the
following.

What if we were to create Pin objects. These pin objects could have
attributes which describe the functionality that the pin supports.

So a pin might include a pointer to a GPIO port, and a bit-mask. It might
also include an ADC #, and adc channel, and a UART number and function (Rx,
RTS, etc).

You could also have names for the pins. Obvious choices are CPU names (i.e.
C13), board names (i.e. X3) and user names. Since there can be overlap
between these, the names should probably go into namespaces.

You'd probably want to describe these pins using some type of python script
which would then generate the pin definitions, the namespaces, and the pin
definitions would all be stored in flash.

The APIs, like GPIO, ADC, and UART would get passed pointers to pin
objects, and there could be various ways of identifying pins.

To support the namespaces, you could have: pyb.Pin.board.X2 and
pyb.Pin.cpu.C13. Users could add their own definitions by doing something
like: pyb.Pin("user-name", pyb.board.X3)

You could have peripheral specific namespaces as well, which makes sense
for things which have their own notions of "channles". What's important is
that the all wind up mapping to a pin object.

Basically, the namespaces are just convenient ways of identifying a pin.
The underlying code ultimately just wants a pin object. The pin objects
would all be stored in flash, and all of the namespaces should be routines
(ala load_attr), and only the user-defined pins would take up RAM.
http://www.davehylands.com

It sure is an interesting topic isn't it. I think the discussion to date is excellent BTW.
I'd appreciate it if we all looked at it as an opportunity to hash out a really good solution. We all have key aspects we care about and we all ultimately want the same thing.

A simple, clear, memory-efficient, extendable interface.

I would like to restate the problem so as to avoid rabbit-holing on solutions. I hope its OK if we step back for a moment.

As usual I'd like to suggest we approach it from several directions with a view to determining the best approach for each type of thing. By this I mean GPIOs are conceptually different to (say) I2C ports. An I2C port has two useful signals and the chip will not allow me to arbitrarily swap which pin is SDA. But for a GPIO - even though they are sharing a port - each pin can have its own dir, PU setting, mode etc. So conceptually they are different even though they are all implemented as port/pin/settings (which may have side effects that affect other pin capabilities). I include ADC, DAC in GPIO basket even though dir is determined - they are pin based.

Critical elements common to stating the problem are (did I miss one, do you agree):

1. every chipset is using port/pin mapping for this.
   • indicates the mappings per chipset belong in C #defined by chipset.
2. every chipset (and sometimes within its variations) does not use the same port/pin for doing the same thing (kind of obvious but it has implications)
   • suggests we map in C the physical to a virtual port for everything.
     • advantage: ports seldom use adjacently incrementing physical pins.
     • advantage: this level of abstraction means code written for one might be ported more easily to another chipset by (say) reassigning how a USB HS (to be topical) port is defined for a chipset.
     • advantage: say we have WXYZ ports for a 144pin device and program uses some pins from all 4 ports - then changing to a diff chipset where two ports can be used, is a number of oneline pin remapping changes to the python code.
     • disadvantage: we have added an abstraction level away from hardware for 'my favourite' chip. If user only uses one chipset - they wonder why we bothered.
3. We want to be sure to expose the capabilities of the chipset. But we don't want to go to an enormous amount of work for each and every chipset before it can be used. We also want to make it easy for users to access basic functionality.
   • expose the registers for every chipset - labelled for each chipset (users will need to refer to chipset docs so naming must reflect that)
   • also expose basic functionality - define what that is for each:
     • i2c, SPI, CAN, UART - just open, read, and write with timeouts.
     • GPIO - pin, dir, active hi/lo.
     • Timer - Basic timebase calc, trigger int on exit.
     • RTC (if available) - set time, read time.
     • Watchdog - set timeout, start, stop, routine to run
     • PCI-X (not on this chipset) - but I include to show falls into same category
   • Yay python - write a python module to get access to weird Timer hierarchy mapping wotsit of chipset A which is not on chipset B. Users who need it - will spend RAM on it. If they don't need it - we want minimal impact.

So how would it be most useful for a user to define a comm port like I2C.
i2c = I2C(1)
That's basically it wrong - see below. Somewhere in the C we define for the chipset what the mapping is for port 1 and what the initialisation routine is - exposed to python - so it can be opened,closed,restarted, readbyte, writebyte, etc.
More useful aspects of i2c such as register setting, 8&16 bit registers, can be handled by a python module. Only loaded if you need to do that, or for convenience when initially developing.

Its similar for many other kinds of ports but GPIOs are a little different:
If we did the above (virtual ports) then we would (in C) have defined Port X as being physical Port A. We would also have defined that PU (say) did nothing (not avial in that chipset) and that to make it an input, high speed, ... was a bunch of C code.
For a different chipset - it'd be diff code and maybe PU would be available and only low speed. So we need to define what is the minimum subset for common use and set those defaulst in C for that chipset. Then allow a platform to expose more, or for a python module to expose even more.
I don't want to add more on GPIOs in this monolog - its gone on too long as it is.

So I restate my initial point. Lets consider:

• Ports and GPIO pins are conceptually different.
• We want the design to work on many chipsets and for simple code to be ported easily to a different chipset (clearly weird Timer(say) chipset capabilities cannot - but otherwise...)
• Common functionality (which is?) defined memory efficiently.
• Python module to expose more functionality than the basic core (which exists to make starting simple projects easy).

If we write what we want the user to see first. Then work out 'best' way to define it.

Uh-oh - couldn't stop myself -
E.g. GPIO StrawMan - Should we have:

temp_pin = some_module.GPIO(X, 4, NO_PU)   # Virtual port X, pin 4. Direction undefined.
# or as we've seen there is no impact to memory:
temp_pin = some_module.GPIO('X', 4, NO_PU)   # string port accessor
pressure_pin = some_module.ADC('Y', 1)  # new class or
pressure_pin = some_module.GPIO('Y', 1, INPUT, 12bit)   # effectively defining it as an ADC
# ADC is like a GPIO but dir is predefined and maybe there is a common option for bitdepth
#  - but maybe not, ..
#  - maybe Virtual ADC1 is 12bit and virtual ADC3 is 8bit, or SAR vs sigmadelta.
#     we need to work this out.
#  - what's the minimal subset, how do we define extensions for diff chipsets
# Define what it looks like - then define how to implement it.

class OneWire:
    def __init__(self, pin):
    ...
    def read(self):
    ...

temp_IO = OneWire(temp_pin)
temp = temp_IO.read()
pressure = pressure_pin.read() # GPIOs would have a read/write defined for input,output
# or have I mixed my metaphors too much and pin should be decoupled from GPIO and ADC
#   even further...? I.e. Have an ADC device which takes a vpin.

# where:
#  - some_module defines a GPIO as a virtual port, pin and direction (as the minimal subset)
#  - an inline mapping is done from virtual port to physical in C HAL file.
# but what about the ADC ?
#
# To convert to new chipset involves changing firstline to correct virtual port/pin for that
#   board or chipset.

So how would it be most useful for a user to define a comm port like I2C.
i2c = I2C(1)
That's basically it. Somewhere in the C we define for the chipset what the mapping is for port 1 and what the initialisation routine is - exposed to python - so it can be opened,closed,restarted, readbyte, writebyte, etc.

Unfortunately, the i2c pins are available on multiple pins, so you still need to configure which pins the signals should show up on.

I2C1_SDA - B7, B9
I2C1_SCL - B6, B8
I2C2_SDA - B11, F0, H5
I2C2_SCL - B10, F1, H4
I2C3_SDA - C9, H8
I2C3_SCL - A8, H7

So merely selecting that you want I2C1 isn't enough. It's similar with many of the other peripherals.

I took a look at what Espruino does and they have python scripts which parse a .csv file and generate something like this (for the STMF3DISCOVERY):

/* PA0  */ { JSH_PORTA, JSH_PIN0, JSH_ANALOG1|JSH_ANALOG_CH1, { 0, 0, 0, 0, 0, 0 } },
/* PA1  */ { JSH_PORTA, JSH_PIN1, JSH_ANALOG1|JSH_ANALOG_CH2, { JSH_AF1|JSH_TIMER2|JSH_TIMER_CH2, JSH_AF9|JSH_TIMER15|JSH_TIMER_CH1|JSH_TIMER_NEGATED, 0, 0, 0, 0 } },
/* PA2  */ { JSH_PORTA, JSH_PIN2, JSH_ANALOG1|JSH_ANALOG_CH3, { JSH_AF1|JSH_TIMER2|JSH_TIMER_CH3, JSH_AF7|JSH_USART2|JSH_USART_TX, JSH_AF9|JSH_TIMER15|JSH_TIMER_CH1, 0, 0, 0 } },
/* PA3  */ { JSH_PORTA, JSH_PIN3, JSH_ANALOG1|JSH_ANALOG_CH4, { JSH_AF9|JSH_TIMER15|JSH_TIMER_CH2, JSH_AF1|JSH_TIMER2|JSH_TIMER_CH4, JSH_AF7|JSH_USART2|JSH_USART_RX, 0, 0, 0 } },
/* PA4  */ { JSH_PORTA, JSH_PIN4, JSH_ANALOG2|JSH_ANALOG_CH1, { JSH_AF2|JSH_TIMER3|JSH_TIMER_CH2, JSH_AF1|JSH_DAC|JSH_DAC_CH1, 0, 0, 0, 0 } },

Yes. I'd forgotten about that. (should have remembered - recently been doing the Timers which are similar.)

We can see that for i2c=I2c(1) there are just 4 possible configs of SDA, SCL.
You could enumerate them like:

• i2c=I2c(1, B7B6) # I.e. one of a choice of 4 possible states [B7B6, B7B8, B9B6, B9B8]
• The problem being that the labels would need to be to the physical ports (so they could make sense of the datasheet) and we just spent some time convincing user to use virtual labels.

Or could add that to the config as required args (which would have to be validated in the constructor by referring to the kind of map table you referred to earlier - but in C):

• i2c=I2c(1, B7, B6) # or I2c(1, 'B7', 'B6') as it takes no addtl room.
• The problem also being that we have addtl fields but its not bad. At least users know that i2c requires two pins.

or even as virtual I2c ports. Only one of which can be active at a time:

• i2c=I2c(1a) # in this case 1a,1b,1c,1d are the mapped as above
• has advantage of being unambiguous.
• The problem being that for large matrices of choices this could get unwieldy.

In all cases you need access to documentation to find out how the ports are mapped on your board. The advantage of the first two being the docs are supplied by the chipset vendor or board supplier and not the micropython port (which the last one requires).

Or could add that to the config as required args (which would have to be validated in the constructor by referring to the kind of map table you referred to earlier - but in C):

i2c=I2c(1, B7, B6) # or I2c(1, 'B7', 'B6') as it takes no addtl room.
The problem also being that we have addtl fields but its not bad. At least users know that i2c requires two pins.

I think that this is the right way to do it. If you look at I2C2, there are 3 pairs which leads to 9 valid combinations.

When you start looking at bitbanged peripherals (like say the LCD) then the number of unique possibilities is astronomic. Some peripherals, like i2c and spi can also easily be emulated in software, and the driver might choose to do software emulation, if the selected pins don't support the hardware functionality.

Yeah - I wasn't including adhoc devices like LCDs which can be connected in so many ways. I was meaning devices the chipset supports directly.
FWIW - I agree case 2 looks the best from what I presented. Hoping others present back alternatives so we can discuss and reach consensus... I think @pfalcon is right about ports/pins. I haven't worked out best way to have virtual ports...

Here's A solution that satisfies the criteria:

1. for each target chipset/board in mpconfigport.h etc the #defines map useful features.
   • as currently.
2. a new module called HAL exposes:
   • the ports as port,pin,args for the useful set of minimal functionality for each device (TBD)
   • all registers to allow maximal control
3. the pyb (or other specific board) module exposes the ports in an efficient way for that board.
   • redundantly to the HAL module,
   • but makes using the board easy for new users.

Migrating code form one board to another then becomes a process of abstracting the lines of code using the (say) pyb module to HAL module(1 line per port/pin). Then changing pins. The creation of a pyb module for a specific board is optional.

Remaining questions:

• how do we make construction of pyb module reuse the HAL module so as to avoid code duplication ?
• whats the minimal set of functionality for the HAL devices ?

Addtl - for @dpgeorge's board where I2c2 is on Y1,Y2 (PB10,11) he could label it pyb.I2c(Y1Y2) instead of pyb.I2c(2). Advantage is the label shows which vpins he is using and so a user is less likely to accidentally(in code) reuse a pin allocated to a port and a GPIO. E.g. in this case Y1,Y2 are both PWM capable and used in USART3. pyb.Gpio(Y1) pyb.Usart(Y1Y2)

E.g.

i2c = HAL.I2C(B, 10, 11)  # (Port, SCL, SDA)
i2c = pyb.I2c(2)   # which is on pins Y1,Y2 (PB10,11)
#or
i2c = pyb.I2c(Y1Y2)

@dhylands I like your idea of having all pins pre-defined and in flash. That way it takes no RAM to make a GPIO object. You just get a reference to the correct one.

As per latest Pin docs, when pin objects are constructed the physical pin is specified via a single identifier, which can be an integer, string or tuple (port, pin), or anything a port wants to implement.