Skip to content

ENH: Added a new Formatter to mpl.ticker and some example recipes #7482

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

madphysicist
Copy link
Contributor

@madphysicist madphysicist commented Nov 17, 2016

  • BinaryIntFormatter: Formats ticks as binary integers. Intended for use with MaxNLocator(integer=True).
  • HexIntFormatter: Formats ticks as hexadecimal integers. Intended for use with MaxNLocator(integer=True).
  • LinearScaleFormatter: Wraps any other formatter and transforms the input to it on a linear scale. The wrapped formatter can be any callable that accepts x and pos as arguments, but actual Formatter instances will get special treatment.

Also, changed the annotation in PercentFormatter tests slightly (non-functional change).

@madphysicist madphysicist changed the title DEV: Added three new Formatters to mpl.ticker ENH: Added three new Formatters to mpl.ticker Nov 17, 2016
@NelleV
Copy link
Member

NelleV commented Nov 17, 2016

What would be the use case for the binary formatter and the hex formatter?

@NelleV
Copy link
Member

NelleV commented Nov 17, 2016

I also don't understand what the linearscale formatter does. Can you add an example?

@madphysicist
Copy link
Contributor Author

I developed the Binary and Hex formatters specifically for plotting the output of a Digital-to-Analog Converters versus the input. The x-axis needed to indicate the input bits in a way that was easier to understand than decimal numbers.

LinearScaleFormatter is a way of transforming the values shown on the axis without modifying the underlying data. It can probably be generalized to an arbitrary tick value-to-display value mapping function, but I found it useful to just specify a linear transformation with two points on the input range and two points on the output range.

A sample use-case would be if you had data acquired with say imperial units like quarts, but wanted to display the axis in liters. If you did not want to provide a transformed dataset, you could wrap your existing formatter with something like LinearScaleFormatter(1.0, 0.946353, ax.xaxis.get_major_formatter()) (1qt = 0.946353L). I expect this is much more useful than the other two formatters. I will work on a piece of example code to add to the gallery.

@QuLogic
Copy link
Member

QuLogic commented Nov 18, 2016

Both the binary and hex formatters could be implemented with (or replaced by?) the StrMethodFormatter.

The example of changing units is something we should get more unit support for, though I feel like much of the underlying infrastructure is there already, but I'm not sure how to use it. Maybe one of the unit examples would be useful.

@madphysicist
Copy link
Contributor Author

Both the binary and hex formatters could be implemented with (or replaced by?) the StrMethodFormatter.

I thought so too at first, but it turns out you have to explicitly cast to an int to use b and x formats. The input to Formatter.__call__ is pretty much always a float, even with MaxNLocator(integer=True).

@QuLogic
Copy link
Member

QuLogic commented Nov 18, 2016

Really strange that it's not defined because you can do 123.0.hex() and get '0x1.ec00000000000p+6' back.

@madphysicist
Copy link
Contributor Author

Yeah. Python 3.5.2:

>>> '{:b}'.format(123.0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'b' for object of type 'float'
>>> '{:x}'.format(123.0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'x' for object of type 'float'

@madphysicist
Copy link
Contributor Author

What I can do is redefine BinaryIntFormatter and HexIntFormatter as a single class and add a paramter type, which would be 'hex', 'bin' or 'oct'.

@madphysicist
Copy link
Contributor Author

@QuLogic I have converted BinaryIntFormatter and HexIntFormatter into a single extension of StrMethodFormatter that mainly just converts the input to an integer before calling the super implementation. Thanks for inspiring that idea.

@NelleV Hopefully the new form is more acceptable. While I agree that the original implementation was a bit domain specific, a formatter that handles the conversion to integer properly is much more generally useful.

I will get the bugs out shortly.

@anntzer
Copy link
Contributor

anntzer commented Nov 19, 2016

For the sake of avoiding adding more very thin classes, I'd suggest simply adding a doc entry suggesting the use of FuncFormatter(lambda x, i: format(int(x), fmt)). This is honestly not that much longer to write than IntegerFormatter(fmt), and much clearer for the reader. Likewise the proposed unsigned kwarg can trivially be implemented by using the - flag in the format string. So I'm fairly strongly -1 on adding such a class.

LinearScaleFormatter is more promising, but should probably be implemented as a general Formatter transformer that simply takes an arbitrary input->output transform callable. Also, following @QuLogic's suggestion I'd like first to know whether the unit machinery is already enough (or close to enough, in which case it should be improved accordingly) to handle this case.

In any case, please make all attributes of these new classes private, unless you have a strong reason to keep them public. Also, please check how LinearScaleFormatter interacts with format_cursor_data.

@codecov-io
Copy link

codecov-io commented Nov 19, 2016

Current coverage is 61.97% (diff: 87.09%)

Merging #7482 into master will increase coverage by 0.08%

@@             master      #7482   diff @@
==========================================
  Files           173        174     +1   
  Lines         56140      56059    -81   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          34743      34743          
+ Misses        21397      21316    -81   
  Partials          0          0          

Powered by Codecov. Last update 45e4a46...c6741ed

@madphysicist
Copy link
Contributor Author

@anntzer. While I agree that IntegerFormatter is a pretty thin wrapper and can generally be omitted, I am not sure that the unsigned keyword can be replaced with a - format (which is the default anyway). In fact, it is part of the reason that I created the class in the first place. unsigned = True does not just remove the minus sign from the number: it causes negative numbers to be treated as n-bit unsigned numbers. Either way, I am fine with making this a recipe in the examples if that is the general concensus.

I will rewrite LinearFormatter as MappingFormatter. The linear transformation can then go into a recipe in the examples section.

Finally, when you say "making the attributes private", do you mean prepending an underscore or something else?

@story645
Copy link
Member

Not sure how the unit infrastructure would help since unit is a bad name for it. The infrastructure is really about registering and working with datatypes and it sounds like this data is in floats/ints.

@anntzer
Copy link
Contributor

anntzer commented Nov 21, 2016

Oh, I completely misunderstood what "unsigned" does, sorry about that. It's arguably a bit more complicated but this can still be implemented as

FuncFormatter(lambda x, i:
    format(int.from_bytes(int(x).to_bytes(int(x).bit_length() // 8 + 1, "big", signed=True),
                          "big"),
           fmt)

which is not necessarily more legible, but now the argument is instead that this feature may be a bit too specialized? (Again I'm all for adding it in the docs.)

And yes I meant that attributes must be prepended with an underscore. Otherwise it basically becomes impossible later to modify the class while maintaining backcompatibility, as users may be relying on whatever public attributes are present...

@madphysicist
Copy link
Contributor Author

OK. I will make all the object attributes private. I would like to keep the transform method public in all the cases though.

'Locator', 'IndexLocator', 'FixedLocator', 'NullLocator',
'LinearLocator', 'LogLocator', 'AutoLocator',
'LinearScaleFormatter', 'BinaryIntFormatter',
'HexIntFormatter', 'Locator', 'IndexLocator', 'FixedLocator',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are non-existent classes in here.

@@ -154,6 +154,15 @@
:class:`PercentFormatter`
Format labels as a percentage

:class:`BinaryIntFormatter`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These docs have gotten out of sync with the PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That last push was mostly an accident. I am overhauling the code based on @anntzer's advice. Thinking about turning some of it into recipes in the docs.

This formatter can use any other formatter to actually render the
ticks.

inRef: number or 2-element iterable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick, there should be a space before the : to format correctly with numpydoc

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last input to this is already a formatter instance, maybe take in a Norm instance as the input here?

If not, I have a slight preference to 4 inputs instead of 2 of mixed type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Norm instance?


# All these values are retained for debugging/extension.
# Only the minima are used explicitly.
self.iMin, self.iMax = unpack(inRef, 'in')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The style tends to be camel case for classes, _ for everything else.

@tacaswell tacaswell added this to the 2.1 (next point release) milestone Nov 29, 2016
@madphysicist
Copy link
Contributor Author

@tacaswell I believe that my latest push addresses your immediate concerns.

@madphysicist
Copy link
Contributor Author

@anntzer , in reference to your request

Also, please check how LinearScaleFormatter interacts with format_cursor_data.

format_cursor_data is defined in Artist, and does not get overriden by Axes or Axis as far as I can tell. There is no direct interaction with a formatter at all.

If you meant the default implementation of _AxesBase.format_coord, it does in fact use the major formatter's format_data_short method. I will therefore go ahead and made proper references to the remaining methods of the underlying Formatter.

@madphysicist
Copy link
Contributor Author

I took all of the advice I got on this thread and made a completely new class called TransformFormatter which allows generic mapping of tick values before plugging them into a different Formatter or formatting function. I then added the two Formatters I had before to an example in the documentation since they are now trivial to implement. The new tests verify, among other things, that format_data_short works correctly.

The only major comment that I am no longer following here is that the two attributes of this new class are both public. I think that in this context it makes sense to do it this way, in the same way that FormatStrFormatter exposes the fmt attribute.

Another possible problem is that this class is sensitive to changes in the underlying API. This is probably fine, but I want to add a note to add the appropriate redirects in TransformFormatter if the Formatter or TickHelper methods ever change. Where would I add such a note to future developers?

Finally, I have a purely technical question. At the moment, the axis attribute is a reference shared by the base instance and the underlying formatter. However, this means that the view and data bounds are not transformed for the underlying formatter. Should I make the formatter attribute always have a _DummyAxis assigned to it with the transformed bounds? Or is the way I currently did it going to work OK?

@madphysicist madphysicist changed the title ENH: Added three new Formatters to mpl.ticker ENH: Added a new Formatter to mpl.ticker and some example recipes Dec 2, 2016
@madphysicist
Copy link
Contributor Author

Could someone please explain why the appveyor test is failing while the same configuration in Travis is passing? Is it something I am missing that can be easily fixed or should I change the test to avoid using unicode strings?

@tacaswell
Copy link
Member

@madphysicist Because windows ;)

@madphysicist
Copy link
Contributor Author

Rebased temporarily onto #7965. This is just to make my life easier when #7965 gets merged. Please ignore until then.

@madphysicist madphysicist force-pushed the new-formatters branch 4 times, most recently from 04469ce to 904bd51 Compare February 7, 2017 03:59
@madphysicist
Copy link
Contributor Author

As far as I can tell, I have addressed all the issues with this PR. Still waiting on approval from @tacaswell and at least one other reviewer though.

@madphysicist
Copy link
Contributor Author

Bump.

- both ends are missing: set ranges to 1.0
- one end is missing: set it to the other end
"""
self._in_offset = 0.0 if in_start is None else in_start
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why aren't you setting in_start=0 and out_start=0 in the init(...)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to be able to test if the parameter was set or not. None allows me to implement all the combinations listed in the docstring. Setting to a number would not allow that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nevermind. I do see your point. Will fix.

self._in_scale = out_end - self._in_offset
else:
self._in_scale = in_end - self._in_offset

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is this

if in_end is not None:
    self._in_scale = in_end - self._in_offset
else if out_end is not None:
    self._in_scale = out_end - self._in_offset
else:
    self._in_scale = 1.0

else:
self._in_scale = in_end - self._in_offset

if out_end is None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this also be rewritten to remove one layer of nesting?

if out_end is not None:
    self._out_scale = out_end - self._out_offset
else if in_end is not None:
    self._out_scale = in_end - self._out_offset
else:
    self._out_scale = 1.0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took me a minute to realize how you just flipped the if so that the else comes first. Slow day. I did what you propose except that I kept the elif checking for in_end is None.

I was thinking that I can actually compute a single scale and offset parameter to minimize the math done in __call__. What it your thought on that?

Copy link
Member

@story645 story645 Feb 16, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ❤️ contrapositives
I like elif in_end is not 'cause it keeps the formulas together and so parallel structure, but :shrugs: not gonna hold up a review over it.

not sure what you mean about single scale and offset...can you clarify?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not really hung up on positives. Now that I've made the change, it does look better that way. Honestly, I am pretty sure I subconsciously avoid contrapositives after having worked in a place where they were discouraged by the coding standard for no really good reason.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can rewrite __call__ as return x * self._scale + self._offset by adding probably about two extra lines to __init__. Personally I think it's worth precomputing everything in __init__, but I am OK with keeping the structure as-is if that is what you prefer.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no preference here, so go with whatever you think is more sensible/maintainable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will go ahead and make the change. I also just realized that when I removed the lines like self._in_offset = 0.0 if in_start is None else in_start, I forgot to replace them with self._in_offset = in_start and similar :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there some sort of test that I can run on my example, like an image test?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made the additional change and ran the code this time to make sure it actually works as intended :)


fig = plt.figure()
ax1 = fig.add_subplot(111)
ax2 = fig.add_subplot(111, sharex=ax1, sharey=ax1, frameon=False)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it necessary to have a shared axis to display this new ticker? That seems to be conflating two separate tasks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not strictly necessary, but the conflation was deliberate. I wanted to be able to display both the original and the transformed values as well as to use this as a demo for #7528. There will be some modifications once that PR gets accepted (which I think is very likely once I find the time to write some additional tests for it).

assert fmt.locs == locs
assert fmt.formatter.locs == xlocs

fmt.set_locs(None)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a teeny test, but it's a different one def test_clear_locs or something

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Tests included.
Example code provided with some recipes from previous attempt.
@madphysicist
Copy link
Contributor Author

I don't think the travis error has anything to do with me...

@madphysicist
Copy link
Contributor Author

'nother bump.

Copy link
Member

@QuLogic QuLogic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, had a chance to look at this. It's mostly good except for the one example that I find a bit confusing.

"""
Sets up the transformation such that `in_start` gets mapped to
`out_start` and `in_end` gets mapped to `out_end`. The following
shortcuts apply when only some of the inputs are specified:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry to say I was very confused by this description; it took me more than one reading to understand what it meant.

First, they're not really shortcuts, they're just rules. Second, the word 'input' is used for both arguments to the class __init__ in this sentence and for arguments to the instance's __call__ in each of the points below.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After commenting on each of these elements, I think they may just be more confusing than it's worth. Is it really necessary to describe all the "shortcuts"? Would a mathematical equation and a set of defaults be more straightforward?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are right. I wrote this when I was trying to figure out how I would want the different cases to work out mostly as notes for myself. Will fix.

`out_start` and `in_end` gets mapped to `out_end`. The following
shortcuts apply when only some of the inputs are specified:

- none: no-op
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'none' should probably be 'no inputs specified'; I thought this was a typo for None at first.


- none: no-op
- in_start: translation to zero
- out_start: translation from zero
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For both of these, translation of what?

- in_start: translation to zero
- out_start: translation from zero
- in_end: scaling to one (divide input by in_end)
- out_end: scaling from one (multiply input by in_end)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be out_end?

- out_start: translation from zero
- in_end: scaling to one (divide input by in_end)
- out_end: scaling from one (multiply input by in_end)
- in_start, out_start: translation
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Translation of what?

def transform2(self, x):
return 2 * x

@pytest.fixture()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can drop the parentheses.


def test_attributes(self, empty_fmt):
# Using == is the right way to compare bound methods:
# http://stackoverflow.com/q/41900639/2988730
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? Both ways work fine. Is there some reason https is preferred?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just so we're clear, I'm going to make the change. I was just curious.


@pytest.fixture()
def fmt(self):
fmt = self.empty_fmt()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could take empty_fmt as an argument to make it clear it uses the fixture as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not know you could do that to non-test methods. Will definitely chain it that way.


@pytest.fixture()
def loc_fmt(self):
fmt = self.fmt()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same with fmt here.

transformed values.

The input may be an instance of
:class:`matplotlib.ticker.Formatter` or an other callable with
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

an other -> another

Some small changes and rework of example transform code.
@madphysicist
Copy link
Contributor Author

@QuLogic Based on your comments, I have greatly simplified the configuration of the linear transform in the example. I hope you like it. I found myself trying to figure out where I was going with the original version and just gave up :)

I made all the other changes you recommended as well.

@tacaswell tacaswell modified the milestones: 2.1 (next point release), 2.2 (next next feature release) Aug 29, 2017
@tacaswell tacaswell dismissed their stale review May 4, 2020 19:17

outdated

@tacaswell
Copy link
Member

@madphysicist Sorry this has completely fallen through the cracks (for 4 years!). Are you still interested in working on this?

@jklymak
Copy link
Member

jklymak commented Sep 8, 2020

This had a lot of input, but I'll close for now as orphaned...

@jklymak jklymak closed this Sep 8, 2020
@story645 story645 removed this from the future releases milestone Oct 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants