Skip to content

[WIP] fix: dtitk interface specs #2463

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Conversation

satra
Copy link
Member

@satra satra commented Feb 23, 2018

@kesshijordan - this involves a bunch of changes. but i don't have a good understanding of the interfaces. i.e. where the inputs can actually be better specified:

  • replace traits.Str with more specific Int/Float/List/etc
  • there were many places where exists=True and mandatory=False were there. these seemed quite contradictory. in nipype exists only goes with File/Directory traits and mandatory is for required inputs. it would be helpful to comment on each spec or simply send a PR to my PR
  • we will probably not merge this till after the monday release

Fixes #2462

also at this point i haven't regenerated the autospecs to simplify reviewing.

@satra satra changed the title fix: dtitk interface specs [WIP] fix: dtitk interface specs Feb 23, 2018
@satra satra added this to the 1.0.2 milestone Feb 23, 2018
@kesshijordan
Copy link
Contributor

Thanks, @satra . I will work on this.

@kesshijordan
Copy link
Contributor

@satra: I made a new branch that I'm incorporating changes into; I'll make a PR to your PR when it's ready and comment on the specs to get feedback.

https://github.com/kesshijordan/nipype/tree/satra_fix_specs

@satra
Copy link
Member Author

satra commented Feb 27, 2018

thanks @kesshijordan

position=0, argstr="%s")
moving_file = File(desc="diffusion tensor image path", exists=True,
mandatory=True, position=1, argstr="%s", copyfile=False)
similarity_metric = traits.Enum('EDS', 'GDS', 'DDS', 'NMI',
mandatory=True, position=2, argstr="%s",
desc="similarity metric")
samplingX = traits.Float(mandatory=True, position=3, argstr="%s",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@satra: some of these functions require parameters to be set on the command line (no defaults), but we provide default values in the input spec (I matched defaults to the example from the DTITK tutorial), fulfilling this requirement. I am redoing the input specs such that usedefault=True and mandatory=False. In general, does that sound like a good idea, or is it better to make the user specify?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if it's a common default value across use cases then setting a default value is fine. however, if it's a required argument, mandatory should always be set to True even if a default value is used. also mandatory=False should not be used.

@effigies
Copy link
Member

Hi @kesshijordan, is this something you would be able to finish up this week? We're aiming to release 1.0.2 next Monday.

@kesshijordan
Copy link
Contributor

I'll work on it, @effigies .

@@ -24,35 +36,34 @@ class RigidInputSpec(CommandLineInputSpec):
default_value=4)
ftol = traits.Float(mandatory=True, position=6, argstr="%s",
desc="cost function tolerance", default_value=0.01)
useInTrans = traits.Float(mandatory=False, position=7, argstr="%s",
useInTrans = traits.Float(position=7, argstr="%s",
desc="to initialize with existing xfm set as 1",
default_value=1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this flag (useInTrans) determines whether or not a previously calculated affine is used to initialize the registration. If it is set to anything, the affine file in the working directory with the ${prefix}.aff will be used. Otherwise, the center of mass is used to initialize. If it's a traits.Bool, setting it to false is still tripping the flag (executes as true) so I don't want to do that because it would be misleading. According to the tutorial, it should be either set to 1 or not set at all.

My thought is to have the field be traits.Int with default 1, but not specify to usedefault for any case (even the affine) so that the user must specify useInTrans=1 to initialize with an affine. Otherwise, it's undefined and the center of mass is used to initialize. Does that sound good?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're not going to usedefault=True, setting a default value doesn't seem to have much point. And it sounds like perhaps this makes more sense as a traits.Bool with argstr="1". That would place "1" in position 7 if True, and omit if False or unspecified. Does that seem reasonable?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Thank you, @effigies .

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since a rigid should be calculated before an affine, I'm setting the usedefault=True to be the case in the affine interface, but not the rigid. Does that make sense or do you think both should require the user to request that the precalculated xfm be used as the initialization?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the sake of flexibility, I probably would not set usedefault here (it becomes more difficult to un-set, so if there's ever a case you might not want the default, best to avoid), but you could use the docstring to show/describe how you expect people will want to use it.

@@ -9,14 +20,14 @@
class TVAdjustOriginInputSpec(CommandLineInputSpec):
Copy link
Contributor

@kesshijordan kesshijordan Mar 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting rid of TVAdjustOrigin because it is redundant (this calls the same command as TVAdjustVoxelspace)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as everything that TVAdjustOrigin does can be done with TVAdjustVoxelspace, that sounds fine to me.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. It calls the command TVAdjustOrigin with a specific set of flags, so it is redundant as an interface.

out_file = traits.Str(desc='output path', position=1, argstr="-out %s",
name_source="in_file", name_template='%s_reslice',
keep_extension=True)
origin = traits.Str(desc='xyz voxel size', mandatory=True,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to set the default of this to be (0,0,0) and set usedefault=True because that is recommended by the tutorial. If a target image is provided, this will be ignored anyway so I think it's fine to do. Let me know if you disagree.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm personally disinclined to add arguments that will be ignored, since that could be confusing to someone attempting to understand or debug a pipeline. You could set it to xor=['in_target'], to make clear that these are mutually exclusive options.

Also, I think you probably want to use numeric data type here, rather than a string:

origin = traits.Tuple(traits.Float, traits.Float, traits.Float, mandatory=True,
                      xor=['in_target'], position=4, argstr='-origin (%g,%g,%g)')

Similarly, in_target should list origin in its xor parameter. (Mandatory is okay in mutually exclusive cases.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are multiple parameters that will be ignored if 'in_target' is provided. You can either provide a target image to match parameters to (origin & voxel size) or give parameters individually, but you do not need to provide both. E.g. one could just want to change the voxel size but not the origin, so mandatory would be problematic in that case, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this look (link, below)? The way I have it, there are defaults with a note in the description that they would be superseded by in_target.

https://github.com/kesshijordan/nipype/blob/satra_fix_specs/nipype/interfaces/dtitk/utils.py

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yes, mandatory isn't necessary, then. I was going off of how this is currently written. My understanding now is: If target file is specified, array size, voxel size, and origin are defined by the properties of the target file and any explicit specifications are ignored; if it is not specified, then the properties of the input file are preserved, except where specified explicitly. So essentially none of these inputs is mandatory except in_file.

It's also unclear to me whether the positions are relevant. Typically, flagged arguments aren't required to be in a specific order. For the sake of simplicity, I'll assume they can be removed.

So, if that understanding is correct, then I might rewrite the input spec as follows (a couple comments within):

class TVResampleInputSpec(CommandLineInputSpec):
    in_file = File(desc="image to resample", exists=True,
                   mandatory=True, argstr="-in %s")
    out_file = traits.Str(desc='output path',
                          name_source="in_file", name_template="%s_resampled",
                          keep_extension=True, argstr="-out %s")
    target_file = File(desc='specs read from the target volume',
                       argstr="-target %s", xor=['array_size', 'voxel_size', 'origin'])
    align = traits.Str('center', argstr="-align %s")
    # This is missing an argstr. I'm not sure what it's supposed to be
    # Would also add a desc to make the options a little clearer
    interpolation = traits.Enum('LEI', 'EI')
    # Note the switch to ints and %d
    array_size = traits.Tuple((traits.Int(), traits.Int(), traits.Int()),
                           desc='resampled array size', xor=['target_file'],
                           argstr="-size %d %d %d")
    # Using %g will almost always give better command strings than %f
    voxel_size = traits.Tuple((traits.Float(), traits.Float(), traits.Float()),
                         desc='resampled voxel size', xor=['target_file'],
                         argstr="-vsize %g %g %g")
    origin = traits.Tuple((traits.Float(), traits.Float(), traits.Float()),
                          desc='xyz origin', xor=['target_file'],
                          argstr='-origin %g %g %g')

(Since you've done some renaming of these input fields, I've taken the liberty of renaming to names that I feel are a little more descriptive. Feel free to reject these changes if you prefer the ones you had.)

Note the xor arguments for target_file, array_size, voxel_size, and origin. That ensures that users aren't mistakenly passing parameters that will be ignored. (Again, I'm assuming that array_size should be included in this group. You can just remove its xor and remove it from target_file's xor list, if I'm wrong.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @effigies ! This is very helpful. The xor is a good idea to make sure the user is aware their inputs are ignored. I've fixed your other notes, here, and will apply them to other interfaces.

Copy link
Member

@effigies effigies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few quick comments, as I believe @satra is busy this week. I trust he'll jump in if I've made any very bad suggestions.

>>> node.inputs.samplingZ = 4
>>> node.inputs.ftol = 0.01
>>> node.inputs.useInTrans = 1
>>> node.run() # doctest: +SKIP
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a node.cmdline with the expected command line, here?

Similarly, this would be good to add to any of the docstrings that are currently missing it.

@@ -24,35 +36,34 @@ class RigidInputSpec(CommandLineInputSpec):
default_value=4)
ftol = traits.Float(mandatory=True, position=6, argstr="%s",
desc="cost function tolerance", default_value=0.01)
useInTrans = traits.Float(mandatory=False, position=7, argstr="%s",
useInTrans = traits.Float(position=7, argstr="%s",
desc="to initialize with existing xfm set as 1",
default_value=1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're not going to usedefault=True, setting a default value doesn't seem to have much point. And it sounds like perhaps this makes more sense as a traits.Bool with argstr="1". That would place "1" in position 7 if True, and omit if False or unspecified. Does that seem reasonable?

>>> import os
>>> filepath = os.path.dirname( os.path.realpath( __file__ ) )
>>> datadir = os.path.realpath(os.path.join(filepath, '../../testing/data'))
>>> os.chdir(datadir)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can remove these prefatory lines. Tests are now run in that directory through another mechanism. (You may need to merge master to make it work.)

Similarly at the tops of other files.

@@ -9,14 +20,14 @@
class TVAdjustOriginInputSpec(CommandLineInputSpec):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as everything that TVAdjustOrigin does can be done with TVAdjustVoxelspace, that sounds fine to me.

out_file = traits.Str(desc='output path', position=1, argstr="-out %s",
name_source="in_file", name_template='%s_reslice',
keep_extension=True)
origin = traits.Str(desc='xyz voxel size', mandatory=True,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm personally disinclined to add arguments that will be ignored, since that could be confusing to someone attempting to understand or debug a pipeline. You could set it to xor=['in_target'], to make clear that these are mutually exclusive options.

Also, I think you probably want to use numeric data type here, rather than a string:

origin = traits.Tuple(traits.Float, traits.Float, traits.Float, mandatory=True,
                      xor=['in_target'], position=4, argstr='-origin (%g,%g,%g)')

Similarly, in_target should list origin in its xor parameter. (Mandatory is okay in mutually exclusive cases.)

@kesshijordan
Copy link
Contributor

Thanks for the review, @effigies . I'm working on another branch incorporating all of the changes and testing the interfaces (not all of them are finished). My goal is to have a complete, functional pipeline that performs tensor registration rigid through diffeomorphic, including all of the helper functions for the release. I will submit a pull request to this branch when I'm done (trying to get this done before Monday!) but I want to get a little further before I ask someone to review it.

@effigies
Copy link
Member

No worries on the time, and thanks for your effort. Do you think this makes sense to push off for next month's release? (And @satra, are the issues that led to this worth holding up this release?)

Also, it might be easier to make a PR against nipype master, and just supersede this PR, rather than merging into Satra's branch, here. Your call, though.

@kesshijordan
Copy link
Contributor

I think pushing it off to next month's release would be better, @effigies. I would like to have some more time to fix and test the rest of the interfaces.

@effigies
Copy link
Member

Pushing to 1.0.3 sounds fine to me. Thanks for the care you're putting into this one.

@satra if you think this should hold up the current release, let us know.

@effigies effigies modified the milestones: 1.0.2, 1.0.3 Mar 26, 2018
@satra
Copy link
Member Author

satra commented Mar 26, 2018

@effigies and @kesshijordan - 1.0.3 sounds reasonable for this.

@kesshijordan
Copy link
Contributor

Thanks, @effigies . Happy to; I'm learning a ton from the feedback (thank you for giving such detailed suggestions and the reasoning).

@kesshijordan
Copy link
Contributor

@effigies are you familiar with this error produced by make check-before-commit? I merged with the current master prior to running it.

py.test --doctest-module nipype
usage: py.test [options] [file_or_dir] [file_or_dir] [...]
py.test: error: unrecognized arguments: -n nipype
inifile: /Users/kesshijordan/repos/nipype/nipype/pytest.ini
rootdir: /Users/kesshijordan/repos/nipype/nipype
make: *** [test-code] Error 2

@effigies
Copy link
Member

Ah, yes. You need to have pytest-xdist installed to run the tests.

@kesshijordan
Copy link
Contributor

kesshijordan commented Mar 26, 2018

Thanks, @effigies !

Nevermind (I think I figured it out). You can't use any name; it has to be a specific filename (diffusion.nii, mask.nii, im1.nii, etc.)

I was getting a consistent error "The trait 'in_file' of a TVResampleInputSpec instance is an existing file name, but the path 'ten1.nii' does not exist." for all of my tests. I changed the in_file to diffusion.nii and it resolved.

    >>> import nipype.interfaces.dtitk as dtitk
    >>> node = dtitk.TVResampleTask()
    >>> node.inputs.in_file = 'ten1.nii'
    >>> node.inputs.target_file = 'ten2.nii'
    >>> node.cmdline
    'TVResample -in ten1.nii -out ten1_resampled.nii -target ten2.nii'
    >>> node.run() # doctest: +SKIP

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants