Skip to content

Variable type conversion #3278

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pekkaklarck opened this issue Aug 23, 2019 · 22 comments · Fixed by #5379
Closed

Variable type conversion #3278

pekkaklarck opened this issue Aug 23, 2019 · 22 comments · Fixed by #5379

Comments

@pekkaklarck
Copy link
Member

pekkaklarck commented Aug 23, 2019

RF 3.1 added automatic argument conversion with library keywords using function annotations (#2890), default values (#2932), and some other means. That has worked very well, and it would be nice if something like that would work also with variables defined in test data. We probably could re-use the underlying argument conversion logic which ought to make it relatively easy to implement this. We also got a similar request related to variables passed from the command line #2946.

Automatically converting numbers like 42 and 3.14 to actual integers/floats would be handy, even when we already support them as variables like ${42} and ${3.14}. Being able to use datatime objects, bytes, and other types supported by the argument conversion that don't have direct variable alternatives would naturally be even more useful (but also less commonly needed). I would expect this to be most useful with lists and dictionaries and especially with nested list and dicts that cannot easily be constructed in Robot's data.

My proposed syntax for this functionality is supporting "type postfix" like :int in variable names like this:

*** Variables ***
${COUNT:int}         42
${GOOD IDEA:bool}    Yes    # Converted to Boolean True
${ANIMALS:list}      ['cat', 'dog', 'horse']    # Python syntax
${NESTED:dict}       {'a': 1, 'b': [{'x': 2}, {'x': 3}]}    # Great with JSON

This syntax is derived from Python's function annotation syntax and also the same syntax that was decided to be used with command line variables in #2946. Created variable names would naturally omit the type suffix and look like ${COUNT} and ${GOOD IDEA}.

In addition to being available in the variable section, the new syntax should also be available also when creating variable otherwise, including in keyword arguments:

*** Keywords ***
Example
    [Arguments]    ${count:int}
    ${nested:dict} =    Set Variable    {'a': ${count}, 'b': [{'x': 2}, {'x': 3}]}
    Set Test Variable    ${DATE:datetime}     2019-08-23 12:03:42

There are some design decision still to be made like should ${X:dict} create a DotDict instance that allows ${X.key} syntax and how to handle possible backwards incompatibility issues. For most parts the design looks pretty good, though, and I think this would be a valuable addition to Robot Framework. If there's enough interest, we can consider adding this already to RF 3.2 scope. To show your interest, vote with 👍 or 👎 and/or add comments.


UPDATE: The final syntax is mostly the same as above, but there needs to be a space after the colon like ${COUNT: int}. The motivation is making the enhancement less backwards incompatible.

@pekkaklarck
Copy link
Member Author

This change is backwards compatible because it's possible that someone has used variables that contain : in their name like ${FOO:BAR}. If we always consider the part after the colon to be the type, then such usage won't work anymore. One possibility is checking is that part a recognized type, but there's still a risk that someone has a variable like ${MY:LIST}. It would naturally be safest to deprecate using colons in variable names first, but that would mean that the actual conversion support needed to wait until RF 3.3. In my opinion the risk for this change causing problems is small enough and benefits of the new syntax are big enough to do the change already in RF 3.2.

@pekkaklarck
Copy link
Member Author

If this enhancement is implemented, it makes issue #2699 pretty useless. The more I think about it, the more I like this one compared to that one. Mainly because this has so many other benefits than just supporting lists/dicts with embedded arguments.

The main questions is, when could this be implemented. RF 3.2 starts to be so much overdue that this issue cannot fit it unless someone else is interested to look at implementing it. RF 3.3. is possible, but also there are so many other high priority features that external help would be highly appreciated.

@pekkaklarck
Copy link
Member Author

We have PR #3385 implementing this so I'll add this tentatively to RF 3.2 scope. That release is so much overdue that I cannot make promises will this issue make into it, though.

@pekkaklarck pekkaklarck added this to the v3.2 milestone Nov 21, 2019
@pekkaklarck pekkaklarck modified the milestones: v3.2, v3.3 Dec 4, 2019
@pekkaklarck
Copy link
Member Author

Unfortunately this is too much work for the already overdue RF 3.2.

@pekkaklarck pekkaklarck modified the milestones: v3.3, v4.0 Feb 7, 2020
LeConTesteur pushed a commit to LeConTesteur/robotframework that referenced this issue Mar 27, 2020
LeConTesteur pushed a commit to LeConTesteur/robotframework that referenced this issue Mar 27, 2020
@pekkaklarck pekkaklarck modified the milestones: v5.0, v4.0 Aug 1, 2020
@mkorpela
Copy link
Member

I think expected type is only needed when variable is receiving a value. This is special case.
I propose that the type should be outside of the body. So we extend current syntax instead of making a rather backwards incompatible change
Like this ${mynumber}:int 3 and not ${mynumber:int} 3

@mkorpela
Copy link
Member

mkorpela commented Sep 12, 2020

And ignore type info on embedded keyword var syntax or escape it.

@pekkaklarck
Copy link
Member Author

I'd prefer the ${name: type} syntax over ${name}: type for these reasons:

  • Looks better to my eye.
  • Easier to implement as the logic to find variables doesn't change.
  • Allows using types with embedded arguments. This is pretty important.
  • Works well also when assigning variable based on kw return value.

As I already commented above, I don't consider backwards compatibility problems that big. Probably best to allow still using colons in names like ${foo:bar} but perhaps that should be deprecated. Alternatively we could require the separator to be : i.e. a colon followed by a space.

@mkorpela
Copy link
Member

This conversion is also a bit different from static typing although the syntax points to that.

Would this converter be a property of the variable or an external thing? How should each of these behave?

*** Variables ***
${myvar: int}   3

*** Keywords ***
Doing something
  ${myvar} =   Calling Library Keyword

Doing something differently
  ${myvar: str} =    Calling Batman

Doing the usual
  ${myvar: int} =    Calling Mom

@pekkaklarck
Copy link
Member Author

It would just be a converter. Creating a variable like ${x:int} would mean converting the value to an integer but ${x} could later be assigned to other type.

@pekkaklarck pekkaklarck removed this from the v4.0 milestone Nov 23, 2020
pekkaklarck added a commit that referenced this issue Apr 11, 2025
pekkaklarck pushed a commit that referenced this issue Apr 25, 2025
See issue #3278. User Guide documentation still missing and some cleanup to be done.
@pekkaklarck
Copy link
Member Author

pekkaklarck commented Apr 25, 2025

User Guide documentation still missing and the code can be enhanced a bit as well.

@pekkaklarck pekkaklarck reopened this Apr 25, 2025
pekkaklarck added a commit that referenced this issue Apr 25, 2025
- Enhance error reporting with embedded args having invalid type.
- Consistent test case naming style.
- Some code cleanup.

Part of #3278.
aaltat added a commit to aaltat/robotframework that referenced this issue May 7, 2025
aaltat added a commit to aaltat/robotframework that referenced this issue May 7, 2025
pekkaklarck pushed a commit that referenced this issue May 7, 2025
Normal type hints should be used instead. Part of #3278.
pekkaklarck added a commit that referenced this issue May 7, 2025
Move validation that embedded args with library keywords don't support
embedded types to a better place. Also move related tests to suite
testing embedded args with library keywords. Related to #3278.
pekkaklarck added a commit that referenced this issue May 15, 2025
Don't capitalize "kind" if it is not all lower case to avoid e.g.
"FOR loop variable" to be changed to "For loop variable".

Related to FOR loop variable conversion that's a missing part of
issue #3278.
pekkaklarck added a commit that referenced this issue May 16, 2025
Missing part of #3278.

Includes also enhancements to variable validation during parsing.
pekkaklarck added a commit that referenced this issue May 20, 2025
Fixes #2946.

Documentation missing, but it will be done as part of documenting
variable conversion in general (#3278).
@pekkaklarck
Copy link
Member Author

Using non-string variables from the command line had a separate issue #2946. That has now been implemented as well and this conversion functionality needs to be documented as part of this issue.

pekkaklarck pushed a commit that referenced this issue May 27, 2025
@pekkaklarck
Copy link
Member Author

The current syntax to create an embedded argument with both a type and a custom regexps is ${name:pattern: type}. This is somewhat inconsistent with the general variable type syntax ${name: type} and even more inconsistent with the CLI variable type syntax name: type:value (#2946). I believe we should change the syntax to ${name: type:pattern} and others on the #devel channel agreed.

pekkaklarck added a commit that referenced this issue May 29, 2025
To some extend related to documenting variable type conversion
(#3278).
pekkaklarck added a commit that referenced this issue May 29, 2025
Earlier syntax was `${name:pattern: type}`, but now it is
`${name: type:pattern}`. This is more consistent with the general
variable type syntax `${name: type}`. Part of #3278.
@pekkaklarck
Copy link
Member Author

I enhanced documentation in 30d87e4. Unfortunately it isn't possible to see generated docs before the final release is out, but here are links to the most relevant places in the documentation source that ought to render somewhat well:

All kind of comments about the docs are welcome and can be written either to here to a thread I'll start about this on the #devel channel on our Slack. This issue can anyway be considered done already now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants