Skip to content

Commit cd78e90

Browse files
mkorpelapekkaklarck
mkorpela
and
pekkaklarck
authored
Topic/list and dict vars (robotframework#3693)
* allow nested list access * allow chained access of list and dict vars * list and dict vars can contain items * remove warning about using list or dict var with items * finder: move validation logic to _validate_value * nested list test fixes * fix utests * tests: remove unused keyword * fix dict var items test * fix dict test * update user guide * Fix nested access with @/& variables when value is not list/dict. * Fix unittests * Support slice with list expansion like @{var}[1:]. This is part of robotframework#3487. * Enhance documentation of list/dict expansion w/ item access. Final (hopefully) part of robotframework#3487. * Ignore eq as dictionary repr is not constant over python versions Co-authored-by: Pekka Klärck <peke@iki.fi>
1 parent 2d6661e commit cd78e90

File tree

14 files changed

+162
-145
lines changed

14 files changed

+162
-145
lines changed

‎atest/robot/variables/dict_variable_items.robot

Lines changed: 5 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -53,20 +53,8 @@ Non-dict variable
5353
Sanity check
5454
Check Test Case ${TESTNAME}
5555

56-
Old syntax with `&` still works but is deprecated
57-
[Documentation] `${dict}[key]` and `&{dict}[key]` work same way still.
58-
... In the future latter is deprecated and then changed.
59-
${tc} = Check Test Case ${TESTNAME}
60-
Old item access syntax is deprecated ${tc.kws[0].msgs[0]} \&{DICT}[A]
61-
Old item access syntax is deprecated ${ERRORS[0]} \&{DICT}[A]
62-
Old item access syntax is deprecated ${tc.kws[1].msgs[0]} \&{DICT}[\${1}]
63-
Old item access syntax is deprecated ${ERRORS[1]} \&{DICT}[\${1}]
64-
Old item access syntax is deprecated ${tc.kws[2].msgs[0]} \&{DICT}[nonex]
65-
Old item access syntax is deprecated ${ERRORS[2]} \&{DICT}[nonex]
66-
67-
*** Keywords ***
68-
Old item access syntax is deprecated
69-
[Arguments] ${msg} ${deprecated}
70-
Check log message ${msg}
71-
... Accessing variable items using '${deprecated}' syntax is deprecated. Use '$${deprecated[1:]}' instead.
72-
... WARN
56+
Dict expansion using `&` syntax
57+
Check Test Case ${TESTNAME}
58+
59+
Dict expansion fails if value is not dict-like
60+
Check Test Case ${TESTNAME}

‎atest/robot/variables/list_variable_items.robot

Lines changed: 10 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -59,24 +59,15 @@ Non-existing index variable
5959
Non-subscriptable variable
6060
Check Test Case ${TESTNAME}
6161

62-
Old syntax with `@` still works but is deprecated
63-
[Documentation] `${list}[1]` and `@{list}[1]` work same way still.
64-
... In the future latter is deprecated and changed.
65-
${tc} = Check Test Case ${TESTNAME}
66-
Old item access syntax is deprecated ${tc.kws[0].msgs[0]} \@{LIST}[0]
67-
Old item access syntax is deprecated ${ERRORS[0]} \@{LIST}[0]
68-
Old item access syntax is deprecated ${tc.kws[1].msgs[0]} \@{LIST}[\${-1}]
69-
Old item access syntax is deprecated ${ERRORS[1]} \@{LIST}[\${-1}]
70-
Old item access syntax is deprecated ${tc.kws[2].msgs[0]} \@{LIST}[99]
71-
Old item access syntax is deprecated ${ERRORS[2]} \@{LIST}[99]
72-
73-
Old syntax with `@` doesn't support new slicing syntax
74-
[Documentation] Slicing support should be added in RF 3.3 when `@{list}[index]` changes.
62+
List expansion using `@` syntax
7563
Check Test Case ${TESTNAME}
7664

77-
*** Keywords ***
78-
Old item access syntax is deprecated
79-
[Arguments] ${msg} ${deprecated}
80-
Check log message ${msg}
81-
... Accessing variable items using '${deprecated}' syntax is deprecated. Use '$${deprecated[1:]}' instead.
82-
... WARN
65+
List expansion fails if value is not list-like
66+
Check Test Case ${TESTNAME} 1
67+
Check Test Case ${TESTNAME} 2
68+
69+
List expansion with slice
70+
Check Test Case ${TESTNAME}
71+
72+
List expansion with slice fails if value is not list-like
73+
Check Test Case ${TESTNAME}

‎atest/robot/variables/nested_item_access.robot

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,5 +33,5 @@ Nested access with non-subscriptable
3333
Escape nested
3434
Check Test Case ${TESTNAME}
3535

36-
Nested access doesn't support old `@` and `&` syntax
36+
Nested access supports `@` and `&` syntax
3737
Check Test Case ${TESTNAME}

‎atest/testdata/variables/dict_variable_items.robot

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ Library XML
55
*** Variables ***
66
${INT} ${15}
77
&{DICT} A=1 B=2 C=3 ${1}=${2} 3=4 ${NONE}=${NONE} = ${SPACE}=${SPACE}
8+
... eq=${{{'second': 'xXx', 'ignore_case': True}}}
89
&{SQUARES} [=open ]=close []=both [x[y]=mixed
910
${A} A
1011
${INVALID} xxx
@@ -108,13 +109,18 @@ Non-dict variable
108109
Sanity check
109110
@{items} = Create List
110111
FOR ${key} IN @{DICT}
111-
Append To List ${items} ${key}: ${DICT}[${key}]
112+
Run Keyword If '${key}' != 'eq' Append To List ${items} ${key}: ${DICT}[${key}]
112113
END
113114
${items} = Catenate SEPARATOR=,${SPACE} @{items}
114115
Should Be Equal ${items} A: 1, B: 2, C: 3, 1: 2, 3: 4, None: None, : , ${SPACE}: ${SPACE}
115116

116-
Old syntax with `&` still works but is deprecated
117-
[Documentation] FAIL Dictionary '\&{DICT}' has no key 'nonex'.
118-
Should Be Equal &{DICT}[A] 1
119-
Should Be Equal &{DICT}[${1}] ${2}
120-
Log &{DICT}[nonex]
117+
Dict expansion using `&` syntax
118+
[Documentation] FAIL This fails
119+
Should Be Equal XXX &{DICT}[eq]
120+
Should Be Equal &{DICT}[eq] first=xxx
121+
Should Be Equal YYY &{DICT}[eq] second=yyy
122+
Should Be Equal xxx values=False &{DICT}[eq] ignore_case=False msg=This fails
123+
124+
Dict expansion fails if value is not dict-like
125+
[Documentation] FAIL Value of variable '\&{DICT}[eq][second]' is not dictionary or dictionary-like.
126+
Log Many &{DICT}[eq][second]

‎atest/testdata/variables/list_variable_items.robot

Lines changed: 26 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
${INT} ${15}
33
@{LIST} A B C D E F G H I J K
44
@{NUMBERS} 1 2 3
5+
@{NESTED} ${{['a', 'b', 'c']}} ${{[1, 2, 3]}}
56
${BYTES} ${{b'ABCDEFGHIJK'}}
67
${BYTEARRAY} ${{bytearray(b'ABCDEFGHIJK')}}
78
${STRING} ABCDEFGHIJK
@@ -155,20 +156,29 @@ Non-subscriptable variable
155156
... literal value, it needs to be escaped like '\\[0]'.
156157
Log ${INT}[0]
157158

158-
Old syntax with `@` still works but is deprecated
159-
[Documentation] `\${list}[1]` and `\@{list}[1]` work same way still.
160-
... In the future latter is deprecated and changed.
161-
... FAIL List '\@{LIST}' has no item in index 99.
162-
Should Be Equal @{LIST}[0] A
163-
Should Be Equal @{LIST}[${-1}] K
164-
Log @{LIST}[99]
165-
166-
Old syntax with `@` doesn't support new slicing syntax
167-
[Documentation] Slicing support should be added in RF 3.3 when `@{list}[index]` changes.
168-
... FAIL List '\@{LIST}' used with invalid index '1:'. \
169-
... To use '[1:]' as a literal value, it needs to be \
170-
... escaped like '\\[1:]'.
171-
Log @{LIST}[1:]
159+
List expansion using `@` syntax
160+
[Documentation] FAIL List '\@{NESTED}' has no item in index 99.
161+
${result} = Catenate @{NESTED}[0] - @{NESTED}[${-1}]
162+
Should Be Equal ${result} a b c - 1 2 3
163+
Log Many @{NESTED}[99]
164+
165+
List expansion fails if value is not list-like 1
166+
[Documentation] FAIL Value of variable '\@{LIST}[0]' is not list or list-like.
167+
Log Many @{LIST}[0]
168+
169+
List expansion fails if value is not list-like 2
170+
[Documentation] FAIL Value of variable '\@{NESTED}[1][0]' is not list or list-like.
171+
Log Many @{NESTED}[1][0]
172+
173+
List expansion with slice
174+
${result} = Catenate @{LIST}[7:] - @{LIST}[:${3}] - @{LIST}[8:2:-3]
175+
Should Be Equal ${result} H I J K - A B C - I F
176+
${result} = Catenate @{NESTED}[0][1:] - @{NESTED}[:][${-1}][::][${99}:-99:-2][:99:1]
177+
Should Be Equal ${result} b c - 3 1
178+
179+
List expansion with slice fails if value is not list-like
180+
[Documentation] FAIL Value of variable '\@{STRING}[1:]' is not list or list-like.
181+
Log Many @{STRING}[1:]
172182

173183
*** Keywords ***
174184
Valid index
@@ -203,6 +213,8 @@ Slicing
203213
Should Be Equal ${sequence}[1:-1:2] ${sequence[1:-1:2]}
204214
Should Be Equal ${sequence}[:] ${sequence}
205215
Should Be Equal ${sequence}[::] ${sequence}
216+
Should Be Equal ${sequence}[:][1:] ${sequence[1:]}
217+
Should Be Equal ${sequence}[:][::] ${sequence}
206218
Should Be Empty ${sequence}[100:]
207219

208220
Slicing with variable

‎atest/testdata/variables/nested_item_access.robot

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ Test Template Should Be Equal
44

55
*** Variables ***
66
${LIST} [['item'], [1, 2, (3, [4]), 5], 'third']
7-
${DICT} {'key': {'key': 'value'}, 1: {2: 3}, 'x': {'y': {'z': ''}}}
7+
${DICT} {'key': {'key': 'value'}, 1: {2: 3}, 'x': {'y': {'z': ''}}, 'nest': {'eq': {'first': 'expected'}}}
88
${MIXED} {'x': [(1, {'y': {'z': ['foo', b'bar', {'': ['ABC']}]}})]}
99
${STRING} Robot42
1010

@@ -50,25 +50,26 @@ Invalid nested dict access
5050

5151
Invalid nested string access
5252
[Documentation] FAIL Tuple '\${STRING}[1]' used with invalid index 'inv'.
53-
${LIST}[1][inv] whatever
53+
${LIST}[1][inv] whatever
5454

5555
Nested access with non-subscriptable
5656
[Documentation] FAIL
5757
... Variable '\${DICT}[\${1}][\${2}]' is integer, which is not \
5858
... subscriptable, and thus accessing item '0' from it is not possible. \
5959
... To use '[0]' as a literal value, it needs to be escaped like '\\[0]'.
60-
${DICT}[${1}][${2}][0] whatever
60+
${DICT}[${1}][${2}][0] whatever
6161

6262
Escape nested
6363
${LIST}[-1]\[0] third[0]
6464
${DICT}[key][key]\[key] value[key]
6565
${DICT}[key]\[key][key] {'key': 'value'}[key][key]
6666
${STRING}[0]\[-1] R[-1]
6767

68-
Nested access doesn't support old `@` and `&` syntax
69-
@{LIST}[0][0] ['item'][0]
70-
&{DICT}[key][key] {'key': 'value'}[key]
71-
&{MIXED}[x][0][0] ${MIXED}[x]\[0][0]
68+
Nested access supports `@` and `&` syntax
69+
@{LIST}[1][2][1] ${4}
70+
@{{[[[1, 1]]]}}[0][0]
71+
&{DICT}[nest][eq] second=expected
72+
second=EXPECTED &{DICT}[nest][eq] ignore_case=True
7273

7374
*** Keywords ***
7475
Literals to objects

‎doc/userguide/src/CreatingTestData/Variables.rst

Lines changed: 48 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -163,15 +163,16 @@ the arguments as explained below:
163163

164164
.. _list variable:
165165
.. _list variables:
166+
.. _list expansion:
166167

167168
List variable syntax
168169
~~~~~~~~~~~~~~~~~~~~
169170

170171
When a variable is used as a scalar like `${EXAMPLE}`, its value is be
171172
used as-is. If a variable value is a list or list-like, it is also possible
172-
to use it as a list variable like `@{EXAMPLE}`. In this case individual list
173-
items are passed in as arguments separately. This is easiest to explain with
174-
an example. Assuming that a variable `@{USER}` has value `['robot', 'secret']`,
173+
to use it as a list variable like `@{EXAMPLE}`. In this case the list is expanded
174+
and individual items are passed in as separate arguments. This is easiest to explain
175+
with an example. Assuming that a variable `@{USER}` has value `['robot', 'secret']`,
175176
the following two test cases are equivalent:
176177

177178
.. sourcecode:: robotframework
@@ -189,6 +190,23 @@ requires its value to be a Python list or list-like object. Robot Framework
189190
does not allow strings to be used as lists, but other iterable objects such
190191
as tuples or dictionaries are accepted.
191192

193+
Starting from Robot Framework 4.0, list expansion can be used in combination with
194+
`list item access`__ making these usages possible:
195+
196+
.. sourcecode:: robotframework
197+
198+
*** Test Cases ***
199+
Nested container
200+
${nested} = Evaluate [['a', 'b', 'c'], {'key': ['x', 'y']}]
201+
Log Many @{nested}[0] # Logs 'a', 'b' and 'c'.
202+
Log Many @{nested}[1][key] # Logs 'x' and 'y'.
203+
204+
Slice
205+
${items} = Create List first second third
206+
Log Many @{items}[1:] # Logs 'second' and 'third'.
207+
208+
__ `Accessing sequence items`_
209+
192210
Using list variables with other data
193211
''''''''''''''''''''''''''''''''''''
194212

@@ -229,6 +247,7 @@ __ `All available settings in test data`_
229247

230248
.. _dictionary variable:
231249
.. _dictionary variables:
250+
.. _dictionary expansion:
232251

233252
Dictionary variable syntax
234253
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -237,7 +256,7 @@ As discussed above, a variable containing a list can be used as a `list
237256
variable`_ to pass list items to a keyword as individual arguments.
238257
Similarly a variable containing a Python dictionary or a dictionary-like
239258
object can be used as a dictionary variable like `&{EXAMPLE}`. In practice
240-
this means that individual items of the dictionary are passed as
259+
this means that the dictionary is expanded and individual items are passed as
241260
`named arguments`_ to the keyword. Assuming that a variable `&{USER}` has
242261
value `{'name': 'robot', 'password': 'secret'}`, the following two test cases
243262
are equivalent.
@@ -251,6 +270,11 @@ are equivalent.
251270
Dict Variable
252271
Login &{USER}
253272

273+
Starting from Robot Framework 4.0, dictionary expansion can be used in combination with
274+
`dictionary item access`__ making usages like `&{nested}[key]` possible.
275+
276+
__ `Accessing individual dictionary items`_
277+
254278
Using dictionary variables with other data
255279
''''''''''''''''''''''''''''''''''''''''''
256280

@@ -284,11 +308,16 @@ are imports, setups and teardowns where dictionaries can be used as arguments.
284308
Accessing list and dictionary items
285309
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
286310

287-
It is possible to access items of subscriptable variables, e.g. lists and
288-
dictionaries, using special syntax `${var}[item]`. Accessing items is an old
289-
feature, but prior to Robot Framework 3.1 the syntax was `@{var}[item]` with
290-
lists and `&{var}[item]` with dictionaries. The old syntax was deprecated in
291-
Robot Framework 3.2 and will not be supported in the future.
311+
It is possible to access items of subscriptable variables, e.g. lists and dictionaries,
312+
using special syntax like `${var}[item]` or `${var}[nested][item]`.
313+
Starting from Robot Framework 4.0, it is also possible to use item access together with
314+
`list expansion`_ and `dictionary expansion`_ by using syntax `@{var}[item]` and
315+
`&{var}[item]`, respectively.
316+
317+
.. note:: Prior to Robot Framework 3.1 the normal item access syntax was `@{var}[item]`
318+
with lists and `&{var}[item]` with dictionaries. Robot Framework 3.1 introduced
319+
the generic `${var}[item]` syntax along with some other nice enhancements and
320+
the old item access syntax was deprecated in Robot Framework 3.2.
292321

293322
.. _sequence items:
294323

@@ -300,21 +329,20 @@ It is possible to access a certain item of a variable containing a `sequence`__
300329
is the index of the selected value. Indices start from zero, negative indices
301330
can be used to access items from the end, and trying to access an item with
302331
too large an index causes an error. Indices are automatically converted to
303-
integers, and it is also possible to use variables as indices. Sequence items
304-
accessed in this manner can be used similarly as scalar variables.
332+
integers, and it is also possible to use variables as indices.
305333

306334
.. sourcecode:: robotframework
307335

308336
*** Test Cases ***
309-
Sequence variable item
337+
Positive index
310338
Login ${USER}[0] ${USER}[1]
311339
Title Should Be Welcome ${USER}[0]!
312340

313341
Negative index
314-
Log ${SEQUENCE}[-1]
342+
Keyword ${SEQUENCE}[-1]
315343

316344
Index defined as variable
317-
Log ${SEQUENCE}[${INDEX}]
345+
Keyword ${SEQUENCE}[${INDEX}]
318346

319347
Sequence item access supports also the `same "slice" functionality as Python`__
320348
with syntax like `${var}[1:]`. With this syntax you do not get a single
@@ -337,14 +365,13 @@ specify the start index, the end index, and the step:
337365
Keyword ${SEQUENCE}[::2]
338366
Keyword ${SEQUENCE}[1:-1:10]
339367

340-
.. note:: The slice syntax is new in Robot Framework 3.1 and does not work
341-
with the old `@{var}[index]` syntax.
368+
.. note:: The slice syntax is new in Robot Framework 3.1. It was extended to work
369+
with `list expansion`_ like `@{var}[1:]` in Robot Framework 4.0.
342370

343-
.. note:: With earlier Robot Framework versions accessing items with
344-
an index or a slice was only supported with variables containing
345-
lists, tuples, or other objects considered list-like. Starting
346-
from Robot Framework 3.2, all sequences, including strings and
347-
bytes, are supported.
371+
.. note:: Prior to Robot Framework 3.2, item and slice access was only supported
372+
with variables containing lists, tuples, or other objects considered
373+
list-like. Nowadays all sequences, including strings and bytes, are
374+
supported.
348375

349376
__ https://docs.python.org/3/glossary.html#term-sequence
350377
__ https://docs.python.org/glossary.html#term-slice

‎src/robot/variables/finders.py

Lines changed: 3 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,7 @@
2525

2626
from robot.errors import DataError, VariableError
2727
from robot.utils import (get_env_var, get_env_vars, get_error_message,
28-
is_dict_like, is_list_like, normalize, DotDict,
29-
NormalizedDict)
28+
normalize, NormalizedDict)
3029

3130
from .evaluation import evaluate_expression
3231
from .notfound import variable_not_found
@@ -46,21 +45,13 @@ def __init__(self, variable_store):
4645

4746
def find(self, variable):
4847
match = self._get_match(variable)
49-
identifier = match.identifier
5048
name = match.name
5149
for finder in self._finders:
52-
if identifier in finder.identifiers:
50+
if match.identifier in finder.identifiers:
5351
try:
54-
value = finder.find(name)
52+
return finder.find(name)
5553
except (KeyError, ValueError):
5654
continue
57-
try:
58-
return self._validate_value(value, identifier, name)
59-
except VariableError:
60-
raise
61-
except:
62-
raise VariableError("Resolving variable '%s' failed: %s"
63-
% (name, get_error_message()))
6455
variable_not_found(name, self._store.data)
6556

6657
def _get_match(self, variable):
@@ -71,19 +62,6 @@ def _get_match(self, variable):
7162
raise DataError("Invalid variable name '%s'." % variable)
7263
return match
7364

74-
def _validate_value(self, value, identifier, name):
75-
if identifier == '@':
76-
if not is_list_like(value):
77-
raise VariableError("Value of variable '%s' is not list or "
78-
"list-like." % name)
79-
return list(value)
80-
if identifier == '&':
81-
if not is_dict_like(value):
82-
raise VariableError("Value of variable '%s' is not dictionary "
83-
"or dictionary-like." % name)
84-
return DotDict(value)
85-
return value
86-
8765

8866
class StoredFinder(object):
8967
identifiers = '$@&'

0 commit comments

Comments
 (0)