Skip to content

Commit f833c84

Browse files
tglsfdctglsfdc
committed
Allow psql variable substitution to occur in backtick command strings.
Previously, text between backquotes in a psql metacommand's arguments was always passed to the shell literally. That considerably hobbles the usefulness of the feature for scripting, so we'd foreseen for a long time that we'd someday want to allow substitution of psql variables into the shell command. IMO the addition of \if metacommands has brought us to that point, since \if can greatly benefit from some sort of client-side expression evaluation capability, and psql itself is not going to grow any such thing in time for v10. Hence, this patch. It allows :VARIABLE to be replaced by the exact contents of the named variable, while :'VARIABLE' is replaced by the variable's contents suitably quoted to become a single shell-command argument. (The quoting rules for that are different from those for SQL literals, so this is a bit of an abuse of the :'VARIABLE' notation, but I doubt anyone will be confused.) As with other situations in psql, no substitution occurs if the word following a colon is not a known variable name. That limits the risk of compatibility problems for existing psql scripts; but the risk isn't zero, so this needs to be called out in the v10 release notes. Discussion: https://postgr.es/m/9561.1490895211@sss.pgh.pa.us
1 parent 41bd155 commit f833c84

File tree

9 files changed

+180
-66
lines changed

9 files changed

+180
-66
lines changed

‎doc/src/sgml/ref/psql-ref.sgml

Lines changed: 22 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -769,18 +769,33 @@ testdb=>
769769
quotes that single character, whatever it is.
770770
</para>
771771

772-
<para>
773-
Within an argument, text that is enclosed in backquotes
774-
(<literal>`</literal>) is taken as a command line that is passed to the
775-
shell. The output of the command (with any trailing newline removed)
776-
replaces the backquoted text.
777-
</para>
778-
779772
<para>
780773
If an unquoted colon (<literal>:</literal>) followed by a
781774
<application>psql</> variable name appears within an argument, it is
782775
replaced by the variable's value, as described in <xref
783776
linkend="APP-PSQL-interpolation" endterm="APP-PSQL-interpolation-title">.
777+
The forms <literal>:'<replaceable>variable_name</>'</literal> and
778+
<literal>:"<replaceable>variable_name</>"</literal> described there
779+
work as well.
780+
</para>
781+
782+
<para>
783+
Within an argument, text that is enclosed in backquotes
784+
(<literal>`</literal>) is taken as a command line that is passed to the
785+
shell. The output of the command (with any trailing newline removed)
786+
replaces the backquoted text. Within the text enclosed in backquotes,
787+
no special quoting or other processing occurs, except that appearances
788+
of <literal>:<replaceable>variable_name</></literal> where
789+
<replaceable>variable_name</> is a <application>psql</> variable name
790+
are replaced by the variable's value. Also, appearances of
791+
<literal>:'<replaceable>variable_name</>'</literal> are replaced by the
792+
variable's value suitably quoted to become a single shell command
793+
argument. (The latter form is almost always preferable, unless you are
794+
very sure of what is in the variable.) Because carriage return and line
795+
feed characters cannot be safely quoted on all platforms, the
796+
<literal>:'<replaceable>variable_name</>'</literal> form prints an
797+
error message and does not substitute the variable value when such
798+
characters appear in the value.
784799
</para>
785800

786801
<para>

‎src/bin/psql/common.c

Lines changed: 66 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -116,19 +116,19 @@ setQFout(const char *fname)
116116
* If the specified variable exists, return its value as a string (malloc'd
117117
* and expected to be freed by the caller); else return NULL.
118118
*
119-
* If "escape" is true, return the value suitably quoted and escaped,
120-
* as an identifier or string literal depending on "as_ident".
121-
* (Failure in escaping should lead to returning NULL.)
119+
* If "quote" isn't PQUOTE_PLAIN, then return the value suitably quoted and
120+
* escaped for the specified quoting requirement. (Failure in escaping
121+
* should lead to printing an error and returning NULL.)
122122
*
123123
* "passthrough" is the pointer previously given to psql_scan_set_passthrough.
124124
* In psql, passthrough points to a ConditionalStack, which we check to
125125
* determine whether variable expansion is allowed.
126126
*/
127127
char *
128-
psql_get_variable(const char *varname, bool escape, bool as_ident,
128+
psql_get_variable(const char *varname, PsqlScanQuoteType quote,
129129
void *passthrough)
130130
{
131-
char *result;
131+
char *result = NULL;
132132
const char *value;
133133

134134
/* In an inactive \if branch, suppress all variable substitutions */
@@ -139,40 +139,74 @@ psql_get_variable(const char *varname, bool escape, bool as_ident,
139139
if (!value)
140140
return NULL;
141141

142-
if (escape)
142+
switch (quote)
143143
{
144-
char *escaped_value;
144+
case PQUOTE_PLAIN:
145+
result = pg_strdup(value);
146+
break;
147+
case PQUOTE_SQL_LITERAL:
148+
case PQUOTE_SQL_IDENT:
149+
{
150+
/*
151+
* For these cases, we use libpq's quoting functions, which
152+
* assume the string is in the connection's client encoding.
153+
*/
154+
char *escaped_value;
145155

146-
if (!pset.db)
147-
{
148-
psql_error("cannot escape without active connection\n");
149-
return NULL;
150-
}
156+
if (!pset.db)
157+
{
158+
psql_error("cannot escape without active connection\n");
159+
return NULL;
160+
}
151161

152-
if (as_ident)
153-
escaped_value =
154-
PQescapeIdentifier(pset.db, value, strlen(value));
155-
else
156-
escaped_value =
157-
PQescapeLiteral(pset.db, value, strlen(value));
162+
if (quote == PQUOTE_SQL_LITERAL)
163+
escaped_value =
164+
PQescapeLiteral(pset.db, value, strlen(value));
165+
else
166+
escaped_value =
167+
PQescapeIdentifier(pset.db, value, strlen(value));
158168

159-
if (escaped_value == NULL)
160-
{
161-
const char *error = PQerrorMessage(pset.db);
169+
if (escaped_value == NULL)
170+
{
171+
const char *error = PQerrorMessage(pset.db);
162172

163-
psql_error("%s", error);
164-
return NULL;
165-
}
173+
psql_error("%s", error);
174+
return NULL;
175+
}
166176

167-
/*
168-
* Rather than complicate the lexer's API with a notion of which
169-
* free() routine to use, just pay the price of an extra strdup().
170-
*/
171-
result = pg_strdup(escaped_value);
172-
PQfreemem(escaped_value);
177+
/*
178+
* Rather than complicate the lexer's API with a notion of
179+
* which free() routine to use, just pay the price of an extra
180+
* strdup().
181+
*/
182+
result = pg_strdup(escaped_value);
183+
PQfreemem(escaped_value);
184+
break;
185+
}
186+
case PQUOTE_SHELL_ARG:
187+
{
188+
/*
189+
* For this we use appendShellStringNoError, which is
190+
* encoding-agnostic, which is fine since the shell probably
191+
* is too. In any case, the only special character is "'",
192+
* which is not known to appear in valid multibyte characters.
193+
*/
194+
PQExpBufferData buf;
195+
196+
initPQExpBuffer(&buf);
197+
if (!appendShellStringNoError(&buf, value))
198+
{
199+
psql_error("shell command argument contains a newline or carriage return: \"%s\"\n",
200+
value);
201+
free(buf.data);
202+
return NULL;
203+
}
204+
result = buf.data;
205+
break;
206+
}
207+
208+
/* No default: we want a compiler warning for missing cases */
173209
}
174-
else
175-
result = pg_strdup(value);
176210

177211
return result;
178212
}

‎src/bin/psql/common.h

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,11 +12,12 @@
1212

1313
#include "libpq-fe.h"
1414
#include "fe_utils/print.h"
15+
#include "fe_utils/psqlscan.h"
1516

1617
extern bool openQueryOutputFile(const char *fname, FILE **fout, bool *is_pipe);
1718
extern bool setQFout(const char *fname);
1819

19-
extern char *psql_get_variable(const char *varname, bool escape, bool as_ident,
20+
extern char *psql_get_variable(const char *varname, PsqlScanQuoteType quote,
2021
void *passthrough);
2122

2223
extern void psql_error(const char *fmt,...) pg_attribute_printf(1, 2);

‎src/bin/psql/psqlscanslash.l

Lines changed: 45 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -242,8 +242,7 @@ other .
242242
yytext + 1,
243243
yyleng - 1);
244244
value = cur_state->callbacks->get_variable(varname,
245-
false,
246-
false,
245+
PQUOTE_PLAIN,
247246
cur_state->cb_passthrough);
248247
free(varname);
249248

@@ -268,14 +267,16 @@ other .
268267
}
269268

270269
:'{variable_char}+' {
271-
psqlscan_escape_variable(cur_state, yytext, yyleng, false);
270+
psqlscan_escape_variable(cur_state, yytext, yyleng,
271+
PQUOTE_SQL_LITERAL);
272272
*option_quote = ':';
273273
unquoted_option_chars = 0;
274274
}
275275

276276

277277
:\"{variable_char}+\" {
278-
psqlscan_escape_variable(cur_state, yytext, yyleng, true);
278+
psqlscan_escape_variable(cur_state, yytext, yyleng,
279+
PQUOTE_SQL_IDENT);
279280
*option_quote = ':';
280281
unquoted_option_chars = 0;
281282
}
@@ -337,9 +338,8 @@ other .
337338

338339
<xslashbackquote>{
339340
/*
340-
* backticked text: copy everything until next backquote, then evaluate.
341-
*
342-
* XXX Possible future behavioral change: substitute for :VARIABLE?
341+
* backticked text: copy everything until next backquote (expanding
342+
* variable references, but doing nought else), then evaluate.
343343
*/
344344

345345
"`" {
@@ -350,6 +350,44 @@ other .
350350
BEGIN(xslasharg);
351351
}
352352

353+
:{variable_char}+ {
354+
/* Possible psql variable substitution */
355+
if (cur_state->callbacks->get_variable == NULL)
356+
ECHO;
357+
else
358+
{
359+
char *varname;
360+
char *value;
361+
362+
varname = psqlscan_extract_substring(cur_state,
363+
yytext + 1,
364+
yyleng - 1);
365+
value = cur_state->callbacks->get_variable(varname,
366+
PQUOTE_PLAIN,
367+
cur_state->cb_passthrough);
368+
free(varname);
369+
370+
if (value)
371+
{
372+
appendPQExpBufferStr(output_buf, value);
373+
free(value);
374+
}
375+
else
376+
ECHO;
377+
}
378+
}
379+
380+
:'{variable_char}+' {
381+
psqlscan_escape_variable(cur_state, yytext, yyleng,
382+
PQUOTE_SHELL_ARG);
383+
}
384+
385+
:'{variable_char}* {
386+
/* Throw back everything but the colon */
387+
yyless(1);
388+
ECHO;
389+
}
390+
353391
{other}|\n { ECHO; }
354392

355393
}

‎src/fe_utils/psqlscan.l

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -699,8 +699,7 @@ other .
699699
yyleng - 1);
700700
if (cur_state->callbacks->get_variable)
701701
value = cur_state->callbacks->get_variable(varname,
702-
false,
703-
false,
702+
PQUOTE_PLAIN,
704703
cur_state->cb_passthrough);
705704
else
706705
value = NULL;
@@ -737,11 +736,13 @@ other .
737736
}
738737

739738
:'{variable_char}+' {
740-
psqlscan_escape_variable(cur_state, yytext, yyleng, false);
739+
psqlscan_escape_variable(cur_state, yytext, yyleng,
740+
PQUOTE_SQL_LITERAL);
741741
}
742742

743743
:\"{variable_char}+\" {
744-
psqlscan_escape_variable(cur_state, yytext, yyleng, true);
744+
psqlscan_escape_variable(cur_state, yytext, yyleng,
745+
PQUOTE_SQL_IDENT);
745746
}
746747

747748
/*
@@ -1415,15 +1416,15 @@ psqlscan_extract_substring(PsqlScanState state, const char *txt, int len)
14151416
*/
14161417
void
14171418
psqlscan_escape_variable(PsqlScanState state, const char *txt, int len,
1418-
bool as_ident)
1419+
PsqlScanQuoteType quote)
14191420
{
14201421
char *varname;
14211422
char *value;
14221423

14231424
/* Variable lookup. */
14241425
varname = psqlscan_extract_substring(state, txt + 2, len - 3);
14251426
if (state->callbacks->get_variable)
1426-
value = state->callbacks->get_variable(varname, true, as_ident,
1427+
value = state->callbacks->get_variable(varname, quote,
14271428
state->cb_passthrough);
14281429
else
14291430
value = NULL;

‎src/fe_utils/string_utils.c

Lines changed: 24 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -425,13 +425,30 @@ appendByteaLiteral(PQExpBuffer buf, const unsigned char *str, size_t length,
425425
* arguments containing LF or CR characters. A future major release should
426426
* reject those characters in CREATE ROLE and CREATE DATABASE, because use
427427
* there eventually leads to errors here.
428+
*
429+
* appendShellString() simply prints an error and dies if LF or CR appears.
430+
* appendShellStringNoError() omits those characters from the result, and
431+
* returns false if there were any.
428432
*/
429433
void
430434
appendShellString(PQExpBuffer buf, const char *str)
435+
{
436+
if (!appendShellStringNoError(buf, str))
437+
{
438+
fprintf(stderr,
439+
_("shell command argument contains a newline or carriage return: \"%s\"\n"),
440+
str);
441+
exit(EXIT_FAILURE);
442+
}
443+
}
444+
445+
bool
446+
appendShellStringNoError(PQExpBuffer buf, const char *str)
431447
{
432448
#ifdef WIN32
433449
int backslash_run_length = 0;
434450
#endif
451+
bool ok = true;
435452
const char *p;
436453

437454
/*
@@ -442,7 +459,7 @@ appendShellString(PQExpBuffer buf, const char *str)
442459
strspn(str, "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-_./:") == strlen(str))
443460
{
444461
appendPQExpBufferStr(buf, str);
445-
return;
462+
return ok;
446463
}
447464

448465
#ifndef WIN32
@@ -451,10 +468,8 @@ appendShellString(PQExpBuffer buf, const char *str)
451468
{
452469
if (*p == '\n' || *p == '\r')
453470
{
454-
fprintf(stderr,
455-
_("shell command argument contains a newline or carriage return: \"%s\"\n"),
456-
str);
457-
exit(EXIT_FAILURE);
471+
ok = false;
472+
continue;
458473
}
459474

460475
if (*p == '\'')
@@ -481,10 +496,8 @@ appendShellString(PQExpBuffer buf, const char *str)
481496
{
482497
if (*p == '\n' || *p == '\r')
483498
{
484-
fprintf(stderr,
485-
_("shell command argument contains a newline or carriage return: \"%s\"\n"),
486-
str);
487-
exit(EXIT_FAILURE);
499+
ok = false;
500+
continue;
488501
}
489502

490503
/* Change N backslashes before a double quote to 2N+1 backslashes. */
@@ -524,6 +537,8 @@ appendShellString(PQExpBuffer buf, const char *str)
524537
}
525538
appendPQExpBufferStr(buf, "^\"");
526539
#endif /* WIN32 */
540+
541+
return ok;
527542
}
528543

529544

0 commit comments

Comments
 (0)