Of course I have a backup!

Random blobs of wisdom about software development

Arrays in Bash

Monday, January 02, 2012

Arrays exist in Bash since version 4.0. However, the capabilities are pretty limited, it is nowhere near other scripting languages like PHP, Ruby, or Python. You can have sequentially indexed arrays, and associative arrays, but, both of them can be only one dimension.

Declaring an array

#!/bin/bash

# Declare a sequentially indexed array
declare -a MY_ARRAY

# There is also a shorthand syntax
MY_ARRAY=()

# You can specify starting values
declare -a MY_ARRAY=('a' 'b' 'c')
MY_ARRAY=('a' 'b' 'c')

# Declare an associatave array, there is no shorthand syntax for this
declare -A ASSOC_ARRAY

# With starting values
declare -A ASSOC_ARRAY=([fruits]="apple banana" [vegetables]="pickle tomato")

As you see, sequential arrays can be declared two ways, either with declare -a , or with the VAR=() syntax. There is a tiny difference between the two: if used within a function, declare -a makes your array local (a local variable is only available in the function it was defined in), while the parenthesis syntax does not. Of course, the quotes are also optional around the values, as long as they don't contain spaces. Associative arrays must be declared with declare -A VAR .

Iterating

Iterating over arrays will feel natural if you are used to Bash, with all of it's quirks and surprises about quoting, and word splitting:

#!/bin/bash

# Iterating over the values of an array
# Outputs: a b c

declare -a LETTERS=(a b c)
for VALUE in "${LETTERS[@]}"; do
  echo "$VALUE"
done

# Iterating over the keys of an array
# Outputs: last first

declare -A NAME=([first]=John [last]=Doe)
for KEY in "${!NAME[@]}" do
  echo "$KEY"
done

Notice the output from the second code block. The order of the keys are 'last', 'first', instead of 'first', 'last'. When expanding with the ${!VAR[@]} syntax, you will not get the keys back in the order you defined them. Actually, I'm not even sure how the order is determined, I tried playing around it with, and it does not seem to be a lexical sort. So keep in mind, that you cannot rely on the order of the keys. Also remember that an array can only be one dimensional, there are no arrays in arrays, so both of these will fail:

#!/bin/bash

declare -A SHOPPING_LIST=([fruits]=(apple orange) [vegetables]=(pickle onion))
declare -a SQUARE=((0 0) (15 15))
# test.sh: line 3: syntax error near unexpected token '('
# test.sh: line 4: syntax error near unexpected token '('

Passing arrays as parameters

This has to be the biggest gotcha. Let's look at what "${ARRAY[@]}" really expands to:

#!/bin/bash

ARRAY=("apple" "banana" "yellow lemon")
printf ":%s:" "${ARRAY[@]}"

# Echoes
# :apple: :banana: :yellow lemon:

As you see, ${ARRAY[@]} expands to the elements of the array, seperated by spaces. If you pass this to a function, it will receive n string parameters, and not 1 array. You could try using the $VAR syntax, like if it was a simple variable, but that will expand to the first element of the array. You cannot preserve, or more like, bash does not preserve the information that a variable is in fact an array, and not a scalar.

Now remember that Bash functions are similar to Perl's subs, as in, you cannot have named parameters, everything will get passed to your function as positional parameters ($1, $2, $n). This should ring a bell now. You have no way of signaling to the function that you are passing an array. You get N positional parameters, with no information about the datatypes of each. There is no way to determine which of them belongs to an array, and which of them is a scalar:

#!/bin/bash

FRUITS=("apple" "banana" "yellow lemon")
VEGETABLES=("onion" "pickle" "red carrot")

dump() {
  printf ":%s: " "$@"
}

# This will print:
# :apple: :banana: :yellow lemon: :onion: :pickle: :red carrot:
dump "${FRUITS[@]}" "${VEGETABLES[@]}"

# The same thing with associative arrays:
declare -A THINGIES
THINGIES=([fruit]="apple" [vegetable]="onion")

# This will print:
# :onion: :apple:
dump "${THINGIES[@]}"

# And this will print:
# :vegetable: :fruit:
dump "${!THINGIES[@]}"

This also means that you have no way of passing in "complete" associative arrays. You either pass in the keys, or the values. There is one tiny trick. If you can guarantee that your function will always receive exactly the same amount of scalar parameters, you can pass in 1, and only one array, and only as the last parameter. This works, because you can pull down the parameters to local variables, shift the positional parameters, and treat the rest as an array:

#!/bin/bash

FRUITS=('apple' 'banana' 'yellow lemon')

purchase() {
  local who="$1"
  local where="$2"
  shift 2
  local what=("$@")
  printf "%s is buying %s, at %s" "$who" "${what[*]}" "$where"
}

# Outputs:
# Alice is buying apple banana yellow lemon, at the marketplace
purchase "Alice" "the marketplace" "${FRUITS[@]}"

Summary

List of things to look out for, when using arrays:

  • Iterating over the keys of an associatve array will not yield the same order as they are defined
  • An array can be only one dimension, there are no arrays in arrays
  • You cannot pass in arrays to a function reliably, except the one case I mentioned
  • Associative arrays cannot be passed to functions, you either pass the keys, or the values

All this make it seem like arrays were an afterthought, kinda like how OOP was in PHP4. Array support is there, but with some huge gotchas. You might argue that "yeah, Bash never meant to handle complex tasks", or "You don't grok the unix philosophy", but I'm not sure. Arrays are here, I don't think it's such a high expectation to be able to pass them around.

This was written by Norbert Kéri, posted on Monday, January 02, 2012, at 21:34

Tagged as:
Brian Chrisman wrote
Not the most glorious of options, but it's possible to export/import/pass associative arrays into bash functions:
<pre>
[bash]
# declare -A bashLevel='([joe]=&quot;5 - journeyman&quot; [bob]=&quot;6 - advanced&quot;)'
# f() { eval $1; echo &quot;single value: ${bashLevel[joe]}&quot;; }
# f &quot;$(declare -p bashLevel)&quot;
single value: 5 - journeyman
#
[/bash]
</pre>

2013-08-16 14:09:00

Post a comment

Providing your email is optional, it is never published or shared, it is only used for auto approval purposes. If you already have at least 1 approved comment(s) tied to your email, you don't have to wait for moderation, otherwise the author must approve your comment.

Please solve this totally random captcha