Of course I have a backup!

Random blobs of wisdom about software development

A workaround for passing arrays in Bash

Sunday, January 29, 2012

In my previous post, I wrote about the shortcomings of arrays in Bash. I think I found a good enough solution, that seems to have a very little footprint, no side effects, and only uses built in functionality.

To recap the problems from the previous post:

  1. Iterating over the keys of an associatve array will not yield the same order as they are defined
  2. There are no arrays in arrays, an array can only hold one dimension
  3. You cannot pass in arrays to a function reliably, except for the one case I mentioned
  4. Associative arrays cannot be passed to functions, you either pass the keys, or the values

While I still don't have a satisfying solution to points one and two, but I think I found one that is good enough for the third and fourth point. It relies on two built in functions, eval, and declare. I'm pretty sure you know the former, because its usage is universally frowned upon in all programming languages, but hear me out. We are only going to eval code that is generated by the shell, which in my opinion qualifies as an exception. In case you haven't used it yet, eval takes a string, and evaluates it as shell code. So eval "X=1" would create variable named X, with a value of 1.

Declare is usually seen when creating arrays, or associative variables, but it also has a handy -p flag, which generates shell code that represents a variable. You give it a variable name, and it will give you back the code that can be used to create an exact replica of it. If you know PHP, this is the equivalent of var_export.

#!/bin/bash

declare -a FRUITS=(apple banana lemon)
declare -p FRUITS

# Echoes
# declare -a FRUITS='([0]="apple" [1]="banana" [2]="lemon")'

Now, if you were to eval the code generated by declare on line 7, you would get the exact variable back. Remember when I told you that we would only eval code generated by the shell? I lied. We need to make a small modification to the returned code, because we need to somehow change the name of the variable, otherwise, we would just be recreating it with the same name (which might not be a problem, since declare creates local variables by default, but will mess up when you use it in the global scope).

#!/bin/bash

ref_array() {
    local varname="$1"
    local export_as="$2"
    local code=$(declare -p "$varname")
    echo ${code/$varname/$export_as}
}

dump() {
    eval $(ref_array "$1" array)
    local key

    for key in "${!array[@]}"; do
        printf "Key: %s, value: %s\n" "$key" "${array[$key]}"
    done
}

declare -A PERSON=([name]="John Doe" [age]=22)
dump PERSON

# Echoes
# Key: name, value: John Doe
# Key: age, value: 22

The ref_array function takes two parameters, first the name of the array, and then another variable name, that the first array should be exported as. The function itself is fairly simple, it takes the code, and replaces the name of the variable with something else you provide in $2, and returns it (The ${variable/replace/with} is the string replace notation in bash). Notice that we are passing in the name of the variables, instead of the values (so don't put a $ before PERSON on line 20) to both the function, and declare, this is not a typo.

Drawbacks

While this is a pretty neat trick, there are drawbacks. You cannot pass an array from a function to a function. More precisely, you must have a name for the variable, that declare -p can resolve, in the current scope. Since variables created with declare in functions will be local (at least by default), you cannot pass it's name to another function, and have declare -p called upon it.

The client code (the code that calls the function) must always eval $() the returned value. We could work around this, if we mangle the returned code a little more:

ref_array() {
    local varname="$1"
    local export_as="$2"
    local code=$(declare -p "$varname")
    local replaced="${code/$varname/$export_as}"
    eval ${replaced/declare -/declare -g}
}

This allows you to do a simpler call, ref_array VAR1 VAR2 , and use $VAR2 after that, by appending a -g flag to declare, which makes the variable global, even in functions (declare created local variables when used in functions, by default), but then we introduce a side effect, namely, we might overwrite variables in the global scope, when passing stuff around, so use it with caution.

This was written by Norbert Kéri, posted on Sunday, January 29, 2012, at 16:50

Tagged as:
raindog308 wrote
Your first problem is not really a problem. Associative arrays don't store order of entry by design. In perl, python, etc. you can't retrieve from associate arrays in order of entry either. If you want that, use an indexed array, or keep both (an indexed array that has order of entry alongside an associate array for lookup by key). The usual way to access associative arrays in some kind of iterative order is to get a list of the keys, sort them, and then iterate over them.

As to the second...Bash doesn't have multidimensional arrays, but there are workarounds. For example, if you want a 10x10 array, you'd ideally do something like somearray[x][y] which won't work. Instead you could:

[php]
printf -v address "%02i %02i" x y
somearray[$address]="some value"
[/php]

Alternatively, write a function to flatten the array (x*10+y) and store it that way.

Good blog entry btw!

2014-01-25 21:19:41

raindog308 wrote
Sorry, that should be

[php]
printf -v address "%02i %02i" $x $y
somearray[$address]="some value"
[/php]

2014-01-25 21:20:28

Jason Antonacci wrote
Great post on aarrays and solutions. What vi colorscheme is that in the screen captures? Thanks!

2014-08-25 18:17:38

Norbert Kéri wrote
It's not a screen capture, it's a wordpress plugin called SyntaxHighlighter that does the highlightning.

2014-09-10 02:14:07

Post a comment

Providing your email is optional, it is never published or shared, it is only used for auto approval purposes. If you already have at least 1 approved comment(s) tied to your email, you don't have to wait for moderation, otherwise the author must approve your comment.

Please solve this totally random captcha